We estimate the stock of human-generated public text at around 300 trillion tokens. If trends continue, language models will fully utilize this stock between 2026 and 2032, or even earlier if intensely overtrained.
  • Toto@lemmy.world
    3·
    5 months ago

    Seems to me the more pressing issue is fixing hallucinations. Also see a barrier of getting access to all that info because of competition.