Skip Navigation

Singularity | Artificial Intelligence (ai), Technology & Futurology @lemmy.fmhy.ml

InternetPirate @lemmy.fmhy.ml

2y ago

Microsoft LongNet: One BILLION Tokens LLM — David Shapiro ~ AI (06.07.2023)

We could have AI models in a couple years that hold the entire internet in their context window.

Machine Learning - Learning/Language Models @lemmy.intai.tech

manitcor @lemmy.intai.tech

2y ago

Microsoft LongNet: One BILLION Tokens LLM — David Shapiro ~ AI (06.07.2023)

9 comments

We could have AI models in a couple years that hold the entire internet in their context window.
That's a really bold claim.
- Also not sure how that would be helpful. If every prompt needs to rip through those tokens first, before predicting a response, it'll be stupid slow. Even now with llama.cpp, it's annoying when it pauses to do the context window shuffle thing.
  
  Yeah, long term memory where ai can access only what it needs/wants is the way.
Anyone know what progress has been made with hallucinations .
- Perplexity has pretty much solved that since it searches the internet and uses the information it finds. But I don't know about any advances to solve it directly in LLMs.

9 comments