Effective Long-Context Scaling of Foundation Models | Research - AI at Meta

Promising increase in context, obviously we've seen other methods like yarn and rope scaling, but nice to see Meta validating some methods and hopefully releasing the models themselves!