Mistral 7B v0.2 Base (released at SHACK15sf hackathon)

GitHub: https://github.com/mistralai-sf24/hackathon
\ X: https://twitter.com/MistralAILabs/status/1771670765521281370

New release: Mistral 7B v0.2 Base (Raw pretrained model used to train Mistral-7B-Instruct-v0.2)
\ 🔸 https://models.mistralcdn.com/mistral-7b-v0-2/mistral-7B-v0.2.tar
\ 🔸 32k context window
\ 🔸 Rope Theta = 1e6
\ 🔸 No sliding window
\ 🔸 How to fine-tune: