Llama-2, Mo’ Lora (proof of concept MOE of LoRAs)
Llama-2, Mo’ Lora (proof of concept MOE of LoRAs)

crumbly.medium.com
Llama-2, Mo’ Lora

https://twitter.com/aicrumb/status/1681846805959528448?t=sG6Xn4p0hodDoB-g7gmuJQ
Very interesting concept, excited to see where it goes