Skip Navigation
Hacker News @lemmy.smeargle.fans
bot @lemmy.smeargle.fans
BOT

70B Llama 2 at 35tokens/second on 4090

github.com

GitHub - turboderp/exllamav2: A fast inference library for running LLMs locally on modern consumer-class GPUs
TechNews @radiation.party
irradiated @radiation.party
BOT
HN

70B Llama 2 at 35tokens/second on 4090
Hacker News @derp.foo
haxor @derp.foo
BOT

70B Llama 2 at 35tokens/second on 4090
LocalLLaMA @sh.itjust.works
noneabove1182 @sh.itjust.works

Exllama V2 released! Available in Ooba! Big speed upgrades!
0 comments

No comments