Skip Navigation

LocalLLaMA @sh.itjust.works

thickertoofan @lemm.ee

4mo ago

Microsoft just released BitNet!

GitHub - microsoft/BitNet: Official inference framework for 1-bit LLMs

Let's go! Lossless CPU inference

4 comments

ELI5 1-bit module. With three attempts, i got nothing out of it, so I assume it's a simpler, more energy efficient model.
- It's a massive performance upgrade, which would make current sized models better and tiny phone-sized models viable. Only problem is that models need to be retrained to use it and afaik, no one significant has done it yet.
- i'm not the smartest out there to explain it but it's like ...instead of floating point numbers as the weights, its just -1,0,1.
Nice. Any additional info on how difficult it was to train this and whether we can expect more? They have a 3B model in the demo video, but doesn't seem like they released that... I mean I'd like something a bit larger.