Any way to prune LLMs?
I don't know about that, but you could try GGML (llama.cpp). It has quantization up to 2-bits so that might be small enough.
minipasila @lemmy.fmhy.ml
Posts 0
Comments 1
I don't know about that, but you could try GGML (llama.cpp). It has quantization up to 2-bits so that might be small enough.