Selfhost an LLM
iii @ iii @mander.xyz Posts 121Comments 2,503Joined 12 mo. ago
iii @ iii @mander.xyz
Posts
121
Comments
2,503
Joined
12 mo. ago
Man Arrested After Fatally Stabbing Three in Roeselare, Known for Domestic Violence
Belgium Needs Eight New Nuclear Power Plants for Climate-Neutral Electricity by 2050, Says Federal Planning Bureau
Flemish Government in Final Push to Secure €1.5 Billion for Budget Balance by Monday
Ongoing problems at Brussels Airport this morning after cyberattack at an external service provider
These Ant Queens Seem to Defy Biology: They Lay Eggs That Hatch Into Another Species
After more than 100 years of mystery and rumours, Masonic lodge in Bruges opens its doors for the first time during Open Monuments Day
European car industry receives subsidies for electric cars, the battle over the end of fossil vehicles in 2035 continues.
One of these projects might be of interest to you:
Do note that CPU inference is quite a lot slower than GPU or the well known SAAS providers. I currently like the quantized deepseek models as the best balance between quality of replies and inference time when not using GPU.