Idea: Selfhosted Interface for LLMs
Idea: Selfhosted Interface for LLMs
Like SearchXNG but for chatbots.
Acts as a trusted inbetween that takes the hit on all the tracking for you and you need no account.
There would be a self-hosted server with a webinterface, where you can choose which of the popular chatbots or hosted open source models you want to ask.
The not free Accounts could be payed with donations maybe?
Basically like duck.ai but without a big company involved.
You're viewing a single thread.
I was looking for something like this. The idea is:
- Run an open model like LLAMA or Mistral on my home server
- Access it from a client on my phone, saving my poor phone's processor and battery.
Does something like this exist?
@mulcahey @not_IO Take a look at: https://openwebui.com/
I'm running the docker container on my home server behind my firewall on an - I think - 6070 GPU (may be a 3070 - it's one of the cheaper, older ones).
It's a wee bit slower than one of the cloud based LLMs, but my data stays local. I like the Qwen model.