I've recently played with the idea of self hosting a LLM. I am aware that it will not reach GPT4 levels, but beeing free from restraining prompts with confidential data is very nice tool for me to have.
Has anyone got experience with this? Any recommendations? I have downloaded the full Reddit dataset so I could retrain the model on this one as selected communities provide immense value and knowledge (hehe this is exactly what reddit, twitter etc. are trying to avoid...)
Thanks! How would you say the learning curve for Ansible is? I want to dedicate a week only to that so I got a little bit of time at the moment. My goal is to automate everything as much as possible and make the whole system low maintainance. I don't have the time anymore to fix stuff or maintain the whole thing multiple times a month.
I know, selfhosting always comes with the risk of everything crashing down, but that risk I'm taking and trying to mitigate it as much as possible with best practices etc.
I recently switched from TrueNAS to Synology for my NAS. TrueNAS had served me well, but I no longer had the time to manage it effectively.
On this occasion, I decided to overhaul my entire home lab, which had gotten pretty messy over the years. As part of this overhaul, I will be discarding my old TrueNAS device due to its high power consumption and bulkiness. I will keep a NUC and another NUC alternative with slightly lower specs, but with 2+ LAN ports.
With this configuration, my plan is to use Proxmox on the NUC as the primary system and use the second NUC as a backup. The backup NUC however ha a dedicated connection via multiple LAN ports directly to the Synology NAS, so that would be ideal to storage intensive tasks.
My primary use case will be running containers and a few VMs for services like Git, Pi hole, backup services and more. Although my Synology NAS supports running containers and VMs, I prefer to keep things separate. I've already taken care of my infrastructure needs and won't be hosting pfSense or similar services.
Since I haven't looked into best practices lately, I'm very interested in learning new technologies like Ansible for automation.
I'm especially interested in understanding how to automate installs and updates while working with containers and VMs. I am considering whether to stay with Proxmox or go for a simpler distribution like Debian, Fedora or others.
Thanks for you insights!