Alright, this may be a bit of a loaded question. But I figured it may provide good insight to both myself and to others. I just came into a pretty beefy server - dual Xeon E5 2699 v3’s (18 cores each), 768 gigs of RAM. Ten front drive bays, 6 of which have 7.68T NVMes and 4 of which have 15.36T SAS drives. I’m thinking the NVMe drives will go into a single RAID 5 or 6 (thoughts?), and the 15360s I plan to use for more sensitive stuff so I’m planning dual RAID 1’s there. Boot drives will be a hardware RAID 1 of dual 1920G SATA SSDs. So again… pretty beefy. I believe this server would cost me ~$100/month to run, although I may try something where I keep it off 6/7 days of the week and only turn it on if I need it otherwise, I’m not sure yet. Thoughts on that are welcome too.
All of that said. I’ve got the power & the storage for some pretty neat projects. But I’ve not delved into anything of this nature before. I’ve heard of Plex, I’ve heard of Jellyfin, but I don’t really know what it all means past that. And I think it would be pretty neat to be able to dump some streaming service subscriptions and make up for a bit of the coin I’d be dumping to power this thing (may also host a Minecraft server with it, lol).
I’m very familiar with Linux/console, so that’s not really an issue. I’m erring towards either Arch or Ubuntu (fight me, I like both).
Thoughts? Ideas? I figured this was a good community to post this in but can remove if it isn’t.
Unraid is a wonderful OS that will let you explore the world of containerized applications and however many VMs you feel like configuring. Spin up and spin down whatever as you please. Terraria. Valheim. Starbound. CounterStrike.
First thing, though: you're going to want your whole goddamn network hooked through that thing. Run CAT 6. Do it right. Buy a Uninterruptible Power Supply that can keep that server humming through the first 10 minutes of a blackout (to gracefully shut down).
Time to look at things like Tailscale, Pihole, Plex. If you're going to run Minecraft then Google "Paper MC". You can replace Google Docs with nextcloud. Play D&D? It's Foundry time. Roll your own Lemmy. Roll your own Mastodon. (Back up your volumes.) Host your own website. Host other people's websites. (Back up your volumes elsewhere.)
All the people in the selfhosting and homelab communities will tell you what to do with that beef.
The biggest reason I personally use and would recommend Unraid is it simplifies everything, specifically around docker.
Deploying docker containers? There are community apps where people have set up scripts so all you have to do is fill in the blanks for your set up and bam, your container is deployed and running.
Managing you can add your own items and fill in your own blanks, or change them and it’ll deploy and remove the old container.
I’ve used portainer, compose, and looked into runtipi for docker management, and tried out windows server, Ubuntu, proxmox, truenas for HV/VE/OS, and while they all had bits I liked they all lacked something, and unraid had it all or a way to have it.
The initial reason was ragged arrays for why I chose it ever the others, but now I like its simplicity, and don’t find myself wanting for more control over anything.
Yes, for someone with limited Linux experience, Unraid is a better choice than Proxmox. And, the experience gained through configuring Unraid will be applicable should you want to move to Proxmox later.
Ragged arrays was also why I chose Unraid. They initially didn't have docker-compose support, you had to jam it in the boot script! Now, they have that very nice Docker management dashboard that I completely bypass because I prefer the CLI.
If you're going to run Minecraft then Google "Paper MC".
Honestly would much rather recommend Fabric unless you're looking to host a large scale public server. Serverside optimization mods like Lithium and Starlight are great and preserve the vanilla gameplay unlike Paper which breaks or disables a lot of mechanics by default.
I wish I could run CAT6, but I rent ! But someday, for sure. Internet speed is absolutely going to be my biggest bottleneck here. But Unraid is a great idea. And somehow, I never even thought of hosting my own website… that’s definitely one of the many moves here. Thanks!
I second the proxmox nomination! I use a fair amount of Ubuntu and Fedora, and with Proxmox, you just spin up whatever you want, whenever you want it. I currently have a machine with a few % of your machine's specs, and I'm running WordPress, jellyfin, pihole, LMDE, and a couple Ubuntu desktops (Mate and Gnome) on different VMs, all at once, like they're running on bare metal.
Kinda, yeah.
It's an open source but commercial product. The stable releases are paid, the beta is free. I've only been running a three machine cluster for a few months now, but it's been absolutely solid despite power outages, internet outages, a hard drive going pop...
That seems like a pretty substantial investment for something you don't have much experience with or a plan for what you'll do with it. I'm not criticizing. I'm actually kind of jealous. Maybe figure out how to host your own lemmy instance!
Lemmy instance would be cool - my biggest concern there is the whole issue Lemmy had w/ CP a few months ago where one person posts it & of course, due to how the fediverse works, that gets downloaded onto everyone’s servers. Seems like it could be a problem.
Otherwise - I do have quite a load of experience on the hardware-side of things, and do love me a good setup. It was more of “I know I’ll do things with it, I just don’t know exactly what just yet”, and after years of lusting after something like this, I’ve finally got the capital to pull it off. Plus a handful of really good deals I got, of course.
I used to have a 16 drive bay DAS and HP Procurve modular switch I scraped from an old managed IT employer. They make really good space heaters in the winter and are good year round as white noise machines for when you sleep.
Unfortunately? Just the internal one, lol. Biggest bummer about this server is that it’s a 1U blade, and all of the PCIe slots are only x8 as the rest of the lanes are more or less consumed by the NVMe slots.
Those are CPU from, like, 2014... they are definitely "beefy" (ie. needlessly powerful regardless of what you plan to do with them) but not more powerful than modern mid-range CPUs (which have way lower TDP).
This is not to mean that the server is necessarily a bad deal (how much is it?)... there's lot of stuff on top of the CPU, but you should definitely factor in the electricity cost (especially if you are not the one paying for it and/or are sensitive to climate issues) and see if you can maybe buy a more "humble" server with a modern CPU and fewer, bigger drives (each hdd consumes more or less the same amount of power, regardless of capacity).
Oh, make sure to check if those old CPUs have hardware-accelearted video codecs if you plan to do transcoding!
I got the server crazy cheap & couldn’t pass up the offer. Figured I’d throw some needlessly powerful CPUs in it because why not, lol. Power isn’t actually nearly as bad as I said… looks like it’d cost ~$40/month to run. Which is way more reasonable.
Drives-wise, they’re all SSDs. So more energy-efficient by a long shot, which is nice.
The only time I remember someone building such a beefy server was big startup opting out of cloud services like aws/azure and working on their own server since they still require devops manpower to deal with cloud anyway and it's cheaper this way for them.
Another thing - any recommendations, ideas, or tips for home setup would also be much appreciated. E.g. do y’all usually set up your own VPN? Haven’t done that before. Host anything locally that you think is a must-have? Thanks again!
Depends on how safe you want to get but you could look into VLANing off all your piracy stuff, then VLANing IoT, then the rest on another for security purposes.
If that’s the case you’d want a good router (Mikrotik for best bang for your buck but most difficult to use, Ubiquiti for the opposite), and a managed switch (I personally love HPE for switches. Their enterprise brand is much better than their consumer stuff). Then you can set that all up in whatever Hypervisor or OS, or whatever you choose to move those all around on the NICs to keep your precious stuff safe.
For set up, you’ll want to look at the *arr stack. Check out trash guides for a getting started, there’s also servarr for even more info. But with those you can set it to auto download movies, comics, tv, books, audiobooks, all sorts of stuff. Then there’s all sorts of ways to feed it to devices and out into the net to others if you choose.
But be very very very cautious about that last part, not just for the obvious reasons of laws and whatnot, but when you start to poke holes for allowing stuff out, you could be allowing stuff in. And there’s lots of people who want in. So setting up your external access with credentials, MFA, certificates if you can, my opinion on those 3 is must, should, could respectfully.
Then you can thing about backups. You should backup your new server once you get it all the way you like of course, but now you can keep your backups of all your computers. So do you want single file backups, directory backups, drive backups, baremetal backups? Some combo? All the above? Who knows it’s all your choice!
Then you can host databases, services, your own smart tech whatever. It’s a blast. Enjoy it all. But I also recommend looking into docker as well! It’s huge as far as hosting a bunch of services.
For drive config, depends on how you plan on using your server, and how you plan in dividing up the data between ssd and platter drives, but if it were my set up I’d do raid-10 for both arrays. Reason? Speed and single fault tolerance. Bigger reason? I don’t trust anything with a single copy. 3-2-1 rule. If you have data you need to have protected that can’t stand an array failure, it shouldn’t only be in the array. But that’s just me. I run multiple servers and keep cloud storage.
My must have is searxng, although anyone can have that in their laptop, it doesn't take many resources nor configuration.
I guess you may gift nextcloud accounts to family and friends too :)
With that much fast storage, I'd consider mining filecoin or something else that rewards low latency file-sharing. And fill it up with your own files over time.