Thanks for this! I guess that settles my case deliberation at least. But man, getting all the things you mentioned for $170 on eBay is crazy good!!! I must just really need to dig in there a bit deeper (it's gonna be too easy to go broke with home networking as a hobby otherwise lol). I guess so long as I am memtesting my RAM and burning in drives right after getting them I should be okay. I think I also am overindexing on trying to get my power draw as low as possible instead of really doing the math out and seeing how long I'd need to be running something 24/7 before the power savings ends up outweighing the price differential. But thanks again!
Thank you! The case rec is super helpful actually, I was considering the Fractal Design Define R5 (though I wasn't sure how to feel about potentially spending the same amount on my motherboard as my case, but people seem to swear by it), so it's great to have confirmation they're actually worth it. I had given PC Part Picker a shot, but it seems like it doesn't quite catch the ECC memory support for some of the AMD systems (maybe because they're unofficial?) I will give it another shot though, and thanks again!
No I think we're aligned! I am not trying to say the "build literally everything" from scratch is a viable alternative. You could go all the way down the rabbit hole of building a compiler, your own programming language, a smelter to refine the metals you need to try to cobble together your own hardware. But of course that is not realistic, which was what I was trying to get at in my comment. Basically, given that it is not feasible to do everything by yourself, at some point it seems you have to decide to trust something to be a functional human and not devolve into solipsism. So the question I am asking is, what are your own evaluations of what is trustworthy? Do you trust coreboot more than AMI? Protectli versus Qotom? It seems to me that we have to make these sorts of evaluations, versus believing that because there is some risk to everything that those risks are all equal. Apologies if I am not being clear though.
Thanks for sharing! Did you end up staying with the stock firmware?
Thanks for this! Agree that coreboot is definitely the requirement that, if dropped, would open up the most other options. So far it sounds like folks are mostly willing to have some faith in stock firmware, which is great as a sanity check for me. Appreciate your response!
Sorry, imprecise wording on my part, I meant build as in build/code from scratch, not build from source!
Thanks so much! I'd seen Hunsn mentioned in a few places as well, so glad to hear that it's working well (and thanks especially for the memory ballooning tip, I'll try to remember that when I inevitably run into issues later).
Thanks so much for sharing this! I think reading through it helps refocus the question I guess I should have asked, which is "Which vendors do people trust more in practice, recognizing that at some point recursive paranoia has to end unless one has the time and skill to try to build literally everything on their own?" And as a question of probabilities, it feels a bit more manageable to try to make a call and move on. I'm sort of thinking of this thread as a way for me to calibrate my current probability estimates with people who know more than I do and have likely thought about this question more than I have. But the reminder that there isn't really going to be any certainty regardless of what I decide is well-taken.
(But if you don't mind my asking, what machine did you end up with from CN? How did you approach firmware?)
Yes, I'm US-based, and you make a great point that it's not as though US brands are inherently trustworthy either. That's why I'm leaning towards an open source (or as open source as possible) firmware, with the understanding that we're stuck with some proprietary blobs at the moment. I suppose I am thinking about it more from a harm reduction lens versus trying to find a bullet-proof solution.
Tl;dr
Very new to home networking, but planning to get some hardware to run OPNsense bare metal to replace my ISP all-in-one.
Requirements: AES-NI support, Intel NICs, supports coreboot, can handle Wireguard both to connect out to my VPN provider and also to allow me to connect back to services at home
Nice to haves: 2.5+ Gbps NICs, resources to support an IPS like Snort or Suricata.
Questions:
- Are people still using cheap AliExpress hardware despite potential security risks?
- If so, do you reflash your firmware? Are you comfortable counting on a script like Flashli, or do you use something like SPI?
- Would you still reflash your firmware even from a more trusted vendor, like Protectli or Deciso?
- What is a reasonable amount to spend on reasonable router hardware?
Some Options I’ve Seen Recommended/Am Considering:
- Protectli Vault Pro VP2420-4 (but open to other Protectli suggestions)
- AliExpress N100/N305 machines (though of course with the aforementioned security concerns)
- Used Thinkcenter M720q (though not sure how the power efficiency compares)
Thanks so much to anyone who takes the time to post your thoughts, I’m extremely grateful! __________________ Hi everyone,
Thanks so much again to those of you who left your thoughts on my thread a month or so ago. Progress on my NAS still continues at a glacial pace (mostly because I want ECC support and an iGPU that supports AV1 decoding on the same machine, but building around the W680 is a little pricey; in fact, might end up considering adding an A380 for AV1 support instead to open up some other options, but that’s a story for a different thread). But I am reaching out now because in the interim I’ve been trying to make progress on some more fundamental network infrastructure while I am figuring out plans for the bigger systems. Of course, even that turned into a bigger project than planned, which is why I am back to get some of your insights.
Just for folks who didn’t see my other thread, I am definitely a home networking noob generally, but am trying to build out my whole home network. In terms of the router hardware I am hoping to get, at minimum it would need AES-NI support and Intel NICs vs. Realtek, as well as support for coreboot for peace of mind (in an ideal world, even libreboot, but my understanding is basically no hardware released within a decade or more would allow that to run). At least 2.5 Gbps throughput would be nice, but not essential. Likely will be paired with a switch, so I’m also not overly concerned about port count (though maybe just for starting out 4 ports would be ideal and I can add in a PoE switch later).
In terms of planned use, I want to use Wireguard both to connect out to my VPN provider and also to allow me to connect back to services at home, as well as a few VLANs. Support for IPS like Snort or Suricata would be a plus, but it seems they can be resource intensive and I’m not currently thinking of them as a necessity.
A lot of resources I’ve seen suggested just grabbing a cheap machine off Amazon or AliExpress is the most cost-effective way to go, but it seems like there are some legitimate security concerns going that route. Are people still buying some of the cheaper AliExpress (e.g. Qotom, Topton, Cwwk) N100/N305 machines for their routers, even with concerns about backdoors (like Horse Shell in TP-Link firmware)? Are you reflashing firmware if you do so (and if you are, are you doing it through SPI vs. a script like Flashli)?
I’m the furthest thing from an expert, but just from a bit of poking around it seemed like if one wanted to reflash firmware in a 0 trust way, it looks like you’d need to either use SPI or JTAG vs. trying to do it through a script (and of course you’d need to take into account whether Boot Guard is enabled).
Would you reserve this treatment just for no-name router brands or would you reflash firmware the same way if you bought from a more trustworthy source like Protectli or even Deciso? Personally, my threat model (just trying to take back some privacy and control over my tech and not trying to stand single-handedly against the NSA) and current (low) skill level make me think I should just opt for a Protectli box that I know will work with coreboot (like the VP2420-4) and then move on with my life, but spending $300-$400 on a router seems like a lot (and perhaps I am robbing myself of the joy of having to figure out how SPI works).
Thanks so much for your thoughts! I remain extremely grateful to have the opportunity to tap into all of your collective wisdom (and hopefully at least save myself a few lessons learned the hard way). As long as its not obnoxious, I am also happy to share my progress and learning as I go in case it can save some time for other folks just starting out. Thanks again!
Thanks for sharing! Will probably try to go this route once I get the NAS squared away and turn back to localLLMs. Out of curiosity, are you using the q4_k_m quantization type?
This is exactly the sort of tradeoff I was wondering about, thank you so much for mentioning this. I think ultimately I would probably align with you in prioritizing answer quality over context length (but it sure would be nice to have both!!) I think my plan for now based on some of the other comments is to go ahead with the NAS build and keep my eyes peeled for any GPU deals in the meantime (though honestly I am not holding my breath). Once I've proved to myself I can something stable without burning the house down, I'll on something more powerful for the localLLM. Thanks again for sharing!
Wow, that sounds amazing! I think that GPU alone would probably exceed my budget for the whole build lol. Thanks for sharing!
This is definitely good advice. I tend to run my laptops into the ground before I replace them, but a lot of the feedback here has made me think experimenting with something much less expensive first is probably the right move instead of trying to do everything all at once (so that when I inevitably screw up, it at least won't be a $4k screw up.) But thanks for the sanity check!
Thank you so much for all of this! I think you're definitely right that probably starting smaller and trying a few things out is more sensible. At least for now I think I am going to focus on putting something together for the lower-hanging fruit by focusing on the NAS build first and then build up to local AI once I have something stable (but I'll definitely be keeping an eye out for GPU deals in the meantime, so thanks for mentioning the B580 variant, it wasn't on my radar at all as an option). But I think the thread has definitely given me confidence that splitting things out that way makes sense as a strategy (I had been concerned when I first wrote it out that not planning out everything all at once was going to cause me to miss some major efficiency, but I feel like it turns out that self-hosting is more like gardening than I thought in that it sort of seems to grow organically with one's interest and resources over time; sort of sounds obvious in retrospect, but I was definitely approaching this more rigidly initially). And thank you for the HDD rec! I think the Exos are the level above the Ironwolf Pro I mentioned, so will definitely consider them (especially if they come back online for a reasonable price at serverpartdeals or elsewhere). Just out of curiosity, what are you using for admin on your MC server? I had heard of Pterodactyl previously, but another commenter mentioned CraftyController as a bit easier to work with. Thank you again for writing all of this up, it's super helpful!
You are both totally right. I think I anchored high here just because of the LLM stuff I am trying to get running at around a GPT4 level (which is what I think it will take for folks in my family to actually use it vs. continuing to pass all their data to OpenAI) and it felt like it was tough to get there without spending an arm and a leg on GPUs alone. But I think my plan is now to start with the NAS build, which I should be able to accomplish without spending a crazy amount and then building out iteratively from there. As you say, I'd prefer to screw up and make a $500 mistake vs. a multiple thousand dollar one. Thanks for the sanity check!
Woah, this is big news!! I'd been following some of the older articles talking about this being pending, but had no idea it just released, thanks for sharing! Will just need to figure out how much of a datahoarder I'm likely to become, but it might be nice to start with fewer than 6 of the 8TB drives and expand up (though I think 4 drives is the minimum that makes sense; my understanding is also that energy consumption is roughly linear with number of drives, though that could be very wrong, so maybe I've even start with 4x a 10-12TB drive if I can find them for a reasonable price). But thanks for flagging this!
Amazing, thanks again for all for all of this! I'll start keeping my eyes peeled for any good deals on 3090s that pop up (though will probably end up prioritizing the NAS build first just to get my feet wet before diving straight into the localLLM world). But thanks again for taking the time to share!
Thanks for flagging this! I'd just passively absorbed second hand the mining rig fears, but you're totally right that it's not as though a regularly used overclocked gaming GPU isn't going to also be subject similar degradation (especially if the miner is intentionally underclocking). I guess the biggest fears then are just physical damage from rough install and then potential heat damage (though maybe swapping thermal pads and paste helps alleviate that?) And of course checking benchmarks for any weirdness if possible I guess...
(Also very curious about all of the HA stuff; it's definitely on my list of things to experiment with, but probably down the line once I've gotten some basic infrastructure in place. Very excited at the prospect though)
Tl;dr
I have no idea what I’m doing, and the desire for a NAS and local LLM has spun me down a rabbit hole. Pls send help.
Failed Attempt at a Tl;dr
Sorry for the long post! Brand new to home servers, but am thinking about building out the setup below (Machine 1 to be on 24/7, Machine 2 to be spun up only when needed for energy efficiency); target budget cap ~ USD 4,000; would appreciate any tips, suggestions, pitfalls, flags for where I’m being a total idiot and have missed something basic:
Machine 1: TrueNAS Scale with Jellyfin, Syncthing/Nextcloud + Immich, Collabora Office, SearXNG if possible, and potentially the *arr apps
On the drive front, I’m considering 6x Seagate Ironwolf 8TB in RAIDz2 for 32TB usable space (waaay more than I think I’ll need, but I know it’s a PITA to upgrade a vdev so trying to future-proof), and I am thinking also want to add in an L2ARC cache (which I think should be something like 500GB-1TB m.2 NVMe SSD); I’d read somewhere that back of the envelope RAM requirements were 1GB RAM to 1TB storage (though the TrueNAS Scale hardware guide definitely does not say this, but with the L2ARC cache and all of the other things I’m trying to run I probably get to the same number), so I’d be looking for around 48GB (though I am under the impression that using an odd number of DIMMs isn’t great for performance, so that might bump up to 64GB across 4x16GB?); I’m ambivalent on DDR4 vs. 5 (and unless there’s a good reason not to, would be inclined to just use DDR4 for cost), but am leaning ECC, even though it may not be strictly necessary
Machine 2: Proxmox with LXC for Llama 3.3, Stable Diffusion, Whisper, OpenWebUI; I’d also like to be able to host a heavily modded Minecraft server (something like All The Mods 9 for 4 to 5 players) likely using Pterodactyl
I am struggling with what to do about GPUs here; I’d love to be able to run the 70b Llama 3.3, it seems like that will require something like 40-50GB VRAM to run comfortably at a minimum, but I’m not sure the best way to get there; I’ve seen some folks suggest 2x3090s is the right balance of value and performance, but plenty of other folks seem to advocate for sticking with the newer 4000 architecture (especially with the 5000 series around the corner and the expectation prices might finally come down); on the other end of the spectrum, I’ve also seen people advocate for going back to P40s
Am I overcomplicating this? Making any dumb rookie mistakes? Does 2 machines seems right for my use cases vs. 1 (or more than 2?)? Any glaring issues with the hardware I mentioned or suggestions for a better setup? Ways to better prioritize energy efficiency (even at the risk of more cost up front)? I was targeting something like USD 4,000 as a soft price cap across both machines, but does that seem reasonable? How much of a headache is all of this going to be to manage? Is there a light at the end of the tunnel?
Very grateful for any advice or tips you all have!
__________________________
Hi all,
So sorry again for the long post. Just including a little bit of extra context here in case it’s useful about what I am trying to do (I feel like this is the annoying part of an online recipe where you get a life story instead of the actual ingredient list; I at least tried to put that first in this post.) Essentially I am a total noob, but have spent the past several months lurking on forums, old Reddit and Lemmy threads, and have watched many hours of YouTube videos just to wrap my head around some of the basics of home networking, and I still feel like I know basically nothing. But I felt like I finally got to the point where I felt that I could try to articulate what I am trying to do with enough specificity to not be completely wasting all of your time (I’m very cognizant of Help Vampires and definitely do not want to be one!)
Basically my motivation is to move away from non-privacy respecting services and bring as much in-house as possible, but (as is frequently the case), my ambition has far outpaced my skill. So I am hopeful that I can tap into all of your collective knowledge to make sure I can avoid any catastrophic mistakes I am likely to blithely walk myself into.
Here are the basic things I am trying to accomplish with this setup:
• A NAS with a built in media server and associated apps • Phone backups (including photos) • Collaborative document editing • A local ChatGPT 4 replacement • Locally hosted metasearch • A place to run a modded Minecraft server for myself and a few friends
The list in the tl;dr represent my best guesses for the write software and (partial) hardware to get all of these done. Based on some of my reading, it seemed that a number of folks recommend running TrueNAS baremetal as opposed to in ProxMox for when there is an inevitable stability issue, and that got me thinking more about how it might be valuable to split out these functions across two machines, one to hand heavier workloads when needed but to be turned off when not (e.g. game server, all local AI), and a second machine to function as a NAS with all the associated apps that would hopefully be more power efficient and run 24/7.
There are two things that I think would be very helpful to me at this point:
- High level feedback on whether this strategy sounds right given what I am trying to accomplish. I feel like I am breaking the fundamental Keep It Simple Stupid rule and will likely come to regret it.
- Any specific feedback on the right hardware for this setup.
- Any thoughts about how to best select hardware to maximize energy efficiency/minimize ongoing costs while still accomplishing these goals.
Also, above I mentioned that I am targeted around USD 4,000, but I am willing to be flexible on that if spending more up front will help keep ongoing costs down, or if spending a bit more will lead to markedly better performance.
Ultimately, I feel like I just need to get my hands on something and start screwing things up to learn, but I’d love to avoid any major costly screw ups before I just start ordering parts, thus writing up this post as a reality check before I do just that.
Thanks so much if you read this far down the post, and for all of you who share any thoughts you might have. I don’t really have folks IRL I can talk to about these sorts of things, so I am extremely grateful to be able to reach out to this community. -------
Edit: Just wanted to say a huge thank you to everyone who shared their thoughts! I posted this fully expecting to get no responses and figured it was still worth doing just to write out my plan as it stood. I am so grateful for all of your thoughtful and generous responses sharing your experience and advice. I have to hop offline now, but look forward to responding to any comments I haven’t had a chance to turn to tomorrow. Thanks again! :)