Container cluster build - budget desktop or mini PCs

the_amazing_raisin

Hi folks,

So I'd like to get a home lab setup for experiments with software developent, containers, kubernetes and generally running a load of web services locally

The basic setup will be 2-3 PCs running a bunch of containers with some storage backend like Ceph or ZFS, mostly depending on my tolerance for pain

Originally I was thinking of getting some office PCs second hand, but they don't seem to come up that often and get snapped up or have very poor specs

So second idea was to build a basic PC and if that works out then just build another as funds allow

I'm also debating one of those mini PCs from the likes of GMK Tec. You can get a fairly decent spec one for around $450. Presumably shipping and customs charges might add a bit but overall they're looking a lot more affordable

Couple a few if those with some USB4 hard drive cages and it should make a decent storage cluster

Obviously a desktop has some advantages in terms of hardware expansion and component replacement, but I'm really struggling to get one into the same price range as the mini PCs

Anyone have any recommendations or thoughts on either option?

Coyler

This is really for you to make a call on. Whatever works for you, works for you. Being honest, I think you are in overkill territory talking about two brand new mini PCs at €450 each and a handful of drives for just a home lab. Just go onto ebay and search for SFF PCs and you'll find a wealth of options. Any Intel 6000,7000 or 8000 series CPU will be absolutely grand for development and clusters. You could easily get 2-3 of those and rely on local SSD storage.

If you want something larger, don't do USB drives. Unraid and Proxmox both support ZFS storage pools. Ceph is good but it's not designed for home labs. Unraid is way more forgiving on what hardware you throw at it. Ceph's a lot stricter. Gun to my head, start with ZFS on Unraid and go from there. You lose nothing apart from $49 on the Unraid license but Unraid is like a small pot, always find a use for it. Find any old PC with 4-6 SATA ports, throw in 4-6 old SATA SSDs and away you go.

the_amazing_raisin

Cheers for that, yeah I'm probably in overkill territory with the hardware

Then again, once you have a container cluster you tend to answer every problem by adding a new container instance 😁

Good shout on the small form factor PCs, I'd originally been looking at tower PCs which definitely jumped the price up a lot. At that point the mini PCs were cheaper and better value

In terms of storage, ZFS is definitely the easiest option I've seen so far, however I believe the OpenZFS doesn't support primary/secondary servers out of the box. I'd been looking at doing some OpenStack work so Ceph would be helpful for that, however it's also designed for massive storage arrays in data centers so is oversized for what I need

I'll look into my options a bit more and check out Unraid

This is all very much in the concept phase, so liable to change a lot as I throw out my ideas

BottomBurp

Another thing to consider is the power consumption. I had a HP Microserver for a homelab and it was using a lot of juice. I moved to a 2010 laptop which was budget when bought new and it's great. Sips power, runs a bunch of automation, media, and other stuff without breaking a sweat.

the_amazing_raisin

https://www.boards.ie/discussion/comment/123863979#Comment_123863979

I suppose that's a point in favour of the mini PCs, they use laptop CPUs so have a much lower TDP

I'd love to get some framework motherboards and 3D print some cases. However that's not exactly cheap and I don't have a 3D printer so that kind of grows the project scope out of control 😂

Nate--IRL--

When I had to play about with this stuff - I just built 4 VMs on a single physical machine and used those for the cluster. Might be cheaper just to buy more memory for your existing computer?

Nate

the_amazing_raisin

https://www.boards.ie/discussion/comment/123864224#Comment_123864224

Oh yeah I could probably fit it onto 1 machine but I want to do some work with clustering which kind of makes a thruple a necessity

I've an old NUC which I'll probably get the containers working on and then I'll look at explanding it out a bit