Building a Lab

Dr4gul4 · 07-07-2020 9:32pm #1

Good evening folks,

So, I currently have an N54L running Synology, it's old, very old ! and I find it's not really up to the task's I set it many years ago.
Coupled with out dated software i now find I have a need to spool up some virtual machines for pentest purposes.

So, I have some ideas in mind, and a budget, something relatively silent to run 24/7 with a single GPU or dual GPU's based on cost and if i can get them working in parallel for hashcat.

Disk space is not a concern really, I dont require a huge amount of space for plex purposes, say 1/2 TB there.

No backup, I dont care if it all falls down, as long as i can rebuild it all again within say 1 /2 days .

I've a feeling 32 Gb of memory and a min of 6/8 cores, or dual chips maybe suit., and some form of SSD for transcoding / cache in plex ( if im correct )

Further details below of course, nothing too fancy case wise, it's not build for drag racing, it's build for power

1. What is your budget? [€1000 - 1250]

2. What will be the main purpose of the computer? [ Esxi / Proxmox host - Plex on lan, Several Vm's for Pentesting, Hascat VM. ]

3. Do you need a copy of Windows? [No]

4. Can you use any parts from an old computer? [ Nope ]

5. Do you need a monitor? [No]

5a. If yes, what size do you need. [N/A]

5b. If no, what resolution is your current monitor and do you plan to upgrade in the near future? [1920x1080/1440x900/etc.] [N/A]

6. Do you need any of these peripherals? [N/A]

7. Are you willing to try overclocking? [Yes/ maybe]

8. How can you pay? [All of the usual methods i guess.]

9. When are you purchasing? [In 30-60 days]

10. If you need help building it, where are you based? [ No building assistance required]

Dr4gul4 · 08-07-2020 7:33pm

Yikes, nobody brave enough to take this one on given my budget maybe ?

Serephucus · 08-07-2020 8:23pm

I have a 1950X and X399 mobo that's not getting much use if they're any use to you as a base?

Dr4gul4 · 09-07-2020 8:18am

Serephucus wrote: »

I have a 1950X and X399 mobo that's not getting much use if they're any use to you as a base?

Tempting, 8 /12 or 16 cores ? Both are still regarded as heavy hitters i believe, if 18 /24 months old if im correct ?

google just told me it's the 16 core version

Coyler · 09-07-2020 6:14pm

Just an FYI, the SSD will do nothing for transcoding. That's one of those tech myths. However, look up how to cache VMs in the SSD though. That really helps.

If you want to improve transcoding, be aware *nix based plex only works with Intel iGPUs and Nvidia cards. But you are putting in GPU, so just get an Nvidia one.

Dr4gul4 · 09-07-2020 7:32pm

Coyler wrote: »

Just an FYI, the SSD will do nothing for transcoding. That's one of those tech myths. However, look up how to cache VMs in the SSD though. That really helps.

If you want to improve transcoding, be aware *nix based plex only works with Intel iGPUs and Nvidia cards. But you are putting in GPU, so just get an Nvidia one.

Fair play for the caching tip at the os level, Had pidegon holed an Nvidia card, would love 4 1080 TI's of course, but, it's not practical ! .

Plex use is really in house only, but im stuck at 720p, cant go higher on the res at the mo, and the version is old, so any boost will be an improvement.

Still not sure on the underlying Os, freenas maybe, had a better look at proxmox, and i see it's nas based ? so, not sure how well that might suit, i'd rather one host, JOBS/ RAID, and carve up the storage, locally, via NFS shares.

Coyler · 09-07-2020 8:22pm

Proxmox or Unraid is your answer here. Proxmox is a hypervisor first and file server second. Unraid is the other way around. However, gun to my head, Unraid is a better choice for a beginner. Much bigger community, more mature and a ton of how-to videos on YouTube. Proxmox is good as well, not knocking it, but it might not be the first option to go for.

As for transcoding, once you have any halfway decent CPU in there even the software transcoding can do several streams so don't fret about it too much. Just be sure to check the Nvidia NVENC support matrix when you are grabbing a GPU.

Serephucus · 10-07-2020 5:17pm

As someone who's using unRAID currently, I can recommend it. I've also played with ESXi, and have yet to play with Proxmox properly.

unRAId should do everything you want. As Coyler says, it's a storage box first, an hypervisor second, but it does all the stuff with VMs you'd expect. Slightly more fancy things like snapshots are possible, but you'll need to go through CLI for that rather than the (very user-friendly) GUI. It has a really nice Docker implementation though that makes setting things up really simple. Have a look at Spaceinvader One on Youtube for tutorials an ideas.

And apologies, I meant to get back to your last post; the 1950X is indeed a 16-core chip.

tazzzZ · 10-07-2020 11:46pm

I third unRAID... it doesnt support all filesystems natively.... as serephucus alluded too ie zfs. but there is a really helpful community for setting that and a multitude of other things up. Only thing is its not freeware (trials are available though). but its been worth every penny to me. even if it has resulted in me spending a stupid amount of money on becoming a data horder!

Also if consider plex transcoding to ram if you can. saved un needed writes to hdd or ssd. Also if plex is in use only in the house surely direct plays are the way to go. no transcoding required and little resources req to do that. just make sure you have players that support the video format.

Dr4gul4 · 11-07-2020 10:05am

Serephucus wrote: »

And apologies, I meant to get back to your last post; the 1950X is indeed a 16-core chip.

1950x and Board is nice, I hope you wont think me rude, but i'd prob aim for new hardware, simply down to life span, that is of course unless you're giving them away for a song

tazzzZ wrote: »

I third unRAID... it doesnt support all filesystems natively.... as serephucus alluded too ie zfs. but there is a really helpful community for setting that and a multitude of other things up. Only thing is its not freeware (trials are available though). but its been worth every penny to me. even if it has resulted in me spending a stupid amount of money on becoming a data horder!

For $60 i'd happily invest, i took a brief look this morning and it does appear to be a good alternative, with great community support.

tazzzZ wrote: »

Also if consider plex transcoding to ram if you can. saved un needed writes to hdd or ssd. Also if plex is in use only in the house surely direct plays are the way to go. no transcoding required and little resources req to do that. just make sure you have players that support the video format.

Jesus, talk about a moment of clarity ! I'll have to check, but i've a feeling it's all direct play on lan, but the bottle neck is on the N54L CPU right now, buffering over 720p .

I'll do some diggin, maybe it's config related, but either way, this old girl need to be sent to the farm soon.

Great advice so far folks, really appreciate it ...

Just need to match up some hardware now soon

Inviere · 11-07-2020 10:17am

I'll 4th Unraid. Serephucus convinced me last year to go this route instead of an all in one Synology unit....I'm sooooo glad I did. I've got Unraid doing more than I originally envisaged, and as such, would have wasted money on a Synology. The community for it is superb I must say, always help on hand if needed.

I've just upgraded the other day to a Plus license, added in a SAS card & two new Ironwolf 6TB drives*....was a little daunting doing it as I haven't messed with the array config since initial set up & I'd forgotten everything, but their forums & subreddit are a wealth of info. Job done in a few mins, parity rebuild overnight last night (swapped my original parity drive WD Red out into the data pool, & put an Ironwolf in as parity instead...damn SMR bs by WD...Seagate for me from now on), and everything back running sweet now.

*Serephucus also laughed at me when I said 18TB would be plenty for me...he was right....again

Dr4gul4 · 11-07-2020 10:27am

Inviere wrote: »

*Serephucus also laughed at me when I said 18TB would be plenty for me...he was right....again

18TB

I've lived off 1 TB for the last 6 years

But of course it's all personal pref, in my case, VM's and pure grunt is more important right now, of course i'll prob put something a little bigger than 1 TB in for storage needs anyway. but performance would be my key focus.

Didn't realize WD RED had gotten so bad these days,

Serephucus · 11-07-2020 12:22pm

Dr4gul4 wrote: »

1950x and Board is nice, I hope you wont think me rude, but i'd prob aim for new hardware, simply down to life span, that is of course unless you're giving them away for a song

That's entirely fair. I'd have a look at ASRock Rack's X470D4U board. I'm not sure if you need remote management for your setup, but I find it really handy.

You could pop any cheap Ryzen CPU in there for the moment, then when you figure out more clearly how much power you need, you can stick a 4th gen chip in there when they launch later in the year.

Inviere wrote: »

I'll 4th Unraid. Serephucus convinced me last year to go this route instead of an all in one Synology unit....I'm sooooo glad I did. I've got Unraid doing more than I originally envisaged, and as such, would have wasted money on a Synology. The community for it is superb I must say, always help on hand if needed.
[snip]
*Serephucus also laughed at me when I said 18TB would be plenty for me...he was right....again

It's really is tough being right all the time... I don't know how I manage really. :cool:

If you do go unRAID, the rule of thumb for storage is go for the single biggest capacity drive you can afford (that isn't an absolutely absurd €/TB). You'll pay a little more for the storage now, but it makes expansion later on that bit easier because you don't have to go swapping parity drives and the like, you need less drives in your box overall, less noise, heat, all that.

tazzzZ · 11-07-2020 3:10pm

I started the same.... i had been using windows home server 2011 up until whenever i changed over... i cant remember exactly but i believe i was sitting at < 4TB storage at that time... i now have an array of 91TB... about another 15TB in unnassigned devices, non array devices (for non unraiders out there), in ssd's and external hdd.

It becomes an absolute money pit if you let it. I am currently using a ryzen 1700 and im getting limitted by PCIe lanes atm. I want to have 2 nvme drives for caching but adding the 2nd is knocking out the last pcie slot on the mobo which is my 10gb nic. So now, like an absolute mental person, im looking at threadrippers and epyc setups... just to add a 2nd nvme... admittedly i do have a second HBA card in mind for a jbod connection and possible ssd zfs array...

You can see by my tangent how much of a rabbit hole this can become. but i literally started off with an old amd phenom 3 core that i unlocked to a 4 core and i got a 3x5.25inch to 5x 3.5inch HDD cage and went from there. Actually still have that whole system just sitting here behind me unused XD

Serephucus · 11-07-2020 3:38pm

tazzzZ wrote: »

I started the same.... i had been using windows home server 2011 up until whenever i changed over... i cant remember exactly but i believe i was sitting at < 4TB storage at that time... i now have an array of 91TB... about another 15TB in unnassigned devices, non array devices (for non unraiders out there), in ssd's and external hdd.

It becomes an absolute money pit if you let it. I am currently using a ryzen 1700 and im getting limitted by PCIe lanes atm. I want to have 2 nvme drives for caching but adding the 2nd is knocking out the last pcie slot on the mobo which is my 10gb nic. So now, like an absolute mental person, im looking at threadrippers and epyc setups... just to add a 2nd nvme... admittedly i do have a second HBA card in mind for a jbod connection and possible ssd zfs array...

You can see by my tangent how much of a rabbit hole this can become. but i literally started off with an old amd phenom 3 core that i unlocked to a 4 core and i got a 3x5.25inch to 5x 3.5inch HDD cage and went from there. Actually still have that whole system just sitting here behind me unused XD

Truer words have never been spoken. I started with 2x2TB drives on Server 2016, running in a basic desktop full of the cheapest hardware I could get my hands on at the time. I'm now in a 4U rackmount case with 84TB array storage, and probably 3-4TB of SSDs and unassigned devices, and am mulling over a second server for GPU passthrough with Parsec and a ZFS array.

I'll say what I said the last time someone asked me about server advise: Ye be warned. :pac:

AlmightyCushion · 11-07-2020 5:50pm

Telling yourself 'I'm just gonna build a small server with a few TBs storage' is probably the same as saying 'I'm just gonna take a little bit of heroin'.

Dr4gul4 · 11-07-2020 5:54pm

I'm just putting it out there, but 'Storage Anonymous' might not be a real thing, but you guys floating about at 70-100 TB , crazy ! I couldn't even imagine holding that much 'data' for in house use these days.....horses for courses

Serephucus wrote: »

I'd have a look at ASRock Rack's X470D4U board. I'm not sure if you need remote management for your setup, but I find it really handy.

God damn, that is a serious board ! I'll have a dig to see where the cpu limitation are on it, plus the cash concerns

tazzzZ · 11-07-2020 6:58pm

Dr4gul4 wrote: »

I'm just putting it out there, but 'Storage Anonymous' might not be a real thing, but you guys floating about at 70-100 TB , crazy ! I couldn't even imagine holding that much 'data' for in house use these days.....horses for courses

We all started out the same way... it will suck you in when you get going! first for me it was about getting a nice automated setup. Then having plex running as fast and as well as possible. then i had a few friends join my plex so i wanted to push the transcoding side of things... then i wanted to have 10gbe networking between my main pc and server.... then.... you see where im going. And dont even get me started on the money im throwing into this to keep myself entertained during lockdown... i ended up changing my whole home networking gear to ubiquiti...

Serephucus · 11-07-2020 10:12pm

Dr4gul4 wrote: »

God damn, that is a serious board ! I'll have a dig to see where the cpu limitation are on it, plus the cash concerns

Then it's a good thing they absolutely don't make a TRX40 board, no sir. Very good thing. :pac:

tazzzZ wrote: »

We all started out the same way... it will suck you in when you get going! first for me it was about getting a nice automated setup. Then having plex running as fast and as well as possible. then i had a few friends join my plex so i wanted to push the transcoding side of things... then i wanted to have 10gbe networking between my main pc and server.... then.... you see where im going. And dont even get me started on the money im throwing into this to keep myself entertained during lockdown... i ended up changing my whole home networking gear to ubiquiti...

I built a whole new server during lockdown. It's dangerous I tells ya.

tazzzZ · 11-07-2020 11:18pm

not to hyjack the thread anymore than we already have... but would you go epyc or threadripper??? bare in mind i dont use it for vms really.... so higher clocks arent as important. i just cant justify the cost of the lowest core threadripper... i can do epyc for alot cheaper it seems. biggest thing im looking for is a good mobo with 10bge and many many pcie slots... asrock epyc server boards look nice.

z0oT · 11-07-2020 11:19pm

Serephucus wrote: »

That's entirely fair. I'd have a look at ASRock Rack's X470D4U board. I'm not sure if you need remote management for your setup, but I find it really handy.

You could pop any cheap Ryzen CPU in there for the moment, then when you figure out more clearly how much power you need, you can stick a 4th gen chip in there when they launch later in the year.

I have that board, and I'm mostly happy with it. It was my introduction to IPMI, and having had it now I would never go back to not having it. Being able to actually debug hardware problems or install an OS remotely is so insanely handy once you get used to doing it that way.

I have had some problems with PCIe passthrough on that board though, and the BIOS revision is seriously lagging from ASRock. They haven't released a BIOS with the newest AGESA version yet, but you can reach out to their support directly to get a newer Beta version if you want. I haven't tried this myself.

For the OS on my system, I went with a headless install of Debian administered entirely via SSH over the command line. Debian is kind of bread and butter for me at this stage, having used it for so long. The cool thing is that I've managed to replicate probably most of the functionality of something like Unraid in doing so, with the main exception being the real time parity.

For me it's kind of fun to really learn the inner workings of things which not having a GUI forces you to do. Some things like PCIe passthrough can be very challenging though which is a downside. Not necessarily recommending going this way, it is of course not for everyone.

I started with my own server in a serious way probably earlier this year. Previously I had an Athlon 5350 setup with a few big HDDs in it that just ran Plex, then I upgraded that to a R3 2200G doing the same thing. The warnings of it being a drug are really spot on. I'm getting seriously tempted to purchase an Eypc based system to get real server class hardware now. You'll never start small and stay small!

As for reccommendations, it might be worth trawlling second hand stuff given the budget. The real server class stuff is the best to get, but the prices can be killer.

Inviere · 12-07-2020 7:48am

Dr4gul4 wrote: »

18TB I've lived off 1 TB for the last 6 years

Hell, if I could store myself on Unraid at this stage I think I would (parity protected of course!). Yeah, the requirements of an Unraid server have a tendency to grow, the more you realise the cool and different things it can do...so your storage needs are inclined to follow suit.

This brings a different problem....parity itself shouldn't be viewed as redundancy...so you're left with a large Unraid array and the need to have its data protected. I use offline usb external backups of everything....so yeah, moar hdd's!!

Inviere · 12-07-2020 8:02am

AlmightyCushion wrote: »

Telling yourself 'I'm just gonna build a small server with a few TBs storage' is probably the same as saying 'I'm just gonna take a little bit of heroin'.

Dr4gul4 · 12-07-2020 4:24pm

Damn storage junkies

Ok, so the basic premise would be, buy the storage, it's cheap, ish. you'll be thankful in the long term.

I've been looking at a few diff builds and see some using M.2 drives ( 2 or 3 where applicable ) and some 2.5 SSD's along with the usual SATA 3/4/5 + TB drives.

As im an unraid noob, I'm wondering how they all fit together, i'm guessing the M.2's could be used to cache OS's and plex data ? the standard SATA's are hard storage. the 2.5 .... boot for unraid maybe ?

Any rough suggestions ? or " you should look at this concept for performance. "

tazzzZ · 12-07-2020 4:55pm

Dr4gul4 wrote: »

Damn storage junkies

Ok, so the basic premise would be, buy the storage, it's cheap, ish. you'll be thankful in the long term.

I've been looking at a few diff builds and see some using M.2 drives ( 2 or 3 where applicable ) and some 2.5 SSD's along with the usual SATA 3/4/5 + TB drives.

As im an unraid noob, I'm wondering how they all fit together, i'm guessing the M.2's could be used to cache OS's and plex data ? the standard SATA's are hard storage. the 2.5 .... boot for unraid maybe ?

Any rough suggestions ? or " you should look at this concept for performance. "

Ok well ill give you my 2 cents on this and others im sure have their own ideas.

Firstly unRAID runs directly off the usb stick so no boot device req. which is great as everything is then storage.

As for cache drives i personnally have found that ssd's have been best for me. I have recently tried M.2(nvme) and i find (or i think) it is getting limitted by the DRAM cache in it. ie when it fills it slows down. This can happen on bigger transfers ive found. And as a result i have had huge fluctuation of speed with them. And even my top end speed with them was similar to the SSD anyway. Im on 10gbe and both seem to max out at about 400MBps but SSD will sustain that for the whole transfer where nvme died off in my case. So in the end i've settled on the following setup..

Array/Parity - HDD
Cache - SSD
docker/VM - nvme (including appdata)

I also assign certain dockers their own appdata drive to ensure other dockers dont effect it. ie plex runs off its own SSD.

Serephucus · 12-07-2020 7:20pm

So unRAID has 4 main device types, if you like:

1) Parity - This can be one drive or two, and gives you that number of drives worth of tolerance in the event of a failure of a drive in your array.
2) Array - Your basic unit of storage. The size of your largest data drive, cannot exceed the size of your parity drive.
3) Cache - Usually a faster drive to store things like VMs and Docker containers, and can also serve as a write cache to speed up transfers to your server. These can be used as a single drive, or a pool of drives using BTRFS to offer RAID 1 redundancy.
4) Unassigned Device - You technically need a plugin for this, but it's such a core feature I'm including it here. It's what it sounds like: A disk you add that's separate from the array and the cache to be used as you like.

Some notes, on cache and storage arrangements:

At the moment, only 1 cache is possible (whether a single drive or a pool), but in the latest version of unRAID (currently in beta) multiple caches are possible. This lets you do things like keep your VMs on a separate disk/disks from the day-to-day stuff you're writing to the array, or from your Docker containers, for example.

There is also currently a bug that's causing massive write amplification on BTRFS cache devices.
(A single cache disk can be formatted as XFS - like the array - or BTRFS, but to be in a pool of more than one device, the drives must use BTRFS)
In my case, this meant I was writing close to a terabyte a day to my cache, which wasn't great, so I dropped a couple of drives and moved to a single XFS cache, with hourly backups of stuff that's cache-only to the array to give some form of protection.
For context, this is probably the worst bug I've seen in unRAID since I've used it, but they're working on it in the betas of the next release at the moment.

Dr4gul4 · 12-07-2020 7:31pm

Fair play guys, i think i've some good clarity on the over all layout now, given the info ye have shared, and some YT video's ..

Back to my hardware search soon ...

Coyler · 13-07-2020 12:40am

Something came to me regarding plex and switching from 720p to 1080p. What router are you using? Chance ares, as tazzzZ says, you will be able to direct play most content anyway and you might be hitting a bottleneck with your router. Best confirm before you buy all this fancy new gear. Now, while we'd all love to get a ubiquiti setup like tazzzZ (Jealous), you can still get some very decent routers for a lot less than you think. Now, you may end up moving the router functionality to your server eventually but good Wifi is just something you have to pay for.

Anyway, what I'm saying is don't forget your network. Your server is only as good as it is.

Dr4gul4 · 13-07-2020 8:46am

Coyler wrote: »

Something came to me regarding plex and switching from 720p to 1080p. What router are you using? Chance ares, as tazzzZ says, you will be able to direct play most content anyway and you might be hitting a bottleneck with your router. Best confirm before you buy all this fancy new gear. Now, while we'd all love to get a ubiquiti setup like tazzzZ (Jealous), you can still get some very decent routers for a lot less than you think. Now, you may end up moving the router functionality to your server eventually but good Wifi is just something you have to pay for.

Anyway, what I'm saying is don't forget your network. Your server is only as good as it is.

On lan only Coyler, Standard unmanaged switch 10/100, backed with Cat6e.

I was doing some investigation over the weekend and can see plex is direct streaming my 720p, I wonder if it's the audio on "other" files that maybe causing an issue with the CPU. i'll do some 1080p tests soon to try and identify whats spiking the CPU.

Either way ! the upgrade is required, the core reason being, i'm running home assistant on a Pi3, which has reached it's limitation unfortunately, so i'd like to move that to a docker + the hashcat etc requirements..

tazzzZ · 13-07-2020 11:09am

Dr4gul4 wrote: »

On lan only Coyler, Standard unmanaged switch 10/100, backed with Cat6e.

I was doing some investigation over the weekend and can see plex is direct streaming my 720p, I wonder if it's the audio on "other" files that maybe causing an issue with the CPU. i'll do some 1080p tests soon to try and identify whats spiking the CPU.

Either way ! the upgrade is required, the core reason being, i'm running home assistant on a Pi3, which has reached it's limitation unfortunately, so i'd like to move that to a docker + the hashcat etc requirements..

Im assuming you just forgot to put the /1000 on the end of the switch bit... either way shouldnt be a bottleneck for 720p.

In fairness its worth the upgrade. plenty of good bargains to be had on hardware out there. In the past I had a very nice middle ground cpu with the 4790S(also a T varient) thats low power and has onboard igpu which you can use for transcoding if thats something you ever do and dont want a seperate gpu for it. I would look for the more modern equivalent. Maybe a 9900t or lower tier or previous gen varient?

The reason id choose this over a ryzen is quicksink works well with plex if i remember correctly and can do even 4k transcodes (but i wouldnt advise this for other reasons). It has a marked tdp of 35W which is nice and managable... even if these are very loose guidance of power usage.

If you have another gpu to use in this then id be going ryzen all day long personally. cost to preformance is just a no brainer on this IMO.

AlmightyCushion · 13-07-2020 11:27am

tazzzZ wrote: »

Im assuming you just forgot to put the /1000 on the end of the switch bit... either way shouldnt be a bottleneck for 720p.

In fairness its worth the upgrade. plenty of good bargains to be had on hardware out there. In the past I had a very nice middle ground cpu with the 4790S(also a T varient) thats low power and has onboard igpu which you can use for transcoding if thats something you ever do and dont want a seperate gpu for it. I would look for the more modern equivalent. Maybe a 9900t or lower tier or previous gen varient?

The reason id choose this over a ryzen is quicksink works well with plex if i remember correctly and can do even 4k transcodes (but i wouldnt advise this for other reasons). It has a marked tdp of 35W which is nice and managable... even if these are very loose guidance of power usage.

If you have another gpu to use in this then id be going ryzen all day long personally. cost to preformance is just a no brainer on this IMO.

I have a 4570k in my current PC. When I eventually build a new pc (probably won't be until next year) I plan on turning my current one into a plex server. Obviously it's not as good as the 4790S but hopefully it's not far off it.

tazzzZ · 13-07-2020 11:44am

AlmightyCushion wrote: »

I have a 4570k in my current PC. When I eventually build a new pc (probably won't be until next year) I plan on turning my current one into a plex server. Obviously it's not as good as the 4790S but hopefully it's not far off it.

Thats more than enough. it has the same igpu as the 4790s, so in theory the hardware accellerated transcoding should be the same.

EDIT

I assume you mean the 4670K??? i didnt think there was a 4570K

AlmightyCushion · 13-07-2020 11:50am

tazzzZ wrote: »

Thats more than enough. it has the same igpu as the 4790s, so in theory the hardware accellerated transcoding should be the same.

EDIT

I assume you mean the 4670K??? i didnt think there was a 4570K

Just checked and you are right, it is the 4670K. I'm pretty sure I've been saying 4570k for years.

Good to know it'll do the job, just got to get the new PC first.

Dr4gul4 · 13-07-2020 12:16pm

tazzzZ wrote: »

Im assuming you just forgot to put the /1000 on the end of the switch bit... either way shouldnt be a bottleneck for 720p.

In fairness its worth the upgrade. plenty of good bargains to be had on hardware out there. In the past I had a very nice middle ground cpu with the 4790S(also a T varient) thats low power and has onboard igpu which you can use for transcoding if thats something you ever do and dont want a seperate gpu for it. I would look for the more modern equivalent. Maybe a 9900t or lower tier or previous gen varient?

The reason id choose this over a ryzen is quicksink works well with plex if i remember correctly and can do even 4k transcodes (but i wouldnt advise this for other reasons). It has a marked tdp of 35W which is nice and managable... even if these are very loose guidance of power usage.

If you have another gpu to use in this then id be going ryzen all day long personally. cost to preformance is just a no brainer on this IMO.

+/1000 prob could go put in a 10 GB switch, but in house traffic is low. I'd love to deploy ubiquity and replace the wifi mesh along the way, but. one step at a time !

If im correct, you cant share the GPU in unraid, it's either assigned or unassigned, and I really just want to get over the 720p limit. But we dont know whats causing that right now, assuming it's transcoding audio maybe. i've never tried 4K as 1080p usually buffers.

any who, the hashcat requirement is driving a separate GPU either way. One of two cards. board/ sli dependant etc...

tazzzZ · 13-07-2020 12:25pm

Dr4gul4 wrote: »

+/1000 prob could go put in a 10 GB switch, but in house traffic is low. I'd love to deploy ubiquity and replace the wifi mesh along the way, but. one step at a time !

If im correct, you cant share the GPU in unraid, it's either assigned or unassigned, and I really just want to get over the 720p limit. But we dont know whats causing that right now, assuming it's transcoding audio maybe. i've never tried 4K as 1080p usually buffers.

any who, the hashcat requirement is driving a separate GPU either way. One of two cards. board/ sli dependant etc...

Have a look at spaceinvaderone's youtube

https://www.youtube.com/channel/UCZDfnUn74N0WeAPvMqTOrtA

more spacifically this video

https://www.youtube.com/watch?v=QoVJ0460cro

It may or may not be what you are talking about in terms of GPU assignment. Again probably another reason to go intel perhaps? the igpu can be assigned to plex while descreate could be assigned to whatever VM or other docker you want.

Dr4gul4 · 13-07-2020 2:47pm

tazzzZ wrote: »

Have a look at spaceinvaderone's youtube

https://www.youtube.com/channel/UCZDfnUn74N0WeAPvMqTOrtA

more spacifically this video

https://www.youtube.com/watch?v=QoVJ0460cro

It may or may not be what you are talking about in terms of GPU assignment. Again probably another reason to go intel perhaps? the igpu can be assigned to plex while descreate could be assigned to whatever VM or other docker you want.

I see where you're comming from now ! so, a concept there would be , what do i need for plex, Igpu would suffice, most of the stream should be direct play, but i could still leverage a seperate GPU ( or two ) for something else like a dedicated hashcat machine then.

I assume there are costings assoicated with this however, intel via AMD, the number of sockets + cores, which means diff boards etc ..

Dr4gul4 · 14-07-2020 12:12pm

So after some initial clicking arround i'm starting here, with little to no research on the hardware or limitations on the board / case etc.

No GPU selected as of yet, but the target would be 1 if not 2 1080 TI's. and no storage selected as of yet, as thats JBOD.

CPU: AMD Threadripper 2920X 3.5 GHz 12-Core Processor (£372.40 @ Alza)
CPU Cooler: be quiet! Dark Rock Pro TR4 59.5 CFM CPU Cooler (£73.97 @ CCL Computers)
Motherboard: ASRock X399 Taichi ATX sTR4 Motherboard (£328.99 @ CCL Computers)
Memory: Patriot Viper Steel 16 GB (2 x 8 GB) DDR4-4000 CL19 Memory (£110.39 @ Overclockers.co.uk)
Memory: Patriot Viper Steel 16 GB (2 x 8 GB) DDR4-4000 CL19 Memory (£110.39 @ Overclockers.co.uk)
Case: Phanteks PH-ES614PC_BK ATX Full Tower Case (£84.97 @ CCL Computers)
Power Supply: EVGA 650 W 80+ Gold Certified Semi-modular ATX Power Supply (£102.40 @ Alza)
Total: £1183.51

Any looming concerns ? or thoughts ?

Serephucus · 14-07-2020 12:15pm

If you're only going 12 core, why not 3900X?

Dr4gul4 · 14-07-2020 12:19pm

Serephucus wrote: »

If you're only going 12 core, why not 3900X?

Damn it ! thats a perfectly good point ...

Dr4gul4 · 14-07-2020 12:23pm

Ok so refactored with new Mobo and cooler.

CPU: AMD Ryzen 9 3900X 3.8 GHz 12-Core Processor (£417.50 @ Amazon UK)
CPU Cooler: be quiet! Dark Rock Pro 4 50.5 CFM CPU Cooler (£72.97 @ CCL Computers)
Motherboard: Gigabyte B550 AORUS MASTER ATX AM4 Motherboard (£289.98 @ CCL Computers)
Memory: Patriot Viper Steel 16 GB (2 x 8 GB) DDR4-4000 CL19 Memory (£110.39 @ Overclockers.co.uk)
Memory: Patriot Viper Steel 16 GB (2 x 8 GB) DDR4-4000 CL19 Memory (£110.39 @ Overclockers.co.uk)
Case: Phanteks PH-ES614PC_BK ATX Full Tower Case (£84.97 @ CCL Computers)
Power Supply: EVGA 650 W 80+ Gold Certified Semi-modular ATX Power Supply (£102.40 @ Alza)
Total: £1188.60

Serephucus · 14-07-2020 12:25pm

Now, to be clear: Threadripper does give you advantages.

You get more PCI lanes, so if you want to add in 10Gb ethernet, more M.2, or a GPU or two to play around with passthrough to a VM, a PCI card for more SATA ports... There are reasons to pick it.

I've gone with the X470D4U, coming from an X399 Taichi. It's limiting, in several factors (as above) but I'm not too bothered because it's to the point now where the unRAID box is the main storage and the 'day-to-day, leave alone and don't screw with it' stuff. I have a second box that I use for testing and breaking things.

You refactored build look decent. I'd ditch the RAM for something else though. There's no point running anything above 3600MHz or so with Ryzen, as once you go about 3733MHz, the infinity fabric (the interconnect between CPU dies) decouples from the memory frequency, and adds latency. I'd also see if you can get a 2x16GB kit, just in case you want to add more down the line.

K.O.Kiki · 14-07-2020 2:10pm

There is a valid reason to buy higher-spec RAM: dropping the speed to 3600MHz & tightening the timings.

Dr4gul4 · 14-07-2020 2:55pm

Serephucus wrote: »

Now, to be clear: Threadripper does give you advantages.

You get more PCI lanes, so if you want to add in 10Gb ethernet, more M.2, or a GPU or two to play around with passthrough to a VM, a PCI card for more SATA ports... There are reasons to pick it.

I've gone with the X470D4U, coming from an X399 Taichi. It's limiting, in several factors (as above) but I'm not too bothered because it's to the point now where the unRAID box is the main storage and the 'day-to-day, leave alone and don't screw with it' stuff. I have a second box that I use for testing and breaking things.

You refactored build look decent. I'd ditch the RAM for something else though. There's no point running anything above 3600MHz or so with Ryzen, as once you go about 3733MHz, the infinity fabric (the interconnect between CPU dies) decouples from the memory frequency, and adds latency. I'd also see if you can get a 2x16GB kit, just in case you want to add more down the line.

And i think the limitations of a NON-TR will be the deciding factor here, the x399 looks amazing from the perspective of expansion, affords me the ability to put in two 1080ti's over time in sli. and possibly a smaller card to dedicate to plex ( should it be required ).

K.O.Kiki wrote: »

There is a valid reason to buy higher-spec RAM: dropping the speed to 3600MHz & tightening the timings.

One could only assume this is a software / bios configuration, but how can it be confirmed cpu perf testing ?

tazzzZ · 15-07-2020 12:45am

Dr4gul4 wrote: »

And i think the limitations of a NON-TR will be the deciding factor here, the x399 looks amazing from the perspective of expansion, affords me the ability to put in two 1080ti's over time in sli. and possibly a smaller card to dedicate to plex ( should it be required ).

One could only assume this is a software / bios configuration, but how can it be confirmed cpu perf testing ?

out of curiosity is there a reason you are set on the 1080ti's? Dont get me wrong they have been amazing gpu's but they are getting old at this point and i think they are very overpriced still. And with the 30XX on the way i imagine the 20 series will drop heavily in price in the coming months.

Also i would defo be saying to go with TR but thats just because im currently being limited by PCIe lanes in my build.

Dr4gul4 · 15-07-2020 8:50am

tazzzZ wrote: »

out of curiosity is there a reason you are set on the 1080ti's? Dont get me wrong they have been amazing gpu's but they are getting old at this point and i think they are very overpriced still. And with the 30XX on the way i imagine the 20 series will drop heavily in price in the coming months.

Also i would defo be saying to go with TR but thats just because im currently being limited by PCIe lanes in my build.

The 1080ti's seem to hold a sweet spot based on price vs return, i'm not interested in 4K gaming, so it's pure MH/s that im really interested in.

That being said, currently im using a VM to run hashcat, with in GPU, and it's fine! well, not fine, it's horrific, so any upgrade to a GPU is better than no upgrade.

Dr4gul4 · 29-07-2020 8:34am

Folks i'm back for some guidance, Case and PSU related, the marketplace is flooded with options.

So far the confirmed spec is ..

AMD Threadripper 1950X
Noctua NH-U9 TR4-SP3
ASRock X399 Taichi ATX
And 4 Dimm's of Memory.

I'm aiming for a mix of drives also, but nothing excessive, and space for something like a 1080ti. ( eventually )

I'd like to put in 120 mm fans , or at the very least have this thing run as quietly as possible, so taking in to account that for the PSU fan also I guess.

Any suggestions, greatly appreciated

Serephucus · 29-07-2020 11:31am

Optimum Tech recently did a video of the best 120mm fans (actually a follow-up to a previous video)
TL;DR - Buy a few Arctic P12s.

Dr4gul4 · 29-07-2020 12:11pm

Serephucus wrote: »

Optimum Tech recently did a video of the best 120mm fans (actually a follow-up to a previous video)
TL;DR - Buy a few Arctic P12s.

I was just looking AT https://www.youtube.com/watch?v=XsMNu0gWlZw

What the hells with all the RGB !! I just want airflow, not to host a damn rave, case manufacturers really make me wonder.

K.O.Kiki · 29-07-2020 2:00pm

Dr4gul4 wrote: »

Folks i'm back for some guidance, Case and PSU related, the marketplace is flooded with options.

So far the confirmed spec is ..

AMD Threadripper 1950X
Noctua NH-U9 TR4-SP3
ASRock X399 Taichi ATX
And 4 Dimm's of Memory.

I'm aiming for a mix of drives also, but nothing excessive, and space for something like a 1080ti. ( eventually )

I'd like to put in 120 mm fans , or at the very least have this thing run as quietly as possible, so taking in to account that for the PSU fan also I guess.

Any suggestions, greatly appreciated

Where are you buying a 1950X?
I don't see them anywhere now except for stupid prices - did you mean 2950X?

For cooler, don't go with the NH-U9 TR4-SP3 - get an ARCTIC Freezer 50 TR (Noctua NH-U14S TR4-SP3 / be quiet! Dark Rock Pro TR4 if you want looks for worse performance) [review @ Kitguru]

I will have a Fractal Design R6 TG on Adverts later in the week.

BloodBath · 29-07-2020 3:55pm

Serephucus wrote: »

Optimum Tech recently did a video of the best 120mm fans (actually a follow-up to a previous video)
TL;DR - Buy a few Arctic P12s.

Recommended these these things ages ago. Nice to see some numbers confirming how good they are.

Can pick up 5 packs on Amazon for about £16 in sales. Crazy good value.

Inviere · 29-07-2020 5:09pm

Is there generally a noticeable difference in noise between 120's and 140's?

Building a Lab

Comments