*Fast* Home File Server

edanto

1. What is your budget? Roughly €1400

2. What will be the main purpose of the computer?

To replace an existing ReadyNAS, which is just too slow to serve files.

We take a lot of photos. There could be 300 25MB Raw files in a folder from a single event, and when looking to find a particular one on the NAS it takes far too long for the thumbnails to load.

I'd want a RAID setup, but I'm not sure which mode gives the quickest throughput.

Currently we have about 3TB of photos, that'll probably grow to 10TB over the life of the box, so I was budgeting about 5 x 3Tb drives (~e800)

I'd also like to use XBMC on it to serve media to a media player, but obviously that's not such a demanding requirements as serving the photos quickly. No issues in terms of networking at home, have a simple gigabit switch and cables run where required.

3. Do you need a copy of Windows? Yes.

4. Can you use any parts from an old computer? Probably not. There is an old machine offered to me, but I think it's too old to use. Worth mentioning Dell Dimension 8300. P4, built for XP!!

5. Do you need a monitor? No

6. Do you need any of these peripherals? Don't need any peripherals

7. Are you willing to try overclocking? Yes, but not sure it's required in this case.

8. How can you pay? Card/cash.

9. When are you purchasing? 3-6 months.

ED E

Are you just RAIDing for throughput and not redundancy? If you want 10TB of files you should really buy 20TB of drives(or aim to build to that).


Large case + 2 raid cards + gigabit/dual gigabit NICs. Processing power wont be required much.

Considered something like FreeNAS vs windows(save a few bucks for another drive)?

Wossack

can you not generate and cache the thumbnails somewhere? those are big files to be generating thumbnails on the fly for..!

frozenfrozen

Proxy? Your bottleneck will always be the files coming off the cameras, so just convert to dng and low quality jpeg at the same time as you import.... that way you've got the ability to quickly browse all of the files in a folder, and no HUGE redundant generation of thumbnails being pushed over the network, which would definitely slow everything down a lot.

edanto

I'm RAIDing for throughput and redundancy. Only needs to be resilient to one failed drive, I'll have a spare one ready to drop in.

Re caching thumbnails, I'm not sure (still researching) what the best thing might be to do there. The photos will be accessed by a variety of machines. A Mac photo editing, a mix of laptops I bring home, tablets and a media player/appleTV we may get for the telly. Or a RasPi. Maybe there is a standard way to store thumbnails that all OSs can use, must look into that.

I'm not wedded to Windows as the OS, but I'm fluent in it and others are more work for me. Reliability and speed are the main requirements.

Any suggestions for which large case/RAID cards combos I could start to look at?

PogMoThoin

Build your own. Why not buy one of these low power HP servers and stick 4x decent drives in there. It is only 150W, will save you money in the long run. The dual core processor it contains is plenty for a NAS. I have one myslef, highly recommend. There is a €120 cashback offer atm.
http://www.hpshop.ie/products.asp?partno=658553-421

Then stick a free NAS operating system on it, no overheads of running Windows
http://www.freenas.org/

edanto

Thanks for the link and ideas.

I don't think that'll work for us, since we already have a 4x2tb nas that is about half full and want to plan for the future.

Would like to find a similar chassis that could take 6 or 8 drives, and then RAID card (s) that will hold it all together.

I've never build a machine from components before so I'm not sure which are good sites to source them from. Any tips?

Wossack

Just wonder what your current NAS is performing like - read/write wise? And do you know if its RAID'd?
It might be too big an ask for consumer level gear to deliver both your required level of performance, and have resiliency.. could end up blowing half your budget immediately on the raid controller alone

_Kaiser_

2nd hand DELL Server would be my suggestion if the noise factor wasn't an issue.

I've previously used a Poweredge 2950 and 1900 for much the same usage (without the XBMC requirement as I use a WDTV for this) and found them great. For about 400 on adverts you'll get one with dual processors and a PERC RAID Controller. Drop in a few SATA drives and RAM (both relatively cheap these days again) and you're good to go.

But as I mention it won't be quiet. The 1900 wasn't as bad as it's a tower chassis so bigger slower fans, but the 2950 is a rack server and you wouldn't live with it in a sitting room (mine lived in a spare bedroom). When I moved to an apartment though I bought a Precision 690 PC which is literally rock solid (weighs a tonne to prove it!!) and transferred all the hardware I needed (CPUs, RAM, RAID card are all compatible) so now it has...

2x Dual Core Xeons @ 3.0Ghz
24GB RAM
PERC 5 RAID controler with 4x2TB drives in RAID 5
2x dual-port Intel GB network cards
Windows Server 2012 DataCentre with 4x Server 2008 R2/2012 Hyper-V Virtual Machines running under it (including a domain controller, Mezzmo media server, WSUS/VPN server and test machine)

Quiet enough that it sits beside the TV and no problems with it at all.

BaconZombie

Have you seen this case:

X-Case - RM 424s Home Server 550mm
Micro +ATX Max 12" x 9.6"
Bays - 24 x 3.5" or 2.5"
120mm Fans
Expansion Slots - 7

https://www.youtube.com/watch?v=0LFyD8dooIo

http://www.xcase.co.uk/x-case-home-server-box-p/case-homeserver24.htm

edanto

Some great suggestions there, guys thanks.

Must admit I'm fairly taken by that 24 bay case! One case to rule them all... just slot more drives into it over the next few years. But maybe I'd be biting off more than I could chew. Must read up more about the various cards he showed in that video.

@Kaiser, I was thinking of something like that initially, and from this thread I'm hoping to learn how to choose the components and configure them so that I get the quickest read speed possible.

Noise isn't an issue, this will be living in the attic.

Our current NAS has 4 X 2TB in RAID X (that's a proprietary RAID setup), it's a Netgear ReadyNAS NV+

My frustrations with it mirror those of this guy.... the damn thing is just so slow.
http://leereamsnyder.com/blog/netgear-readynas-nv-review

Finder’s "Copying files" dialog was telling me it would take multiple days to move a hundred or so gigabytes of files, an operation that would take 3 minutes tops with a plugged-in hard drive. It’s stupid slow.

And, worse, any little network hiccup will ruin – not interrupt partway, but totally negate – any transfer in progress. Do you trust your network to stay up flawlessly for days at a time?

Even if you do get files on there, get used to Finder or Explorer hanging on you for several minutes while the NAS reads a folder with a lot of files. You can’t do a whole lot else with your computer while Finder hangs. Well, you can reach for that whiskey.

Are there ways to speed things up? You betcha! There’s an FAQ with some ideas, and the forum has lots of contradictory and anecdotal advice. Just hop into some configuration windows and sometimes a goddamn Linux command line on the ReadyNAS, my router, and my computer to adjust context-free values for network settings with nonsense names like “jumbo frames”, TCP, ACK, and MTU. Easy, right?

Figuring out what those mean will only get you so far. All these network settings affect each other, and there ain’t no such thing as a free lunch. Change one setting and big files will transfer faster, but lots of little files will be much slower. Modify another setting to compensate and [sad trombone] you're back to where you started. Change a different one to speed up your local network and your general web browsing will be slower.

In trying to find the right balance, you can tinker with these settings for days. I know; I did. You can get sucked in, and it’ll drive you mad.

So, this project is an effort to learn what I might do to improve on that. I'm a total novice when it comes to building servers generally, and RAID cards in particular. Every card in that video above was new to me.

All the advice so far is really helpful, thanks.

Cuddlesworth

Hi OP,

Not sure of your exact specifications, but it does look like you have reached the limitation of a normal NAS. But I don't think your into full blown SAN territory as this is a home setup in the end. Servers and huge SAN setups are full blown power hogs and will cost you far more in the long run. Your also limited to a 1gig connector, so 100mbs throughput. It sound like IOP's is a killer for you, although your current NAS is bad at both.


I would recommend the HP N40L listed above in this thread. Its cheap and with a decent raid card, you don't need CPU to get great results.

Put a 4gig usb key inside it and install FreeNAS onto it.
Bump the RAM up to 8 gig for local caching.
Install a decent raid card such as the LSI 9750-4i4E and plug the internal SAS connector of the four port bays into it.
Backup your files and move your existing hard-drives into the new unit. Or buy three 3-4 tb capacity drives for it(card can take larger drives), build a array, port all the current files over. This will add a level of cost though.

For expansion, you can buy external low power enclosures and connect them. Such as this one.

Two box's, one unit. Large array of up to 32tb currently with the usual Raid5 loss and really good speed/power usage. Should be just in budget as well.

Serephucus

Just to mention:

Start -> Run -> gpedit.exe

User Configuration -> Administrative Templates -> Windows Components -> Windows Explorer

Find and disable “Turn off the display of thumbnails and only display icons on network folders”. This will allow thumbnail generation on mapped network drives in Windows 7. Won't speed things up for newly added photos, but you'll only have to browse to each folder once to generate them locally.

edanto

Thanks for that tip. The main issue tends to be on the OH's Mac, so I might look around and see if there's a similar option in Mac.

I wonder if there's a way to generate a folder of thumbnails in each folder that both Mac and PC systems would be able to read quickly.
Haven't found anything about it yet.

ED E

http://blog.medin.name/2012/04/22/thumbnail-1000s-of-photos-on-a-synology-nas-in-hours-not-months/

Birra linux OP?