1Tb external storage (follow up to Backup! Backup!)

CraggyIslander

Following on from Valentia's Backup! Backup threat from the other week, I've been doing some research and found a European supplier of external drive cages. (no VAT/import hassle)

This is just a quick poll, to see who would be interested and if it is worthwhile to organise in bulk and split our shipping costs.

To create a 1Terabyte external storage unit, with RAID 5 (i.e. if one HD fails, no data is lost) one would need the following:

External Enclosure
Controller Card
5 of these

Including shipping, the grand total would come to 652.61 Euro. Of that about eighty euro is on shipping, which only marginally goes up if ordering multiples so if buying 2 units, the total would drop to around 610 each.

Afcourse if one splashed out on 500Gb disks, the raid 5 unit would be 2Tb!!

5uspect

TBH I'd just buy one of these and run it in RAID 1.

The 1.5 TB is around €600 and the 1TB is about €400. (thats in RAID 0 so RAID 1 is half that capacity but everything is duplicated) RAID 5 seems like overkill.

rymus

the maxtor does seem to be more logistally sensible..All in one unit, no farting around with parts. Set it to raid 1 and plug in your firewire 800.

Of course the other side of that is that the maxtor isn't strictly upgradable. With a self build you can always swap out disks or add more as your needs change. Spose if you didn't want to go all out for raid 5, you could go with 4 x 500GB disks in a raid 10.

CraggyIslander

fair enough 5uspect, although how accessible are the disks if one should fail and needs to be replaced? My WD mybook hasn't got a removable backplate or other discernable screws. :rolleyes: Now I just buy a 40Euro sharkoon case and roll my own

Being a tech head, I like the thought of extensibilty (i.e start with 3hd's installed and add more as needed). The decider for me would be the transfer speed advantage that sata II has over USB.

Roen

Definitely interested. What's the craic on buying a Maxtor...are they States only cos you'd be facing a hefty Vat bill if they are. Do they have a UK or Irish shop?
The self build one looks better from that point.

rymus

I'll check our (my workplace) supplier. Might be able to get a bulk discount if enough people were interested.

Sebzy

Before people run off and buy a storage system consider this

lets say 5 250GB SATAII disks in raid 5 config to give a total of 1TB storage

Pro's
- This would give you very good data security
- Large storage space

Con's
- Complex to setup and configure for unskilled peeps
- Cost of 5 disks
- If one disk from the raid set fails it has to be replaced with a nearly ideticle disk which down the line lets say one year it may cost a premium to get or they may not be avalable as their produced for small office and home use. You may have to replace all the disks to get raid 5 back.


Seb.

Valentia

Can I ask what might be a silly question? Why would Raid 5 (must google that in a minute) be overkill when with raid 1 you need 4 500Gig disks to achieve what 5 250 Gigs. I think I'd prefer Freds arrangement. At least it can be DIY'd in the future. I'll vote yes but I'd go for the 2Tb option with maybe 350 or 400 GB disks.

rymus

4 x 500gb's in a raid 5 with an iomega rev autoloader and a gigabit ethernet card humming away happily under the stairs in a small fridge

oh and maybe 2 spare 500gb drives as spares. I'd hate to have 1.5TB's of space unusable because I couldn't get a spare drive at some point in the future...

2 x 500gb disks in raid 1 gets ya 500gb
5 x 500gb disks in raid 5 gets ya 2tb

CraggyIslander

rymus wrote:

4 x 500gb's in a raid 5 with an iomega rev autoloader and a gigabit ethernet card humming away happily under the stairs in a small fridge

Who's the bigger tech head?;)

rymus wrote:

oh and maybe 2 spare 500gb drives as spares. I'd hate to have 1.5TB's of space unusable because I couldn't get a spare drive at some point in the future...

Hopefully SATA II (or compatible) drives won't be obsolete in 2 years (look how long IDE is lasting) but the joy of raid 5 is....... you can replace a 500Gb model with next years 1Tb model and still be OK (granted 500Gb of the 1Tb wouldnt be used)

sebzy wrote:

Before people run off and buy a storage system consider this...

Not suggesting anyone rushes into this, only bringing it up as a viable alternative As I'd mentioned in the original thread, I'd be more than happy to help anyone with installation/configuration. (for the price of a pint that is )

For simplicity there is the option of hot-swappable trays (an extra 100euro i think) so the case would never have to be opened again. With that option one could also start with 3disks and add an additional one or two later

5uspect

Fred, I didn't mean to say that your idea is at all bad, its sounds like a fun build and would be great for lots of people to back everything up.

I suggested the maxtor purely as the selfish single user option.
The maxtor also seems a better option mainly, as has been pointed out by Rymus, its portable, small(ish) and as I've found out with my 200 gig One-Touch II nearly indestructible! (Roen I see two screws on the back of mine for DIY fun that I'll avoid for now!)

rymus

its certainly a viable option; I've been meaning to get around to building one for the last 2 years or so in fact.

For the average Joe or Josephine that knows sfa about raid levels, sata or controller cards though, the 1tb maxtor is probably the best bet. I've got a 300gb maxtor one touch at the moment that I use to backup my stuff and I'd love to replace it with a mirrored something or other. For conveneice, I may end up going with something that has firewire 800. Having said that, in a few years I know I'll end up with a half height rack and a few nas boxes or whatnot in it.

It appears I can get a lacie 1 or 1.2tb unit or units if there is interest. Anyone want to research the raidability of those?

nilhg

Am I correct in thinking that Fred's suggestion connects directly to one pc only?
We did a bit of an upgrade to our backup storage here recently using one of these
http://www.komplett.ie/k/ki.asp?sku=331223
and are happy enough at the moment with it (you need to upgrade the firmware first though).
We also have a variety of USB2 disks scattered around connected to individual PC's and soon to the network via a nslu2.
Our network is far from typical as it was originally setup to share a very expensive broadband connection to 4 houses spread over a distance of half a mile or so. That broadband supplier has since been replaced by plain dsl but as a happy consequence we have proper offsite backup which is very important.

rymus

if you setup a fileserver with a few disks and a raid controller (and a neat little linux install) you could invest in a gigabit switch & NIC for the server and share the box over your entire network. Course if you got one of those maxtor or lacie boxes you could do the same, albeit with the likes of a linkstation or other device that allows you to network a usb hard drive.

Valentia

Jazus this is a great thread

I don't understand a shaggin' word ye are saying. With regard to Raid 5...if a disk gets banjaxed is the data safe?? That's the bottom line to me. Raid 0 seems like Russian roulette to me. Raid 1 seems like wasting half the space. Does the extra drive in Raid 5 keep some sort of a record of file structures or something so that the "map" can be rebuilt when a drive is replaced or am I way off the mark?

CraggyIslander

yep it would connect to only 1 pc at a time. Sharing files or just a load of extra space was not what I had in mind tho. Keeping precious raw files, psd files, etc as safe as possible from harddisk failure is my only concern here (archiving to DVD and storing elsewhere still recommended for fire/theft/etc)

At a reasonable price and without the need for manual or scripted backups, the week you forget to run your backup is when your disk dies.... it's murphys law

It's fast enough to work directly on files on the raid disk for editing etc. External USB drive, mirrored or not are imho too slow for this, so they get used for irregular backups mainly.. and murphys law loves those :eek:

magicbastarder

raid five sacrifices a disk. but one can go and you retain your data.
access is slower in a raid 5 array with a dead disk, cos the missing data has to be 'calculated'.

CraggyIslander

Valentia wrote:

Jazus this is a great thread

I don't understand a shaggin' word ye are saying. With regard to Raid 5...if a disk gets banjaxed is the data safe?? That's the bottom line to me. Raid 0 seems like Russian roulette to me. Raid 1 seems like wasting half the space. Does the extra drive in Raid 5 keep some sort of a record of file structures or something so that the "map" can be rebuilt when a drive is replaced or am I way off the mark?

Raid 5 in a nutshell:

All data and so-called parity info is stored evenly across all disks. If a disk fails and you replace it then the parity info is used to recreate the missing part of the file (or if the disk held the parity info then the parity is recreated).

So yep if a disk gets banjaxed your data is safe... if a second one fails before its replaced then its still up the creek without paddle. Hence the hot-swap trays, no need to open the box to replace a drive.

There are other options like raid 6 where up to two drives can fail and data still safe or raid5 with hot spare.... if a disk fails, a spare one automatically takes over... but you'd need a soeper doeper controller for those.

Valentia

Deadly, thanks. I'm with the Raid 5 jobbie so......and maybe the maxtor boyo for the hols. No, no, I'm just joking about the maxtor.

nilhg

Fred, your suggested setup looks great but I find it hard to imagine people archiving 1TB (or even 500GB) to DVD. No backup solution can be complete without some provision for offsite storage so I think people should budget for at least one usb2 drive which can be left in work or a neighbours or relatives house.

rymus wrote:

if you setup a fileserver with a few disks and a raid controller (and a neat little linux install)

This describes perfectly the dlink DNS-323 I linked to above, also has an itunes server and a media server.

There are a huge range of devices out there to suit all budgets and situations, and hopefully this thread will encourage people to get setup with something

rymus

maybe if you saved up a bit you could get a HP DLT drive and a small fireproof safe.

Maybe run some CAT6 to your neighbours house, put a 2TB NAS in their attic and sync yours to it every night after you've done your backup to DLT? Or are we getting too extravagant here?

Best options so far seem to be;

1. For the bone idle and/or technophobes
* Buffalo Terastation (capable of raid 5)
* Maxtor One Touch (raid 1)

2. For those willing to experiment
* Build your own NAS box and invest in a gigabit switch
* Try to build a 'compact as possible' raid array with usb connectivity

Cheapest, probably the maxtor.
Foolproof and functional; the Buffalo
For those who never RTFM and never call tech support; build your own.

Yes? No?

I think if I was going to go into a shop tomorrow and buy something to satisfy my needs, it'd be the 2tb terastation (simply for it's raid 5 and ease of use), I'd get a 5 port gigbit switch to speed everything up a bit and of course a copy of lightroom to help catalog the new beast

magicbastarder

pretty soon, it'll be less work to re-take all the photos than get backup storage sorted.

Valentia

One more small q. If a disk exterminates irself on Raid 5 do you get a warning notice?

I'm beginning to remind myself of somebody I know with all these questions...........

rymus

generally you'll get a flashing light or a beep or some kind of white flag waving...

For example, the couple of times a disk has died on our servers in the office, the light on the drive turns red and you get a message in the OS or on reboot. Just pop in a new drive and wait for it to rebuild.

Diarmuid

Is it not true to say that you will really need 6 x 250Gigs for the RAID 5? 5 in the array + 1 ready so if/when one drive fails you can pop in the spare drive. Running on 4 disks in the array while waiting on your replacement to arrive would not be ideal in my eyes.

rymus

it's not nessecary to have an online spare.. it's just beneficial.

Of course it's not ideal to run with a knackered drive, but the point is it will run with a knackered drive while you're off sourcing another one..

JDxtra

RAID 1 is overkill for home use (even home professional). RAID 5 is fine for storage of media files. You can also have a "hot spare" allowing you to continue with a resiliant configuration even after 1 drive failure.

However, all this is worth nothing if somebody breaks into your house and lifts the lot...

magicbastarder

how much data do people actually have to back up?
i've about 70GB of photos.

Valentia

JDxtra wrote:

However, all this is worth nothing if somebody breaks into your house and lifts the lot...

Let them try. We keep our dog only marginally fed in anticipation of just such an event!!

rymus

note to self, dont break into house to steal 5D

CraggyIslander

hmmm 5D + couple L lenses.... just remember to bring large raw steak for the doggie

rymus

laced with sleeping tablets

DotOrg

magicbastarder wrote:

how much data do people actually have to back up?
i've about 70GB of photos.

I've currently about 600GB of photos spread over two PCs

magicbastarder

that's about 30,000 photos at an average of 20MB per file.
that is nuts.

rymus

with that amount of data to potentially lose, do you have any kind of raid setup or external storage dotorg?

DotOrg

magicbastarder wrote:

that's about 30,000 photos at an average of 20MB per file.
that is nuts.

600GB is about 100,000 photos, some raw files, mostly jpg files. it also includes the edits of the original files

everything is on two internal hard drives, not in any raid format,just two regular hard drives that i copy from one to the other and then also, they're backed up onto external hard drives that aren't stored with the computers in case of fire or robbery

i plan to also back everything onto high quality DVDs too so i'll eventually have 4 different copies of the same files

rymus

youre not too badly off so.. Wouldnt it be nice if santa brought you a 2tb nas though...

magicbastarder

DotOrg wrote:

about 100,000 photos

5uspect

Shift + Delete

leinsterman

I have my system on Raid 0 ... because I want performance, but Raid 0 does not back data up : It "stripes" data across two physical drives so that when you access the computer is effectively reading data from two drive simultaneously ... meaning twice the read speed ... great if running a system hog like CS2 ... do this with two raptor drives and your system runs incredibly fast ...

... I also have two additional single drives internally for a total of 1TB ... I keep my pictures on the raid array when processing and move it to one of the two spare drives later ...

However for back up ... I just bought a "My Book" 500GB external drive ... and I use a little freeware app called "allway sync" ... total cost around €180 from the very nice guys at elara.ie (who I do recommend) ... OK it takes ages to back up the first time ... but that is simplified by allway, it takes 5 minutes to set up ... you pop it on and go to bed ... next morning bob's your uncle ...

I keep the external drive in cold storage away from nasty power and computer glitches and only take it out to back everything up ... works fine for me ... and costs less ... OK a bit on the clunky side ... but Raid can be a bit on the difficult side for the non techie ...

If I were a pro and my living depended on it ... I would buy a NAS (Network Attached Storage) configure it for RAID (if you get one from Buffalo this is pre-configured) or the Terrastation. If you want a price to compare I can ask Kin at Elara for a group discount on a terrastation, my book or NAS ... PM me if interested ...

kjt

Great thread. At the moment I'm doing all manual backups and it's
starting to become annoying.... thus putting me off doing them
often. My last backup was probably 2/3months ago, if my system
was to die I'd loose SO much important data. I've got 500gb internally
and three external drives @ 300gb each... so 1.1Tb. I think I really
want to go with raid5 or maybe setup my own files server with raid.
Some good suggestions flying around

Diarmuid

leinsterman wrote:

I have my system on Raid 0 ... because I want performance, but Raid 0 does not back data up : It "stripes" data across two physical drives so that when you access the computer is effectively reading data from two drive simultaneously ... meaning twice the read speed ...

...and twice as likely to die.

Interesting read on HD reliability based on Google's recent paper.

kjt wrote:

At the moment I'm doing all manual backups ... so 1.1Tb. I think I really
want to go with raid5 or maybe setup my own files server with raid.

I would think a previous posters suggestion to hit the delete key would be more sensible. I would prefer to permanently delete 60% of my photos (the sh1t ones) than to loose the 40% that were good, because I didn't have a backup.

leinsterman

Diarmuid wrote:

...and twice as likely to die.

Interesting read on HD reliability based on Google's recent paper.



I would think a previous posters suggestion to hit the delete key would be more sensible. I would prefer to permanently delete 60% of my photos (the sh1t ones) than to loose the 40% that were good, because I didn't have a backup.

well my raid system is three years old ... so going by the article I must be due a failure ... I did have to replace one of my raid array drives recently but this was due to a broken edge connector ... and the reason for this was my fault ... I was able to recover the array.

Anyways ... everything is backed up so bring it on ..

BostonB

What you need is a regular backup and a copy of your data in more than one place. Delete all the stuff you don't need for sure. But back up first! Raid won't help you if someone nicks the computer or the drive, or you have a fire etc. If the RAID controller fails (as does happen) you'll need the exact same controller to get your data off your RAID.

I would use two external drives. One for short term back up and the other for long term back you can move to a different location or put in a firepoof safe etc. The drives should be different makes. You should use some proper software that sync's or backups you files and automates the process.

I use http://www.karenware.com/powertools/ptreplicator.asp which is free and simple to use.

I have a bout 500GB of data and my sync takes about anywhere between 5-15 mins depending on how many changes there are. I run a synch one a week to one drive and about once a month to the other drive. Every so often I backup some critical stuff to a DVD or CD aswell. Every so often I run through all my data and delete all the obsolete data.

kjt

Linky

Might be good for some people

Takeshi_Kovacs

kjt wrote:

Linky

Might be good for some people

yes this might be a nice addition, especially considering it has external SATA connection...

CraggyIslander

kjt wrote:

Linky

Might be good for some people

4 of those, a controller with 4 external sata ports and you could use them in raid 5! with a bucketload of poweradapters and cables