backig up photos.... yet again

CraggyIslander

Apologies for bringing up an old topic, but we all know the importance of backing up our images, right?

There's been various threads on the different methods for backing up / archiving and their pro's and con's. For me personally, just having a couple of external HD's and archives on highgrade DVD's is not enough peace of mind. So I will be building myself a NAS server with 8 ports.. do realise not everyone is a techhead like me

While looking for some parts, came across this:

http://www.komplett.ie/k/ki.aspx?sku=332317

Looks like enterprise raid storage is finally becoming affordable and usable for joe bloggs.

Paulw

Nice box, and then all you need are the disks to go inside.

I think single external storage disks are cheaper and just as effective for storage.

Personally, I store everything on 2 external disks (and still have most on my internal PC disk too).

rymus

the only thing that's better than those nifty little boxes o' storage are the x-raid NAS appliances. Removes the need for formatting & reconfiguration when it comes time to either add more hard drives or change for bigger drives. Does all the changing of raid levels and initialising new disks on the fly apparently.

I'd say that's where the smart money goes. - http://www.komplett.ie/k/ki.aspx?sku=336971

Single external disks are fine and good but the beauty is in RAID. If you've got all your photos on one external disk and that goes, you're boned. If you have it on two seperate external drives, you're probably wasting money. Get one RAID 1 external disk instead. Or splash out and get a honkin great NAS appliance. I've got one 1tb external drive in RAID1 at the moment. Probably going to move to NAS within the year and unless anything better comes out in the meantime, I'll be buying the readynas at the link above..

jlang

Not sure about justifying the cost of NAS yet. Maybe in a few years when I actually need terabytes.

When you can buy a few 500GB external drives for E120 each you can have on site and off-site backups rather than having all your eggs in a E1000 basket under your desk.

nilhg

Windows Home Server is going to hit the shops quite soon either as software you can install on a machine yourself or installed on something like this.

Its designed to be useable by non techies, and while the proof of the pudding will be in the eating, it does seem to tick most of the boxes for a decent home backup solution, offsite storage excluded.

It might be worth waiting to have a look and see how it goes.

Borderfox

I had a good chat with the guys and girls from www.putplace.com and seen their demo also and have to say it looks right up my street for backups of work. They will be running a beta shortly and are accepting signups?

rahtkennades

If you're looking for a cheap NAS solution, this looks promising.

leinsterman

I considered all sorts of funky and fancy configurations ... but to be honest I went for a cheapo solution ...

I have two disks in my machine which I keep manually sync'd up using freeware solution Allway Sync ... usually about once every 2-3 weeks

I then also use Allway to sync an external HD ... which I sync every other months or so ... the external HD is kept off my system and unplugged unless it is being used to sync ...

not 100% safe ... but safe enough I suspect ...

burning to DVD's is not something I would consider ...

rymus

leinsterman wrote:

burning to DVD's is not something I would consider ...

Why not?

TelePaul

RCNPhotos knows a guy who takes film shots of his digital prints. Total technophobe! An dedicated external HD is a pretty good option.

CraggyIslander

burning to DVD is a solid archiving option, just be sure to use the right media.. I've been using this page as a guide.

I've used a similar CD-R guide from the early days and the very first batch of discs (philips gold something) I bought 15 years ago (when cd-burning happened at 1X speed) are still in perfect nick. Same cant be said for a far more recent batch of memorex cds that turned out to be fakes. The media identifier utility they have is a must.

Here's a solution thats overkill but this nerd recently did anyway: bought a brand new 320Gb regular SATA2 drive, temporarily installed and formatted it, then copied everything of my 2 external HD's (where everything thats been archived to DVD eventually ends up).

The drive is filled up completely with raw files, psd's, flac backups of my favourite CD's and iso backups of my games and apps discs. This HD is now safely tucked away in a cool dark dry place.... Will be doing the same with the next 320Gb batch

PaulieC

TelePaul wrote:

RCNPhotos knows a guy who takes film shots of his digital prints. Total technophobe! An dedicated external HD is a pretty good option.

wow! you'd have to wonder why doesn't he just use film to take pictures...

TelePaul

cambo_gueno wrote:

wow! you'd have to wonder why doesn't he just use film to take pictures...

Yeah, he sounds kinda mad. RCN will tell you the details!

feileacan

can anyone suggest a good website to read up about raid systems, id be interested in the device from Komplett. do all the bays have to be filled with drives or can it work on a minimum ?

Cameraman

Good introduction to RAID here

Paulw

I think some people just go too far.

In 5 years time, will you still be able to read the raw files produced today? Maybe/maybe not. How many of your photos will you really want to use again? Of the thousands of images you store, might you use 100?

Why is burning to DVD any better than storing on an external disk?

There are even times when I think I'm being too cautious, by using 2 external disks.

But, to each their own.

CraggyIslander

this one's a reasonable breakdown of it.

Especially love the pic of the 36x 320Mb disks from 1992 at the bottom

Or wikipedia

You'll need to fill at least 3 bays to use RAID5. To calculate how much usable space you would have, use (n-1) * size of HD (all hd's have to be equal size or it will use the smallest size of the set).

i.e. 3 1Tb drives would give you 2Tb usable and 4 x 1Tb drives would give you 3Tb usable.

10-10-20

Can I just stick in my spoke here on the backup matters... I have considerable experience dealing with RAID and backup solutions, and I have delivered bad news to many companies when their be-all and end-all RAID solution fails on them and they loose their company.

I have gizillions of photos stored on my 'server' which is a retrofitted Dell Precision workstation and this is backed up using a custom script onto an external USB hard drive, with incremental archives occurring weekly into different folders.

OK, so what's an incremental backup and why is it in bold...?
Well, one of the issues that I initially had was that I did a straight like-for-like copy onto my USB disk, overwriting the previous changes and just having a point-in-time mirror of my server. If then for example, I was then to either corrupt or delete a photo and a backup was to occur between the time of the actual corruption and when I noticed the corruption, I wouldn't have a method of recovering the file from the backup. So, it's not really a backup, it's more of a secondary mirror space.

An incremental backup is when you take an initial mirror of your data (so a complete backup) and then at points after this (daily, weekly), take a backup of the differences between the last incremental backup and the current state. Or as Wikipedia states... "For instance, following our full backup on Friday, Monday’s tape will contain only those files changed since Friday. Tuesday’s tape contains only those files changed since Monday, and so on."
http://en.wikipedia.org/wiki/Incremental_backup

Therefore if file corruption occurs once again and you don't notice for a couple of weeks, you will actually have a copy of the corrupted file and the original either in the initial 'mirror' backup or the incremental.

So, don't lull yourself into a false sense of security by having an 8 drive RAID-5 with a hotspare and a rebuild rate of 50Mb/sec... the backup solution IS AS important as the backup hardware.

TelePaul

10-10-20 wrote:

Can I just stick in my spoke here on the backup matters... I have considerable experience dealing with RAID and backup solutions, and I have delivered bad news to many companies when their be-all and end-all RAID solution fails on them and they loose their company.

I have gizillions of photos stored on my 'server' which is a retrofitted Dell Precision workstation and this is backed up using a custom script onto an external USB hard drive, with incremental archives occurring weekly into different folders.

OK, so what's an incremental backup and why is it in bold...?
Well, one of the issues that I initially had was that I did a straight like-for-like copy onto my USB disk, overwriting the previous changes and just having a point-in-time mirror of my server. If then for example, I was then to either corrupt or delete a photo and a backup was to occur between the time of the actual corruption and when I noticed the corruption, I wouldn't have a method of recovering the file from the backup. So, it's not really a backup, it's more of a secondary mirror space.

An incremental backup is when you take an initial mirror of your data (so a complete backup) and then at points after this (daily, weekly), take a backup of the differences between the last incremental backup and the current state. Or as Wikipedia states... "For instance, following our full backup on Friday, Monday’s tape will contain only those files changed since Friday. Tuesday’s tape contains only those files changed since Monday, and so on."
http://en.wikipedia.org/wiki/Incremental_backup

Therefore if file corruption occurs once again and you don't notice for a couple of weeks, you will actually have a copy of the corrupted file and the original either in the initial 'mirror' backup or the incremental.

So, don't lull yourself into a false sense of security by having an 8 drive RAID-5 with a hotspare and a rebuild rate of 50Mb/sec... the backup solution IS AS important as the backup hardware.

Seems like extensive and very good advice. Maybe you could talk about retrieval software?

10-10-20

TelePaul wrote:

Seems like extensive and very good advice. Maybe you could talk about retrieval software?

Are you referring to retrieval software for usb keys and such or complete backup applications?

Marcus

I might just add a note here about RAID 5

BE CAREFULL with RAID 5.

Hard drives from the same batch "tend" to go wrong within quick succession of each other. In a RAID 5 scenario, this means that after a drive goes wrong you may bee looking at a second failure within a very short space of time. So it's essential that you replace a faulty drive immediately... Preferably have a spare "ready".

RAID 10 allows up to 2 drive failures, but you only get 50% capacity. It really depends on what risk level you are comfortable with!

CraggyIslander

good point 10-10-20, seen plenty of raid failures and examples where the daily backup tapes were never even tested.... until it was too late.

To me the raid is for peace of mind for keeping the files where they're easily accessible and reasonably safe. It does not negate the requirement for backing up and storing files off-site (either on HD or DVD or even tape)

My prefered backup strategy would be the differential one, where each file that has been changed since the last full backup is backed up.

10-10-20

iampixie wrote:

I might just add a note here about RAID 5

BE CAREFULL with RAID 5.

Hard drives from the same batch "tend" to go wrong within quick succession of each other. In a RAID 5 scenario, this means that after a drive goes wrong you may bee looking at a second failure within a very short space of time. So it's essential that you replace a faulty drive immediately... Preferably have a spare "ready".

RAID 10 allows up to 2 drive failures, but you only get 50% capacity. It really depends on what risk level you are comfortable with!

Hi iampixie,
I know where you are coming from on this one, but I can't say that I wholeheartedly agree that mixing disks in a RAID-5 is preferable to having a matched set. Yes, there are times when there can be manufacturing defects or firmware issues which can reduce the lifetime of a disk, but in all, mixing disks is not usually done at the initial point of sale. Replacement disks can be different as long as some guidelines are adhered to. Keep your firmwares and drivers up to date...!

Fred C. Dobbs wrote:

My prefered backup strategy would be the differential one, where each file that has been changed since the last full backup is backed up.

Yes! That's a discussion all of it's own thread! Depending on your backup space, backup window, speed and requirements you have to balance your method. I chose Incremental as I then know that I only have one copy of each file per backup set, and not multiple. But, differential means a short recover time from a complete disk failure... so... around we go...

Marcus

10-10-20 wrote:

Hi iampixie,
I know where you are coming from on this one, but I can't say that I wholeheartedly agree that mixing disks in a RAID-5 is preferable to having a matched set. Yes, there are times when there can be manufacturing defects or firmware issues which can reduce the lifetime of a disk, but in all, mixing disks is not usually done at the initial point of sale. Replacement disks can be different as long as some guidelines are adhered to. Keep your firmwares and drivers up to date...!

Yes... Mixing disks presents its own set of risks! I was merely pointing the high risk nature of RAID 5.

Have you ever seen http://baarf.com/.

(We use RAID 10 with both hot and cold spares)