Data explosion !

Morrisseeee

In 4 years while using a 4MP camera (with movies) I have ~2GB of data.
In the last few months, with the D_SLR I have exploded to 12GB of data.......wow! (just RAW power I guess).:eek:

kensutz

In one match I could shoot 8GB of stuff.

Calina

oh God.

please.

I am bad. Last night I copied 70gig of data to offline storage.

stcstc

i have two backups running on my server each day, each one is 800gig of data backed up onto LTO3 drives

and this is the server in my house !!!!

dakar

The first pc where I work had a 2gb hard drive. This was considered ludicrous overkill at the time (about 15 years ago). Who could ever fill 2 whole gigabytes of data! Fast forward to now and I know I carry at least 10 times that between cf and usb memory on me when out with my camera. I presume the next 15 years will see a similar shift!

AnCatDubh

I only thought about this earlier today. I took delivery on Monday of an 8GByte SD card which gives me less images with the new camera than the previous 4GByte did in my previous camera (thats mega pixel madness for ya). It is fun though to select a 3MPixel image format and see that the camera gets *stuck* at 9999 images to take

I am beginning to see the data explosion and resulting backup as a real issue for me going forward. I mean i bought a 40GB portable hard disk a little while back to basically have some extra capacity on my netbook and maybe download the SD card if I were somewhere that i needed to. The 40GB hard disk will only take 5 downloads of my new SD card.

I wonder if backing up locally is a good idea - i'd worry about media in general. I've seen catastrophic failures because of media and many resulting Doh! moments. In terms of backup strategy, throwing hard disks at it is fine so long as you have full resilience and no single point of failure and are prepared every couple of SD card downloads to throw another hard disk (or two) into the array - could be troublesome in the final analysis (also maybe not).

There are online solutions if you are prepared to spend a little - for US$50 per annum, Carbonite.com will automatically back up your files with unlimited storage. Its secure and available on demand. If you were to put a new disk or two into an array each year you'd probably easily spend the US$50 that they are asking. Then i wonder if you have primary storage based on a media solution local to your machine (hard disk, dvd, blu ray, etc...) - is your backup actually safe. Heavens forbid that fire visits any of us but it may happen or a catastrophic failure of technology, or darn it - you make a mistake. You may be secure if you have offsite replication but jezzz life shouldn't be so bloody complicated.

I haven't settled on a solution yet. For me, CD's, DVD's, and disk drives are becoming a disaster - there is overhead to maintaining them locally which i ask myself do i need the hassle. From the nuts and bolts side of it i'd have no problem with any of the technical capability or capacity required to do it. I just think there must be better solutions - online perhaps but then again i'm only thinking about them at the moment

Whilst taking a stroll down memory lane, in 1988 i *upgraded* from a dual 360k (yes k) 5.25" floppy drive to a 20MegaByte hard disk which sounded like there was a bird trapped inside which chirped every time you accessed something on it. No - not a typo 20MB rather than 20GByte. So let me see, by todays standards if it were around today I would get approx 1 complete RAW image onto it!

Maybe the specifications and capabilities of our media change but the numbers and problems haven't really shifted that much.

An interesting thread and one that everyone should contemplate methinks.

ThOnda

Well, on-line back up. How long would it take to upload one 8GB memory card? A night or two? :pac:

Shiny

On-line backup is fine as long as the company remains in business.
They could be here today, gone in 3 years down the road.

Some people suggest raid. This is all well and good until it comes to
upgrade. If the controller dies after 3/4 years you have to buy an
exact replica or it will not work due to the fact that RAID is not
a standard amongst companies.

At the moment I do a direct backup from my main photo drive onto
another drive on the computer. Basically a glorified RAID 1.

At the same time I am in the process of uploading all my photos to
flickr in jpg form but as I have said above how long this will last is
anyone's guess.

Backing up to DVD is fine for those with small collections of photos
or those who have a good work flow and have been backing up
periodically as the photos come in.

At the moment I'm only at around 200GB's of photos which is easy
enough to back up. I reckon (Hope:)) that hard drive size is going
to advance alot faster than my photo collection ever will so I can
continue the 1:1 backup thingy. Backing up movies/tv is another story.

TelePaul

Probably use about 30MB of storage on a given shoot...

Shiny

TelePaul wrote: »

Probably use about 30MB of storage on a given shoot...

That equates to about 3/4 shots in RAW....:D

You mean 30 GB ?

AnCatDubh

I wouldn't worry too much about the time taken to back up - that will obviously depend on the quality of your internet connection which i appreciate will vary (greatly) around the country and this may not be for everyone. As far as i know, online systems can be configured to use 'idle' time and throttle its internet usage accordingly which should minimise the inconvenience of the time that will be taken.

The issue of the backup merchant being around in the years to come is a serious one though. I recall only recently receiving a notification from an online service which i was no longer using to say that they were having 'issues' with their ISP and were changing their ISP and that I should download all data which I stored with them, and re-upload at a later point when the change over to the new ISP occurred. In this case i didn't really use the online service so it didn't matter to me but i wouldn't like to have a couple of hundred gig of images backed up with them only to receive such a notification.

The other point about the data explosion is the capacity of your existing hardware to process the images in PS, Lightroom, Silkypix, Gimp etc... I've noticed a big performance hit and i haven't gone at it with a vengence as yet. Have we the hardware capacity to processes the ever increasing GB of image files - As ThOnda says; it may be a batch overnight job.

TelePaul

Shiny wrote: »

That equates to about 3/4 shots in RAW....:D

You mean 30 GB ?

No no...three or four's about normal.

Cabaal

not sure what I have used, maybe 300GB off hand. need to buy myself anoither 1TB drive for backups