13
\$\begingroup\$

My main computer is a laptop, so disk space is at a premium. I have been taking RAW photos on and off for 18 months and I'm worried about disk space now. What options do I have for backing up my files? Burning the whole lot to DVD seems like a laborious process. I've had bad experiences with external harddrives. Are there other options I'm missing?

  • Burning the whole lot to DVDs seems laborious and awkward
  • External hard drives look like a reasonable option, but I've had one break on me, so I don't want my only backup to be on the one drive. Two external harddrives?
  • Online backup seems like a possibility, but I imagine that it will be expensive to house ~40GB+ of photos.

Are there other options I'm missing? Within the above three categories, what advice can you give me about how to proceed?

[edit: an ideal solution would be open source and work natively in linux]

Improve this question
\$\endgroup\$
7
  • 5
    \$\begingroup\$ This is a good question in general, but I don't think it's specific to photography. We tend to have large amounts of irreproducible data, but from the point of view of backups, I don't think there's anything unique. This question would be better for superuser.com , or given your edit, for unix.stackexchange.com. \$\endgroup\$
    – mattdm
    Jan 2, 2011 at 12:44
  • \$\begingroup\$ See also photo.stackexchange.com/questions/2138/storage-recommendations \$\endgroup\$
    – Joanne C
    Jan 2, 2011 at 13:59
  • \$\begingroup\$ What is the capacity of your current HDD and how much space is used? \$\endgroup\$
    – Zaz
    Jan 2, 2011 at 16:44
  • \$\begingroup\$ Also relevant: photo.stackexchange.com/questions/292/… \$\endgroup\$
    – Craig Walker
    Jan 2, 2011 at 20:41
  • \$\begingroup\$ if using an external hd, definitely look into something automatic like Syncback! \$\endgroup\$
    – stijn
    Jan 3, 2011 at 16:55

17 Answers 17

Reset to default
9
\$\begingroup\$

You have to have an onsite solution (easy, quick and cheap) and also an offsite storage (if something happens to your house (flooding, fire, robbery), you lose everything).

The onsite solution is easy. As said, external drives are the best, you will then have tons of applications to help you synchronise this. Avoid manual backups as you'll end up forgetting this and realize it when you'll need to recover the files.

For the offsite solution, I went to Amazon S3. It's extremely reliable and quite cheap. Check their pricing. at the end, this small fee is nothing to keep your most important files secure. From their monthly fee calculator, storing 40GB will be around 5$ a month. You'll also have to pay an even smaller fee for transfers (uploading a GB is 0.10$).

It will be the equivalent of a sandwich per month to keep your files secure and always available in case of issues with your laptop and onsite storage.

Improve this answer
\$\endgroup\$
1
  • \$\begingroup\$ I like Carbonite.com as well. $59/year, unlimited space... but it is per computer and no network drives. For a single computer... it works good for me. Also you are able to access the files remotely via their web app or mobile phone apps... in the end +1 to LudoMC... you need on and off-site. \$\endgroup\$
    – RiddlerDev
    Jul 19, 2011 at 16:54
6
\$\begingroup\$

I'd say that external hard drive and online backup are the only realistic options:

  • External hard drive - cheapest solution; note that even if the external hard drive does break, you will still have a copy of the data on your computer (costs about $50 for a 500GB drive).

  • Online backup - safest, but most expensive solution

    • Dropbox: $9.99/50GB/month or $19.99/100GB/month (i.e. 20¢/GB/month), handles incremental syncing automatically; see features.

    • Amazon S3: Starting at; storage:14¢/GB/month, upload:10¢/GB, download:15¢/GB. Syncing must be handled by the user; no built-in syncing function.

My advice is to stick with an external hard drive; it pays for itself in half a year if you would otherwise be using Dropbox. Your previous bad experience is probably due to chance.

As a side note, you could look into RAID, although setting up a RAID array is quite complicated and will only be possible if your laptop has a second drive bay.

If you're running out of space on your laptop's HDD I would recommend buying a bigger HDD for your laptop, then using your laptop's HDD as a backup drive. You can buy an enclosure for the backup drive for about $20. The process of transferring the data and the drives themselves is quite simple but will require some technical knowledge which you can find help with at superuser.com.

If you're not willing to mess about with drives then you'll have to either buy two external hard drives, or use an online backup service such as Dropbox.

Improve this answer
\$\endgroup\$
9
  • 1
    \$\begingroup\$ The point is, since hard drive space is limited, I'd delete the originals from my laptop so the backup would actually be the only copy of the files. That's why I'm worried about HD failure. \$\endgroup\$
    – Seamus
    Jan 2, 2011 at 13:04
  • 1
    \$\begingroup\$ If your data is valuable, use both of these. The local copy is convenient, and protects you in case the online service suddenly goes out of business. And the online copy protects you against both drive failure and more tragic scenarios. (The previous experience probably was chance, but it's the sort of chance that is in fact inevitable and worth planning for.) \$\endgroup\$
    – mattdm
    Jan 2, 2011 at 13:45
  • 1
    \$\begingroup\$ There are fireproof enclosures available as well as at least one fireproof safe that I found with a USB cable of all things that allows for a USB drive to be connected from inside the safe. So, if online is not an option because of ongoing cost, one of these might add some additional security for a more local solution. \$\endgroup\$
    – Joanne C
    Jan 2, 2011 at 14:58
  • 1
    \$\begingroup\$ Well, for the little drives that are powered from USB, probably not an issue. Of course, as solid state drives get bigger and cheaper, I think it becomes even less an issue. The tech is improving. :) \$\endgroup\$
    – Joanne C
    Jan 2, 2011 at 17:32
  • 7
    \$\begingroup\$ I'd like to point out the common saying "RAID is not a backup". Even an external hard drive is a much better backup, because it can be configured to be safer (e.g. if you or a virus accidentally deletes a file it can keep a copy) and it is not susceptible to hardware malfunctions like a failing power supply zapping hardware components. Still, online backup is the best because it protects against catastrophes like fires. I can't afford real online backup, but I put jpeg copies of my favorite photos somewhere online. \$\endgroup\$
    – rm999
    Jan 3, 2011 at 5:40
6
\$\begingroup\$

Two standard hard disks is the simplest and most economical backup solution. One disk is kept off site in case your house burns down. You can use regular (and cheap) internal hard disks if you have external SATA ports on your machine. If you have somewhere to keep the backup disk at work taking it home one evening a fortnight works well.

Every 12-18 months replace the disks with new ones with double the capacity, to keep up with your expanding collection of photos.

This method works great under linux with a little shell scripting!

Improve this answer
\$\endgroup\$
4
  • \$\begingroup\$ +1, this is also my solution. I use Unison \$\endgroup\$
    – labnut
    Jan 2, 2011 at 20:46
  • \$\begingroup\$ I'm not familiar with Unison though I will check it our - for various OCD-esque reasons I like to be in total control of my backup solution so I prefer to write my own software, in addition to owning the hardware. Fortunately writing a script to sync to files isn't too hard! \$\endgroup\$
    – Matt Grum
    Jan 3, 2011 at 3:04
  • \$\begingroup\$ I have a script to do this; see reidster.net/software if you're interested. \$\endgroup\$
    – Reid
    Jan 4, 2011 at 14:15
  • \$\begingroup\$ rsnapshot is another good solution, it's a wrapper around rsync. \$\endgroup\$
    – chris
    Jul 19, 2011 at 23:36
5
\$\begingroup\$

Like others here I believe in layers of protection for my data

  1. RAID - it isn't backup but reduces the risk of a device failure losing all my data
  2. Removable hard disk, local backup (2 TB)
  3. Remote backup at CrashPlan ($3/month for unlimited storage - right now I have some 140 GB of data backed up)

At some point I plan on augmenting 3 by making use of the CrashPlan ability to back up to a friend's computer that also runs CrashPlan.

Two things that I think make CrashPlan worth looking at, beyond the low cost. They'll post you a blank disk (one TB ISTR) to perform an initial backup to, to avoid having to do that first backup across the Internet. Then they'll post a disk with your backups on it to allow you to recover faster.

Improve this answer
\$\endgroup\$
5
  • \$\begingroup\$ RAID without proper RAID-controller is only useful for speeding the system up. Software RAID is not secure and leads to nice f***ups. \$\endgroup\$
    – Leonidas
    Jan 4, 2011 at 0:33
  • \$\begingroup\$ @Leonidas - Hardware raid controllers also contain software, and they are not infallible. With a OS-level raid you at least have a few options when things go wrong. \$\endgroup\$
    – Console
    Jan 4, 2011 at 9:27
  • \$\begingroup\$ @Leonidas - references? I've run multi TB software RAID arrays without problems for years now. Indeed, I've had more f*** ups with hardware RAID controllers. \$\endgroup\$
    – Cry Havok
    Jan 4, 2011 at 10:27
  • \$\begingroup\$ Reference? As your anecdotal evidence, my anecdotal reference plus the knowledge, that SW-RAID suddenly involves processor, main--memory and bus too instead of only dedicated controller and HD. Simple MTBF-calculation + Murphys law. \$\endgroup\$
    – Leonidas
    Jan 4, 2011 at 12:00
  • \$\begingroup\$ @Leonidas, that dedicated controller adds a processor, memory etc. Indeed, you've added complexity and potentially made a failure more likely. I think Murphy's on my side on this one ;) Of course, if you've got something you can point me at I'm interested in your experiences, though I'd suggest that this is better taken on the likes of SuperUser/SeverFault than here. \$\endgroup\$
    – Cry Havok
    Jan 4, 2011 at 13:35
4
\$\begingroup\$

I don't consider online backup a realistic option at this point. the cost effectiveness for larger collections isn't there. If you stop and look at how long it'd take to restore after a catastrophic failure, the time aspect is a killer right now. Maybe some day.

hard disk is your only realistic option, and is cost effective. Multiple copies, and offsite storage are your best options.

Iv'e written about this -- see:

http://www.chuqui.com/2009/11/more-than-you-wanted-to-know-about-backups/

http://www.chuqui.com/2009/11/some-more-thoughts-on-backups/

http://www.chuqui.com/2009/12/following-my-own-advice-on-backups/

http://www.chuqui.com/2010/03/what-to-do-when-you-realize-youre-running-out-of-disk/

that last one talks about the specific situation -- sitting on a laptop and realizing you're running out of disk, and what I did about it. To protect yourself from serious data loss isn't difficult, but it means getting serious about doing something and building habits and a work flow that makes it happen. it's easy to get lazy and let it slide, and then realize you've got holes in the backups just when you need them...

Improve this answer
\$\endgroup\$
2
  • \$\begingroup\$ Worth looking at CrashPlan - they'll post you a (next day delivery even) disk with your backup on it to allow you to recover faster. \$\endgroup\$
    – Cry Havok
    Jan 3, 2011 at 19:15
  • \$\begingroup\$ +1, working with semi-reliable (see Wikileaks) online-storage-providers over not necessarily existing connections with no real possibilites for proper encryption (of possibly private pictures) is no real backup strategy at all. Some duplicated HDs with a nice TrueCrypt will always fit into a bank vault and your parents home. \$\endgroup\$
    – Leonidas
    Jan 4, 2011 at 0:29
3
\$\begingroup\$

I had the exactly the same problem - but with over 100GB of photos. I used to keep backup on the external HDD but after fire alarm, I realized I want an off-site backup. I've set up a server in another location and now I'm rsyncing photos. All my machines are running Linux - so far it works well.

Improve this answer
\$\endgroup\$
2
\$\begingroup\$

Lets admit the fact that things fail, and if you really want a photo to last forever, then printing is the only option and another discussion about what type of printing lasts!

Short Term

If you want to build your own solution I would suggest one of the following:

Either

  • 1x White Box PC that is Dual Core and 2Gb RAM or higher
  • 1x500+Gb HDD for main OS
  • Several 1xTB HDD for data repository

With this you can now install GNU/Linux (or FreeNAS) onto the primary HDD and then use ZFS (via fuse if using GNU/Linux) to setup the other HDD in ZFS Raid. This gives redundancy and ensures that your photos are safe from bitrot, and is the most robust solution given current technology and options.

OR

A Thecus NAS that can take care of this for you and save you the headache of having to worry about building your own PC.

We wont worry about performance as we are more interested in ensuring a data is robust for the short term.

After you have this you now need a way of getting this data of-site for an extra redundancy plan. This essentially boils down to using any of the following:

  • Replicate afore mentioned setup and sync across the Internet (using a VPN of course!)
  • Use a smaller external HDD (with mirrored drives for redundancy) to replicate to and rotate your offsite storage
  • Use Blueray DVD's to burn your catalog to and storing these off-site. Remember to always do a complete burn as this mediums life is probably around 2-5 years.

Long Term

I have no real answer here and its something myself that I am worried about. This is the problem with the digital age. In 25 years am I really going to be able to either access my data or even be able to read my data?

I have no plan for this myself and am basically running on blind faith that holographic storage will be available before the end of my lifetime as anything magnetic/optical as a storage medium is vulnerable to the elements of a period of time

Improve this answer
\$\endgroup\$
4
  • \$\begingroup\$ This sounds like a very paranoid solution. I'm also not sure why you're worried about what'll happen in 25 years; a mirror RAID set can keep your data safe indefinitely. \$\endgroup\$
    – Zaz
    Jan 3, 2011 at 13:26
  • 1
    \$\begingroup\$ @Josh - I believe only if you have the high end products that can scrub your data to ensure checksums match. I believe Raid 6 will not protect against bitrot but am happy to be proven wrong! \$\endgroup\$
    – Wayne
    Jan 4, 2011 at 0:23
  • \$\begingroup\$ Long Term - it's the bits that matter, not the media. Copy the data to new media every few years you'll be fine. \$\endgroup\$
    – Reid
    Jan 4, 2011 at 14:16
  • \$\begingroup\$ We are probably heading off topic now, but writing data every x months/years will not solve a bitrot issue. Raid 6 can cope with loss of sectors to write too, but not about corrupt bits in a byte I do believe it will happily replicate that to any sector that can be successfully written to/striped too \$\endgroup\$
    – Wayne
    Jan 5, 2011 at 9:31
2
\$\begingroup\$

While external storage might be a good solution, it is still in-house, if anything happens, you will still lose everything (having two disks and keeping them seperated still requires more work).

I made the decision to research online backup providers. I didn't need synchronization, so it's a little bit cheaper (bigger packages) in general.

If it's just for home use I can recommend MozyHome. It will cost $4,99 a month, for unlimited storage (am now using ~80GB).

If you decide on a backup solution, please let us know, everybody is always looking for better ones.

Kind regards,

Matthias Vance

Improve this answer
\$\endgroup\$
2
\$\begingroup\$

I went through this question a while back and decided to go with offsite storage with a local convenience copy. I have a complete blog post about it on my blog but to paraphrase, my solution was the following:

  1. Take lots of pictures
  2. Import pictures into Lightroom Catalog called "Originals"
  3. In Lightroom use the Publish feature to upload the originals straight out of camera shots to SmugMug which has unlimited file storage size and is hosted on Amazon S3 servers.
  4. Import and edit the photos in Lightroom Catalog called "Working"

While I have everything locally, I also have originals (JPG format) stored on SmugMug. For really great photos I keep the RAW files as DNGs on my JungleDisk site (also hosted on Amazon S3 servers).

One can just as easily do all of this using open source tools or just the web portals for SmugMug and JungleDisk. I have placed files offsite in case something really bad happens and have also made the accessible without "my computer" in case it goes kaploowe.

Improve this answer
\$\endgroup\$
0
\$\begingroup\$

My solution is to use Apple Time Machine, this way I haven't to configure anything else and the backup is up to date. When I import my RAW files to my Mac, Time Machine makes and instant copy of them. Tine Machine is a differentialn (or incremental) backup, so it only stores the new things.

Improve this answer
\$\endgroup\$
1
  • 2
    \$\begingroup\$ Already good but not enough. Should you have an issue in your house (robbery, fire...) you lose everything. \$\endgroup\$
    – LudoMC
    Jan 2, 2011 at 16:42
0
\$\begingroup\$

You're mixing up backups and a simple memory extension:

You will need at least two copies of each file on separate devices to have what is called a backup. I'd recommend an external HDD for this. Your previous experience with an external HDD only confirms how necessary this is - internal hard drives are not technically different from external ones, so your internal one could fail just the same way.

Furthermore, in case you've got too little memory on your internal hard drive, you'll need two external ones.

Improve this answer
\$\endgroup\$
0
\$\begingroup\$

Backup is one of those things where everyone has a different strategy. For me burning DVD's isn't an option anymore because it's way too small (maybe if i got a blu-ray), so nowadays I have an hybrid of the other two options:

  1. External drives: Automatically backup photos to external drives and I'm planning to create a small NAS/backup server at home to extend this.
  2. Online storage: Regularly upload a smaller subset of my photos, usually the ones I picked to work on, things I uploaded to stock or prints

But whatever your choice is remember: it must me an automated process and there should be several copies

Improve this answer
\$\endgroup\$
0
\$\begingroup\$

An external drive that is a bit beefier and safer thank a single typical external hard-drive would be to have multiple disk drives cloned for redundancy. These come in pre-built units such as this buffalo model:

link text

Also there is a great free software to automate copies or files/ folders to multiple locations. I use it in web server farms and its never let me down. Here is a link: link text

Improve this answer
\$\endgroup\$
0
\$\begingroup\$

USB Drives + Carbonite.com

First of all: well done on asking a very important and (sadly) rarely asked question!

I use Carbonite.
It costs something like GBP 5 a month for unlimited backups from one PC.
The only downside is how long it takes to do the initial upload. After that, it's OK.

In combination with this, I have 2 500GB USB drives.
I keep one at my day-job office, and swap them over whenever I do a backup.

Improve this answer
\$\endgroup\$
0
\$\begingroup\$

There are actually three issues you need to deal with:

1) Lack of space on the laptop to store all images

2) Backup of all images

3) Off-site storage of all images

So I'd suggest a pair of external hard drives, and a fire-rated safe.

Keep 3 months worth of data on the laptop (or some other period that works for you, depending on the quantity of data you have), and sync to the external harddrive as needed. I use rsnapshot, but there are other similar solutions to do this automatically. Now you have 2 copies of the most recent data, and 1 copy of the historical data.

Now, once a week or so, swap the external drive with the drive in the safe. Now you have 2 copies of the historical data, and 2 copies of the recent data. The only data that's at risk is any data created since the last swap, and it's only at risk for fire/theft. You can change the swap frequency (or ensure that every time you take pictures, you back it up to both drives) if that's really a concern for you.

Total cost for this can be very reasonable - Costco has several small fire & water rated safes for under $100.

Improve this answer
\$\endgroup\$
0
\$\begingroup\$

Looks like your interests just overlapped with Slashdot:

http://ask.slashdot.org/story/11/07/20/0121248/Ask-Slashdot-Best-Offline-Storage-Method-For-Large-Archives

Pay attention to the responses.

Improve this answer
\$\endgroup\$
0
\$\begingroup\$

Please read this concerning online backups - depending on your Internet provider, you may rue it. This is precisely what happened to a photographer who used Carbonite for his backups: The Day Comcast’s Data Cap Policy Killed My Internet for One Year

Personally, I use digikam for organizing my Photos on Gentoo Linux, I use the synchronizing ability of the file manager Krusader for easy synchronization in the following manner:

  1. I import photos from the card to the digikam collection via digikam.
  2. Once the import is finished, before making any changes, I use a previously defined synchronization profile in Krusader to synchronize the backup external harddrive that is always attached. Ideally, this disk should not be left attached as it is then subject to wear and tear at the same rate as the original disk.
  3. I have a portable harddrive which I normally keep at work. At regular intervals, I take it home and synchronize in the same manner as above. At this point and at this point only, I delete the originals from the SD card.
  4. I keep a separate digikam album of any photos I want to tweak, I never touch the originals save for deleting shots I deem unusable. I try to do that in-cam, before I import. The whole idea is to prevent a single point of failure, or a double failure from destroying your data.

As such, I would suggest looking into Krusader, especially if you are using KDE.

Improve this answer
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.