40

My Mom has over 400GB of photos on a quickly dying 2011 IMac. We are getting her a new computer but I am hoping to set up some system where she can safely store all these files in an easy to access format. Currently her photos are scattered throughout her computer in files and applications like the defunct iPhoto and the new Photos. She also has some 200GB of photos on an external disk.

My parents don't want to store the photos only on the machine as they are afraid of losing the machine and not having a backup. My idea was to hook them up with a google drive which would sync the computer in the background, how do other photographers solve this problem?

TL;DR

  • I need a user friendly way to store and backup over 400GB of photos
  • How should I move over 400GB of photos from an old dying computer to a new one?
  • 32
    What prevents you from putting all photos on the existing external disk? – null Dec 10 '18 at 19:35
  • I think Windows has some user-friendly mechnism for declaring a drive a backup drive and then keeping the backups of specified directories updated. Does Mac not have something like this? Or maybe a third party program exists. Keeping the photos on the hard drive and having 1 or 2 backup hard drives is definitely the best in my opinion (but I'm currently using git to synchronize my files including many photos across computers, so that's nothing for your mom unless she happens to be a programmer :D ). – Nobody Dec 10 '18 at 22:55

14 Answers 14

54

The general reccomendation is a 3-2-1 Backup strategy, meaning you have 3 copies: 2 local, 1 offsite. Here's one way:

  • Get a NAS appliance (like a Drobo) to protect against a single disk crash
    • (or just an external disk, with no single disk failure protection)
  • Add an Apple Time Machine to get your 2nd local copy.
    • If Time Machine is not for you, get another external drive and some other software (Crashplan can do this) to make a local backup.
    • The point of this 2nd copy is (1) quicker restore, (2) immediate backups while the cloud catches up, (3) do you trust your cloud backup service completely?
  • Use some service like CrashPlan, BackBlaze, Amazon Prime, ... to do offsite backups.
    • This protects you from fire, theft, flood, lightning, ...

The downside to these off-the-shelf solutions is they do not protect against bit rot on the disks. To get that, best I can tell, you'll need to do something custom like periodically running par2 or deploying a custom FreeNAS appliance with ZFS.

Also keep in mind that a mirror is not a good backup -- if you mirror new files and also sync the deletion of files, you'll have the ability to recover from a disk disaster (assuming your disaster wasn't the kind that made files look like they were removed!), but your recovery path from user error (oops, I deleted something I shouldn't have!) is very limited.

You also want to retain file history (not just the latest copy!) to protect yourself from crypto trojans. Most cloud services do this. Your 2nd local copy should do this as well.

For moving the photos, I would copy them using rsync. The benefit of rsync is if it is interrupted, it can be restarted and will skip files already copied.

  • 20
    @PeterW. Pardon my French, but f**k Time Machine. It uses a weird structure with hard links to folders and cannot be easily read by any other program. If any step fails during the recovering process, the whole process stops and fails. CarbonCopyCloner is much more reliable. – Eric Duminil Dec 11 '18 at 11:24
  • 2
    For offsite backups, you could simply use yet another external HDD and leave it somewhere else (e.g. at a friend's place or at neighboor's) and swap it every time you visit your mom. – Eric Duminil Dec 11 '18 at 11:26
  • 1
    @EricDuminil So what? You're not supposed to browse the Time Machine volume's file system anyway, so how the backup's structured is utterly irrelevant. It's perfect for this purpose. – Peter W. Dec 11 '18 at 11:49
  • 18
    @PeterW So what? Well, having an unreliable backup is worse than having no backup at all because it gives a false sense of security. I've seen TimeMachine fail 3 times already. And when it fails, it fails spectacularly. A single corrupt file is enough to break TimeMachine. And then it becomes very relevant how the data is written because you need complex script to find the missing data and ignore the broken links. With other backup systems, the only consequence of a corrupt file is losing one file at most. – Eric Duminil Dec 11 '18 at 12:28
  • 4
    @PeterW.: A power surge when the backup disk is connected is enough to wipe 2 disks at the same time. A fire can destroy 2 disks at the same time and a burglar can steal 2 disks at the same time too. The stress induced by losing one disk can be enough to have a brainfart and somehow mess up the backup disk too. One "backup" done in the wrong direction can also delete files on both disks. So we definitely agree on this one : "a secondary offsite backup service" is badly needed. – Eric Duminil Dec 11 '18 at 12:50
22

Perhaps I'm a bit old-school in this way, but I personally avoid storing my stuff on "somebody else's computer" (a.k.a. the "cloud").

I would just buy (at least) two external drives of sufficient capacity - storage is cheap these days. Back up all your images onto one, and then make a copy of that drive on the other, so you have two copies. Generate a cryptographic hash of each file (MD5, SHA1 or something similar). Periodically test each drive by reading every file and comparing hashes, so that you know when you start to experience bit rot or the drive starts failing (bad sectors, etc.) and have time to acquire another drive which you can seed from the other still-working drive.

Depending on your platform, there's various software to assist with this (e.g. rsync or unison on Linux, both of which I'm pretty sure are available in some form for either Mac or Windows as well).

For best protection, keep one of these drives offsite - safe deposit box or have a friend or family member keep it for you. That will greatly reduce the risk of losing both drives at once.

  • Worth reading: pcworlda.com/article/2984597/storage/… – xenoid Dec 10 '18 at 21:57
  • 32
    To quote OP: "I need a user friendly way to store and backup over 400GB of photos". Speaking as someone who is fully comfortable with the command line, hashing, etc., and I think a manual hash comparison procedure is crazy. Let alone a layperson who couldn't even consolidate/organize their photos in the first place. – Alexander Dec 11 '18 at 3:34
  • 1
    @twalberg No part of a backup (apart from maybe the physical movement of an off-site backup) should be manual. And a naive rsync would delete the file on the destination drive, if you're not careful. All in all, way too manual, way too complex, and way too error prone for even savy people, let alone a lay person.. – Alexander Dec 11 '18 at 4:33
  • Shameless self promotion: I created Archiverify to help automate exactly this workflow. I use it myself, but I may be the only one, so please test before using on your precious photos :). – Dan Dec 11 '18 at 9:54
9

The easiest and faster way to backup is an external drive. 400 Gb is not that much and 1 Tb hard drives are pretty cheap.

You could backup on two external hard drives and store one in a different place like a family member, in case (let's hope not) something happens like a burglar.

The other option is online storage, but 400 Gb is too much for a free account so you are probably forced to buy... like 50 years of storage. But remember to have a decently strong password.

Both methods have pros and cons.

8

With Amazon Prime you can store an unlimited amount of Photos incl. Raw Files!

https://www.amazon.de/b?ie=UTF8&node=12153288031

  • And you get the other benefits of Amazon Prime as well – SztupY Dec 12 '18 at 14:19
  • 1
    An English link for the few of us who don't speak German. ;) – FreeMan Dec 13 '18 at 12:33
6

Get Google Photos.

It's free, it's easy, and once you set it up then future backups are automatic.

  • 4
    +1 For personal photos. For uncompressed quality photographs space is not unlimited. – Pere Dec 11 '18 at 15:39
  • 1
    @Pere Correct. Based on the post I think it is safe to say that the mom is not a professional photographer nor digital artist. – MonkeyZeus Dec 11 '18 at 15:44
  • Well, I'm afraid a lot of people in this site aren't professional photographers but wouldn't indulge in storing their work anywhere with less than professional quality storage. In fact, for people investing a lot in their photograph equipment it wouldn't make sense to store their photographs in a service that compresses images with some loses. However, for people taking photographs with mid-low digital cameras or just mobile phones, Google Photos is fine. – Pere Dec 11 '18 at 15:50
  • @Pere That is an ambitious statement. I would argue that even enthusiasts or hobby-ists would think Google Photos is fine for their JPEGs. I have yet to notice the slightest difference between an original photo and the lightly compressed one on Google Photos. If you have a JPEG which proves otherwise then please provide a link and I will test it out. – MonkeyZeus Dec 11 '18 at 15:56
4

Don't be me and store them all on a hard drive. Back them up online as well. It'll suck when you lose the drive or break it. A physical backup only, is not the right solution

4

Let me share my solution for the same purpose. I've built a NAS server, which is running at my home. It has two big drives that are working in a RAID 1 solution and i'd like to point out that RAID makes sense in such applications, since in case any of the drives fail i have another one and some time to replace it to regain redundancy. This is not a backup solution and never was. The third drive is for the operating system, which in this case is a Debian Linux with Openmediavault installed. Openmediavault is a software that turns a computer into a NAS and allows you to share directories, configure raid and backups, etc. I wanted something quiet and something that will not drain much energy. Now for the more interesting part. My data flows like this:

-data gets archived chronologically by my script that reads EXIF data and knows when a photo/video has been taken. It then ends up on two drives working in RAID -data gets backed up to an external drive (tar) -data gets encrypted, divided into parts and sent over night to the "cloud". Since i have parts smaller than 15GB i can send them all to Google without having to pay for subscription, but i still consider paying.

Storing data in only one place is not a good solution.

There is no reason for others to build a NAS on their own. There already are ready and nice solutions out there. I did it because i wanted to play around with it deeper and script it's operation.

2

Treat your images like investments, spread your portfolio to minimise risk!

You should use a mixture of on site and offsite backups, so if one storage medium fails, you will still have the others.

Cloud storage is very good for this, because large cloud companies, such as Google, Amazon and Microsoft have much better backup policies than a casual home user can achieve without significant expense.

I personally use Google Photos, because they provide unlimited free storage space, and they automatically categorise all of your photos, so they're easy to search. In my opinion, this is good value for money. They have utilities that make syncing your backups to the cloud trivial, so everything gets backed up no matter what.

I also have my photos backed up onto a home NAS, with Raid 1 redundancy. This is useful if anything ever happens to Google. I can also access them from anywhere that has an internet connection. (Remember, Raid by itself is not a backup solution, it merely gives you some extra redundancy. You should always pair it with other backup solutions).

As a third option, I get free cloud storage with my ISP. I have some limited backups there, of very important photos.

There is always risk. All of your backup locations could fail at roughly the same time. You just want to weigh the risk, vs the expense of improving your redundancy, and adjust accordingly.

2

I believe you should go for a three 1TB USB3 drives from at least two different known manufacturer (WD, Segate, Hitachi, Samsung) which give a long manufacturer warranty. Copy everything to the three disks (two for actual backup and one working copy attached to the computer you use to see the photos). They should cost less than $250.

And if they really have meaningful value to you, you should upload the photographs, after locally encrypting them, to BackBlaze’s B2 Cloud Storage, which will serve as an off premises frozen and resilient backup for emergencies only.

Storing 700GB with them will set you back $3.50 per month and a one time charge of $7.00 for each restore of the full 700GB, unless you need to receive a drive with all of the encrypted photos through FedEX or UPS which is a bit more expensive.

1

I would like to address

How should I move over 400GB of photos from an old dying computer to a new one?

Copying 400 GB over a WLAN can take considerable time, especially if one of the involved machines (the old computer? the old router?) uses old and slow WLAN hardware. If that is an issue you have two options:

  • Connect both computers with a LAN cable to the router. That should give you at least 100 MBit/s, independent of other devices on the WLAN. Then share drives on the network.
  • If the computers cannot be moved easily you can attach the target disk to the old computer via USB, using a USB hard disk case; or you try and physically move the disk from the old machine to the target machine. That will be particularily efficient if the target computer can be a desktop machine, which can usually easily hold another hard disk. This requires at least some basic familiarity with computers (how to open a case, how to connect a hard disk, obviously). If the old computer is a laptop it must be opened, and the hard disk must be removed. That can be difficult (find all the screws, detach the two halves of the snapped-together casing). The laptop disk will need an adapter for the desktop computer. But it is all fairly doable and will be faster than other solutions.
1

For storing amounts of data that large, especially in the realm of photography and video, RAID drives (Redundant Array of Independent Disks) are a reliable choice, albeit more expensive.

All RAID arrays consist of multiple drives that spread out your data across multiple disks, either to increase performance, provide data redundancy, or both. If you are just storing the files and are not constantly reading and writing the data to and from the disk, setting it up in RAID 1 is optimal data redundancy, meaning if one drive fails, you have a backup on the second drive. There are several other fault-tolerant configurations, the other main popular one being RAID 5, which calculates and writes "parity data" across 3 or more drives. Using RAID 1, you lose 50% of the combined drive storage; with RAID 5 you only lose 1/n (where n is number of drives) at the expense of calculating the parity data.

Note that RAID 0, which is used to increase read and write speed across multiple drives, is NOT fault-tolerant.

If you’re looking for a cheaper solution, a simple external hard drive will work fine, but doesn’t protect your data if the drive fails.

With that much data, online storage likely isn’t optimal, especially if you’re needing to upload and download data constantly. It also doesn’t fit well into typical photography workflows through Lightroom and Photoshop (though I’m not sure what cloud storage options Adobe is offering these days, but with 400gb I bet it would be well more expensive and troublesome than getting a RAID).

Hope this helps.

  • 6
    RAID is really not useful at all for this application. It makes sense when you want realtime/online redundancy for data that's continuously updated, to minimize or eliminate downtime when hardware fails. It does not serve as a backup, which is what OP needs. – R.. Dec 10 '18 at 22:32
  • 2
    @R.., I have a small RAID system on which my computers periodically back up their hard drives. If one of my computers goes belly up or, as has happened in the past, I've replaced a computer, I simply restore the backup to the new computer thus backing up my photos (and all the rest of my data as well.) I don't understand why a RAID "does not serve as a backup" because that seems to be exactly what I've been doing for years. – CramerTV Dec 11 '18 at 0:08
  • 3
    @CramerTV because if someone breaks in and steal every electronic in your house, you don't have a copy of anything. – Fábio Dias Dec 11 '18 at 0:46
  • 2
    @CramerTV: What Fabio said. Also, if you run malware on your PC and it decides to delete all your files, you don't have a copy of anything. A backup is offline or append-only (so it can't be modified/deleted by malware) and physically located somewhere other than the original copy. – R.. Dec 11 '18 at 3:38
  • 5
    @CramerTV RAID doesn't protect you against a failure in the RAID controller, a house fire, a software bug, ransomware, or user error (accidentally deleting a file, and deleting the "orphaned file" on the backup volume with a misconfigured sync). History is a key element of backups. Suppose a file F is made then deleted shortly after. Even if the "latest" view of the backup shows F as non-existent, it's absolutely critical that F be stored and accessible on the backup drive (e.g. via an hourly incremental backup) – Alexander Dec 11 '18 at 4:38
1

My current approach is to use two external hard drives for local backup, one automatic and one manual, combined with Backblaze B2 for remote backup.

Photos are initially stored locally on an SSD, grouped into one or more folders per day. Post-processing, if necessary, is performed on the files stored there. Files on the SSD are automatically backed up to the first of the external drives using the File History feature built into Windows. (Mac users can use Time Machine to do the same.) The File History layer contains the most up-to-date backups and is rarely behind by more than a few hours (typically, the drive is plugged in and a backup manually initiated shortly after the files are loaded onto the system).

These folders are then regularly copied to the second external hard drive, which is intended to contain a readily-accessible and complete archive of my photos. Older folders of photos are periodically removed from the local SSD to free disk space, but only after they have been backed up to both drives. This step is performed less often and may be a few days to a few weeks out of date (depending on when I get to it), though the data is still on the system SSD and the File History drive.

Later on, the folders on the second external drive are rolled into encrypted archives, each containing one calendar month's worth of photos, and uploaded to Backblaze B2 using a program called Cyberduck. Only whole months of photos are uploaded at a time, so the data there can be a month or more behind the local storage, but any data that is yet to be uploaded would still be redundantly stored locally.

My File History drive may fill up faster than the main archive drive because it covers all my personal data on my system, not just photos, and may store copies of photos that are being worked on or deleted. As such, this does mean that older backups files on this drive may need to be deleted. By the time this happens, though, the data would already be on the second archive drive and on Backblaze B2, and I generally don't care about older intermediate copies stored on the File History drive.

In all cases, at least two copies of all photos are maintained, and many will have three copies.

Diagram of backup system


This backup scheme has proven itself before. I've had the archive drive suffer unrecoverable filesystem corruption at one point due to a botched USB transfer. Downloading the archives only costed a few dollars, and while the B2 archives didn't contain the most recent photos, I was able to retrieve the rest of the images from the File History drive.

  • I was writing about BackBlaze almost a the same time you did. Coincidence. – abetancort Dec 11 '18 at 23:27
  • As of this writing, I have about 550 GB of data in Backblaze B2. Costs less than $3 a month, a small price to pay for peace of mind. – bwDraco Dec 11 '18 at 23:40
1

Buy a USB blu-ray optical disc burner!
My preference is the Pioneer BDR-XS06 USB3 slot-load burner.
I'm in the middle of backing up roughly 400GB onto the BDXL 100GB discs.
Takes like 3 hours per disk. I got a ten pack for around fifty bucks on amazon.
"Verbatim BD-R XL 100GB 4X White Inkjet Printable, Hub Printable - 10pk Spindle"
If you want the archive to be permanent, pay a little more for the M-Disc media.

The biggest pain is sorting what you want on each disc, I suppose.

EDIT:

  • I should point out, I've got a Synology DS718+ with a pair of 8GB DDR3 laptop sticks in it, and two WD 10TB disks in SHR (btrfs), as the backup source.
  • The optical discs are the secondary offsite backup; now stored in a little fire-safe "good for 30 minutes at 1550f", with a second copy to be mailed off to another state.
  • Yet another boon for optical media -- shipping 1TB rust platters is a good way to end up with a box that jingles when UPS gives it ye olde 8-foot-drop. Mail a set of photo backup discs to family members elsewhere. Pray you never have to recover from them.
  • Rockstor, FreeNAS, Nas4Free, and the rest are nice if you can support yourself -- but if you don't know bash from cmd, buy a synology or drobo, save yourself the headache.
  • Drobo/Synology support having a USB optical burner plugged in and accessible from the webUI
  • Drobo is more mac-like, Synology's more windowsy (even in it's webui)
  • You can also use a simple linux box with ubuntu 18.04, and samba4. Raspberry Pis should be avoided (Cheap USB), Odroids and RockPi4's fine. LePotato's highly recommended. Any other Amlogic's probably fine too. Stay away from Allwinners.
  • Avoid cloud storage unless you don't mind having every picture you've taken shared on all the pirate sites / celebgate/icloud-swiped. Just sayin'. If you gotta pick one, go with backblaze or tarsnap.
  • Cryptolocker viruses are nasty. Few of 'em can infect NAS boxes now.
  • Kinnnda hard for them to encrypt an already-burned optical disc though.

If in doubt? Buy a Drobo. They're more or less bulletproof to your average mac user.

All the water in the world can't sink a ship unless it gets inside. Plan for failure.

  • 8
    Not only is this more expensive than an external portable 1 TB hard drive, you also have to buy the burner, and you have to sit and wait while burning and swap the discs. – pipe Dec 11 '18 at 9:36
  • This is probably not the right solution for a very layperson, but in principle optical media are good for long term archival so have an upvote. This would require the person to first organize the photos, then maybe sort by date and burn the oldest ones each time they accumulate to the size of a disc. – Nobody Dec 11 '18 at 9:59
  • @Nobody - You are correct. An additional point to consider is that while waiting to gather enough data to make burning a blu-ray disk worthwhile, your data will still need another backup method to protect it from loss. For that, some combination of external HDD or cloud storage (as suggested in other answers) would be a great solution. – FreeMan Dec 11 '18 at 15:50
  • 3
    @Nobody "optical media are good for long term archival" - I'm not convinced. It perhaps depends on the quality of media, however, I have some DVDs I used for backup 15+ years ago and they have since visibly decayed! I have HDDs that have lasted longer. – MrWhite Dec 11 '18 at 17:18
  • 1
    I'm not sure about BDR longevity, but many CDRs and DVDRs have an appallingly poor lifetime, and may show corruption in a few years. Good archive quality ones, stored correctly, will last better, but that's not what most people use. – Dan W Dec 12 '18 at 10:58
1

As of 2018, there hasn't been a single instance of a major cloud storage service (Google Drive, Dropbox, Onedrive) losing any customer data, despite numerous allegations about their unreliability from various sources on the Internet. On the other hand, there are thousands of cases every single day of users losing their own local backups every single day, even those who use a more secure backup option such as a RAID array. Therefore my recommendation is to store your files on two separate cloud storages and use a local copy for quick access only.

protected by AJ Henderson Dec 11 '18 at 17:55

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.