What is the best way to synchronize Adobe Lightroom databases between two computers?

Question

What is the best way to synchronize Adobe Lightroom databases between two computers (a desktop and a laptop)?

I'd like to be able to edit photos on either machine, and preferably not copy the whole 3+ gb database on each sync.

Answer 1

I've been synchronizing collections spanning up to 60GB and ten thousands of small and large files for years now. This includes my LR catalogs and pictures since I started photography. I'll be honest though, this is not a tool a lot of people use, it is not trivial to setup but it will solve your problem. Once it is up and running, it is very easy to work with and it will perform its duty quickly and effortlessly. As a software engineer, this is one of the top 3 pieces of software I use daily and can't live without.

The tool is Unison File Synchronizer.

Unison

Unison is a file-synchronization tool for Unix and Windows. It allows two replicas of a collection of files and directories to be stored on different hosts (or different disks on the same host), modified separately, and then brought up to date by propagating the changes in each replica to the other.

• It's free, run on both Windows and Unix, and even works across platforms.
• It can deal with updates to both replicas of a distributed directory structure.
• Updates that do not conflict are propagated automatically.
• Conflicting updates are detected and displayed (more about that below).
• Unison is resilient to failure. It is careful to leave the replicas and its own private structures in a sensible state at all times.
• It works between any pair of machines connected to the internet. It is careful with network bandwidth. Transfers of small updates to large files are optimized using a compression protocol similar to rsync.

Large files

This later point is relevant to your problem. If a large file has been updated (say, your LR catalog), it will not copy the whole file over. It will instead transmit a smart, compressed subset (a delta) representing the difference between both files. For example, if I modify the Artist name on a MP3 or add some keywords to a JPEG file, it will not send the whole song or photo over, but only a very small portion of it. Granted, bigger changes happen in your LR catalog, but never quite big enough that they require to send the whole file over. This makes sense because your catalog is a SQLite database (a SQL db on disk); for performance reasons it is in the database engine's best interest not to shuffle every bits around.

Renaming

Unison has no problem renaming files and directories. Files are not identified by names actually, but by a unique signature. This is very convenient.

Compared to Microsoft Live Sync

Similar to Microsoft Live Sync, Unison does not keep an online copy of your files (thank God), which means both machines need to be connected to the internet.

Unlike Microsoft Live Sync (from what I understand), it does display potential conflicts and its user interface will let you decide which replica should "win". That is, if you made changes on both sides, it will let you mark if the 'left' side should have priority over the 'right' side (see screenshot below), or vice versa. You can do so file by file, replica by replica, on a subset of files, etc. You can even, for text files, have Unison launch a tool that will let you resolve the conflict manually at the contents level (say, using WinMerge).

Thumbnails and Previews

Now what about thumbnails and previews? Do not synchronize them. You do not need to, I never did, and LR is smart enough to regenerate a preview if it's missing. This has never been a problem for me in 2 years.

Besides your RAW and JPEG files of course, which piece of data should you synchronize to make sure you can work with LR on both sides? There are 3 components to consider, to my knowledge:

• Your LR catalog file (Lightroom 3 Catalog.lrcat), which is where LR stores its metadata, keywords, rating, labels and develop settings. Remember, LR is non-destructive, it doesn't touch your original files but store every single change you made to a photo in its catalog instead (unless you instruct LR to dump these settings back to your XMP/DNG/PSD files, but that's another story and not a problem anyway). Your catalog is the important file to sync.
• Your Previews directory (Lightroom 3 Catalog Previews.lrdata), which is where LR stores intermediate representation of your photos for faster viewing (1:1, low res, thumbnails, etc). Do not synchronize this folder.
• Your Adobe Camera RAW Cache (ACR Cache). Every time you open an image in Camera Raw the full resolution of the image must be loaded into Camera Raw. As you can imagine, this can be pretty processor intensive. The Camera Raw cache will cache recently opened images to make re-opening them faster. This cache is actually used by LR and Photoshop if you have both. Do not synchronize this folder either, its content will be regenerated automatically.

You can change the default location of your catalog, previews and Camera RAW Cache from the Preferences. This is how I moved my catalog to a specific user directory (my HOME directory, with personal data), and this is the very directory I sync. You don't have to do that, but it makes my life easier.

Installation

As I mentioned earlier, installing Unison is tricky, check the user manual online, and comment if you are still in the blue. It does involve running a SSH server on one of the computer (if you sync between two), traditionally using Cygwin or a subset of it. This is the tricky part, but SSH is an open, robust, safe and secure solution. I trust this set up more than running Microsoft Live Sync or Dropbox, which are essentially client/servers too.

Once your SSH server is running, you are pretty much all set. You only need to write a small text file called a 'profile' that will tell Unison which folders need to be sync'ed. I realize it's not very user friendly, but it does have a powerful syntax that lets you ignore files using regular expressions or simple patterns. Here is what the top of a profile looks like, for instance:

root = d:\users\barre
root = ssh://barre@foo.net/d:\mystuff

The first line specifies where my local replica is, and the second where my remote replica can be found, on the foo.net (fictitious) machine at home for example, in the mystuff directory. This is even simpler if you stay within a local network. I use Unison to sync between 4 machines at 3 different sites.

ignore = Path {Pictures/Lightroom/Lightroom 3 Catalog Previews.lrdata}
ignore = Regex Pictures/Lightroom/.*\.lock
ignore = Regex Pictures/Lightroom/.*-journal

These 3 lines instruct Unison to ignore my LR thumbnails/previews, as well as any temporary lock/journal files found in the Pictures/Lightroom sub-directory (relative to the root directory specified above).

And that's about it.

Running Unison

The first time you run Unison takes quite a bit as it computes hash signatures for all the files under its new jurisdiction. This is key to help computing differences quickly. The next time you run it, it's astonishingly fast. The user interface is a bit rough because Unison is an exotic beast written in OCaml + GTK. Just pick your profile from a list and Unison will check your local replica for changes. At the same time it will call itself through SSH on the other replica to compute remote changes. Wait a few seconds and it will display a list of changes and potential conflicts to help you reconcile the two, if needed. If there is nothing to resolve, just click "Go" and sync happens automagically.

In this screenshot below I’m synchronizing my local machine (first column), with my remote machine named KANEDA. The second column, in between, is what Unison thinks should be done.

As you can tell:

• I deleted one file on my local machine (First Friday…jpg), and the change will be propagated from left to right (the arrow), thus deleting the file on the remote.
• I changed both my LR catalog and a DNG on the remote. The arrow is pointing from right to left, i.e. these changes will be propagated to my local replica.
• I changed my .emacs.el file on both replicas, in a way that can’t be merged. Unison tells me at the bottom when the modification happened on both sides, the files size, etc. The question mark in the middle denotes that it’s up to me to make a choice. This line is selected, so I can just press the left or right arrow key to decide which side should win. I can even override any other row.

This situation doesn’t happen very often to me anymore, 99% of the time I don’t have any choice to make, I just review the situation quickly, press “Go”, and Unison synchronizes both replicas.

In Conclusion

I've used this tool for years and I swear by it. I synchronize and backup music, photos, documents and personal data using Unison, a task that would be impossible to do efficiently manually. It supports very large collections and thousands and thousands of file. It handles renaming, smart sync of large files and works quickly. It never corrupted a single one of my files.

This is still a file-level synchronization tool. It is not aware of the subtleties of your catalog, only Lightroom does. If you made changes to your catalog on both sides, you will have to make a choice between the two. At least the whole file is not copied over, and the user interface makes it easy to tell Unison which one should win. As somebody pointed out in this post, a bit of discipline should take care of this problem. Your catalog is one complex file. Even if a general-purpose tool used high-level SQL commands to inspect your catalog’s database and try to synchronize it with another, I still wouldn’t trust it over Lightroom.

Good luck.

Answer 2

I got an upvote to this answer today which is usually a reminder to me to go visit old answers or questions to see if they have withstood the test of time.

This one both has, and has not, withstood this.

Lightroom now (I don't remember when it was introduced) has a feature that allows it to synchronize changes you do with photos into separate XMP files.

You can turn this feature on by going through:

• Edit->Catalog Settings
• 3rd tab, named Metadata
• Enabling the checkbox for "Automatically write changes into XMP"

Now, after you have done this, the following things will happen:

1. All existing photos in your catalog will:
   • Either get a second .XMP file alongside it on disk
   • Or 2, be changed to include metadata changes inside the file, this is possible for some file formats and I know Adobe DNG is one of them, possibly JPG's as well.
2. Any future changes you do in Lightroom will be saved in the same spot as in point 1.

You can now synchronize the entire folder structure containing your photos, and those .XMP files (or photos with the metadata inside), but not the catalog using any such synchronization software that I mention below, like Dropbox, Google Drive, Jotta cloud, OneDrive, etc.

Be aware that my advice against file conflicts still hold if you switch between two machines and do changes, but now the changes has to be done to the same photos for the conflicts to occur, the conflicts are no longer at catalog level.

Instead of synchronizing the catalog, set up a separate catalog on each machine you want to work with your photos on, importing all folders from their synchronized location on disk without moving them.

Then, whenever you switch to a different machine, do the following to bring it up to speed:

1. Find your library folders, these are at least in my installation located on the left side, in the Library module, under a pane named "Folders"
2. Right-click the topmost folder and select "Synchronize Folder"
3. Ensure "Import new photos", "Remove missing photos from catalog" and most importantly, "Scan for metadata changes" are all checked.
4. Click "Synchronize" and let it do its work

This has the benefit of synchronizing any of the changes Lightroom can do to your photo into the catalog on this machine, including:

• Anything you do in the develop module
• Tags, keywords

There are probably some parts not being synchronized, such as stack configurations and whatnot, but most of your catalog should synchronize just fine.

Here's also a very useful, at least to me, change to the process that I could introduce with this.

I have 3 computers:

1. A powerful Windows desktop computer
2. A powerful iMac
3. A not-so powerful MacBook Pro

The MacBook Pro has one very big bonus though, I keep it with me much more often than I am at home with my desktop based computers. Sadly it does not have enough storage built in to keep all my photos locally, but Dropbox to the rescue.

I can synchronize only the latest months, or projects, locally using Dropbox' selective sync feature, and then synchronize the local catalog. This will remove, from my local catalog, photos I no longer have locally but add in all photos I just synchronized down to my computer.

I can then work on my MacBook with the photos, safe in the knowledge that any changes I do will be synchronized through Dropbox back to my two desktop computers at home, ready to be synchronized into their respective local catalogs.

This has allowed me to not only work cross machine on my photos, but also be mobile, which is a big bonus.

NOW! If you came here looking for advice on how to synchronize things like this, but without Lightroom, then the rest of my answer is the best I can give.

---

Older answer

If we remove Lightroom from the equation, we reduce the question down to this:

• How can I synchronize changes to large files between multiple (2 in this case) computers, without just copying the whole file each time

Now, let's say you're 100% disciplined. You only ever make changes to the file on one computer at a time, and you allow the synchronization software run and do its job before you change the file on any of the other computers.

Warning, read further below for a BIG caveat

In this case, there are many software programs that can do what you want, but since any software that runs on only 1 of the computers will have to read the file on any of the other computers to work out what changed, the only viable solution is one that has software running on both ends, ie. both machines.

Software like this includes:

• DropBox
• Microsoft Live Sync

and there are many other. Basically you install the software on all the machines involved, set up synchronization, which usually means registering the folder on one of the machines as "I want to synchronize this folder", and then visiting all the other machines and mapping the synchronized folder (that the software knows about) to a folder on disk. After that, it's all automatic. Changes on one computer gets propagated to the others.

Microsoft Live Sync does not keep an online copy of your files, which means synchronization only happens when computers are online and can talk to each other, whereas DropBox keeps an online copy so you can update on one machine, let DropBox synchronize, turn the machine off, and then turn on another to get the changes downloaded.

Now, the problem here, and you'll find this holds true for all file-level synchronization tools, is that at some point, your disciplination skills waver, and you end up having changes on more than one computer. There is no way for the above type of software to work out how to merge these files. Microsoft Live Sync will flag it as a conflict, and then go about making sure all unique copies of the files are available on all the machines. This essentially means one of them will be renamed, and then both will be downloaded to the machine they're not on.

I don't know how DropBox handles conflicts, but I can tell you it won't be able to merge.

So now you have 2 copies of your catalog.

It gets worse. The Lightroom catalog also contains many smaller files, like thumbnails and such, and what you might end up with is that changes to those on multiple computers gets cross-synchronized to other computers, but end up not corresponding to the catalog they belong to.

Example:

• Catalog + smaller files A, B and C on machine 1 is changed
• Catalog + smaller files D, E and F on machine 2 is changed

After synchronization:

• Conflict on catalog files
• A, B and C copied to machine 2, but does not correspond to the catalog any more (remember, conflicting catalog changes)
• D, E and F copied to machine 1, and does correspond to the catalog (at least, one of them)

So let me give you what I consider is a good advice: Don't use file-level synchronization tools for this. It might work in the beginning, but at some point you're left with a restore from backup and hours (or days) worth of lost time.

If, on the other hand, you can find true Lightroom catalog synchronization tools (I've looked but haven't found any yet), that should work. The lightroom catalog is (or at least was in earlier versions) an SQLite database file, so replicating the individual changes to the contents is perhaps doable, but hard, and requires precise tools that understands the database type and most likely the content.

Other ways you can do what you want:

• Use RDP or similar to work from one machine, but keep everything on just 1
• ... suggestions?

Answer 3

You could try a different strategy: I use a small large capacity (1TB) USB3 drive and take that wherever I go. These are smaller then a wallet nowadays and quite fast at 7200RPM and USB3. I use Cobian backup freeware to make sure the contents are always backed on my primary workstation and then everytime I plug it back in. You can take the backup a step further and use an online backup service. Personally I use Egnyte for work ($50/ year for 1TB) where I have a use a similar setup for my GIS data. (Egnyte is a File Sharing service with backup capabilities) It takes a long time to run the backup the first time but it's fairly quick (and seamless) thereafter.

Answer 4

I'm currently developing a tool to synchronize catalogs between computers especially when one is a PC and the other one is a Mac. It currently focuses on the hardest part: making a catalog created/modified on one computer usable on the other despite the differences in layouts. Although work in progress, I use it for about six months with success. So you may want to have a look to LRSync

Answer 5

I am not sure if you are talking about Adobe Lightroom or Apple Aperture. Assuming you are talking about Lightroom, since you used that in the title and in the tags of your question, I would recommend using Microsoft Live Sync.

Live Sync Info

Live Sync is a system that allows you to share folders between your computers through the Microsoft "cloud" network. Synchronization is automatic, lightweight, and allows nearly unlimited files. The current "beta" version of Microsoft Live Wave 4 supports the following:

• Up to 200 synchronized folders
• Up to 100,000 files per folder
• A maximum size of 40Gb per file

Thats a grand total of 800,000,000 Gb, or 800 petabytes of data. When syncing, Live Sync only syncs the differences between your shared folders. If you added 100 photos and edited 50 more, it will sync only those photos and the related lightroom data, along with the lighroom collection database. Only modified parts of files, along with patching details, will be transferred, making sure syncs are as quick as possible.

Possible Drawbacks

Live Sync may not be the right tool for the job. If you need to sync considerable amounts of data (many gigs) very quickly, Live Sync may be too slow. Syncing huge amounts of new data on a regular basis will overload your allotted bandwidth that you have to use with Live Sync, so your syncs can take a very long time. If you only need to sync a few hundred megs here and there throughout each day, it will serve you well. If you need to sync many gigs every day, it may not perform well enough to be useful.

Portable Drive Solution

If you do need to sync multiple gigs per day, then a better solution than Live Sync would be a portable external hard drive. To keep things quick, I would try to get a portable drive that supports E-SATA for maximum speed. Just move your lightroom collection and your photography to the portable drive, and set up both your desktop and your laptop to use the portable drive as your collection storage area. The drawback with this solution is you need to be able to physically move a hard drive between the two systems, which may not be ideal. A nice, high-speed, high-density drive like a Hitachi or a Seagate Barracuda will keep things fast.

Answer 6

You can always "export as catalog" from your laptop and then "import from catalog" it into your workstation later if you want to consolidate everything there.

Both options are under the file menu.

Answer 7

Although the other answers here are very good, they all have synchronization problems (which they address in some detail).

Another approach (really easy to use, but a bit tricky to setup) would be to buy a NAS (network attached storage) which is effectively a small pre-built and configured server that you attach to your main gateway/router - usually with an Ethernet cable.

You then access it as a network drive on your local system and you also (usually) get a static IP address and URL for it (usually by using a service like dyndns.com).

Once you or your local geek get this setup, you put your LightRoom and any other files on it. You then access this one copy from your home system, your notebook, or from any computer on the Internet (with encryption, password security, etc.).

This way you have just one set of data so there are no synchronization issues at all (as long as two people don't try to update it at the same time). You can access it securely from anywhere.

Since it's not stored on the computer you're working on, you have more options for running backups of the data without loading down the computer you're working on. In fact, most NASs that support more than one disk drive will allow you to set them up as RAID 1, 5, or 10 or even (much more simply) to just to use the two (or more) drives separately as mirrors of your data so that you end up with multiple copies of your data on different drives more or less automatically. You can think of it as your own personal cloud storage that no one else has access to (unless you explicitly give it to them).

This takes a bit of expertise to setup, but once it's done, it's very easy and flexible to use - for any data that you need to access from more than one computer or even just for data that you want to backup, but still have access to.

Answer 8

I use Resilio Sync (former BTSync): https://getsync.com

It is a tool that runs in a background and synchronizes files between two running computers. Like Dropbox, but without central server - so it's free, automatic (and has really fast transfers over local network), but you must ensure:

• that both computers are running
• that you don't open Lightroom on second computer until changes from saved catalog are synced to it