2
\$\begingroup\$

In Lightroom CC, there is an option to automatically save all changes not only to the database, but also to a sidecar .XMP file. This can be useful if one wants to use the RAW files in other programs with adjustments and metadata intact. As far as I know, it can also be useful to recover this information in case the Lightroom catalogue ever becomes corrupt (automatic backups should make this superfluous, but it doesn't hurt either). So it seems like every Lightroom user should turn this option on.

However, I somehow feels wrong to clog up my folders with thousands of xmp files that I'll probably never use. I can't really justify that feeling though, since they don't take up any space worth mentioning compared to the RAW files themselves and if I ever need to get rid of them for some reason, I can do so with a simple cmd command.

So, are there any downsides to activating this option? Like, performance-drops, slower start-up, or anything else? If there is any reason for me not to use that option I would like to know. Thank you!

Improve this question
\$\endgroup\$

4 Answers 4

Reset to default
4
\$\begingroup\$

feels wrong to clog up my folders with thousands of xmp files that I'll probably never use

You say "Lightroom CC," by which I assume you also have Photoshop. Every time you say "Edit in Photoshop," you use the XMP data. Photoshop loads it when it loads the photo, and then saves it back out when you save the edited photo.

When you tell Lightroom to "Edit original," this is how Lightroom's changes to the photo get sync'd over into Photoshop. Photoshop will typically end up saving the changes to a different file format than it received the original photo in (PSD or TIFF, most commonly) but the photo metadata still needs to be written out as XMP metadata, else things like camera data get lost in the edit. Despite the edit, you still want the photo marked with your exposure settings, camera model, exposure date, etc.

A number of Lightroom plugins also operate by editing the XMP metadata. If the XMP metadata is out of sync with respect to the Lightroom catalog metadata, Lightroom will put a little arrow badge on the photo to call out the conflict. If you then tell Lightroom to load the metadata from disk, you wipe out any metadata changes in Lightroom's catalog, because Lightroom can't merge the two copies. Alternatively, if you tell Lightroom to overwrite the copy on disk, you lose your plugin's changes to the metadata.

My advice: take the speed hit and leave this option on, always.

Bonus tip: although we have said that writing XMP data takes time, it might not immediately be clear to you how much time. When editing a single photo, it's near-instant. But, if you have a deep and wide keyword hierarchy and change one of the core keywords, such that the XMP metadata for a huge number of photos have to be updated, it can take hours. I tell you this not to talk you out of keeping the "Automatically write changes into XMP" setting turned on, but to point out that there is still a use for Cmd/Ctrl-S with the option turned on: to force Lightroom to save the metadata for a given photo or set of photos immediately when you know you're going to edit it in an app that needs current metadata.

Beware also that Lightroom will pause and resume XMP writes when the app is closed, so if everything isn't written to disk when Lightroom exits, it might resume what it left undone when you start it back up. This is done near-silently in the background; you can't assume that because you restarted Lightroom that all XMP metadata changes are flushed to disk. If you need to be able to see what's going on, select "Metadata Up-to-Date" from the filter bar's preset drop-down menu, then watch the Metadata Status column: if Lightroom is busy writing changes to disk, you'll see the contents of that column changing.

As for the "thousands of files" part, Lightroom only uses separate XMP files with file types that have no way to embed XMP metadata into the file itself. DNG, PSD, TIFF, and JPEG allow embedded XMP metadata. About the only time you see separate XMP files are when you're not letting Lightroom convert your digital camera's raw files into DNGs. Not only does DNG save you from having to keep that separate XMP file associated with the raw photo file, it's probably smaller than your camera's raw file format, without losing much, if anything.

Improve this answer
\$\endgroup\$
5
  • \$\begingroup\$ If your camera includes information from masked pixels or any other data from the sensor saved in the maker notes section of the EXIF info you lose it when you convert to DNG. Some cameras include such information that gets stripped when converting to DNG, other cameras don't. So it really depends on the particular camera in question. With Canon .cr2 files it effectively bakes in the black point by discarding the data from the masked pixels. \$\endgroup\$
    – Michael C
    Commented Jun 24, 2016 at 2:44
  • \$\begingroup\$ @MichaelClark: DNG got support for masked pixels in the second version, support for which first shipped in Lightroom 3.3. I've been shooting CR2s since the 20D, and I've never noticed any practical loss from converting to DNG. \$\endgroup\$
    – Warren Young
    Commented Jun 24, 2016 at 2:50
  • \$\begingroup\$ You'll never notice any loss if you only edit with Adobe products because they've always ignored the data in the maker notes section. But other raw processor can and do use that info in a variety of ways. For instance from the last comment of the article you cite in your answer: "I used to use DNG and it worked great – until I had a problem with a Hasselblad file (.FFF), which had been converted to DNG. There was a serious moire pattern in an article of clothing. I tried several methods to remove it, but eventually had to give up on this image, as it was unusable in it’s current state..." \$\endgroup\$
    – Michael C
    Commented Jun 24, 2016 at 2:54
  • \$\begingroup\$ "...A few months later, a folder with the original .FFF files, turned up. I ran them through Hasselblad’s “Phocus” software and was able to remove the moire with just one slider." \$\endgroup\$
    – Michael C
    Commented Jun 24, 2016 at 2:54
  • 1
    \$\begingroup\$ Thanks for the elaborate answer! I always keep the original .cr2 files from my camera, so I will get tons of XMP files. But I guess I'm convinced of using that option now \$\endgroup\$
    – MoritzLost
    Commented Jun 24, 2016 at 16:09
2
\$\begingroup\$

So, are there any downsides to activating this option?

Some people apparently experience minor performance impact when the data is written to the disk. I see no difference on an SSD. Obviously, you will have your folders littered with XMP files with Adobe specific data and questionable usefulness outside of LR/ACR.

Improve this answer
\$\endgroup\$
1
  • \$\begingroup\$ You don't notice a performance impact using an SSD until it wears out prematurely from all of the write/rewrite cycles every time you save changes to a DNG file if that's the format you are using. \$\endgroup\$
    – Michael C
    Commented Jun 24, 2016 at 2:49
0
\$\begingroup\$

Assuming you keep your raw files in your camera's native format (e.g. .nef, .cr2, etc.) the sidecars can lengthen the file each time more data is written to them. Under certain conditions this can lead to fragmentation when many files were written sequentially when imported.

On the other hand, if you write the data directly to a DNG file (that you created when you converted your native raw files to DNG) the entire file is rewritten each time you save changes. This increases the chance of data corruption with each read/rewrite cycle and can also cause faster wear to flash memory storage devices such as SSDs.

Improve this answer
\$\endgroup\$
6
  • \$\begingroup\$ DNGs do have file validation via a checksum; proprietary raw files do not. dpbestflow.org/DNG#introduction eg. Many consider that a sizable benefit for long term storage and batch validation. YMMV \$\endgroup\$
    – 211Oakland
    Commented Jun 24, 2016 at 16:49
  • \$\begingroup\$ Modern file systems like ZFS, ReFS, and Apple's upcoming APFS do not rewrite files in place. Replacement blocks are written elsewhere on the disk, and only when the data are committed do the old blocks get freed. The OS can crash, the app can crash, or the PC can crash at any point in this process, and you will be left with either the old file or the new file, never a corrupt partially-written file. I keep my Lightroom masters on ZFS. \$\endgroup\$
    – Warren Young
    Commented Jun 24, 2016 at 17:17
  • \$\begingroup\$ As for fragmentation, modern filesystems also solve that problem internally. The only filesystem in common use that needs manual defragmentation is NTFS, and it's hard to argue that a quarter-century old design is "modern." Even HFS+, long in the tooth though it is now, is newer. ZFS doesn't have a defragmentation tool because it only becomes a problem with nearly-full disks, and is solvable even then. \$\endgroup\$
    – Warren Young
    Commented Jun 24, 2016 at 17:57
  • \$\begingroup\$ As for wearing out SSDs, modern high-quality SSDs are more durable than they used to be, what with wear leveling, large spare sector pools, intelligent controllers, TRIM, etc. SSD lifetime is not nearly as big a problem as it used to be. Compared on bits written, SSDs can outlast spinning disks these days. That is, if an SSD writes 10x more data in the same time as an HDD, you've gotten 10x more use out of it in that time. \$\endgroup\$
    – Warren Young
    Commented Jun 24, 2016 at 23:27
  • \$\begingroup\$ You can insist that NTFS is not modern, and you'd be correct. But it is still one of if not the most common file system used to store data by the vast majority of people whose primary occupation is not in the information technology sector. And the statement in the answer is qualified with "Under certain conditions." \$\endgroup\$
    – Michael C
    Commented Jun 24, 2016 at 23:30
0
\$\begingroup\$

Writing changes to the files, whether sidecar or directly into the images (dng or jpeg) has a significant impact on performance when you apply changes to a lot of images at once.

Just try to add a tag to 10000 images in one go, your hard disk will start working for quite a while.

Writing to the image instead of a sidecar file just exacerbates the problem.

Improve this answer
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.