1

I have pretty much the opposite question to this one.

I recently needed to upgrade my computer. On my new computer, I still have all of the documents from my old computer, copied over in the same directory structure. This means that within my darktable working location, I have all of my original raw files, and all of my sidecar files (thankfully). Unfortunately, the fresh version of darktable doesn't automatically know that, for example, IMG_1234.CR2.xmp should be applied to IMG_1234.CR2. So none of my previous developing and editing work shows up anymore.

Obviously, I could manually apply each sidecar file to its corresponding image file to get my edits back, but across thousands of photos, that would be tedious and error prone. Is there a way that I can automate this by filename across my whole photo collection from within darktable?


What I tried

For what it's worth, I tried a couple things based on reading the online manual, but it didn't quite do it. From section 2.2.7 Sidecar files:

Once an image has been imported into darktable the database entries take precedence over the XMP file. Subsequent changes to the XMP file by any other software are not visible to darktable – any changes get overwritten the next time darktable synchronizes the file. This behavior can be changed in the preferences dialog (see Section 8.2, “Core options”). On request darktable looks for updated XMP files at startup and offers a choice whether to update the database or overwrite the XMP file.

... and from section 8.2 Core options:

LOOK FOR UPDATED XMP FILES ON STARTUP

Check file modification times of all XMP files on startup to find out if any got updated in the meantime by some other software. If updated XMP files are found a menu opens for the user to decide which of the XMP files to be reloaded – replacing darktable's database entries by the XMP file contents – and which of the XMP files to be overwritten by darktable's database. Activating this option also causes darktable to check for text sidecar files that have been added after import time – see option “overlay txt sidecar over zoomed images” in Section 8.1, “GUI options” (default off).

I tried selecting the "LOOK FOR UPDATED XMP FILES ON STARTUP" option, then restarting darktable, but this only worked for one photo. I'm not sure, but I think it's because the sidecar files are all from before the photos were imported into the new install, so darktable ignores them because it thinks they're old.


Kludgy last ditch option

I'd prefer to find some way native to darktable to get this done, but failing all else, I might try the following from my base darktable working directory (I'm in a Linux environment):

for i in $(find . -name "*.xmp")
do touch $i
done

After this, I'd try the "LOOK FOR UPDATED XMP FILES ON STARTUP" option again. This approach makes me vaguely nervous.

  • 1
    Good question. "This approach makes me vaguely nervous." makes me suspect you're working without a reliable backup. Is that the case? If you have a backup you could take 100 photos, do the kludgy thingy and check if the "XMP files on startup"-option works. – Saaru Lindestøkke Sep 29 at 15:06
  • @SaaruLindestøkke Good idea. I do have backups, but it'd be kind of a pain to have to restore everything again. Mostly I'm just skittish. The kludge approach seems like it should be pretty low risk, I just worry that there's some obscure thing I didn't consider, and running this command would delete all my photos and reformat my hard drive and crash my car and burn down my house. – Gern Blanston Sep 29 at 15:52
1

I think your suspicion that the date of the sidecar files plays a role is correct. The manual states (bolding mine):

Look for updated xmp files on startup

Check file modification times of all XMP files on startup to find out if any got updated in the meantime by some other software. If updated XMP files are found a menu opens for the user to decide which of the XMP files to be reloaded – replacing darktable's database entries by the XMP file contents – and which of the XMP files to be overwritten by darktable's database. Activating this option also causes darktable to check for text sidecar files that have been added after import time – see option “overlay txt sidecar over zoomed images” in Section 8.1, “GUI options” (default off).

So it seems you need to have XMP files that have a modification time that is later than the last database modification time.

Roughly the steps I would have in mind would be:

  1. Create a copy of +/- 100 of your photos and XMP files into a separate folder
  2. Import the copies into darktable
  3. Set the "Look for updated xmp files on startup" options in darktable.
  4. Close darktable.
  5. Loop over your XMP files and touch them to update the modification time only (hence the -m option):
for file in $(find . -name "*.xmp")
do
    touch -m $file
done
  1. Open darktable. I expect it now to replace the database entries by the XMP file contents.
| improve this answer | |
1

Reimporting pictures will apply existing sidecar files.

Conversely, replacing the newly created database with the previous database is another option when file locations have not changed.

Darktable stores all your edits in the sidecar files. It stores metadata about the sidecar files in an SQLite database.

When Darktable opens it uses the SQLite database to generate the light table view. As Darktable runs it uses the SQLite database for sorting and searching. It updates the SQLite database when ever images are imported.

When upgrading computers or reorganizing the file system, I prefer to rebuild the SQLIte database by reimporting everything into a new empty database. This purges metadata about pictures in locations that no longer exist or that are irrelevant to my needs (for example duplicates in backup locations).

| improve this answer | |
  • I wonder why it didn't apply the sidecar files when I imported the pictures in the first place. – Gern Blanston Sep 29 at 21:22
  • @GernBlanston Did it work when you tried again? – ben rudgers Sep 29 at 21:30
  • Well, I didn't really do anything at all, but the next time I opened darktable, all my edits had been applied to my photos. I have no idea why. Apparently the answer was "wait a couple hours and try again, maybe it'll fix itself." – Gern Blanston Sep 29 at 21:50
  • @GernBlanston I am glad it worked out for you. – ben rudgers Sep 30 at 2:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.