0

I'm using darktable-cli to export photos from my library. I started to get an error with the latest version(2.6.1). It's a segmentation fault with error

[init] the database lock file contains a pid that seems to be alive in your system: 16768
[init] database is locked, probably another process is already using it

I'm always getting this error if darktable is running. And if darktable is not running, most of the times. I have a script running 4 instances darktable-cli in parallel. As I mentioned I didn't have this error with the previous versions.

I don't know how darktable works internally, but I feel like there shouldn't be a need for library for the cli. Is there a need for it?

As a workaroung I tried changing the library for the darktable calls. But --library option is ignored. Darktable still worked with the default library. The command is:

darktable-cli raw-file xmp-file jpg-file --width 1920 --height 1080 --core --conf plugins/imageio/format/jpeg/quality=50 --conf plugins/imageio/storage/disk/overwrite=true --library /tmp/darktable.db

Is this command argument correct?

4

I'm not using Darktable myself, so I can't check it myself but this might be a re-occurrence of a bug that has been fixed two years ago. You might file a bug-report with them and while it's been discussed/fixed, you can revert back to the previous version to keep your stuff functional.

Edit: Checking the documentation you might try to call darktable-cli explicitly setting the library to :memory:, so that it looks something like this:

darktable-cli [...] --library ':memory:'
  • --conf --library ':memory:' didn't work neither – rosencreuz Mar 25 at 6:30
  • @rosencreuz Shouldn't that be --core --library ':memory:'? What's the error message? – Lothar Mar 25 at 7:45
  • --core --library ':memory:' also doesn't work. Same error. Very long error message starting with [init] the database lock file contains a pid that seems to be alive in your system: 1915 [init] database is locked, probably another process is already using it and last line is segmentation fault (core dumped) darktable-cli test.jpg --width 1920 --height 1080 --core. – rosencreuz Mar 26 at 17:30
2

Unfortunately, you can't run multiple instances of darktable at a time.

If your script is running 4 instances of darktable-cli in parallel, there's your error. You need to serialize your script to run at most one darktable-cli at a time. Having some kind of file locking mechanism that waits for the old darktable-cli to exit is needed. An example of such a program is flock.

For example, flock /some/where/somefile.txt darktable-cli raw-file xmp-file jpg-file --width 1920 --height 1080 --core --conf plugins/imageio/format/jpeg/quality=50 --conf plugins/imageio/storage/disk/overwrite=true --library /tmp/darktable.db ought to work.

1
darktable-cli $src $xmp $export [options] --core --library ':memory:' --configdir '/dev/shm/$uniquevalue'

It seems that there's also a data.db in the --configdir parameter (default: $HOME/.config/darktable) which also gets locked by a darktable session. This probably holds user preferences and styles, but if an xmp sidecar is specified then it's not important. (One could probably copy the data.db into a unique instance if necessary.)

Here's a snippet using GNU parallel:

parallel "darktable-cli {} somesettings.xmp export/{/} [opts] --core --library ':memory:' --configdir '/dev/shm/{/.}/' ; rm -rf /dev/shm/{/.}" ::: *.jpg

Darktable will create a new data.db in a dir with the source filename exclusively for that darktable-cli session and remove it after exporting.

Notes for this example:

  • My source images are jpgs. Change the *.jpg to a glob matching your sources (or pipe it to parallel).
  • {/.} strips out dirpath and extensions from files, so that can be used to specify output file type
  • I'm creating the temporary configdir in memory (/dev/shm) for speed; each instance is ~152KB
  • --configdir ':memory:' creates a literal :memory: directory, so doesn't have the same magic as --library :)

Hope this helps, I feel like this is just a workaround and that darktable-cli should probably not lock or open the data.db as read-only.

  • Some questions: (1) what does {/} without the . do? (2) what does the ::: do? – juhist Jul 7 at 6:49
  • {/} Basename of input line. ::: arguments Use arguments from the command line instead of stdin (standard input). And I want to thank you @anomalyconcept for the elegant workaround, I have a script to parallel process raw files from sd card (copy, backup to external storage, develop using darktable, then create smaller "web" versions) and that darktable bug/feature(?) introduced recently forced my parallel invocation to have --max-procs 1 for a while. Was checking again to see if they fixed it and found this answer. Already using it while the bug is fixed, back to usual performance. – Jorge Nerín Aug 3 at 18:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.