3

When adding photos to a catalog Lightroom 6 goes through it's usual building of standard previews. As Adobe designed this process to run in the background leaving you with enough resources to work at your photos at the same time, the preview building allocates only 50% of the CPU time (uses all cores).

I usually add photos in large batches and use this time to take a break and make some tea while Lightroom chews through them. Naturally, I'd like previews to be built faster rather than having resources available for other tasks during the process.

Is there a way to make Lightroom utilize 100% CPU for standard preview building, the same way it does when exporting RAW images for example?

EDIT: to make things clear, Lightroom uses up 100% CPU and all the cores/threads (my CPU is 2 cores / 4 threads) for any other task it needs to, including exporting photos, working with adjustment brushes in develop mode (which tend to be CPU heavy edits) and rendering 1:1 previews. There is no problem with the system/hardware, everything works perfectly fine.

The issue in question only concerns building of standard previews and as far as I know is there by design. The question is if it possible to overcome this Lightroom design limitation in any way.

  • 1
    I have no idea how the Windows scheduler works, but are you sure that this is the case, and it's not just that the process is IO-bound? – mattdm Apr 25 '17 at 11:35
  • I's not. Using an SSD instead of 7200 rpm HDD results in the same CPU load. – lightproof Apr 25 '17 at 11:41
  • 4
    This might be better on superuser.com, since it's really about the computing aspect rather than photography or photographic workflow per se. – mattdm Apr 25 '17 at 12:22
  • @mattdm the windows scheduler works on a priority basis so isn't the culprit here. The process could run at essentially 100% even with idle priority if you're not doing anything else. Maybe it maxes out on cores - coded to run on up to 4 but the OP has 8 or something like that. – Chris H Apr 25 '17 at 16:22
  • @ChrisH I have 2 cores :) (i7-3520M) – lightproof Apr 25 '17 at 17:30
2

I have experimented extensively with Lightroom and performance, but have no knowledge of its internal workings (and you would need Adobe to comment there).

It APPEARS that Lightroom has some sections of code that cannot execute in parallel. Over the years it has improved, but is not there yet. I have recently built a new system for photography which is all SSD, has separate controllers (one U.20 to get it off SATA), has 64G of memory. I tried to eliminate all bottlenecks, and it still tends to run about 50-75% busy building previews. In testing this I see no queue on disks at all, and certainly no memory pressures.

I believe you get slightly better utilization if you start multiple jobs on separate files, but it is not a full solution, and it does not seem to help to start 3, 4, etc. processes. Two seems to get all there is to get, and it is not much. My empirical impression is that certain activity is single threaded (probably by critical sections) and there is nothing the user can do.

Now that said, there is a lot you can do in general, for example: if your disks are busy, separating preview/ACR cache from images on separate disks (especially if spinning) and/or controllers, and in turn separate from catalog (a symbolic link is needed to separate preview cache from catalog), having adequate and fast memory (I have found it is difficult in single images (vs pano merges) to get LR to use over about 8-12G), choosing the smallest preview that you need ("auto" works pretty well), and finally by far the most important: using a CPU that has the fastest single code speed you can get.

If you use Intel that supports Hyperthreading, I find it performs slightly better with Hyperthreading off, not on (i.e. no "fake" cores).

A corollary of all this is that getting more, slower cores is not very helpful; getting few (say 4) faster cores helps much more. With significant code single threaded (apparently), as a benchmark to choose CPU's for LR, the single core processing seems the most important.

I do think 4 is better than 2; I certainly see at least three cores active at times. I do not think 6 would be better, much less 8 (this may change in later versions, and as mentioned is more about single core speed than number).

I also tried faster memory (clocking at 3000mhz vs 2400mhz) and found it helped slightly (about 10%). Faster GPU performance ONLY helps develop screen sliders, it will have no impact (as of the current version, 2015.10) with preview builds. Overclocking the CPU gives almost linear improvement in speed. Obviously overclocking anything may reduce stability.

For what it is worth, one often requested feature that is not in Lightroom that would help is the ability to use the embedded preview instead of buiding one. For many, with high volumes and low keeper rates (i.e. lots to cull), it is just too slow building previews to use. My personal solution for this was to stop using Lightroom for culling; I cull, crop and straighten outside with a faster tool that uses the embedded preview (Photo Mechanic in my case but there are many of them), and only take shots that have a high likelihood of being keepers into Lightroom.

  • It's a LR design limitation. I edited the post to make things more clear. As for hardware, I have 16Gb of RAM and an i7-3520M, which is a full-voltage laptop CPU with 3.6/3.4 GHz top single/multi core clock speed. It's not a quad-core of course, but Lightroom runs very good on it. I don't use GPU acceleration (Intel drivers produce artifacts in LR and it doesn't speed the workflow a lot). I cull, rate and do all metadata processing in Photo Mechanic as well, but I still add all the remaining photos to LR after I delete all straight bad shots. – lightproof Apr 25 '17 at 19:19
1

I assume you are probably memory- or cache-bound. Your i7-3520 has 4 MiB shared cache, meaning that both cores share the 4 MiB L3 cache. Because your CPU is hyperthreaded, to the degree that each SMT thread acts as a "CPU", your L3 cache is shared amongst 4 "CPUs".

The particular size of the cache isn't important, nor is the 50% utilization that you are observing particularly telling. That is, don't assume that because 50% is the same as "one half", that there is some "divide by 2" inefficiency going on. The utilization might as well be 42%, or 61.803%, etc.

Let's think for a moment about the data that Lightroom is working on while trying to build previews. These are low-res versions of the real image, meaning there is some sort of downsampling or N:1 interpolation going on to create the smaller previews. Therefore, the preview building is operating quickly over a large dataset, skipping or quickly averaging nearby pixel values, rather than performing pixel-by-pixel heavy computation that might occur when editing in the Develop module.

Additionally, we have no idea how the image is represented in memory in Lightroom. Personally, I would assume the data is stored as RGB triples, probably 16 bits per pixel (or more) per color. (Note that this has nothing to do with how the image is stored on disk, or whether it was captured RAW or JPEG. Those are file storage details.)

So assuming the images are represented in memory as uncompressed triples of 16-bit data, each hyperthreaded CPU "core" can only operate on at most, ideally, √(1 MiB / (3 color/pixel) / (2 byte/color)) ≈ 418 px square regions. However, the time cost of populating the cache regions with image data from RAM is a lot longer than it takes for the CPU to operate over some fraction of each ~ 420 × 420 pixel region.

@Linwood's answer is quite good in covering his performance measurements and observations of Lightroom's overall multithreaded performance. I'd like to add this multicore performance study of Lightroom CC/6 by Matt Bach at Puget Systems, a custom performance computer builder. Of note is the observed performance increase per core utilized when generating 1:1 previews, up to about 5 or 6 cores. After 6 cores, there is no more per-core utilization to be gained.

Also of note in Puget Systems's test is that the Lightroom tasks that gain the most bang-for-the-buck increase of number of cores (not hyperthreads) are:

  1. Exporting images to disk
  2. Conversion from RAW to DNG
  3. Generate 1:1 Previews

The other image editing and library tasks don't benefit as much from scaling up the number of cores.


Back to your particular system, if we could hypothetically alter certain parameters of your computer, here's what we would likely see:

  • Decrease CPU clock rate: assuming the RAM speed is the same (1600 MHz), you'd see an increase in utilization, because the fraction of time loading data to/from RAM compared to the amount of time executing instructions would fall. You'd spend a bit more time in execution, as opposed to load/store in RAM. Of course, your real aggregate throughput would be lower, so this is not desirable.

  • Turn off hyperthreading: you'd probably see a slightly higher CPU utilization percentage, but your net throughput of number of previews generated per minute would fall. I don't know if these are offsetting gains/losses though. Linwood's experience would suggest that there would be at least a slight net gain.

  • Double the number of physical cores (no hyperthreading): CPU percentage would be similar (assuming the cores shared L3 cache).

  • Double the L3 cache size: This would probably realize the largest observable gain, both in CPU utilization and in net throughput.


Bottom line, however, is that no, you can't do anything to increase your CPU utilization on your system during Lightroom preview generation, with the inputs you're feeding it.

0

If your system has 2 cores, and Lightroom is pegged at 50% CPU usage, then I can make the following observations:

  1. Lightroom is using 100% of one of the cores. So that the import to Lightroom is bound by the speed of the CPU. This is a limitation of your hardware.

  2. Lightroom has not been written to take advantage of multiple-cores. This is a limitation of Lightroom.

As such I think you have run into a combined limitation of Lightroom and your system. And you can't change how your copy of Lightroom works!1 So the only way of improving things is get a system with a faster CPU.


1. Although a different version of Lr may make difference. But I know nothing about what version of Lr you are using and how Adobe's development process is going. In fact I am trying to get away from Adobe anyway, so I don't even want to know!

  • I have edited the post to make things a little more clear. – lightproof Apr 25 '17 at 19:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.