Where does this noise come from and how to get rid of it

Question

I am using Darktable as my RAW processing software. In darker pictures that have been taken with an ISO of 800 or more, I frequently encounter great issues with luminance noise.

The following two images (one of clouds and one of pure sky) are cuts from a recent photo that I took. There clearly is some noise, especially in the second picture.

After doing some standard edits like changing exposure and adding contrast, the noise becomes more notable as can be seen in the next two images. Here, color denoising was already applied but nothing for luminance.

And finally, this is what Darktable's denoising filter makes of this. There are heavy artifacts that are arguably even worse than the original noise.

Those artifacts look similar to what a phone camera puts out when you take a picture too dark with it, so I assume it's not a completely obscure problem that I am encountering here.

When I tried to replicate something similar in Adobe Lightroom, the denoising was so much better (see next two images). So apparently, it is possible to implement a better denoising procedure in theory.

I wonder, is there a step that I am missing in my editing workflow which causes these artifacts to be there, or is it simply a bad algorithm that is used by Darktable and there is nothing I can do to improve the image?

Answer 1

Many of darktable's algorithms are significantly different from those typically used by other image processing programs. If you cannot get the results you want from darktable, consider trying other raw processing software, such as RawTherapee or UFRaw.

• The images are consistent with the use of noise reduction filters with different "strengths", where the darktable filter is much stronger than the one from Lightroom. If there are user-adjustable parameters in darktable, try changing them to reduce the strength of the filter.

• Different algorithms detect and treat noise differently. The darktable images have some characteristics that are consistent with median averaging.

• Some processing steps, such as increasing contrast and sharpening, are known to increase the appearance of noise. The darktable images appear to at least have had a greater contrast boost than the Lightroom images.

Answer 2

My guess is you're using denoise (profiled). If so, then you should read darktable's manual section on profiled denoise.

At the time of my original reply the recommendation was to use 2 instances of the module to avoid the effect shown in your examples above with their characteristic look (smeared and "painterly"):

• Use non-local means for luma noise, combined with blend mode ligthness or HSV lightness.
• Use wavelet for chroma noise, blend mode color or HSV color.

This would still work if using older version of darktable (pre-3.6); in this case you would apply two instances of denoise-profiled, one to reduce luma noise the other to reduce chroma noise, with opacity of the blending ranging from 30 to 100%.

With darktable 3.6 this is no longer recommended and using the wavelets algorithm in Y0U0V0 color mode you can now reduce luma and chroma noise with one instance.

The results are equal and sometimes better than what I used to achieve with Lightroom. Lightroom's NR is obviously easier to use.

Answer 3

By curiosity, I tried the state of the art denoiser BM3D [0] on the very first image from the OP; I used as parameters a putative white noise with variance of 0.0012, and find the result excellent: The color and luminance noise are gone and the fine structures of the clouds are revealed.

[0] Mäkinen, Ymir, Lucio Azzari, and Alessandro Foi. "Exact transform-domain noise variance for collaborative filtering of stationary correlated noise." 2019 IEEE International Conference on Image Processing (ICIP). IEEE, 2019.

Answer 4

Denoising basically replaces each pixel with a weighted average of itself plus the neighboring pixels.

There are multiple parameters, and in LightRoom, you can set them independantly: the size of the circle of pixels that are considered, the weighting of the pixels based on distance, and the aggressiveness / strength of the overall approach.
From my experience, the default settings are pretty well chosen in LR, so you simply move the slider until you like the result. I don't know DR well enough to compare, but probably it has similar or equal settings available somewhere.