Advantages of complex demosaicing algorithms on images to be scaled down

Question

Let's say I have a 20MP RAW image, that I know will be used scaled down. I can think of several reasons why I would know that for sure: it will be published on the web, it will only be displayed on screens/media of limited resolution, file size restrictions etc. I can always keep the RAW, but let's assume I want a downscale version at the moment, with less than quarter the size, so <5MP.

I understand that 20MP of Bayer data represents somewhat more information than a 5MP RGB, so it is generally worthwhile to exploit this and try to reconstruct a full 20MP RGB image. This is still an approximation however, and several advanced algorithms have been in use for demosaicing, to try to tackle the artifacts created in this step.

Now my intuition would say, that in the <5MP case in my example, demosaicing could also be done with a trivial 4-pixel grouping (with averaging on the green channel) or something similar, and it would still yield a reasonable result for 1:4, that could be scaled down further as needed. Alternatively, it would even be possible to do it in a one-step downsampling method with arbitrary target dimensions.

In two of the widely used RAW editors (darktable and RawTherapee) I was not able to find an option to set the size target of demosaicing, they always do full resolution it seems. First of all, of course it would be appreciated if anyone could point to a software package that has this option. But I think the main question remains: are there any advantages of downscaling from the full resolution, even if it was reconstructed from Bayer data? Would it be possible, that the feature I have mentioned is just simply not implemented because it does not matter much when scaled down further, and it can be done in two steps with similar results anyway?

Answer 1

Demosaicing is a bit more complex than you think and the relative position of the R/G/B sensels is important. If you take groups of sensels instead the algorithm would be different. Combine that with the idea that people who use RAW usually want to milk all the data from the sensor and authors won't have much incentive to do the additional programming.

Answer 2

The advanced "Aliasing Minimization and Zipper Elimination" (AMaZE) demosaicing algorithm explicitly has "Zipper Elimination" in its name, and zippering is a non-local mosaicing/demosaicing artifact that will have effects even after downscaling.

Answer 3

Your idea of "four pixel grouping" doesn't work for several reasons. But the primary one is a BIG one.

The colors of our color filter arrays (typically a Bayer mask, but some camera makers use other patterns, such as Fuji's Trans-X sensors) DO NOT match the colors used by our output devices.

All of the cute little Red-Green-Blue checkerboard diagrams all over the internet notwithstanding, Bayer masks do not have filters that are those colors!

Although we call them "red", "green", and "blue", the colors of most Bayer masks are:

• 50% "green" pixels that are centered on around 530-540 nanometers and significantly sensitive to light ranging from about 460nm to past 800nm and the edge of the infrared range. The "color" of 540nm light is perceived by most humans as a slightly yellow green color.
• 25% "blue" pixels that are centered on around 460nm and significantly sensitive to light ranging from the non-visible ultraviolet range to about 560 nm. The "color" of 460nm light is perceived by most humans as a bluish-violet color.
• 25% "red" pixels that are centered on around 590-600nm and significantly sensitive to light ranging from about 560nm to well into the infrared range. The "color" of 600nm light is perceived by most humans as a yellowish-orange color. (What we call "red" is on the other side of orange at about 640nm).

_{The above image is a microscopic capture of a sensor that has had part of the Bayer mask removed. As you can see, the "red" filters are actually yellow-orange and the "blue" filters are as much violet as they are blue.}

Let's call the blue-violet filter centered on 460nm "Blue" or "B".
Let's call the slightly yellow green filter centered on 530-540nm "Green" or "G".
Let's call the yellow-orange filter centered on 590-600nm "red" or "R".

Then let's call the Red, Green, and Blue colors used by our emissive displays that are centered around 480nm, 525nm, and 640nm, respectively, R, G, and B.

The R, G, and B values for each pixel must all be interpolated from the raw values of the sensels covered with "R", "G", and "B" filters because "R" ≠ R, "G" ≠ G, and "B" ≠ B.