1
\$\begingroup\$

Let's say I have a 20MP RAW image, that I know will be used scaled down. I can think of several reasons why I would know that for sure: it will be published on the web, it will only be displayed on screens/media of limited resolution, file size restrictions etc. I can always keep the RAW, but let's assume I want a downscale version at the moment, with less than quarter the size, so <5MP.

I understand that 20MP of Bayer data represents somewhat more information than a 5MP RGB, so it is generally worthwhile to exploit this and try to reconstruct a full 20MP RGB image. This is still an approximation however, and several advanced algorithms have been in use for demosaicing, to try to tackle the artifacts created in this step.

Now my intuition would say, that in the <5MP case in my example, demosaicing could also be done with a trivial 4-pixel grouping (with averaging on the green channel) or something similar, and it would still yield a reasonable result for 1:4, that could be scaled down further as needed. Alternatively, it would even be possible to do it in a one-step downsampling method with arbitrary target dimensions.

In two of the widely used RAW editors (darktable and RawTherapee) I was not able to find an option to set the size target of demosaicing, they always do full resolution it seems. First of all, of course it would be appreciated if anyone could point to a software package that has this option. But I think the main question remains: are there any advantages of downscaling from the full resolution, even if it was reconstructed from Bayer data? Would it be possible, that the feature I have mentioned is just simply not implemented because it does not matter much when scaled down further, and it can be done in two steps with similar results anyway?

\$\endgroup\$
3
  • \$\begingroup\$ Anecdotally; sometimes it's better to actually lose some data at downsize, let things get a tad fuzzy… trying to keep things as crisp as possible can give some seriously crunchy images at smaller sizes, as they fight with the screen resolution. You never know what your audience is going to be viewing on. [i have absolutely no mathematical proof of this, merely observation; including that bilinear can be 'better' than bicubic for some purposes, contra to received wisdom] \$\endgroup\$
    – Tetsujin
    Mar 22, 2022 at 17:43
  • \$\begingroup\$ @Tetsujin I think what you're describing is aliasing, where high frequency detail creates artifacts as you shrink an image. It gets worse the more you shrink. Deliberately blurring an image can help, because it gets rid of the high frequency detail. \$\endgroup\$ Apr 23, 2022 at 4:17
  • \$\begingroup\$ Simple averaging isn't a good way of demosaicing, because the colors get offset from each other by the Bayer filter. I worked out the math to do demosaicing simultaneous with a 2x shrink at stackoverflow.com/a/49062949/5987 \$\endgroup\$ Apr 23, 2022 at 4:22

3 Answers 3

2
\$\begingroup\$

Demosaicing is a bit more complex than you think and the relative position of the R/G/B sensels is important. If you take groups of sensels instead the algorithm would be different. Combine that with the idea that people who use RAW usually want to milk all the data from the sensor and authors won't have much incentive to do the additional programming.

\$\endgroup\$
1
\$\begingroup\$

The advanced "Aliasing Minimization and Zipper Elimination" (AMaZE) demosaicing algorithm explicitly has "Zipper Elimination" in its name, and zippering is a non-local mosaicing/demosaicing artifact that will have effects even after downscaling.

\$\endgroup\$
1
\$\begingroup\$

Your idea of "four pixel grouping" doesn't work for several reasons. But the primary one is a BIG one.

The colors of our color filter arrays (typically a Bayer mask, but some camera makers use other patterns, such as Fuji's Trans-X sensors) DO NOT match the colors used by our output devices.

All of the cute little Red-Green-Blue checkerboard diagrams all over the internet notwithstanding, Bayer masks do not have filters that are those colors!

enter image description here

Although we call them "red", "green", and "blue", the colors of most Bayer masks are:

  • 50% "green" pixels that are centered on around 530-540 nanometers and significantly sensitive to light ranging from about 460nm to past 800nm and the edge of the infrared range. The "color" of 540nm light is perceived by most humans as a slightly yellow green color.
  • 25% "blue" pixels that are centered on around 460nm and significantly sensitive to light ranging from the non-visible ultraviolet range to about 560 nm. The "color" of 460nm light is perceived by most humans as a bluish-violet color.
  • 25% "red" pixels that are centered on around 590-600nm and significantly sensitive to light ranging from about 560nm to well into the infrared range. The "color" of 600nm light is perceived by most humans as a yellowish-orange color. (What we call "red" is on the other side of orange at about 640nm).

enter image description here
The above image is a microscopic capture of a sensor that has had part of the Bayer mask removed. As you can see, the "red" filters are actually yellow-orange and the "blue" filters are as much violet as they are blue.

Let's call the blue-violet filter centered on 460nm "Blue" or "B".
Let's call the slightly yellow green filter centered on 530-540nm "Green" or "G".
Let's call the yellow-orange filter centered on 590-600nm "red" or "R".

Then let's call the Red, Green, and Blue colors used by our emissive displays that are centered around 480nm, 525nm, and 640nm, respectively, R, G, and B.

The R, G, and B values for each pixel must all be interpolated from the raw values of the sensels covered with "R", "G", and "B" filters because "R" ≠ R, "G" ≠ G, and "B" ≠ B.

\$\endgroup\$
3
  • \$\begingroup\$ of course the RGB primary colors are different from the perceived colors of the bayer filters - that's because their function is different. color science is more advanced than "lets take a photo through a green glass and then project it with green light". this color difference has nothing to do with the need of demosaicing and with the reason why 2x2 RGGB grouping is inferior to more sophisticated algorithms. think of the foveon - no cross-pixels interpolation, we could say it is a pure subpixel grouping (like the OP wanted) and it works correctly. what's different, then? geometry. \$\endgroup\$
    – szulat
    Jun 12, 2022 at 11:05
  • \$\begingroup\$ @szulat But there are other disadvantages of Foveon sensors, which this question nor answer really address at all. The second layer gets less light than the first. the third layer gets even less light than the second. So each layer is a stop or more less sensitive than the layer above it. That's why they are so noisy and why they haven't been widely adopted. \$\endgroup\$
    – Michael C
    Jun 12, 2022 at 21:41
  • \$\begingroup\$ @szulat You can say, 'Of course the RGB primary colors are different..." all you want. But the simple truth is that the misconception that the cones in the human retina, the filters on a Bayer mask, and the three colors emitted by RGB displays are the same three shades of red, green, and blue is widely held by many. Most also believe that 'red' cones and filters detect ONLY red light, 'green' cones and filters detect ONLY green light, and 'blue' cones and filters detect ONLY blue light. These misconceptions lead to a LOT of misunderstandings about how camera sensors, demosaicing, etc. work. \$\endgroup\$
    – Michael C
    Jun 12, 2022 at 21:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.