Why is a Bayer filter with three colors(RGB) preferred over one with two colors and unfiltered (RBNeutral)?

Question

The Bayer filter consists of red, green and blue filters placed before a sensor array to enable deduction of the color of light incident on the sensor. A downside of this method is that roughly 2/3 of the light is not detected.(exact amount depending on the light and configuration of the filter)

An alternative Bayer filter configuration might have only two colored filters(Red, Blue) and a third unfiltered region(Neutral). The green value would be inferred from Green = Neutral - Red -Blue.

I suspect this alternative design would have the following features:

Pros:

1. This filter configuration would reject less light -> better overall sensitivity
2. One less material used in manufacturing process

Cons:

1. Greater color noise
2. Edge artifacts (cannot correctly determine the color if full RBN pixel not uniformly illuminated)

So, my question is, what is the main reason we don't see products containing the alternative color filter described above?

Answer 1

Due to overlap between the responses of the Bayer filter dyes it's closer to 50% light loss (one stop) than two thirds.

Your scheme would work but at the expense of colour accuracy and colour resolution (the demosaicing process would be less constrained and more error prone). Combined with the fact the sensitivity gain is rather modest (up to 25% depending on the layout), this explains the prevalence of RGB Bayer sensors.

A better approach would probably be to widen the response of all dyes so you get a CMY sensor. However the same drawbacks apply and you still don't gain much in terms of sensitivity.

Answer 2

There are various alternative colour systems that have been tried in the past but ultimately it comes back to Bayer being 'good enough' and other technologies not panning out. The 'dual pixel' design from the early Fuji DSLRs springs immediately to mind as being a great idea which ultimately didn't deliver enough benefit (although the dynamic range was vastly better than the other DSLRs on sale at the time).

You have a few technical challenges to overcome - you need pixels of differing sensitivities to be collocated (because the N would have far more light hitting it than RB) which makes engineering more difficult, not massively more, but enough to add cost so there needs to be a benefit. You also have to do more processing.

You have some physics in that the light that hits the Neutral will be a broad spectrum which will include UV and IR. Filtering only reduces those wavelengths it does not remove them entirely.

You have barriers to adoption as you'd need applications that work with raw files to be able to handle your different raw system. People will implement it, but it would have to really prove it's worth first. And since you're going to be viewing/editing on something that is RGB anyway, you've added work for something that is essentially invisible.

But ultimately beyond everything there has to be a payoff and you can bet that in labs at Nikon, Canon, Sony, Sigma, Fuji, Global Foundries, TSMC and the like they've all thought long and hard and done plenty of experiments which come to the conclusion that the benefits (if there are any) of a system like the one proposed wouldn't justify the costs.

Answer 3

The type of mosaic which tyou are talking about is RGBW. Telling about why it is not used is complex and I may misinform you. I may point at least one thing which in my opinion is most important.

So, my question is, what is the main reason we don't see products containing the alternative color filter described above?

Every channel of output image is constructed from all three channels of sensor output - they are mixed using matrix conversion. The farther the spectral response of a sensor is from LMS response the bigger is the error which that conversion introduces. White is kind of worst scenario of this. Suppose that you succeeded in making W channel which equals R+G+B. Then, to get each component from that channel using adjacent values you will need to compute R=W-G-B and so on. This will introduce very big errors and will probably cancel all the benefit from having W channel at all. Same applies to CMYK sensors: they require very strong conversion coeffificents which introduce the colour errors.

The CMYK and RGBW cameras will produce better mono-converted image for the same amount of light but RGB sensors will produce better RGB image even though their quantum efficiency is like half of CMYK sensors - it just happens so because of noise sources.

My guess is that optimum sensor mosaic (i,e, producing smallest noise for a given amount of light) would be 1:1:1 RGB instead of 1:2:1 RGB and the responses would be close to LMS response but I cannot backup this statement ATM.

A downside of this method is that roughly 2/3 of the light is not detected.(exact amount depending on the light and configuration of the filter)

---

Due to overlap between the responses of the Bayer filter dyes it's closer to 50% light loss (one stop) than two thirds.

Both statements are a far cry from real world. Canon 5D Mark II looses from 2/3 to 4/5 of light at the peaks of sensitivities and, when compared to ideal mono (unmasked) sensors, it looses a whoopping 9/10 of all light (i.e. the mono image from Bayer camera will have approximately same noise as the image from mono camera made from 1/10 of the light).

Check out the scale on the left of the graph:

More modern cameras may be better at it but I do not have the data to compare. There are causes for light loss othen than mosaic pattern.