6

The Bayer filter consists of red, green and blue filters placed before a sensor array to enable deduction of the color of light incident on the sensor. A downside of this method is that roughly 2/3 of the light is not detected.(exact amount depending on the light and configuration of the filter)

An alternative Bayer filter configuration might have only two colored filters(Red, Blue) and a third unfiltered region(Neutral). The green value would be inferred from Green = Neutral - Red -Blue.

I suspect this alternative design would have the following features:

Pros:

  1. This filter configuration would reject less light -> better overall sensitivity
  2. One less material used in manufacturing process

Cons:

  1. Greater color noise
  2. Edge artifacts (cannot correctly determine the color if full RBN pixel not uniformly illuminated)

So, my question is, what is the main reason we don't see products containing the alternative color filter described above?

  • Are you sure you can determine G from N-R-B??? dont forget the bare sensor is sensitive to a wide wavelength spectrum, not just R/G/B... – Digital Lightcraft Aug 5 '15 at 10:46
  • @DigitalLightcraft you could indeed calculate the green value, the sensor will already have IR and UV blocking filters in front of it. This is in essence how the Foveon sensor works, the top layer is sensitive to red, green and blue. – Matt Grum Aug 5 '15 at 12:37
  • yes the invisible spectrum is blocked, but surely all of the other visible wavelengths inbetween reg, green, blue would skew the results, as the estimation of green would be based purely on the luminance of ALL received light...?? im interested to know the answer to this one ! :-) – Digital Lightcraft Aug 5 '15 at 12:39
  • This may be of interest... "Panchromatic cells": en.wikipedia.org/wiki/Bayer_filter#.22Panchromatic.22_cells – Digital Lightcraft Aug 5 '15 at 15:46
  • @DigitalLightcraft Thank you. It is interesting two see that many variation other than RGB have made it to market. However, other than CYM used by Kodak quite some time ago, all variations have made use of four colors. – Brian Kubera Aug 6 '15 at 12:27
4

Due to overlap between the responses of the Bayer filter dyes it's closer to 50% light loss (one stop) than two thirds.

Your scheme would work but at the expense of colour accuracy and colour resolution (the demosaicing process would be less constrained and more error prone). Combined with the fact the sensitivity gain is rather modest (up to 25% depending on the layout), this explains the prevalence of RGB Bayer sensors.

A better approach would probably be to widen the response of all dyes so you get a CMY sensor. However the same drawbacks apply and you still don't gain much in terms of sensitivity.

2

There are various alternative colour systems that have been tried in the past but ultimately it comes back to Bayer being 'good enough' and other technologies not panning out. The 'dual pixel' design from the early Fuji DSLRs springs immediately to mind as being a great idea which ultimately didn't deliver enough benefit (although the dynamic range was vastly better than the other DSLRs on sale at the time).

You have a few technical challenges to overcome - you need pixels of differing sensitivities to be collocated (because the N would have far more light hitting it than RB) which makes engineering more difficult, not massively more, but enough to add cost so there needs to be a benefit. You also have to do more processing.

You have some physics in that the light that hits the Neutral will be a broad spectrum which will include UV and IR. Filtering only reduces those wavelengths it does not remove them entirely.

You have barriers to adoption as you'd need applications that work with raw files to be able to handle your different raw system. People will implement it, but it would have to really prove it's worth first. And since you're going to be viewing/editing on something that is RGB anyway, you've added work for something that is essentially invisible.

But ultimately beyond everything there has to be a payoff and you can bet that in labs at Nikon, Canon, Sony, Sigma, Fuji, Global Foundries, TSMC and the like they've all thought long and hard and done plenty of experiments which come to the conclusion that the benefits (if there are any) of a system like the one proposed wouldn't justify the costs.

  • That is a good point about the inconvenience of the N channel always having much higher signal. – Brian Kubera Aug 6 '15 at 10:06
  • I cannot find any mention of this 'dual pixel' in Fuji DSLRs. Is it the same as one of EXR or X-Trans mentioned on the wikipedia page: en.wikipedia.org/wiki/Bayer_filter? – Brian Kubera Aug 6 '15 at 10:08
  • 2
    @BrianKubera He's talking about the Fuji Super CCD SR: dpreview.com/articles/6851251325/fujisuperccdsr – Matt Grum Aug 6 '15 at 12:21
  • @MattGrum - that's the one... I knew I should have referenced it at the time! – James Snell Aug 6 '15 at 12:25
  • @BrianKubera - well that will teach you for using Wikipedia. :) – James Snell Aug 6 '15 at 12:27
1

The type of mosaic which tyou are talking about is RGBW. Telling about why it is not used is complex and I may misinform you. I may point at least one thing which in my opinion is most important.

So, my question is, what is the main reason we don't see products containing the alternative color filter described above?

Every channel of output image is constructed from all three channels of sensor output - they are mixed using matrix conversion. The farther the spectral response of a sensor is from LMS response the bigger is the error which that conversion introduces. White is kind of worst scenario of this. Suppose that you succeeded in making W channel which equals R+G+B. Then, to get each component from that channel using adjacent values you will need to compute R=W-G-B and so on. This will introduce very big errors and will probably cancel all the benefit from having W channel at all. Same applies to CMYK sensors: they require very strong conversion coeffificents which introduce the colour errors.

The CMYK and RGBW cameras will produce better mono-converted image for the same amount of light but RGB sensors will produce better RGB image even though their quantum efficiency is like half of CMYK sensors - it just happens so because of noise sources.

My guess is that optimum sensor mosaic (i,e, producing smallest noise for a given amount of light) would be 1:1:1 RGB instead of 1:2:1 RGB and the responses would be close to LMS response but I cannot backup this statement ATM.

A downside of this method is that roughly 2/3 of the light is not detected.(exact amount depending on the light and configuration of the filter)


Due to overlap between the responses of the Bayer filter dyes it's closer to 50% light loss (one stop) than two thirds.

Both statements are a far cry from real world. Canon 5D Mark II looses from 2/3 to 4/5 of light at the peaks of sensitivities and, when compared to ideal mono (unmasked) sensors, it looses a whoopping 9/10 of all light (i.e. the mono image from Bayer camera will have approximately same noise as the image from mono camera made from 1/10 of the light).

Check out the scale on the left of the graph: Canon 5D Mark II QE

More modern cameras may be better at it but I do not have the data to compare. There are causes for light loss othen than mosaic pattern.

  • 1
    FWIW, Fujifilm actually uses 2:5:2 RGB. – mattdm Jun 23 '16 at 14:42
  • @mattdm: I did not know about it and it is correct. However, I limited the scope to Bayer sensors, and X-Trans isn't one. Also, 2:5:2 is even worse than 1:2:1. – Euri Pinhollow Jun 23 '16 at 14:49
  • Yeah, I actually brought it up because that's, as you say, "worse", but it's an interesting comparison because AFAIK it's a Sony sensor underneath, similar to the highly-regarded one used in the 16mpix Nikon/Pentax/Sony cameras — but with the different pattern. – mattdm Jun 23 '16 at 14:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.