1
\$\begingroup\$

From what I understand, camera sensor photosites have 'filters' on them so that each can receive only either red, blue or green light.

Example:
When there is an overexposure of only pure blue light, why does the overexposure come out as white? Shouldn't it be completely blue, like around (0, 0, 255) and not white (255, 255, 255)? What causes the red and green colour?

\$\endgroup\$
3
  • 2
    \$\begingroup\$ Filters aren't perfect; "only" red, blue, or green is, in fact, "mostly" red, blue, or green. \$\endgroup\$ Dec 10, 2021 at 14:23
  • \$\begingroup\$ Not even "mostly" for the "red" and "green" filters, which have a huge amount of overlap. They're not actually red, green, and blue, either. They're a slightly blue violet, a slightly yellowish green, and somewhere between yellow and orange. The specific color of the "red" filter is more varied from one camera manufacturer (or even camera model) to the next. The colors of Bayer filters are NOT the same colors as the actual "red", "green", and "blue" that emissive displays (monitors, TVs, etc.) use to create a similar response by the human eye/brain vision system. \$\endgroup\$
    – Michael C
    Dec 10, 2021 at 23:39
  • \$\begingroup\$ @PeteBecker you wrote only a few words, but it's an answer, not a comment. Don't shy away from posting short answers - they're just as valid. \$\endgroup\$
    – osullic
    Dec 16, 2021 at 23:34

5 Answers 5

2
\$\begingroup\$

From what I understand, camera sensor photosites have 'filters' on them so that each can receive only either red, blue or green light.

Your understanding, along with the same understanding held by a lot of other folks, is not entirely correct. Or, as Roger Cicala once remarked about a different misunderstanding in one of his lensrentals.com blogs, "Some things I wrote about years ago, I now realize are, um, well, less correct than I would have liked."

Here are the two most significant things that are flawed in that very common understanding:

  • The three color filters on a Bayer mask do not fully block other colors of light. Each allows some light from the full visible spectrum through. What each filter color does is allow more of one part of the visible spectrum through than other parts of the visible spectrum.

Remember when we used to put colored filters in front of our lenses while shooting B&W film? We did this to change the relative tonal values (gray scale brightness) in the monochrome image between differently colored objects in the scene. If we used a red filter to take a picture of a forest all of the green leaves on the trees didn't show up as pure black in the resulting image, they were just a darker shade of grey, compared to red objects, than they would have been had we not used the filter.

The same is true of the filters in a Bayer mask. All of the photons that get through - regardless of whether they are vibrating/oscillating at frequencies on the blue end of the visible spectrum, the red end, or anywhere in between - release energy when they strike the sensor. That energy is accumulated as an electrical charge. When it is read out the sensor measures the total charge collected by each photosite. Each photosite (a/k/a a pixel well or sensel) has a single energy value that is measured. This value includes the total energy from all of the photons that got through the filter in front of it. A photosite behind the "red" filter does not discriminate between the energy created by a "red" photon that passes through the filter from the energy created by a "blue" or "green" photon that passes through the filter. The "red" filter allows proportionally fewer "green" and "blue" photons through than "red" photons, but the energy from all of those that do pass through are added to a single charge.

Here's a graph of the sensitivity of a typical Bayer masked CMOS sensor. Notice that the response by photosites beneath each color filter includes overlap with the response of the other two filter colors. I've added vertical lines to show the actual wavelengths of colors emitted by most display screens. Most use only RGB, but there are a few that also have a fourth yellow channel. Any other color we perceive when we look at a color display is the result of our eye/brain system's combined response to light that is centered on those three (or in rare cases, four) colors.

enter image description here

Notice that the peak sensitivity of the "blue" filter is actually somewhere between blue and violet, the peak sensitivity of the "green" filter a slightly yellow shade of green, and the peak sensitivity of the "red" filter is much closer to yellow than red. So in a sense, our cameras are really YGV (Yellow-Green-Violet) as much as they are RGB. Our color reproduction systems (monitors, printers, web presses, etc.) are what are RGB, CMYK, or some other combination of colors.

This mimics, to a degree, the relative sensitivity and the overlap between the types of cones in the human retina.

enter image description here

Without this overlap between the sensitivities of the types of retinal cones, our human eye/brain vision systems could not create color at all! The color we perceive for a specific wavelength of light is not intrinsic in the nature of the light, it's created by our vision system. Other species may or may not perceive the same wavelength at all, just as we do not perceive infrared, radio, x-ray, etc. wavelengths of the fuller electromagnetic spectrum within which visible light is one narrow band.

For more about how both the human vision system and our cameras create "color" out of the portion of the electromagnetic radiation spectrum we call "light", please see: Why are Red, Green, and Blue the primary colors of light? (Hint: There are no primary colors implicit in the nature of light.)

An astute observation of the colors each type of our retinal cones are most sensitive to as well as the peak sensitivity of a silicon-based CMOS sensor behind the color filters on a Bayer mask leads us to the other major misconception about the "RGB" filters on a Bayer mask.

  • The colors of the filters on a Bayer mask are NOT the same three colors as the RGB pixels used by emissive displays such as computer monitors, televisions, and smartphone screens, all of the cute little checkerboard drawings with red, blue, and green squares posted on the internet notwithstanding.

This is a lie:

enter image description here

This is what a the colors of a Bayer mask actually look like under high magnification:

enter image description here
Above is an image of part of a sensor from a Sony α7S that has had part of the Bayer mask removed.

These are not the same colors that our emissive displays use for Red, Green, and Blue.

The peak sensitivities of the Bayer mask filters are at about 450nm, 540nm, and 590nm, respectively.

The peak sensitivities of our retinal cones are at about 420nm, 534nm, and 564nm, respectively. Our L-cones (L is for long wavelength, not Large) have peak sensitivity to lime-green light at 564nm, rather than to red light at around 640nm!

Emissive displays generally aim for emitting Red, Green, and Blue at around 480nm, 530nm, and 640nm, respectively.

When there is an overexposure of only pure blue light, why does the overexposure come out as white? Shouldn't it be completely blue, like around (0, 0, 255) and not white (255, 255, 255)? What causes the red and green colour?

Again, two main things:

  • Very few light sources in the real world are pure colors. Most have a fairly wide band of the visible spectrum in their emissions.

Incandescent bulbs, for instance, emit some of the entire visible spectrum, but they emit much more in the area around 2700K than they do on the bluer end of the spectrum around 10000K. The sun as seen from the Earth's surface after the atmosphere has filtered the light passing through it also emits the entire visible spectrum. This is not that surprising, since what we call the 'visible spectrum' is only a narrow range of the entire electromagnetic spectrum. After all, the range of wavelengths we designate as visible light are the wavelengths our human vision system evolved to perceive because they are the wavelengths most prominent in sunlight after it has been filtered by the Earth's atmosphere.

enter image description here

  • Even if there were a pure blue light source emitting at, say, 460nm some of that light would make it through all three of the color filters on the Bayer mask.

If the light is dim enough, or if the camera's exposure is set to not fully saturate any of the sensor's photosites, then the signal from the "blue" filtered photosites would be much stronger than the signal from the "green" and "red" filtered photosites. The signal from the "green" and "red" filtered photosites may be so weak as to be indistinguishable from the noise floor generated by the camera's electronics. When the camera or postprocessing application demosaics the raw sensor data to create color information the Blue channel will be dominant and the Green and Red channels will be very weak. We may get an RGB value something like (2,4,187).

If we increase the brightness of the light until the "blue" filtered photosites are right at the maximum saturation point, the "green" and "red" filtered photosites would still have the same proportionally less energy collected compared to the "blue" filtered photosites. Our RGB value will now be something like (3,5, 255).

But if we then continue to increase the brightness of the light the blue channel can not increase any more. It's already measuring at maximum signal. Any additional increase in brightness or exposure will continue to raise the Red and Green channel values, but the Blue channel can not be increased beyond 255. Even if we have 1,000 times more light than needed to give us a Blue channel value of 255, the Blue channel will still be 255. But with 1,000 times more light, the Red channel and the Green channel will also receive enough energy from the blue photons striking it to peg out at 255 and 255, respectively.

This is a bit simplified, as the raw values are usually 12-bit or 14-bit numbers. But the same logic holds. If the signal for a photosite is fully saturated and digitally encoded with a 14-bit value that maxes out at 16,383 and we then increase the amount of light allowed into the camera ten-fold for the next exposure, the value will still be 16,383 and no higher. Meanwhile the photosites that measured somewhere around 1,640/16,383 in the former exposure will increase their response by ten-fold and in the latter exposure will also measure 16,383 as well.

It's like water buckets. Once a bucket (the "blue" filtered photosites) is full we have no way of measuring how much additional rain falls and spills out of the full bucket onto the ground. But if it continues to rain then other buckets (the "green" and "red" filtered photosites) on the other side of the yard that are partially protected by a tree overhead (the "green" and "red" color filters) that aren't nearly as full when the "blue" bucket begins to overflow will continue to accumulate more water. If it rains long enough or hard enough they, too, will eventually become full.

Once all three sets of photosites filtered by the different colored filters are fully saturated, we have no way of measuring which ones received more light than the others, because all of them received enough light to fill completely up.

When all three channels are fully saturated we get pure white. It matters not that there is much more blue than red or green light striking the sensor. As long as there is at least just enough to fully saturate each color channel we will see that area rendered as white.

For other related question here at Photography SE that say much more about this, please see:

Why do pure colors (red/green/blue) become a mixture of colors when converting raw?
Why are Red, Green, and Blue the primary colors of light?
Why is it that when the green channel clips, it turns into blue?
Why does BRIGHT red in direct sun come out orange?
RAW files store 3 colors per pixel, or only one?

\$\endgroup\$
1
\$\begingroup\$

The white colour is created during the demosaicing stage, where an algorithm interpolates between adjacent red, green, and blue pixels according to the bayer filter matrix to create a viewable full-colour image.

If all pixels in an area are saturated, irrespective of the colour of the filter in front of them, it's a pretty good guess that the light at the pixel's location was actually sufficient to saturate all three channels, so it will be assigned (255,255,255), put simply.

If you happend to expose with actual monocolour light (such as pointing a laser at the camera), the spillage due to imperfect filters (see the graphic in this answer) might still be enough to saturate the other pixels as well.

\$\endgroup\$
1
\$\begingroup\$

This is one of those areas where common terminology confuses the correct terminology/function.

Pixel stands for Pix Element (picture element) and they are not biased towards a single color... E.g. each pixel on your screen is made up from a red+blue+green LED (or similar)... This is a super macro of the screen on my cell phone, and each RGB LED group is/creates a single image pixel.

enter image description here

The photodiodes on the camera sensor are color filter to be red, or green, or blue, centric. And they are co-located to where a pixel will be; but they are not stand alone pixels in the final *color image. Their individual pixel colors are determined by demosaicing; mathematically calculating the relative color/brightness of surrounding photodiodes in order to determine an output RGB value per point/pixel.

I have seen it stated that a single pixel's color is determined by an RGBG quad of the bayer filtered photodiodes on the sensor... that's oversimplification at best, but it does highlight the difference.

enter image description here


*a photodiode on an unfiltered monochrome (B&W) sensor could also be the resultant pixel in the output image.

\$\endgroup\$
1
  • 1
    \$\begingroup\$ The colors of the "red", "green", and "blue" filters of a Bayer mask are nowhere near the same colors as the Red, Green, and Blue pixels used by emissive displays, all of the cute little RGB checkerboards posted on the internet notwithstanding. This is particularly true for "red", which is actually a yellowish orange color at around 590nm for most digital sensors as opposed to 640nm for most emissive displays. \$\endgroup\$
    – Michael C
    Dec 10, 2021 at 23:54
1
\$\begingroup\$

As you know, the heart of the digital camera is its image sensor. This chip has a surface covered with light sensitive sites called “pixels” (picture elements). These are covered with transparent colored filters (red, green, and blue). The job of these filters is pass light that only matches up with the color of the filter and reject all light that is a miss-match. Thus, a red filter passes red and rejects green and blue. The blue filter passes only blue, and the green filter passes green. This arrangement fractures the light from the subject into three images that comprise what is termed the light primary colors.

In a perfect world, the primary-colored filters would pass only their color however in reality they leak and pass a small quantity of unwelcome colored light.

In an overexposed condition, the resulting image will be rendered too light (washed out, void of detail). If the overexposure is moderate, the washed-out image will likely also be off-color. If the overexposure is acute, likely portions or perhaps the entire image will be washed-out white.

Additionally, you might look up the term “clipping” which happens when the exposing light intensity is above the maximum. The result is a uniform area of maximum brightness resulting in a total loss of image detail

\$\endgroup\$
1
  • 1
    \$\begingroup\$ If "primary colored" filters had no overlap they could not reproduce anything that resembles what the human eye/brain vision system perceives. Our entire perception of color is based on the overlap of the sensitivities of the different types of retinal cones. The narrow portion of the EMR spectrum that we call visible light has no more color implicit in it than other areas of the electromagnetic radiation spectrum such as X-rays or radio waves. Color is created by the vision system that perceives certain wavelengths of EMR. \$\endgroup\$
    – Michael C
    Dec 11, 2021 at 2:35
0
\$\begingroup\$

The process that converts the Bayer filter pattern on the sensor into a full color image is called Demosaicing. This can be a complex process depending on what image attributes are being optimized. Most algorithms are optimized for the resolvable resolution on grey-scale resolution charts, because that determines both the measured resolution and the objective sharpness of the final images. This may come at the cost of color accuracy.

This means that the demosaicing algorithm may choose a less colorful result for overexposed areas, i.e. white. If you have a raw converter that lets you choose different demosaicing options, you should experiment to find out which option gives you the most accurate color for overexposed areas.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.