Can all colors be described with RGB?

Question

If one could pick ideal sensitivities for the RGB filters of a camera sensor, would it be possible to capture any color as humans see it?

Answer 1

Have a look at this introduction to color perception and reproduction. It also contains a comparison of CIE, RGB and CMYK gamuts at the bottom, where CIE represents what the eye can do and RGB and CMYK what cameras, monitors and printers can do.

In your detailed question, you basically ask, if choosing different RGB filters would accurately model human color perception, which it doesn't:

The human eye is very adaptive, so that cameras for example have difficulties with situations with extreme contrast (where one would use HDR imaging) or low light situations where humans experience a loss of color vision. So it would increase the accuracy or be better model of the set of colors we are able to perceive, but the adapted RGB model would still have limitations.

In addition, being able to accurately model or measure what humans perceive does not solve the problem of creating the same stimulation of color vision in other humans.

That means using a different set of RGB filters will only "cure" the data acquisition of color, but not the reproduction. Your monitor and printer must also be able to reproduce that.

In color reproduction there are other issues present, like 8bit vs 16 and 32bit per channel in sensors, file formats and monitors, calibration of color in output devices, and non-linear perception and adaptivity of your eyes to an extreme range of color and brightness again, for example due to ambient light.

Another issue is that of texture, which can make it difficult to reproduce things like Gold and Silver surfaces correctly.

Please check Poynton's color FAQ.

Answer 2

The colors used in a Bayer filter are already centered as closely as possible to the three wavelengths of light to which human eyes are most sensitive. How sensitive each color is relative to the other two is determined by how the raw data from the sensor is processed. Changing the multipliers used for the red-filtered and blue-filtered pixels is normally how this is accomplished. The green channel is normally maintained at a multiplier of 1.0.

To really understand what is going on, we need to realize that neither the filters on a Bayer mask nor the color receptors in our eyes have hard cutoffs between red, green, and blue receptors. This is all covered in-depth in this answer to RAW files store 3 colors per pixel, or only one? We'll cover a few of the highlights here.

Here is a graph that demonstrates normalized response curves of the three types of color receptors in the human retina. The wavelengths at which the color receptors in the human retina are most sensitive closely models the wavelengths at which each filter in a Bayer mask is most transmissive. Notice how much overlap there is between the three - particularly between green and "red" (which is actually most sensitive to a lime-green color).

Raw files don't really store any colors per pixel. They only store a single brightness value per pixel.

It is true that with a Bayer mask over each pixel the light is filtered with either a Red, Green, or Blue filters over each pixel well. But there's no hard cutoff where only green light gets through to a green filtered pixel or only red light gets through to a red filtered pixel. There's a lot of overlap. A lot of red light and some blue light gets through the green filter. A lot of green light and even a bit of blue light makes it through the red filter, and some red and green light is recorded by the pixels that are filtered with blue.

Since a raw file is a set of single luminance values for each pixel on the sensor there is no actual color information to a raw file. Color is derived by comparing adjoining pixels that are filtered for one of three colors with a Bayer mask. But just like putting a red filter in front of the lens when shooting black and white film didn't result in a monochromatic red photo, the Bayer mask in front of monochromatic pixels doesn't create color either. What it does is change the tonal value (how bright or how dark the luminance value of a particular color is recorded) of various colors by differing amounts. When the tonal values (gray intensities) of adjoining pixels filtered for the three different colors used in the Bayer mask are compared then colors may be interpolated from that information. This is the process we refer to as demosaicing.

The same is true of the color receptors in the human eye.¹ "Green" receptors detect a lot of the same light that "red" receptors do and vice-versa. What we perceive as color is based on the relative differences between response in the red, green, and blue receptors in our retinas.

Too often we equate a particular wavelength of light as intrinsically having a particular "color." The truth is that the "color" a particular wavelength of light is perceived as is constructed by our eyes and brain. Animals with differently attuned color receptors differentiate between various wavelengths of light differently than humans do. Some animals only have monochromatic vision. Some have as few as two different types of color receptors. Some animals have more than the three different types of color receptors we humans posses.

"Color" as we perceive it is not an intrinsic physical property of light. In fact, what we call "visible light" is just the narrow band to which our eyes are chemically responsive from within the overall electromagnetic energy spectrum. There's no real intrinsic difference between visible light and non-visible radio waves other than the fact that our eyes are chemically responsive to the wavelengths of the electromagnetic spectrum that we call "light" and our eyes are not chemically responsive to the wavelengths we call "radio." There are "colors" that we perceive that cannot be produced by a single wavelength of light. Magenta, for instance, is the color that we perceive when we see light that is made up of a combination of wavelengths from both the near-infrared and near-ultraviolet light at opposite ends of the visible spectrum.

What is most different between color receptors in human retinas and pixels in a Bayer masked sensor are the shapes of the response curves, particularly in the areas of lower response for each type of color receptor as the wavelength of the light source moves further away from the wavelength to which that receptor is most sensitive.

_{¹Different people can vary slightly with regard to the exact wavelengths at which the receptors in their retinas are most sensitive.}

But no one sees with their eyes. We see with our brains.

Just as a camera must do a lot of processing to the data collected by a camera sensor to create a color image, the brain does a lot of processing to the signals it receives from the retinas. It adjusts the relative weight that each set of color receptors is given relative to the others. That's why we perceive an object to be the same color(s) under various types of lighting. If the lighting source is limited enough in its spectrum, though, our ability to do so begins to break down. For example, under very limited spectrum red light it is impossible for our eyes to tell the difference between a red shirt and a white one.

Our brains also adjust the relative brightness of various areas in a scene so that we can perceive details over a very wide range of brightness levels. The brain builds a mental model of our surroundings. It collects the "data" from the dark areas differently than it collects "data" from the bright areas. The brain controls the iris in our pupils and changes the iris' diameter as we scan a scene so that the brightness of the light striking our retinas from areas of varying brightness is normalized to a degree.

There are two very big considerations that complicate any type of camera, digital or analog, from reproducing the full range of the colors our eyes can perceive.

The first is that we still must view the results of a camera's capture of light using our eyes. That means that what the camera has captured based on the light it recorded must be reproduced to create reflected (print) or emitted light (electronic screen) that stimulates the receptors in our retinas the same way the light captured by the camera would have. At this point in time cameras can capture a much wider range of information than what our display methods are capable of simultaneously displaying. So even if the camera could capture the entire range of light that our eyes respond to, our display technology can't reproduce that entire range. Part of what we do in the raw conversion process (or darkroom) is decide how much of the total information captured by the camera we want to squeeze into the narrower capability of our display methods.

The other is that our eye/brain can't look at a two dimensional photograph and adapt to the light it is reflecting (or emitting) in the same way our eye/brain would have adapted to the actual scene recorded by the photograph. We lose the advantage of our stereo vision that helps our brain perceive depth and distance as it builds an internal "3D model." We don't adapt the same way to the different areas of the light and dark parts of the image. We don't adjust to the different sources of light and their various color temperatures and spectral distribution illuminating the photo in the same way we can to an actual scene. Our pupils don't generally expand or contract as we scan the different areas of a high contrast photo. (Perhaps if we are sitting in an IMAX theater showing a scene that is very dark on one side and very bright on the other they would.) The light under which we are viewing the image also influences the colors that we perceive the image to be. That is why there are designated standard viewing conditions for doing critical examination of photographs. Light that is polarized in a natural scene is not polarized in the same way when it is captured by a camera and then displayed as a photograph. Many of the things our brain uses to perceive color in an actual scene are absent in a photograph.

Answer 3

In photography we mix or hold back colors in varying proportions and with varying intensities. We can approximate most spectrum colors. By “spectrum” we are talking about those colors that are produced by refection of white light via a prism. Using available filters we can even produce magenta (red + blue) and numerous shades of purple, these are colors not seen in spectrum.

The faithful color image is dependent on getting a color match. There are two types of color match. One is a match method whereas a match is obtained when the two colors are seen via light of various wave lengths in the same proportions. Another match method is to adjust the component energies that are different, to obtain the required effect on the human visual receptor system causes us to see a match. The distention between the two systems is important because color photography is based on the latter. In other words, we can’t with dye or pigment duplicate the actual physical stimuli resulting when we are actually looking at an object via the camera. We can however, by manipulating light, simulate this action.

In photography we image and display that image using the three light primaries which are red, green and blue. However, these fail when we print on paper. To make a faithful print we must use the subtractive primaries which are the complement (opposite) of the three light primaries. These are blue’s complement which is yellow -- red complements which is cyan (blue + green) -- green’s complement which is magenta (red + blue). In photo science we have never been able to procure the exact shades to make filters and dye and pigment.

We can come close but the faithful image still eludes us.