Why do we use RGB instead of wavelengths to represent colours?

Question

As we know, the colour of a particular beam of light depends on its frequency (or wavelength). Also, isn't that the information which is first captured by digital cameras? Then, why do we use formats like RGB (or CMYK, HSV etc.) to represent colours digitally?

Answer 1

I think there are some misconceptions in prior answers, so here's what I think is true. Reference: Noboru Ohta and Alan R. Robertson, Colorimetry: Fundamentals and Applications (2005).

A light source need not have a single frequency. Reflected light, which is most of what we see in the world, need not have a single frequency. Instead it has an energy spectrum, i.e., its energy content as a function of frequency. The spectrum can be measured by instruments called spectrophotometers.

As was discovered in the nineteenth century, humans see many different spectra as having the same color. Experiments are done in which light of two different spectra is generated by means of lamps and filters and people are asked, are these the same color? With such experiments, one verifies that people don't see the spectrum, but only its integrals with certain weighting functions.

Digital cameras capture the response to light of sets of photodiodes covered with different filters, and not the fuller spectrum that you'd see with a spectrophotometer. Three or four different types of filters are used. The result is stored in a raw file output by the camera, although many people suspect that raw files are "cooked" to a greater or lesser extent by camera manufacturers (camera sensors are of course highly proprietary). The physiological responses can be approximated by applying a matrix transformation to the raw data.

For convenience, rather than using approximations to physiological responses, other types of triples of numbers are employed to name colors, e.g., Lab, described in https://en.wikipedia.org/wiki/Lab_color_space (but note warning on page). One must distinguish triples which can express the full range of estimated physiological responses from others, like RGB, which can't. The latter are used because they do express the colors which computer screens can display. They are the result of conversions from triples like Lab, or from raw data. CMYK is for printers.

Answer 2

The goal of the imaging engineer has always been to capture with the camera a faithful image of the outside world and present that image in such a way that the observer sees true to life picture. This goal has never been achieved. In fact the best images made today are frail. If this goal were to be achieved, you would need sunglasses to comfortably view an image of a sunlit vista.

You are asking why cameras don’t capture the entire span of radiant energy that created the human visual response. Why does the modern camera only capture three narrow segments that we call the primary light colors which are red, green and blue?

The answer falls in the category of how we see, namely the human visual response. Over the years there have been many theories proposed regarding how humans see color. So far all have failed to give a satisfactory explanation of every aspect of how we see colors. The wave lengths span that our eyes are sensitive to covers the range of 400 to 700 millimicrons. It is no accident that earth’s atmosphere is transparent to this range.

When we stare at a light source, we cannot distinguish any one particular wave length unless it is presented alone. When we look at a white light source, we are unable to isolate and identify any specific color. Our eye/brain combination interprets the color of the light without analyzing what makes up the mix of frequencies. Capitalizing on this, scientists have proven by experimentation that by mixing only three colors in varying proportions, almost all colors can be produced. In other words, presenting to the human eye, in varying intensities, a mix of red, green and blue, most spectrum colors can be reproduced, not exactly but a close approximation. This was the work of Thomas Young (British 1773 – 1829) titled the Young Theory of Color Vision.

Building on Young’s theory, James Clerk Maxwell (British 1831 – 1879), showed the world the first color picture photography produced. In 1855 he used three projectors and superimposed the three images projected on a single screen. Each projector was fitted with a colored filter. The three images were each one of the three light primary colors, namely, red, green, and blue. The film images projected were made by taking three separate pictures on three pieces of black and white film, each exposed thru one filter of the three light premiers.

Since that day in 1855, innumerable methods to make and display color pictures have been explored. Early color motion pictures projected feeble color images using just two colors. Edwin Land (American 1909 – 1991) founder of Polaroid Corp. experimented making color pictures using only two primary colors. This has remained a laboratory curiosity. So far, the most faithful color images are made using the three color primaries. However, one man, Gabbriel Lippmann (French 1845 – 1921) made beautiful color images that captured the entire visual light spectrum. He devised a method that employed black and white film with a mirror backing. The exposing light penetrated the film, hit the mirror and was reflected back into the film. Thus the exposure was made via two transits of the exposing light. The image comprised of silver arranged with a spacing equal to the wave length of the exposing light. When viewed, the film only allowed light to pass that matched the wave lengths of the exposing light. One could behold a full color picture that contained no dye of pigment. Unique and beautiful, the Lippmann process remains impractical. Our film and digital cameras fall back to the method used by Maxwell. Perhaps, if you study human vision and color theory, maybe you will be the one that advances our science and obtain the first truly faithful image.

Answer 3

You said,

this is the information that is captured at first by digital cameras.

That is not correct. By themselves, sensors on most digital cameras respond to a broad band of frequencies of light, beyond what humans can see into the infrared and ultraviolet spectrum. Because sensors capture such a broad spectrum of light, they are terrible discriminators of light wavelengths. That is, roughly speaking, digital sensors see in black and white.

For most camera sensors¹, in order to capture colors, colored filters are placed in front of the sensor, called a color filter array (CFA). The CFA turns each sensor pixel (sometimes called a sensel) into a primarily red, green, or blue light sensor. If you were to view the raw sensor data as a black and white image, it would appear dithered, somewhat like a half-toned black-and-white newsprint image. Zooming in at high magnification, the individual pixels of the image would have a checkerboard-like appearance.

Interpreting the individual squares of the raw image data as red, green, or blue as appropriate, you will see a color dithered version of the image, similar to a color half-toned newsprint article.

^{Bayer color filter array, by user Cburnett, Wikimedia Commons. CC BY-SA 3.0}

Through a process called demosaicing either when saving the image data in camera, or in post-processing on a computer, the array of color data is computationally combined to create a full-resolution RGB color image. In the demosaicing process, the RGB value of each pixel is computed by an algorithm that considers not only the pixel's value, but the data in nearby pixels surrounding it as well.

• See also: RAW files store 3 colors per pixel, or only one?

Then, why do we use the RGB format to represent colours digitally?

We use a trichromic color model because that's how humans perceive colors. From Wikipedia'a Trichromacy article,

The trichromatic color theory began in the 18th century, when Thomas Young proposed that color vision was a result of three different photoreceptor cells. Hermann von Helmholtz later expanded on Young's ideas using color-matching experiments which showed that people with normal vision needed three wavelengths to create the normal range of colors.

Thus, we build cameras that capture what we can see, in a somewhat similar fashion to how we see. For instance, for typical photography that aims to capture and reproduce what we see, it makes little sense to also capture infrared and ultraviolet wavelengths.

• See also: Can all colors be described with RGB?

---

1. Not all sensors use a CFA. The Foveon X3 sensor, used by Sigma DSLRs and mirrorless cameras, relies on the fact that different wavelengths of light penetrate silicon to different depths. Each pixel on the X3 sensor is a stack of red-, green-, and blue-detecting photodiodes. Because each pixel is truly an RGB sensor, no demosaicing is required for Foveon sensors.

   The Leica M Monochrom is an expensive black-and-white only camera that does not have a CFA on the sensor. Because there is no filtering of incoming light, the camera is more sensitive to light (according to Leica, 100%, or 1 stop, more sensitive).

Answer 4

The reason cameras and displays work in RGB is because our retinas work that way.

Since our eyes encode colors with those components (RGB), it is a very convenient system (although certainly not the only one) to encode not only pure-wavelengths (which form a more or less deterministic combination of retinal response for each chromatic component), but also mixed colors.

The rationale would be "if any color combination can only be delivered to the brain as a combination of three components, I can cheat the visual system by presenting only a given combination of those isolated, pure components (via RGB display) and let the visual system decode them as if they were the real thing.

It is interesting to note that, since we are trichromats, most color systems are three-dimensional in nature (Lab, HSV, YCbCr, YUV, etc.), not because of intrinsic physical properties of color, but instead because of the very way our visual system works.

Answer 5

An attempt to answer simply:

• We cannot practically capture enough information to store a complete breakdown, frequency by frequency, of all the different wavelengths of light present, even just within the visible spectrum. With RGB we can describe the colour of a pixel using just three numbers. If we were to capture the entire frequency spectrum of light, every single pixel would require not 3 numbers, but a graph of data. The data transmission and storage would be immense.

• It's not necessary for our eyes. Our eyes don't just see three single wavelengths, but instead each of our "red", "green" and "blue" receptors capture partially-overlapping ranges of light:

The overlap allows our brain to interpret the relative strengths of the signals as varying colours in between the primaries, so our vision system is already pretty good at approximating an actual wavelength given only the relative signal strength of the three primaries. An RGB colour model reproduces adequately this same level of information.

Answer 6

There are two interacting reasons.

Reason (1) is that the eye (usually) receives multiple wavelengths of light from any given point [so to speak]. White light, for instance, is actually [as a rule] a mixture of many diverse wavelengths; there is no "white" wavelength. Similarly, magenta (often called "pink" nowadays (via "hot pink") ) is a mixture of red and blue, but without green (which would make it appear white). Similarly again, something that appears green might have some lime and some cyan components.

Reason (2), then, is that RGB is how the human eye works — it has red, green and blue sensors.

Thus, combining (1) and (2): to get the human brain to interpret the light signals the same way as it would interpret the original signals, they have to be encoded in its terms.

For instance, if (conversely) the original were (what a person would perceive as) white light, but it were encoded using, say, violet and red sensors — just the two — the reproduction would appear to the human eye as magenta. Similarly, but more subtly or finely… white light that was a mixture of a full range of colours… if this were encoded using, say, violet, yellow and red sensors… this reproduction would appear to the human eye as not a pure white — as (offhand) a yellow-ish off-white. Conversely, it would appear as a pure white to an imaginary alien [and indeed possibly to some real animal] with the same sensors (viz. violet, yellow and red) in its eye.

By the same token… if the original were white — that is, a mixture of a full range of colours — then a human eye perceiving this would encode this in terms of only red, green and blue… and a reproduction using only red, green and blue (in the same proportions) would appear to human perception as a pure white — the point being that information is lost in both cases, but the end result appears perfect, because the losses correspond. Unfortunately, they will correspond exactly only if the sensors [RGB] in the camera have sensitivity curves exactly the same as the sensors [RGB] in the human eye [noting that each sensor is activated by a range of colours] — if, for instance, a lime colour activated each of the red, green and blue sensors by exactly the same amount, across the two cases.

Answer 7

The short answer: Because wavelength is a single value, and the entire range of colors we can perceive is not representable by a single value, any more than the dimensions of a rectangular solid can be represented by a single measurement.

To continue the analogy - you could quote the solid's volume, but there are many different solids with the same volume.

RGB, CMY, HLS, etc., all use three "dimensions" because that's now many you need to adequately describe colors as seen by humans.

Wavelength equates to Hue in the HLS system, but it can't tell you lightness or saturation.

Re "Also, isn't that ([wavelength]) the information which is first captured by digital cameras?", no, it isn't.

As others have noted digicams capture relative intensities of red, green, and blue. (And some have used at least one additional color to give better discrimination in the critical red-to-green region.) Directly measuring the frequency of incoming light would be far more difficult. We just don't have cheap sensors that can do that, certainly not ones that we can make in a grid of several million of them. And we'd still need a way for the camera to measure lightness and saturation.

Answer 8

tl;dr: It is way much easier to detect light on three broad parts of the spectra than analyse the frequency accurately. Also, the simpler detector means it can be smaller. And third reason: the RGB colourspace is mimicking the principles of opperation of human eye.

---

As Max Planck proved every hot body emitts radiation with various frequencies. He sugested and proved that the energy is radiated in bursts, called photons, not contiuously as was supposed before. And from that day, physics was never the same. The only exception is ideal LASER/MASER that emitts radiation of only one frequency and discharges (neon bars,...) emitt radiation with several isolated frequencies.

The distribution of intensities over the frequencies is called spectrum. Similarilly, the detectors also have their spectra, in that case it is distribution of the detector's response to a radiation of normalized intensity.

As it was already noted, the white light is white because our eyes are evolution-callibrated to see the sunlight, ranging from far infrared to ultraviolet, as white. Leafs, for example, are green because they absorb all the frequencies except for the part, that we see as green.

Of course, there are detectors that can gather the spectra and extract the information. They are used in optical emission spectroscopy and x-ray diffraction and fluorescence techniques, where the chemical composition or microstructure is evaluated from the spectra. For a photography it is overkill; except for the astrophotography, where we want to evaluate the "chemical" composition but the images are "translated" to fake colours. These detectors are accurate and huge or small but inacurrate and you need much more computation power to analyse it.

Human eye, or any other eye, is not that case. We do not see the chemical composition, or bonding states, of the object. In the eye there are four different "detectors":

• colourless: These are the most sensitive and they work for all visible frequencies. Without them you would not see anything in night.
• reds: These are most sensitive in low frequency region. That's why hot things are glowing red first.
• greens: These are most sensitive in higher frequency regions. That's why the hot things turn from red to yellow when heated further.
• blues: These sre most sensitive in high frequency region. That's why the heated things glow white when heated much more. If you could heat them more and more they will start glowing light blue.

If we look at rainbow, or CD or DVD, we will see colours turning from red to violet. The light beams for a given part of the rainbow are of one perticullar frequency mostly. The infrared beams are invisble to our eyes and they do not excite any cell in the retina. Increasing the frequency, the beams starts to excite the red "cells" only and the colour ic seen as red. Increasing the frequency the beams excite "red cells mostly" and the little bit the "greens" and the colour is seen as orange. Yellow beams excite the "greens" a bit more...

The sensors in cameras, CCD or CMOS, are excited by light beams of any frequency, to take a picture our eyes will see as colour we are just mimicking the human eye - we use, for example, Bayes filter. It consist of three colour filters with transmission spectra intentionally simillar to the cell types of our retina.

The light reflected from a yellow paper illuminated by the Sun exites the "reds" fully (100%), the "greens" fully (100%) as well and slightly the "blues" (5%), so you see it yellow. If you take a picture of it, simillar, say the same, excitation is gathered by the camera. When looking at the image on the screen, the screen sends 100 red photons, 100 green photons and 5 blue photons over a really short period of time towards you. The excitation levels of your retina will be simmilar to the excitation caused by direct observation and you will se a photograph of yellow paper.

There is another problem to be solved if we want to reproduce the colours. Using RGB colourspace we need only three types of light sources per pixel. We can have three colour filters (LCDs work like this), we can have three types of LEDs (LED and OLED panels use that), we can have three types of luminophors (CRT used this). If you want to reproduce the colour fully, you would need infinite ammount of filters/sources per pixel. If you want to use simlify the information colour-to-frequency it won't help either.

You can also try to reproduce the colour by its temperaure. I suppose you will be able to reproduce only red-orange-yellow-white colours and you would have to heat each pixel to temperatures around 3000 K.

And in all that theoretical cases your eyes will still translate the actually true colour to its RGB signals and pass it to your brain.

Another problem to solve is how to store the data? The conventional 18MPx RGB image consists of three matrices 5184x3456 cells, each point with 8-bit size. That means 51 MiB of uncompressed file per image. If we want to store the full spectra for every pixel, say in 8-bit resolution, it will be 5184x3456x256 übermatrix resulting in 4 GiB uncompressed file. That means storing intensities of 256 different frequencies in range of 430–770 THz, that means resolution of 1,3 THz interval per channel.

Totally not worth the effort if I may say...