What *exactly* is white balance?

Question

When setting a white balance configuration, we adjust the temperature and green-magenta shift to a wavelength-intensity distribution of light that correlates most closely with the actual distribution of light emitted from the light source illuminating our scene.

What I don't understand is by what means our camera uses this information to change the way it records the RGB colour data. Assuming that this ideal distribution illuminated our sensor evenly, we would expect white/grey objects to exihibit a particular Red/Green/Blue intensity over the whole of the sensor, and I assume that this pattern would be mapped to equal RGB values in the process of white balance correction. I'm just guessing here though.

• How exactly is the raw data of the RGB photosites on the sensor converted into pixel RGB values using the white balance modelled distribution of light? If the red, blue and green channels of a little patch on the sensor each collect the same number of photons, then why isn't this represented by a pixel with equal RGB values? Why do we 'correct' this by distorting the values according to the light source?

• If the white balance is chosen correctly, won't the light source appear to be pure white? This is at odds with the fact that light sources clearly do not appear pure white in general.

• If I want an image not to represent the colours of objects accurately, but to include the colour-casting that my vision is subject to, then what white-balance configuration will achieve this? Is there a sort of global 'neutral' setting which doesn't alter colour casting? For example, white objects do not appear white in a dark room with the red safety light on. I don't want them to appear white in my photos either.

The two parameters of white balance configuration (temperature and magenta-green shift) alter what the camera thinks is the wavelength-amplitude characteristic of the scene's lighting. How does it use this information (the formulae; what it's aiming for in principle) to alter the luminance of the RGB channels?

Answer 1

Our eyes and brain do things on a daily basis that make LSD's effects seem relatively tame.

One of the things our brains do is a color balancing activity of their own. No one knows why for certain, but its theorized we do it so that it would be easier to track prey as they dodge in and out of shadows (prey reflect the blue sky while in the shadow, so they become bluer). Regardless of why, our brains do it.

This is remarkably obvious if you are a diver. Reds get cut out by the water column rather rapidly. In fact, at 30m, red is a camouflage color. However, we don't perceive this when we dive. We think we're seeing perfect colors. Hold up a white card in 30m of water, and it looks "white" to you.

Now take a picture of that card. The camera sees raw photon counts. It's going to call it like it is. Far fewer red photons will hit the camera, so it will record less red in the picture. No problem!

The need for color balancing comes when you try to view those photos when you aren't 30m under water. Your brain will do its color balancing thing, like it did underwater, but now it does it with respect to the perceived lighting in the room. If you're in a reasonably lit room, your brain will tune itself to perceive a white object (like the unprinted white rim around a photo) as "white." Now the picture looks horribly blue. This is an accurate model of how many red photons hit your eye when you were at depth, but now your brain is no longer color correcting for it.

The solution is white-balancing. You pick a "white" object in the picture (which is actually a bunch of blueish pixels), and declare "I want people to think this is white." The software does some color mapping to effectively do what your brain was doing before. Once printed, this region of pixels takes on the color of the light in the room (usually rather yellowish), but now your brain properly does its corrections, and you perceive white!

That's almost the end of the story. This works remarkably well for printing. On a screen, the brain has a little more trouble making good guesses at color corrections because the brightness of the screen does not scale with the light in the room around you. If you're editing a photo professionally, its common to pick a room with very constant lighting, and "color balance" the monitor so that things it shows as "white" show up as "white" when printed!

Answer 2

Why do we 'correct' this by distorting the values according to the light source?]

Because your visual system responds to relative changes in the intensities of different colors, whereas the camera sensor records absolute intensities. If you stand under a sodium streetlight for a while, you get used to that light being "white" even though it's quite a different color than sunlight is. And sunlight itself changes color depending on time of day, atmospheric conditions, etc., but most of the time we think of sunlight as "white" too.

If the white balance is chosen correctly, won't the light source appear to be pure white?

I don't think the correlation is that direct. Consider an incandescent lamp lighting a room -- most of the light illuminating the objects in the room is probably reflecting off the walls and other objects before it hits the objects you're looking at and bounces into your eye. So you need to take the wall color, etc., into account. If you adjust the white balance in the camera to make a sheet of paper look white in a photo, a picture of the light source might still look a little off white because the rest of the room plays a role. (Usually, though, if you take a picture of a bare light bulb, you get something very white just because it's overexposed.)

If I want an image not to represent the colours of objects accurately, but to include the colour-casting that my vision is subject to, then what white-balance configuration will achieve this?

That's what RAW does -- records exactly what the sensor sees without adjustment. It also records the white balance setting, though, so your software can make an appropriate adjustment when rendering the image.

Answer 3

The general answer to what you're wondering about is that there's a big difference between the simple photometric scene recorded by our eyes or a camera, and the results of filtering this raw data through human perception processes. One human perceptual phenomenon that might be closely related to what you're asking about may be this one, whereby even the quantity of light can influence our subjective impression of its "warmness" or "coolness".

Hopefully there will be better answers, but it's a place to start pondering just how complex the situation is. :)

Incidentally, I strongly suspect that the ability to be conciously aware of the variations in light sources varies quite a bit among people, and probably can be "learned" to some degree once you start paying attention to it... at least, I know that I'm far more aware of it than I used to be.

ADDITIONAL THOUGHT: In answer to your last point, it seems to me that even when we want to capture an impression of the light color in a scene, the literal, "objective" impression of the camera is still too strong, since our impressions are probably being "corrected" to at least some degree, even when we're aware of light color. The best subjective result is probably usually achieved by splitting the difference, so to speak.

Answer 4

What exactly is white balance?

'White' doesn't have a color balance/white balance. Light sources have a color balance. The amplification of light collected by a camera's sensor needed to make something look or reproduce as white has a color balance. Light of any color temperature/white balance with a full enough spectrum can be made to look white in a photo. It can also be made to look orange, blue, red, or any other color we wish to make it look by adjusting the amplification of the red, green, and blue channels in the image we have taken under that light. We call the total channel amplification for the three color channels in photographs the white balance.

Different light sources emit light at different color temperatures and tints. Even "white light" sources that emit light that includes most or all of the visible spectrum usually have most of their light centered on various color temperatures. If these light sources are what are known as 'black body radiators' the light they emit is determined by their temperature as measured in degrees Kelvin. The glowing gases on the surfaces of stars, for example, are black body radiators. So are most metals when heated until they begin to glow, then melt, and finally turn to vapor if heated hot enough. The scale of temperatures that produce specific colors from black body radiators is expressed in degrees Kelvin and is one axis of the color wheel that moves from blue on one side to amber on the other side. This is what we refer to as Color Temperature.

But color temperature is only a single axis across the 360° color wheel. What we call white balance includes the entire color wheel. Light sources that are not black body radiators can emit light that is a color not found along the color temperature axis. Such light may be more magenta or it may be more green than the nearest color that falls along the color temperature axis. We sometimes call this green←→magenta axis tint or color tone. In order to fully express the dominant color of a light source we not only need to define its location along the blue←→amber color temperature axis, but we must also define its location along the green←→magenta tint axis that is perpendicular to the blue←→amber axis. (When we use only a color temperature to properly describe a light source, it is because the tint of that light source is neutral - that is, it falls on the color temperature axis with no bias towards green or magenta.) Most natural light sources emit light that falls along the color temperature axis.

We still haven't fully described the nature of the light from a light source when we have defined the amount of blue←→amber and green←→magenta that is the most dominant component of that light, though.

Not only do light sources emit light centered on a certain wavelengths (that our eyes/brain interpret as certain colors), but some sources emit light that has a broader range of wavelengths/colors than others. Tungsten light bulbs, for example, emit light that is centered at around 3000K. But some quantity of almost the entire range of wavelengths of visible light are included in the light from a tungsten bulb. It's just that the light given off by a tungsten bulb is dominated by the range at around 3000K. Sodium vapor lights, on the other hand, emit a very narrow spectrum of light at around 2500K. But high pressure sodium vapor lights do not emit any light at all in some very broad segments of the visible spectrum. Pretty much all of the light they emit is very near to 2500K. Sources that emit a more limited spectrum of the range of wavelengths we call visible light are even more problematic when we try to do white balance correction to get accurate color of the objects they illuminate. If a light source is not emitting any blue light at all, there won't be any light at all for blue objects to reflect. If there is no blue signal to amplify, it does not matter how much we amplify the blue channel, we'll see no blue (other than the false blue caused by the camera's read noise in the blue channel).

The adjustments we make between the raw information collected by the camera and the photo we want to end up with that makes something look white is not a color temperature per se, it's a compensating filter that adjusts the relative strengths of the red, green, and blue components in the picture so that the red, green, and blue values are equal for the objects we wish to appear white or neutral gray. We assign a color temperature number (5500K) or white balance name (cool fluorescent) to a certain set of multipliers because it is the appropriate one needed to compensate for a photo that was taken under light that was centered on that color temperature and with that tint. If the light used was very blue, then we must apply a very orange filter to correct the blue tint of the light. That is why even though 10000K light is very blue when we move the slider in our raw processing app all the way to 10000K it makes things shot under more yellow light look orange. That is why even though 2500K light is very warm when we move the slider in our raw processing app all the way to 2500K it makes things shot in more yellow light look very cool.

Again, at any particular color temperature setting, we may also need to alter the green←→magenta axis setting that runs roughly perpendicular to the blue←→yellow axis on a color wheel in order to make a particular object look white. This is because not all light sources emit light that falls exactly along the color temperature continuum defined by the temperature, in degrees Kelvin, of a black body radiator. For example, the LED lighting currently used for stage illumination in a lot of small night clubs can have a much more magenta tint to it than a black body radiator will emit at any temperature. Typical old style fluorescent lights, on the other hand, emit a much greener tint than a black body will radiate.

When we alter the color temperature setting of a photo we have taken, we don't change the color of the light that was present when the photo was taken. Rather, we change how much each of the RGB channels is amplified compared to the other two RGB channels.

A white balance setting is a set of multipliers for the red, green, and blue channels that is appropriate to apply to a photo taken under light of a specific color temperature and tint. This affects what color various objects in the photo will appear to be, but it doesn't change "their white balance" because those objects do not have a white balance - the light that was illuminating them has a white balance.

If we photograph a white object under light that is 2700K, we need to apply a 2700K color temperature setting for that object to look white in our photograph. If we photograph the same object under light that is centered on 8000K then we must apply a color temperature setting of 8000K for the object to look white in our photograph. If we apply RGB multipliers (i.e. a color temperature setting) appropriate for 5000K light to the first image taken under 2700K lighting the white object will look yellow/orange, if we apply RGB multipliers appropriate for 5000K to the second image that was taken under 8000K lighting the white object will look blue.

The term white balance is also used to describe the way we attempt to correct color casts in photographs taken under those various types of light sources.

Remember when we said different light sources emit light at different color temperatures and white balances? This affects what colors the things they illuminate appear to be. It affects the color that our eyes and brains see them as. It affects the color that our cameras see them as, too. Although our cameras are designed to mimic the way our eyes and brains create color, they don't do it exactly the same.

Our eye/brain systems are incredibly good at adapting to various sources of lighting, particularly those that have been found in nature since the dawn of time (remember those black body radiators?). They also do fairly well with those artificial sources we have invented that closely mimic such natural light sources. Our brains can compensate for the differences in light sources and we perceive most objects to be the same color under different types of light sources.

Cameras, however, must adjust the bias they give to the red, green, and blue channels in the images they capture. Unless we have told the camera, via a setting such as 'daylight' or 'shade' or 'fluorescent' or 'tungsten', what the color of the light source is it has to make an 'educated guess' based on clues in the scene. When scenes don't give the expected clues, such as when the brightest parts of the scene are not a neutral/white color, the camera can often get it wrong. Another scenario that can often fool cameras in a different way is when most of the frame is a uniform brightness which the camera will attempt to expose as a medium brightness halfway between pure white and pure black.

---

So how does this all work out?

Imagine that you have a completely dark room with no windows. In that room are three separate light sources. One emits pure blue light, one emits pure green light, and one emits pure red light. Now go into that room with four cards in your hand: a pure blue, a pure green, a pure red, and a pure white one.

• When only the blue light is on there will be no light the correct color for the red and green cards to reflect and so they will look black. The blue card and the white card will both reflect only blue light and will look identically blue. If we took a photo under such light there would be no way to discriminate between the blue card and the white card in the resulting photograph.
• When only the green light is on there will be no light the correct color for the red and blue cards to reflect and so they will look black. The green card and the white card will both reflect only green light and will look identically green. If we took a photo under such light there would be no way to discriminate between the green card and the white card in the resulting photograph.
• When only the red light is on there will be no light the correct color for the blue and green cards to reflect and so they will look black. The red card and the white card will both reflect only red light and will look identically red. If we took a photo under such light there would be no way to discriminate between the red card and the white card in the resulting photograph.
• When the red and green lights are both on, there will be no light the correct color for the blue card to reflect and so it will look black. The red card will look red. The green card will look green. The white card, however, will be a combination of both the red and green light it is reflecting and will appear to be yellow. If we took a photo under such light we could discriminate between the red, green, and white cards but with the total absence of blue light there would still be no way we could make the white card appear white by only varying the amplifications of the red, green, and blue channels in our photo.
• When the red and blue lights are both on, there will be no light the correct color for the green card to reflect and so it will look black. The red card will look red. The blue card will look blue. The white card, however, will be a combination of both the red and blue light it is reflecting and will appear to be purple/magenta. If we took a photo under such light we could discriminate between the red, blue, and white cards but with the total absence of green light there would still be no way we could produce white only by varying the amplifications of the red, green, and blue channels in our photo.
• When the green and blue lights are both on, there will be no light the correct color for the red card to reflect and so it will look black. The green card will look green. The blue card will look blue. The white card, however, will be a combination of both the green and blue light it is reflecting and will appear to be aqua. If we took a photo under such light we could discriminate between the green, blue, and white cards but with the total absence of red light there would still be no way we could produce white only by varying the amplifications of the red, green, and blue channels in our photo.

Now imagine that our three light sources are each on a rheostat and can be independently varied in brightness. If we turn the blue light on at 20%, the green light on at 60% and the red light on at 100% we'll have light that looks very much like that from a tungsten bulb with a very warm tint. If we took a photo of our four cards under such light they would all appear to be different colors but the colors would be shifted towards red. The key difference from before, though, is that now we have at least some light of each color with which to work. If we adjust the camera's amplification of each color channel so that the red light is only amplified at 20%, the green light at 33%, and the blue light at 100% we would wind up with each color having the same brightness for our white card and it would appear to be white.

The HUGE disadvantage to doing it this way is that now none of the colors are any brighter than 20% of what we could have gotten if all three lights had been adjusted to 100% and all three color channels had been amplified at 100%! If we decide to amplify our photo by an additional 500% in post processing to make it look like 100% RGB amplification of 100% RGB light, we'll also amplify our camera's read noise by 500%! That's why it is always preferable to get the lighting as close as we can to what we want before we expose the photograph.

---

How exactly is the raw data of the RGB photosites on the sensor converted into pixel RGB values using the white balance modelled distribution of light?

The thing to keep in mind is that the filters in a Bayer mask are not absolute. Neither are the three types of cones in the human retina!

Some red light gets through the green and blue filters! Some green light gets through the red and blue filters! Some blue light gets through the green and red filters! It's just that more red light than green or blue gets through the red filters. More green light than red or blue gets through the green filters. More blue light than red or green gets through the blue filters. But every photon (regardless of what wavelength of light it is oscillating at) that makes it past the Bayer filter and down into each pixel well is counted the same as every other photon that makes it down that pixel well. Raw data from the sensor is a single monochrome luminance value for each pixel well (more properly called a sensel).

In much the same way, all of the cones in our retinas have some response to all wavelengths of visible light. It's just that the overlap between green and red is a lot closer in our eyes than in our cameras.

If the red, blue and green channels of a little patch on the sensor each collect the same number of photons, then why isn't this represented by a pixel with equal RGB values?

The reason a camera can't just always use the same weighting is that the color of various light sources are different. Our eyes and brains usually compensate for these variations in the color temperature and white balance of different light sources. Our cameras need a little more guidance. If the camera is set to 'Auto WB' it will use the information it collects in the scene to guess the correct setting. The most basic cameras usually do this by assuming the brightest thing in the picture is white. Modern cameras have become very sophisticated in the ability to guess correctly most of the time. But certain scenarios are still difficult for them to interpret properly. Thus, cameras also give the user the ability to set the color temperature and white balance manually.

Why do we 'correct' this by distorting the values according to the light source?

Because when the light from various light sources reflects off of white objects the reflected light does not contain the same amounts of red, green, and blue compared to the light from other various light sources reflecting off of the same white objects. The colors of the objects in our photo are already 'distorted' when the light strikes the sensor, based on the color of the light source illuminating the scene we photographed. We do white balance correction to counteract the 'distorted' colors caused by the imperfect light source.

If the white balance is chosen correctly, won't the light source appear to be pure white? This is at odds with the fact that light sources clearly do not appear pure white in general.

The "correct" WB for a given light source is an amplification of the R, G, & B channels that is more or less reciprocal to the strength of each in the light source. If the light source has more red, we amplify the blue channel more. If the light source has more blue, we amplify the red channel more.

If I want an image not to represent the colours of objects accurately, but to include the colour-casting that my vision is subject to, then what white-balance configuration will achieve this?

It will depend upon the light source and the colors of the objects the light source is illuminating. A good place to start would be somewhere about 1/3 of the way along the color temperature axis between the temperature of the light source and about 5200K ("daylight").

Is there a sort of global 'neutral' setting which doesn't alter colour casting?

No. Your eyes and brain always adjust one way or another to different light sources. Your camera does not adjust unless the white balance is changed. If you have the camera set to Auto White Balance the camera, instead of the photographer, will 'choose' how it is adjusted.

For example, white objects do not appear white in a dark room with the red safety light on. I don't want them to appear white in my photos either.

In a case where the lighting is very limited in its spectrum, adjusting saturation will usually have a greater effect on the perceived color than adjusting white balance. If there's only red light in the image, no amount of amplifying green and blue will change that very much.

---

Further Reading

For an extreme example of how proper white balance, particularly along the magenta green axis can affect the color (and more) of a photo, please see this answer to Blown out blue/red light making photos look out of focus (Several example images are included in the answer)

For how correcting white balance and using selective color adjustments when converting from raw can vastly improve the end result over letting the camera do it please see: Lots of noise in my hockey pictures. What am I doing wrong? (an example including screen shots of the settings used to process the raw file are included)

For more on how to set fine adjustments of white balance beyond color temperature in-camera (or, with many cameras, even when using AWB) please see: How to cancel purple stage lighting on subjects? (several example images are included in the answer)

What's the color temperature of target illumination of white balance?
What is white balance in a camera? When and where should I use WB?
What is the meaning of "white balance"?
Why are high white balance temperatures redder when warmer objects are bluer?
RAW files store 3 colors per pixel, or only one?
Why does my white picture have a blue hue?
What is the difference between auto white balance and custom white balance?
Are there reasons to use colour filters with digital cameras?
How do I find the right whitebalance for a night cityscape?

Answer 5

Electronics and the human mind are different things. As already mentioned, our eyes adjust the lighting/scene for us.

Light, in physics, are wavelengths. In wavelengths are different frequencies. These different frequencies determine the colour. Below is a very simplified example of the relationship between colours and wavelengths:

From: http://science.hq.nasa.gov/kids/imagers/ems/visible.html

From this, you can understand that different light sources emit different frequencies. Please see another simplified graph:

From: http://micro.magnet.fsu.edu/primer/lightandcolor/lightsourcesintro.html

Cameras can actually capture more than our eyes. This is where white balance comes in. For a camera to show what our eyes are seeing, it adjusts the white balance.

If I want an image not to represent the colours of objects accurately, but to include the colour-casting that my vision is subject to, then what white-balance configuration will achieve this?

Automatic white balance. If the results of your camera don't satisfy, change the white balance. You might learn something!

Remember, cameras are very sophisticated these days. But not as sophisticated as the human body.