Why do we actually need to adjust white balance?

Question

The answer seems obvious: without white balance, we would have bad color reproduction, because different lighting would produce lots of different color tints. Our eyes adjust for the color tints so we can reconstruct the real colors of objects, so cameras need to adjust white balance too.

But that seems strange. We clearly can perceive color tint in scenes: everybody can see that incandescent lamps are yellowish, while fluorescent lamps are very white/slightly blue. But with auto white-balance, the color tint is removed in the photograph. Both incandescent lighting and fluorescent lighting become white.

And though our eyes do adjust to color tint, why don't they adjust when looking at a photograph? Why does the camera need to do work that the eyes would already do?

This seems to imply that to get accurate color reproduction - including color tint that we perceive and thus want to capture, just set the white-balance to daylight, all the time.

But white-balance evidently is necessary. Even in a room with terrible incandescent lights that give off a strongly perceptible yellow cast, the image on the digital viewfinder still looks much more correct with the white-balance on automatic, than with it on sunlight! I just stood there messing with the camera for quite a while, and I'm still really confused why this is the case. Why would the viewfinder in the room, which shows an image without yellow tint, look correct literally right next to objects illuminated with a strong yellow tint? And when I put the camera on sunlight, the screen suddenly shows a WAY stronger yellow tint than the actual room, even though my yellow-adjusted eyes should shift both the room and the screen back to white, no?

Is there something about screens and photographic paper that make our brains/eyes "turn off" our internal white-balance correction?

Answer 1

The perceived color of an object depends on two elements: the intrinsic color of the object, and the color spectrum of the light shining on it.

A red apple for example, will appear nearly black with a pure blue light shining on it. Depending on the difference in spectral density of different lights, the absolute perceived color of the red apple will change, it isn't constant. But because we have knowledge of what color the apple really is, our brain adjusts our color perception so the red apple is what we expect.

White balance is the tool to make the output of the camera reflect the post-processing our brains do.

When we look at a photograph or a screen, our visual cortex applies its white balance depending on the lights in the room and your knowledge and preconceptions of what the intrinsic colors of items should be, but it's not equipped to make extra, special adjustment knowing it's looking at a photograph. When the white balance of the photo or screen is different than the environment you are in, the resulting colors look strange, e.g. the red apple's perceived color is different than what your brain expects in your rooms lighting.

You say everyone can see that incandescent lamps are yellow, but that's not strictly true. You have knowledge of the lights in comparison to other light sources which is why you think it's yellow, but that can easily be fooled. I could put you in a room, new to you, with only incandescent bulbs, and I could make the lights seem virtually any color, by carefully selecting the colors of the paint and other objects to trick your visual cortex into applying an incorrect white balance. If I had a bunch of objects in the room that are ordinarily white, but are in fact tinted a particular way, your brain will adjust its white balance correction so they are, and that can result in perceiving the incandescent lights a different color. The infamous blue/gold dress is an example of the phenomenon at work.

The Wiki page on Color Constancy has more explanation as well as some sample images that can further illustrate the concept.

Answer 2

Our eyes adapt to ambient light. When you look at a photograph, the lighting in the image does not necessarily match ambient light. Our eyes and brain work together to get a global understanding of what we see which is why optical illusions work on us. The mind thinks it know what is darker/lighter or closer/further and then people design drawings to trick the brain.

This works for white-balance too. Our brain ignores the lighting because it knows what things should look like under it. A photograph however breaks this because it can present a differently lit scene in a display or print and so we have trouble resolving this difference.

Answer 3

The receptivity curves of our eyes' receptors and of a digital camera's receptors are different. We have three different kind of receptors from which we manufacture our color impression. Light sources emit energy over a spectrum of frequencies/wavelengths, surfaces reflect different at different wavelengths, and our receptors have different sensitivities at different wavelengths. One saving grace is that intensitivities tend to change comparatively gradually and many light sources are modeled after sunlight which has the spectrum of a hot black body filtered through the atmosphere. Now the sky is blue due to dispersion, and the sky's blue has been misappropriated from the sunlight as seen in space. The overall result is missing only some blue, but if the sky is clouded or just the sun obstructed (because we are in the shade), the spectrum for lighting is different.

Incandescent bulbs are similar in nature to sunlight but start out not as hot and are missing the dispersive action of the atmosphere. Fluorescent lights tend to have lots of intensity concentrated on narrow wavelength bands.

This is a complete nightmare for producing, say, car paint. You can create substitute paints that blend perfectly under sunlight but will look totally patchy under sodium vapor street lights.

Strongly pigmented colors tend to have highly wavelength-dependent refractive behavior. Fine art museums are some of the few environments crucially dependent on incandescent lighting.

Color film chemistry depended on creating a mix of sensitivities closely matching those of the human eye. The color filters of digital cameras work differently (and tend to make better use of light). In the end, you need to match the output to what a human eye might perceive and you need to make assumptions about the light source and the reflective surfaces. Those assumptions are coded into the white balance. With significantly pigmented surfaces and/or different sources of light in one picture, a single white balance might even not be enough (raw editing programs tend to have color corrections for shadows as an option).

There have been even cameras with more than 3 different color detectors (Sony DSC-FSF828 had a blue, emerald, red, green matrix instead of blue, green, red, green) but the loss in resolution for the important green receptors did not convince users that the better behaved white balance was worth the trouble and such developments did not really last long enough to mature.