How to estimate colour temperature of the illuminant from a photograph of a standard colour chart?

Question

I am working on a digital still camera. I need to compute the colour temperature of the illuminant programmatically using some algorithm (not using a third party tool from Adobe or etc.).

In my case images are captured using my experimental/under development camera, and the illuminant is not a standard illuminant with a known colour temperature. I need this for colour correction/white balancing. I am looking to estimate Colour Temperature (degree Kelvin) of a light-source illuminant based on some statistics of the RGB data or the Raw Bayer data of the captured image of say a standard colour chart (24 colour patches) under this illuminant.

How can I estimate the colour temperature from this colour chart data?

Any pointers or reading material or algorithm which talks about it would help.

Answer 1

To keep things simple, use a neutral gray card instead of a color checker. Or use only a neutral patch of the color checker. You have to identify in the image the area that corresponds to the relevant patch/gray card, then average separately the R, G and B raw pixels on this area. Make sure none of those pixels is saturated, otherwise start over again with a lower exposure.

You end up with an (R, G, B) triplet which is the color of your illuminant in your camera's native color space. Convert then to the CIE standard color space using a formula like

⎛X⎞   ⎛            ⎞ ⎛R⎞
⎜Y⎟ = ⎜ 3×3 matrix ⎟ ⎜G⎟
⎝Z⎠   ⎝            ⎠ ⎝B⎠

where the matrix is the color matrix of your Bayer sensor, which you hopefully already know. From here, you can compute the correlated color temperature using standard formulas.

Answer 2

What your looking to do can be a very complex problem space. Color theory, color correction/white balance, etc. is a very complex field. To make sure I understand, you are trying to create an algorithm that can properly set the white balance of a scene based on the illuminating light source in the scene. It might be possible to do what you need to do, and many of the algorithms you would need to use should be available these days.

Some points to consider, and factor into your processing:

• No in-scene light source
  • For example: The sun illuminating a scene from behind the photographer
  • Another example: Off-scene flash or other lighting
  • This is probably the biggest issue, and would make it pretty much impossible to do any light-source based color correction
• Multiple light sources
  • It is possible for there to be multiple light sources with the same color temperature
  • Of more importance is multiple light sources with differing color temperature
  • Key Question: What do you want to happen if a scene is illuminated with multiple light sources of differing color?
• Ambient vs. point light sources
  • Example: The sun illuminating a room through a window, with additional lighting illuminating the shaded parts
  • Key Question: What source do you want to white balance on?
• Light sources of radical color
  • Sometimes you have scenes illuminated by primary light sources of radical colors
  • For example: red, green, blue, purple, yellow, etc.
  • The Blue/Yellow/Red plane can be considered extremes of normal color temperature
  • The Green/Purple/Magenta plane could be considered the extremes of tint
  • No saturated color, primary or secondary, makes a good target for color balancing a scene, as it would likely produce pronounced color cast in your image
  • Key Question: What do you do when your scene has no common ambient or primary lighting, and is entirely or primarily lit by radical colors?

If we exclude no in-scene lighting and radical colors from the problem space, it is probably possible to identify and measure the color temperature of in-scene light sources. There are some fairly advanced shape detection algorithms available these days that, provided you have a decent base or known light source shapes, would make it possible to identify all the light sources in a scene. Once you have identified each light source, you could probably use some kind of tristimulus calculation to determine the color temperature of the light source, taking the green, red, and blue subpixel values or RAW bayer pixel values with proper weights to generate your three luminance peaks. An average of samples from the area of the light source (as discovered by your earlier identification step) could produce a fairly accurate color temperature reading (but probably not entirely accurate, as digital data will always be less accurate than original analog data).

Your algorithm becomes particularly complex when you involve non-standard or radical scene illumination. Extremely saturated light sources won't make good white balance targets, since they are far from the "white" range of a color gamut. It is very important to know what part of your scene is most "neutral" so that you can pick a proper "gray or white" to base your white balance on, as that is fundamentally how white balance works. It may be possible to color balance your scene based on a different base color, such as red, however I don't know exactly how you would need to modify existing mathematical formulas to accommodate such an approach (there is a fair amount of complex math involved in color theory.)

The most difficult situation is when there are no in-scene light sources. If the scene is illuminated by an off-scene light source, it would be pretty much impossible to calculate its color temperature. If you wanted to take a fairly radical approach to camera design, you might put some basic light metering sensors on all sides of the camera body. That would allow light to be measured from all directions, however it would not be entirely accurate, and the gains for all the extra complexity would likely be minor. Another approach might be to create a hotshoe device that could connect to multiple spot meters that could provide accurate illumination readings to the camera for any light source you point a meter at. Scenes with multiple light sources or sources with radical color would still make for a very complex algorithm.

Trying to determine the color temperature of your scenes light sources from a simple photo of a color correction card, for example, would be extremely difficult if not flat out impossible. You would have to make too many assumptions about, or you would have to know ahead of time, what kind of light was illuminating your scene. This is generally why white balance/color corrections are usually done by setting the scenes white balance based on a known neutral color in a scene. Photographing a landscape twice in the same lighting, once with and once without a color correction card, allows you to pick a valid neutral or pure white color during post processing, which will set proper color balance. I myself have the X-Rite ColorChecker Passport, which includes two rows of color correction swatches. One row is for landscapes and the other for portraits, and they progress from slightly cool to slightly warm color patches. I can set the kind of color balance that I want for my scenes, which tends to be warmer for landscape photography, and more neutral white toned (5500k-6500k) for things like wildlife or macro.

If you are doing something considerably more specialized, you may be able to define a smaller problem space within which you can create an algorithm that does what you need. The problem space would have to be fairly narrow, with rather well-defined parameters and limits, in order to generate enough well-known data to help the algorithm make the correct decisions when white balancing a scene. Any scene without primary light sources within the frame is going to be very difficult to do any automatic white balance or color correction.

Answer 3

Am I understanding you correctly that the purpose of this is that you're building a new digital camera? I.e. working on developing firmware, and that sort of thing?

If so, there appears to be some good information on wikipedia's Color Temperature article, specifically the Calculation section.

This is not something I've actually done before, so my knowledge is limited. I should think, though, that a key part of this is knowing what the image is of. So, shooting a white or neutral gray card, with even lighting, would be an obvious choice. And your choice of a standard colour chart is also good, so long as your software knows which part of the card any given set of pixels that you're going to do calculations on is based on. So if you're planning to do a bunch of shots of that card in different lighting situations, you'll want to either make some sort of a rig that can keep the card in the same place relative to the camera for each shot, or determine the position of the card (perhaps with edge detection and/or similar such methods).

Of course, you'll also want to check your results against known values... a colorimetric light meter or other such device(s) may be required purchases to really do this right. And there are different sorts... those meant for photographers (or videographers or motion picture photographers) seem to turn up with one google search, while ones that may be more oriented towards laboratory also exist. I know someone with a meter, for example, that has a little (1cm diameter, or so) sphere at the end of a rod, which then hooks up to a USB port on a computer, and measures intensity of light at various frequencies, giving a true spectrum analysis of a light source. I'm not sure exactly what that device is called, but I could probably find out, if that's useful.

Of course, those devices have to do it somehow, too, so maybe just learning the theory and math of it will do the trick? :) Various useful (and I'm sure some less useful) items can be found with a google search. I imagine you have some serious research ahead of you, beyond the scope of what you'll be able to get hard answers to here. Hopefully this gives you some starting points, though.