White balance math for white, black, and grey color cards

Question

This question got me very close to my answer, but I'm looking for more precision: What is the math behind white-balance, and how to choose it?

I have an image with white, grey, and black color cards in the background and I would like to white balance my image (using math rather than an "auto" button from a photo editing software.

In my image, the white card has RGB values of (181, 176, 141) and the black card has values of (64, 61, 38).

If I follow the answer from the question I linked above then I would adjust the pixels by (255/181, 255/176, 255/141) to make the white card actually white (255). But this would turn the black card into (90, 88, 69) rather than (0, 0, 0).

So my question is, what is the best mathematical adjustment to use to correct for both the black and white cards. I have not used the grey card yet, but adjusting to match it would be good too. Again, I'm looking for the mathematical adjustment since I'm trying to do this in a rigorous scientific way. Identical white balance correction between all my images is very important.

Here's an example image if it's helpful. Note that the glare from the overhead light will not be present in the pictures I actually care about.

Answer 1

Why are you trying to match 3 values with a linear relation?

Consider only the red channel. You have 3 reference values (taken from your original photo): Rw for the white color, Rg for the gray and Rb for the black.

You want to find a function mapping Rb to 0, Rg to 128 and Rw to 255. With 3 references, you can't use a linear function if you want a perfect match, you have to use a polynomial function with a degree at least of N-1, N being the number of references to match.

The grey reference (here 128) is related to an 18% grey card (more info: Why is 18% grey considered to be in the middle for photography?). It may be different for your grey.

The function ax² + bx + c = v (1) is of degree 2, exactly what you want for a perfect match ("fitting") with 3 values. Now, you want to find a, b and c such as (1) is verified for Rb, Rg and Rw. In other words, you have 3 equations:

• a (Rb)² + b (Rb) + c = 0
• a (Rg)² + b (Rg) + c = 128
• a (Rw)² + b (Rw) + c = 255

Solving this equation is easy (but not related to photography).

Once you have found a, b and c you can map any red value to a "calibrated" red. You may have negative values or values greater than 255, be careful.

An example maybe?

If Rb=10, Rg=100 and Rw=254, solving for (1) will yield a=-0.002449, b=1.6916 and c=-16.671 (need help ? Wolfram Alpha) so (1) <=> f(x)=-0.002449*x²+1.6916*x-16.671.

On the absciss (x-axis) lies your original values, on the ordinate (y-axis) the "calibrated" ones.

As you can see, all original values bellow 10 are mapped to negative values and should be set to 0... and values beyond 254 are mapped to values greater than 255. In fact, the [0,255] interval is mapped to [f(0),f(255)] = [-16.671,255.44].

If you don't want to clip your values (=losing information), you should use a new function to map [-16.671,255.44] to [0,255]. This function can be linear, such as ax+b.

You can replace (1) with any polynomial function of degree 2, you will get differents results.

If you really want to use a linear relation, say ax+b=v, you will need to find a and b minimizing errors between expected values (0, 128 and 255) and "calibrated" ones.

For more information on Polynomial Interpolation, take a look at https://math.stackexchange.com/questions/7125/polynomial-fitting-how-to-fit-and-what-is-polynomial-fitting or in Wikipedia (https://en.wikipedia.org/wiki/Polynomial_interpolation and https://en.wikipedia.org/wiki/Curve_fitting)

Answer 2

The values you specify are the sRGB values. You should transform back to a linear colorspace and there make sure that the components scale linearly (in sRGB they should actually also scale linearly to a reasonable good approximation). In linear RGB, I find that the white card is (117, 110, 67), the black card is (12, 11, 4.5) and the gray card is (50.5, 46.5, 19). The R and G components scale linearly, but not the B component.

This means that the transform to linear RGB is not correct, some tone curve has been applied to the B component. You can try to deal with this problem using the method in Olivier's answer, but you should be able to do the conversion from the raw file to the (s)RGB output correctly, otherwise a lot of your efforts go into undoing whatever non-linear tone curve has now been applied. You can then just scale the components linearly in RGB and transform to sRGB.

Answer 3

The proper way of doing white balance is using only one reference (white). If you cannot get desirable results with only one reference, you will only introduce false colour if you try to use multiple references.

If you have a tint on the whole image (like, constant amount of coloured water between target and camera), you should subtract the tint light from image data, and then apply white balance (channel multiplication).

Your scientific way of getting decent colour for an object lying under a layer of coloured luminant water would be:

0) set camera to RAW recording

1) set up white balance using white card, photographed separately

2) photograph black and white card separately under same lighting in manual exposure mode with desired exposure

3) place them inside water and photograph them there

4) export images with Dcraw without any modification with command dcraw -w -T -4 -o 4 image.raw where image.raw is actual image name. Examplary bad export from Adobe Camera RAW with completely neutral settings - this is why you should use dcraw. (These are tone curves which I expected to be straight)

dcraw -w -T -4 -o 4 *.raw is also valid (all images will be converted).

5) you should take pixel values of black and white patches both separately and underwater (bS, bW, wS, wW respectively) using RawDigger or ImageJ and solve an equation system:

bW=B*bS+T wW=B*wS+T

for each component (R, G, B)

This will give you T (tint, constant linear addition to each colour, the luminance of the water) and B (white balance multiplicator).

6) Then, to normalise colour, you should subtract T and divide result by B for each image (GIMP has arythmetic modes for layers, development versions of GIMP have support for 16bit).

You will end up with image with normalised colours. However, colours won't be spot on like photographed without water (digital cameras never give precise colour nonetheless).

You won't get any good results if you do not use RAW images.

Protip: if your black card has any pixel values above zero when photographed separately it is not actually black and you should not make it (0,0,0).

I am ready to go into details as soon as you acknowledge that you are actually ready to do all of this.

Answer 4

There have been a few algorithms for white balance. The philosophy of course is Not about the color of the subject, but is instead about the color of the light on the subject (which we never know accurately of course, possibly excepting bright sun)

Photoshop Auto Levels is an older common method, and it equalized the highest tonal value in the three histogram channels, aligning these three max values to all be the same max value. It has a option to specify the boundary between signal and noise, default is 0.1%, meaning weaker tones are ignored. This makes the assumption that the brightest tones should be from neutral white, perhaps not often true.

Raw software has a better WB tool where we click a known neutral spot (a white balance card, or even a gray card included in the test picture in same light), which tells the computer "this spot is neutral, make it be neutral." And it does, neutral meaning equal RGB components, i.e., no color cast. The result is that spot will have equal RGB components of the average value of the three. This is as true and accurate as our known white card is accurately neutral.

In your example of RGB(181, 176, 141), you would make that spot be (181+176+141)/3 = (166, 166, 166), if you want to make that spot be neutral (no color cast). If that spot on that object were in fact truly a known neutral color, then your picture will have no color cast, in that light.

Gray cards can work, even black can work, but 18% gray cards have no neutral color specification. And they are pretty dark to show color casts well. White is a good choice, if it is actually an accurate neutral white. Be careful to not overexpose and clip it.