3
\$\begingroup\$

I've been reading this great article on how digital cameras convert raw data to sRGB images. In the last section (section 7) it discusses how Adobe's DNG converter does the conversion using color matrices or forward matrices. The description in this section is consistent with the DNG specifications (which contain more detail though). Equation 75 on page 32 in the first article describes how color matrices and forward matrices are related. $$ \mathrm{ForwardMatrix} = \mathrm{CAT}_\mathrm{AW\rightarrow D50} \cdot \mathrm{ColorMatrix}^{-1} \cdot D^{-1} $$

The formula seems very logical but I wanted to test it for my camera, a Fuji X-T4. So I used the Adobe DNG converter to convert one of my RAF files to DNG. The resulting file contains both color and forward matrices. Using linear Bradford CAT (and \$D^{-1}\$ found by applying color matrix to the XYZ coordinates of A and D65 respectively) doesn't make this equation equal. I've provided the full example for illuminant D65 (forward/color matrix 2) below:

forward_matrix2 (extracted from Exif data):
 [[0.397  0.418  0.1493]
 [0.219  0.7369 0.0441]
 [0.1021 0.0017 0.7213]]

color_matrix2 (also from Exif data, normalized for D50):
 [[ 1.34689632 -0.63542688 -0.11807664]
 [-0.4244      1.2136      0.2371    ]
 [-0.0575128   0.12920548  0.5929768 ]]

color_matrix2 (normalized for D65):
 [[ 1.26067768 -0.59475141 -0.11051822]
 [-0.39723296  1.13591403  0.22192256]
 [-0. 05383124 0.12093467  0.55501868]]

ie color_matrix2 * D65_XYZ = D65_RGB has G = 1
 [0.48307219 1.         0.6740461 ]

D contains the white balance multipliers,
so that's the inverse of previous line,
we want D^{-1} so that's just the previous line:
 [[0.48307219 0.         0.        ]
 [0.         1.         0.        ]
 [0.         0.         0.6740461 ]]

Linear Bradford CAT from D65 to D50 is given by:
 [[ 1.04788256  0.02292279 -0.0501786 ]
 [ 0.02959226  0.99045781 -0.01706769]
 [-0.00924256  0.01505579  0.75190228]]

Thus the right hand side gives:
 [[ 0.48394277  0.55680221 -0.07654498]
 [0.17040624   1.09923601 -0.26964225]
 [0.0056379   -0.13067634  0.94993844]]

This doesn't match the forward matrix.

So I'm wondering if a) I'm doing something wrong, b) a CAT other than linear Bradford is used (although that is the recommended one), or c) there's an issue with the profile.

I'd appreciate it if somebody could point out an error. Alternatively, it would be great if somebody could upload DNG files which contain color matrix and forward matrix exif data (ideally for a camera other than a Fuji X-T4) so I can test this formula on another example.

Thanks for your help!

Note, there are some clarifications below. But if you have some DNG files whose exif data contain forward matrix and color matrix, please share.

EDIT: all data I'm using (forward/color matrices) comes from the DNG file's metadata

EDIT: More details / clarification on the process:

ColorMatrix is a matrix that maps XYZ values to the corresponding camera RGB values. Color matrices are normalized for D50, which means that ColorMatrix * D50_XYZ results in an RGB triple whose maximum is 1 (typically green). For example, ColorMatrix2 * D50_XYZ = [0.56584913, 0.99997731, 0.5628982].

ColorMatrix2 is optimized for illuminant D65 which means that ColorMatrix2 correctly maps the XYZ coordinates of D65 to the camera's sensor values you get when a D65 spectrum shines on it. Applying ColorMatrix2 to D65 XYZ gives [0.51610984, 1.06839071, 0.72014459] whose maximum is larger than 1 because ColorMatrix2 is normalized for D50. Renormalizing results in camera RGB values for D65 of [0.48307219, 1, 0.6740461].

ColorMatrix^-1 maps camera RGB values to the corresponding XYZ values.

D contains raw channel multipliers. That's the coefficients that map the camera RGB values of the illuminant to [1, 1, 1]. So for illuminant D65 with camera RGB values [0.48307219, 1, 0.6740461] the raw channel multipliers are [2.07008397, 1, 1.48357805] because multiplying them together (component-wise) gives [1, 1, 1]. That also means that D^-1 is [0.48307219, 1, 0.6740461] which is the same as D65's camera RGB values.

So let's apply the right hand side of the equation to camera RGB coordinates [1, 1, 1] (using matrices for D65, which is color/forward matrix 2 in DNG files). First D^-1 applied onto [1, 1, 1] results in the camera RGB coordinates of the illuminant, D65 in this case, which is [0.48307219, 1, 0.6740461]. Second ColorMatrix2^-1 maps the camera RGB coordinates of D65 to the XYZ coordinates of D65, which are [0.95040117, 1, 1.08874301]. Third the CAT takes the XYZ values of D65 to those of D50 [0.9642, 1, 0.8249].

So applying the right hand side of the equation to camera RGB coordinates [1, 1, 1] gives D50 XYZ. This is exactly the definition of what the forward matrices do. In fact, applying the left- and right-hand sides to [1, 1, 1] gives the same result (D50 XYZ). But since the matrices are different, camera RGB values other than the illuminant will result in different XYZ values. So for some camera RGB value X,

$$\mathrm{ForwardMatrix} \cdot D \cdot X \neq \mathrm{CAT} \cdot \mathrm{ColorMatrix}^{-1} \cdot X$$

Given that both forward matrix 2 and color matrix 2 are optimized for the same illuminant (D65) I'd expect them to give the same result. That's what I mean by inconsistent. (but maybe there's something I'm missing, hence my question)

\$\endgroup\$
4
  • \$\begingroup\$ DNG can also include a lookup table and built-in Adobe DNG Converter profiles have colour lookup table. I do not remember specific term for it but I totally expect it to invalidate your equation if you are using image data for your analysis (it's not entirely clear to me if you are using metadata only or not). \$\endgroup\$ Commented Aug 6, 2023 at 5:51
  • \$\begingroup\$ hey, all data I'm using (forward/color matrices) comes from the DNG file's metadata, extracted through exiftool. Which colour lookup tables are you referring to? Maybe the Hue Saturation Value map? If I read the specification correctly then that is only applied after the conversion to XYZ (under D50) - so I wouldn't have expected that map to impact the equation. Or am I misreading the specification? \$\endgroup\$
    – mpr
    Commented Aug 6, 2023 at 6:36
  • \$\begingroup\$ Actually, I'm not sure about my answer at all anymore. \$\endgroup\$ Commented Aug 6, 2023 at 10:24
  • \$\begingroup\$ Ok, thanks for trying! If you have any DNG files with color and forward matrix exif tags please share. Then I could see if the process works with yours or if there's some consistent difference which could help identify the error. \$\endgroup\$
    – mpr
    Commented Aug 6, 2023 at 10:33

1 Answer 1

1
\$\begingroup\$

I came across your question because my astro camera sensor is the same as the one used by your camera, and I needed to get the matrices for color conversion.

I read the paper you cited and everything seems to be logically correct. However, I encountered the same problem as yours - equation 75 does not hold. I got an example image for your camera from DPreview - studio shot comparison website: https://www.dpreview.com/reviews/image-comparison where you can select your camera model and download RAW image files as well as jpegs. I also got an example image for the camera that the author of that paper used (Olympus E-M1), and I found the same situation. Regrettably, the author did not attempt to test his equation 75 with the actual exif data from his image.

Then I read the Adobe DNG specification document (v 1.6.0.0) to try to figure out the reason of the discrepancy. The only extra matrices that are involved in the conversion from camera to XYZ_D50 are the ones called AB and CC -- Analog Balance and Camera Calibration matrices. The DNG exif data for the Fuji has AB = identity, and CC = diag(1.0347, 1, 0.9481), which is very close to identity. Disregarding CC (and AB which is ID), the equations given in the DNG spec doc agree with those in Rowlands' paper. I checked also including the CC matrix, but that does not make a significant difference, naturally.

So what is the conclusion? The DNG spec doc says that if the Forward Matrices are given in the exif data, one should use those to make the conversion to XYZ (and then RGB), whereas if they are not given, then one should use the conversion equation based on the Color Matrices. (In another forum someone conjectured that the color matrices are used just for normalization purposes, so we shouldn't expect them to be consistent with the forward matrices.)

An additional clue comes from the exif data in the JPG image downloaded from the same DPreview site. There, you find Red, Green and Blue matrix columns. Putting these together and transposing, you get a matrix that has to be a forward matrix, as it sends the vector (1,1,1) to the XYZ coordinates of D50. This is probably the conversion that was used to create the JPG image, and it is similar (but not exactly the same) to the Forward Matrices in the DNG file (probably interpolated to the exact scene white). According to https://www.strollswithmydog.com/determining-forward-color-matrix/, the matrix \$M_{\mathrm{sRGB}}^{-1} \cdot \mathrm{CAT}_{D50\rightarrow{D65}} \cdot \mathrm{ForwardMatrix}\$ is called "color matrix" in the DxOMark website https://www.dxomark.com/, as it takes white-balanced raw sensor data to sRGB. So some flexibility in the terminology should be accommodated. Fortunately, the above website (strollswithmydog) does make all the detailed calculations showing that his forward matrix agrees with the values given in the DxOMark website, so we can be pretty confident that the Forward Matrices are the ones to be used, if given in the exif data.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.