1

I need to compare the color reproduction accuracy of 2 cameras quantitatively, in RGB space or Lab space. In my set-up, I use the same standard lighting source, same lens(same focal length and same aperture), use the one-time auto white balance for the 2 cameras.No gain for both cameras. The images of a Macbeth color chart are taken by these 2 cameras, then the color values of each color patch are compared with the ground truth value.

But my question is: the exposure time for each camera is adjusted to avoid over-exposure, thus different for two cameras. The color intensity of the images from these 2 cameras depend on the exposure time. Then how can I compare the color difference ?

  • Are the cameras from same manufacturers? – Crowley Apr 24 '17 at 13:02
  • Yes, those cameras are from same manufacturers. I have a lens that can be mounted on the cameras, and I fixed the aperture and focal lens for the 2 cameras. Now the only difference is the exposure time. I don't know if I still can compare those 2 cameras if the exposure time are different. – Zhao Huang Apr 24 '17 at 13:05
  • Exactly what 'standard lighting source" are you using? Even moderately high end studio flashes have color output variation from one flash to the next. – Michael C Apr 24 '17 at 16:51
1

While I don’t think a minor difference in shutter time will be a major influencing factor, I suggest setting both test cameras to the same f-number and shutter speed, in manual mode. Should you observe differences in the exposure levels, mount neutral density filters on the camera that yields the greater exposure. For many years I managed an operation that made test films for the photofinishing industry. These were super accurate as to color and density. They were used worldwide to calibrate high speed photofinishing color printing machine, so I have some experience in these matters.

Use ND filters to balance the camera with the greater exposure to the other. I suggest you use Kodak Wratten Neutral Density filters. These are of high quality and as neutral as you can get. Some should be available on the web or you can procure others.
ND filters are labeled in density units. These seem cryptic however the values are exponents of base 10 logarithmic notation.

0.0 = No density change used to balance a system keeping the number of filters used the same.

0.025 = 1/12 f-stop = 1.06X

0.05 = 1/6 f-stop = 1.12X

0.10 = 1/3 f-stop = 1.26X

0.15 = 1/2 f-stop = 1.41X

0.20 = 2/3 f-stop = 1.58X

0.30 = 1 f-stop = 2.0X

Let me add -- Use constant voltage transformer to power the lighting system. The slightest voltage change will greatly effect the color output of your lighting system.

  • I agree with this for film. In digital, on the other hand, you have so many software intermediate corrections on-body that actual exposure settings don't make it reproductible, unless both cameras run the same firmware on the same processor. So I would stick to choosing pictures which have the same white level, out of a bracketing serie. – Aurélien Pierre Apr 25 '17 at 23:33
1

There is a somewhat dry ISO standard procedure for doing this experiment that you can reference. Identical exposures is not required for the test.

ISO 17321-1 Colour characterization of digital still cameras (DSCs)

"Fixed exposure settings shall be selected to provide peak output levels between 50 % and 90 % of saturation"

The reason is that in the color difference algorithm, both your colorchecker values and the ground truth values are normalized by the exposure of the white test patch. They don't need to be the same.

enter image description here

I didn't read the entire color difference algorithm but effectively somewhere it will normalize the values so that the CIE Y tristimulus value of the white patch is 100, and everything else is relative to that.

However, you might be interested in knowing that most color difference algorithms come with a luminance independent color error metric, that only measures chroma differences. These metrics should work even if the source colors have different luminances:

  • CIE 1994: ΔE*94 , which includes Luminance L*, and ΔC*94 , which omits L*.

  • CMC: ΔECMC , which includes Luminance L, and ΔCCMC , which omits L. Widely used in the textile industry for matching bolts of cloth.

  • CIEDE2000: ΔE00 , which includes Luminance L*, and ΔC00 , which omits L*. This is the emerging standard as well as the most accurate color difference metric. Its acceptance has been slowed by the complexity of its formula. Although it is less familiar that the other equations, it is the best choice in the long run.

Whichever metric you choose, remember that they give different numbers. It is important to be consistent and always specify which measurement you are using.

https://www.iso.org/standard/56537.html (paid) http://www.gamutvision.com/docs/gamutvision_equations.html

0

Maybe do mot adjust anything at all. Set both cameras to manual, use very same setup and very same lens, if possible.

If you compare, say, Canon 700D to Canon 5D mark III, use one EF lens for both bodies. If you compare, say, Canon 700D to Nikon D5300, try to use vintage M42 lens with adaptors for Canon and Nikon mounts.

If you have all apperture, shutter and ISO set to same values you should have simillar images. When there are differences in resulting RAW image, there are differences in either sensor sensitivity and in-camera processing. If all the other parameters are kept the same.

With the very same postprocessing, you can directly compare the results.

  • Do you mean also keep exposure time the same for cameras? In my measurement, I want the images just below overexposure, so I adjust the exposure time to achieve that, then the expousre time are different for 2 cameras. If i use same exposure time for 2 cameras, maybe it's OK for one camera, but overexposured for the other one. – Zhao Huang Apr 24 '17 at 13:22
  • Yes, keep all shutter speed, apperture and ISO the same. Then it is the difference betwee the two. These values shall be interchangable among cameras and manufacturers. The notation was developped to describe the setup; With exposimeter (dedicated device) you will measure the scene and acording to results you will set your camera up. Whatever the camera is, you shall get same exposure. – Crowley Apr 24 '17 at 13:28
  • ISO are same for them. Because I use different exposure time for 2 cameras, for example the white patch in resulting images has different intensity, this intensity contains not only the color reproduction difference, but also the effect of different exposure. I don't know if I understand correctly. If so, what can I do to compare the cameras. – Zhao Huang Apr 24 '17 at 13:32
  • I look at the images I took earlier. One camera with exposure time 400 us, will take the image in which the white patch is [250,250,249], but the other camera needs exposure time 800us to achieve the similar result. If I use same exposure time, either one camera is under-exposed or one camera is over-exposed – Zhao Huang Apr 24 '17 at 13:39
  • With the very same setup? Ligting, shutter, apperture, ISO (sensitivity/gain) and manual mode... There is something wrong with one of the cameras... – Crowley Apr 24 '17 at 15:26
0

Assuming you compare RAW files, comparing raw color rendition on two cameras makes no sense. Every sensor has a different response to color and light, and none is perfectly linear. In real life (JPGs files, photo editing softwares), the RAW values are adjusted through a calibration profile, which corrects the color deviations of the sensor, based on curves or lookup tables, to match real colors. No sensor (in cameras as in industry) is ready to work as-is, you always have to calibrate it. (Full disclosure : I'm a trained metrologist).

Assuming you compare JPEGs files, again using the same settings makes no sense (different sensors, different integrated softwares, different response). Usually, the L value of the white patch of a color chart is between 96 and 98. Just make an exposure bracketing with both cameras and retain only the files whose white patches are the closest to the expected value. In other words, compare the pictures wich have the same white level, thus the same actual exposure.

BUUUT… what you will compare will NOT be the actual color accuracy of the sensors but the accuracy of the color profiles used to calibrate the sensors, inside or outside the camera. So again, I don't know why you would need to compare the color accuracy of two cameras, but it sounds useless to me since you can make your own profiles (it's just software) and since the sensor raw data is always to be adjusted anyway.

Some other specifications might be more relevant to compare, such as the noise level in RGB channels or the color depth. More details on DXO : https://www.dxomark.com/About/Sensor-scores/Use-Case-Scores

  • The cameras are from same manufacturer. And I use the same SDK for the 2 cameras to generate PNG images, so the profiling from RAW to PNG is the same for them two. I just want to compare, under the same setting, which camera can generate better color reproduction. But now it seems that I can not keep the shutter speed the same for 2 cameras, I can keep the setting completely same for them. – Zhao Huang Apr 25 '17 at 8:13
  • Same manufactuer does not imply same sensors, same exposure algorithms and same processor. The settings don't matter, different cameras may have different software strategies to manage proper exposure and dynamic range (exposing to the right of the histogram or protecting highlights instead). I don't know which SDK you are using, but still… the profiling cannot be the same for the two cameras or you don't have any profile at all, which is not a real-life situation. Why do you need to perform such comparison ? – Aurélien Pierre Apr 25 '17 at 23:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.