0

I'm in the market for a camera with better high-ISO capability than my current ones. There are more than enough reviews on the internet that measure cameras' high ISO performance. They usually pit one camera against another at particular ISO.

However, I don't think such method is just. It is well known that camera bodies have different "real" ISO sensitivity than their nominal one, and some brands have tendency to inflate the ISO number. So in order to assess the high-ISO performance in real life situation, you need to consider the camera's "effective" ISO, which is also available at some review sites, like DXOMark.

This makes me wonder. Why don't people run high-ISO performance test by using the camera's own auto-ISO mode, with the aperture and shutter speed fixed at the same parameter for each camera? If camera A's ISO is so superb that it only needed 800 in the same available light while others needed 1600, it's not fair to compare camera A's ISO 1600 against other cameras' ISO 1600. You should forget the number and compare at the ISO that was necessary for each camera. This method does depend on the camera's own auto exposure metering, but I think auto exposure meter has less discrepancy between brands than the actual ISO sensitivity. This method will catch two birds with one stone too: you can see which brand tends to jack up the ISO, and you can truly see how the camera does at the given low light situation when the aperture and shutter speed are fixed and the camera is asked to do the best it can with its high ISO.

  • You say that "it is well known that [...] some brands have a tendency to inflate the ISO number". Could you provide a reference for this? It's not something I'm aware of. – Philip Kendall Nov 4 '13 at 19:23
  • There is a lot of talk about it on the internet. For example, dpreview.com/forums/post/50713232. I think you can google for other brands. – Anon Nov 4 '13 at 22:26
  • Interesting. However, I'd be far more trusting of the actual DPReview findings of 1/3 to 1/2 a stop rather than the postings of randoms on the Internet. Any other brands you think are particularly "bad" here? – Philip Kendall Nov 4 '13 at 22:36
2

There are several reasons we don't do that. The primary issue when testing cameras is isolating variables and that requires making parameters which are not being tested the same. Even so, everyone's definition of the same is actually not the same! Essentially, this is the point of your question: You are questioning that for ISO sensitivity.

Not all cameras have a TAv mode and, even when they do, each camera imposes its own limitations such as the maximum sensitivity and ISO step-size which may end up using values which are more noisy than necessary. Metering systems differ greatly too. Even if two digital cameras have the same sensitivity, a different ISO may be chosen. Contrarily to your assumption, metering varies considerably more than ISO.

  • So you're saying it's more important to keep the arbitrary number that manufactures use to label sensitivity the same, than it is to keep the actual sensitivity the same? – Matt Grum Nov 4 '13 at 12:15
  • Knowing the real sensitivity is important which is why DxOMark measures them but letting the camera select ISO will give much worse comparisons. The arbitrary number as you says is off by nothing to perhaps a full-stop but metering for the same scene can differ by much more (1-3 EV is common) between cameras. – Itai Nov 4 '13 at 19:48
0

You are correct that ISO numbers are in general not comparable in most cases you can't rely on the camera metering to do the same thing between different camera brands

There really is nothing stopping a camera manufacturer adding an extra zero to all the ISO value displayed onscreen without changing anything else in the sensor, and then making the claim "our camera produces cleaner files when set to ISO 1000 than brand X does when set to ISO 160", despite the exposure times being the same between the two!

However I don't think you can easily get round the problem by using auto-ISO, as neat as that solution may be, you'll run into problems with the camera's metering system's having different behaviour.

The best testing methodology is to set the shutter speed and aperture* to the same value for all camera models and then try several different ISO settings, choosing the one that gets closest to clipping the brightest part of the scene (without overexposing). The results should then be normalised during RAW conversion to obtain identical (or close to) brightness values in the final image.


Going further down the rabbit hole it's worth noting that the relative sensitivity of each colour channel varies between sensors, sometimes but as much as a stop. So one camera's blue channel at ISO 100 might be the same sensitivity as a different camera's blue channel at ISO 200, even if their green channels are identical in terms of sensitivity.

The test as outlined above should be repeated three times for each colour channel. As you can see it is quite difficult providing a really fair test.

* As lens focal length and aperture values are generally only correct (or almost correct) at when the lens is focussed to infinity, the test should also be carried out using an infinitely large test scene, for example The Universe.

  • At least some review sites do check the claimed ISOs from cameras; see e.g. here from DPReview's take on the 70D. I'm not sure I've ever seen DPR ever say anything other than "match the marked ISOs within 1/6 stop accuracy", which at least limits the amount of stretching a manufacturer can do for ISO levels. – Philip Kendall Nov 4 '13 at 14:59
  • "There really is nothing stopping a camera manufacturer adding an extra zero to all the ISO value displayed onscreen" -- Surely it means something that ISO is, well, an ISO standard? Evidently camera vendors aren't required to ratify their ISO speeds by a third party, but surely somebody would object if they fuddled the numbers too much. – JohannesD Nov 4 '13 at 18:38
  • 1
    @JohannesD You'd certainly think so wouldn't you, however the ISO standard defines sensitivity in terms of the camera's metering, which isn't covered by an international standard, so by fiddling your metering algorithm you can effectively choose whatever numbers you like for the ISO sensitivity! – Matt Grum Nov 5 '13 at 10:31
  • @MattGrum Ah, figures :P Thanks for the clarification. – JohannesD Nov 5 '13 at 13:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.