How to test noise?

Question

I'm just looking for what's the best technique to test for noise on a camera.

Like what settings are best? What type of photo is best? What lenses should I test with? Like those sorts of things. I've felt for a long time that the noise on my 30D is pretty bad and I'm looking at upgrading, so I'm trying to either prove/disprove myself. I've read/heard that 30D's are bad but I need to test for myself.

I did a quick test earlier today.
Photos were shot in RAW on the Canon EOS 30D at f/1.8 on the EF 50mm f/1.8 lens with the 580EX II on ETTL.

http://photo.heyvian.com/30d-noise-test.html

Answer 1

You can measure the noise ratio at different ISO settings in a fairly simple way and use Gimp (Photoshop) to visualise the results. Below are the results with my camera, a Pentax K7.

ISO 100 is used as the basis for comparison.
The Noise Ratio is a dimensionless number showing the increased noise above that of ISO 100.

If you photograph a perfectly uniform grey surface, the standard deviation of the pixel values should be exactly zero. Any deviation from that should be noise of different types.

But, in practice we can't create a perfectly uniform grey surface. So we photograph the same surface at all ISO settings and use the ratio of increased standard deviation above that at ISO 100 to measure the ratio of increased noise.

One can see below, that my K7, for all its virtues, is not a stellar noise performer.

Procedure

Take test photos as follows:

• photograph a uniform dark surface at each ISO setting as follows.
• set the aperture to the maximum.
• focus on infinity, we want the image as blurred as possible.
• determine the normal exposure and decrease the shutter speed by 2 EV.
• mount the camera on a tripod, each photo must cover exactly the same area.
• set your camera to raw.

Process the test photos as follows:

• set the raw convertor to no profile, linear input, linear output, default exposure (not auto).
• open the result in Gimp and select the central 500 x 500 pixel area. It is important to select the same area in each photo.
• Select Colors|Info|Histogram and note the mean and standard deviation readings.
• Calculate the coefficient of variation = (std. dev.) / mean.
• Calculate the Noise Ratio = (coefficient of variation of test sample) / (coefficient of variation at ISO 100).

Results for Pentax K7

To help visualise the results I show the extracted noise at each ISO by by overlaying the test layer over the base layer at ISO 100 and setting the layer mode to Grain Extract. This is used as the background of the histogram, below. Note how the histogram spreads out as the noise increases. Performance up to ISO 400 is good but noises creeps in from ISO 800 onwards.

ISO 100 standard deviation, the background shows extracted noise.

ISO 200 standard deviation, the background shows extracted noise.

ISO 400 standard deviation, the background shows extracted noise.

ISO 800 standard deviation, the background shows extracted noise.

ISO 1600 standard deviation, the background shows extracted noise.

ISO 3200 standard deviation, the background shows extracted noise.

ISO 6400 standard deviation, the background shows extracted noise.

Answer 2

To prove how bad things are :) you need to set an exposure which will give a dark output without under-exposing. Use something with fine-details or a texture. Noise is always most apparent in dark regions and its effect it to destroy fine-details.

To see how good things are ;) do the same with a bright exposure, without over-exposing of course. Use a slightly out-of-focus target.

Any reasonable lens will do. Set it to its sharpest aperture, usually 2 stops down from wide-open. Focus manually and shoot with different ISOs, letting the shutter-speed vary. Use a self-timer or cable-release to make sure the softness you see is caused by noise.