What are good test shots for finding out the capabilities of a camera?

Question

I have a range of cameras from a (frankly awful) kiddie camera to a hybrid (Olympus Pen) and I want to take some pictures to test their (comparative) limitations.

So I want to take the same photos with each camera, load them on to my computer, and be able to say: "This is clearly better than that" or "There's not a lot to choose between them" (so it's a rough comparison). Also, in the absolute: "Never use the kiddie camera for photographs that ..." (to be honest, I suspect that that one could end after the fifth word).

Thus my question: What sort of photographs should I be taking for this experiment, and what sort of feature will each photograph test?

To add a sniff of motivation, I bought the kiddie camera for (surprisingly enough) one of my kids to get a bit of practice with taking photographs. But it is absolutely awful and seems to only take reasonable pictures with lots of light and the subject an exact distance from the lens. I recently picked up a (very) cheap compact with the intention of swapping it for the kiddie camera (with suitable admonitions that this one can't be dropped) but I'd like to test it myself before handing it over as replacing a rubbish camera with a rubbish camera won't get me those highly coveted Daddy Points. So in this particular test, I want to compare those two with (maybe) the Olympus providing some sort of "gold" standard (I'm aware that by the standards of this site, the standard provided by the Olympus won't be quite gold, but then iron pyrites is underrated in my opinion). But I'm also interested in the wider question of taking test shots myself to compare cameras to learn more about what each one is capable of (or rather, to learn what each one is capable of when I'm taking the photos - which is why I'm not interested in online comparisons of the cameras).

Answer 1

There's quite a lot of different photographic flaws that you can test for. For each of these tests you may get different results at the widest angle compared to the longest zoom.

1. Autofocus - Try setting up a high-contrast subject like a black & white test card and use autofocus from different distances while the camera is on a tripod to take shots. Check the sharpness of the results.
2. Vignetting - Use a plain card under even lighting and check how much difference there is between the light center and darker corners.
3. Distortion - take a straight-on shot of some graph paper, and measure how distorted the lines are at the edges or in the middle.
4. Auto White Balance - take shots of white paper in different lighting conditions (sunlight, cloudy skies, normal indoor lighting, tungsten lighting) and check how close it gets to white
5. Representation of colour - take shots of a standard test card which has both brightly saturated colours and flesh tones, and compare the results with what you like

There's also different capabilities that might restrict the choice of camera - e.g. macro, depth of field, flash, manual focus, etc.

Ultimately, testing this stuff yourself is interesting but might be hard work. For comparing specifications, there are a few different online solutions. For comparing test card type images there are fewer options as taking consistent shots is very difficult.

Answer 2

I would put it through a series of situations:

• Low light/Noise Handling - take a shot in relative darkness to see how the camera handles noise and a slow shutter speed.

• High contrast - take a shot with a big difference in light and shade to test the dynamic range.

• Detail - take a shot of something with fine detail to test the resolution and sharpness.

• Action - take a shot of something fast moving like a sports event to test how the camera handles it.

Of course, the final shot is only part of it. The Olympus might get the best shot, but if it's a pig to handle then is it worth it? Etc. etc.

Answer 3

I very big fan of the approach I took in Is the Panasonic Leica Summilux DG 25mm f/1.4 as good as the name implies?.

That is:

1. Make sure you have a long enough time to really get a feel, and get over first impressions. Issues which seem awful at first may be minor once you learn to adapt or compensate, and features which seem cool might not be useful in practice. A week minimum, and a month is better.
2. Take the gear out and use it as you normally would. Maybe a little more heavily, but nothing wrong with that.
3. Bring back your results and go over them every evening. Look at what you like and what you don't.
4. When you find specific things you have questions about, then you might take a few test shots. In my example, I used backlit leaves wide open to investigate purple fringing, and a simple real scene (while I was at the playground while my kids played) to test bokeh. If you're worried about sharpness or centering based on what you see in your real photos, then you might take some test shots.
5. With that knowledge in hand, go back out and make more real-world photographs, avoiding any problem areas and playing to the camera or lens's strengths. See if you're happier with the results.

If you are looking for specific technical things to check out, take a look at What image-quality characteristics make a lens good or bad?. For a camera body, you might look at JPEG processing and noise, but mostly that is about features and handling rather than tests.

Rigorous, scientific testing takes both the right setup and a degree of meticulousness that is hard for the average — or above average — photographer to do. My advice is to leave that to the review sites, where people have made that their profession.