How to perform a proper lens comparison test?

Question

I already have a 50/1.4 lens and plan to buying a more expensive version of this lens which only just came out and still not available in most parts of the world. Since this is a brand new lens, there is a large online community that is eager to know how these two lens compare. What is a proper way to do lens comparison? What tools would I need? And is it possible to do it at home?

Should I be printing out test charts? If so where? And what about lighting conditions, would it matter? What are some comparison shots I should prepare? Center vs corner and others?

Answer 1

You can look at DxO Labs description of their testing protocol. Be sure to click on next at the bottom of each page to read the entire document.

As the above link indicates, for testing to be meaningful it must take place in strictly controlled and reproducible conditions. This includes seemingly insignificant things such as temperature and humidity, as well as more obvious factors such as lighting that can be uniformly measured. There are standard test charts, such as ISO12233 targets, but the results are only meaningful if they are used under uniform lighting that conforms with the test standards as set out by the International Organization for Standardization (ISO).

Answer 2

My suggestion is to avoid test charts. For these to be really meaningful, conditions have to be carefully controlled, and when comparing models of lenses generally, it helps to have multiple copies of each lens.

When the technical reviews from professional or dedicated amateur technical reviewers come out, people can start obsessing over the numbers. I think a better thing to do is look for situations which demonstrate the general, practical performance of each lens and take comparison shots.

See What image-quality characteristics make a lens good or bad? for ideas of things to look for and situations to look for them in.