6

I have an image data set consisting of pairs of JPG and RAW files. I was tasked with transforming JPG pixel coordinates (x, y) to RAW pixel coordinates (x_raw, y_raw). By comparing RAW and JPG images visually, it is clear that the device undistorts the images at some point of the image processing pipeline (RAW->JPG). Now, to undistort an image, you basically need a camera matrix and parameters of distortion (e.g. radial) that you get by calibration. This means that the device must store these parameters somehow.

My question is: how do cameras perform undistortion and how are calibration parameters derived?

Does each camera sensor get calibrated during the manufacturing process (and the parameters are written to memory of the device)? Or, perhaps, camera sensors of the same model store the same parameters that were derived by averaging parameters of many sensors?

4
  • 1
    It appears you want to know how cameras perform lens correction so you can perform the inverse. Since camera manufacturers tend to not disclose their algothms, the parameters are not generally useful, even if you are able to obtain them.
    – xiota
    Oct 5 '20 at 21:42
  • What is the purpose of mapping corrected JPG coordinates to uncorrected RAW coordinates?
    – xiota
    Oct 5 '20 at 21:45
  • 2
    How precise do you need it to be? What's the ultimate goal and the real problem you are trying to solve? This might be a case of an XY problem where there might be a better solution to the overall problem than to try to address this specific one.
    – jcaron
    Oct 6 '20 at 11:08
  • You'll find a complete description in the document linked from this answer: photo.stackexchange.com/a/72792
    – Szabolcs
    Oct 6 '20 at 14:18
17

Distortion is produced by the lens, not the sensor. So the distortion correction you describe is also referred to as lens correction.

There are different distortion models (equations). The parameters depend on the specific distortion models used. Different cameras and software may use different distortion models. Camera manufacturers tend to not reveal the distortion models they use. So even if you are able to extract the parameters, they would likely be of little use.

The correction parameters may be taken from a lens model (more equations) that describes the behavior of specific lenses. Or they may be found empirically, by photographing scenes and making measurements. Usually a representative copy of a lens is used. (This appears to be how Adobe and lensfun databases are built.) Results are good enough, and averaging parameters from multiple lenses does not necessarily produce desirable results.

The distortion models used by Hugin are used to perform lens correction in open-source software (eg, lensfun). Adobe appears to use different models. (See What model does Lightroom use for lens correction?)

To solve your problem (the inverse of lens correction), you should be able to use Hugin to calculate the transformation. Another option is to manually map every coordinate, but that is likely to be time consuming and unnecessary for most photographic purposes.

4
  • 1
    _"you should be able to use Hugin"_—but this won't necessarily reproduce the transformation done by the in-camera processing, so be careful.
    – Ruslan
    Oct 5 '20 at 19:41
  • Why “map every single pixel”? A dozen pixels should be enough to get a quite decent distortion model. Oct 6 '20 at 8:42
  • No. If the model is not good enough, you refine it. IIRC, Hugin's model has 3 parameters. Refining is as simple as adding terms to a polynomial. If you can't get a good fit with 5 or 6 parameters, you are probably doing something very wrong. Oct 6 '20 at 18:14
  • @EdgarBonet I'm not telling OP to map every pixel. For photography, Hugin is more than good enough, but OP has not specified how accurate the mapping needs to be. There is always error in the model, which Hugin reports. If OP's tolerance for error is 0, without knowing the camera distortion model, the alternative is mapping every pixel.
    – xiota
    Oct 6 '20 at 18:19
4

There are different types of distortion but they are always due to the lens. Geometric distortion is often corrected by image processing in the camera to compensate for compromises done during lens design. Those are known at design time for the construction of the lens and usually not by measurements on each individual lens which is why there will often be slight residual distortion, meaning that the optics might have 4% barrel distortion but the corrected image still has 0.2%, for example.

Many cameras have controls enable or disable certain correction. Often Geometric, Vignetting and Chromatic Aberrations. While I usually leave the later two on, I always turn off Geometric because correcting for this has an impact on framing. When the image is undistorted, it will produce a non-rectangular view which gets cropped so that it results in a rectangular image. If you are trying to remove the undistortion, you will be missing parts of the image!

The distortion data is often stored in the camera for each lens. For this reason, often when a new modern lens comes is released, camera makers often release a firmware to support the new lens. The lens will still work without that firmware in most cases but there will be no corrections. There are a few cases where the lens does not work but those are rare and have nothing to do with distortion, usually it is because a new feature are type of focus, control, aperture, etc has introduced.

On modern electronic platforms, it is possible that some lenses could store their own data to avoid having a F/W update needed when a new lens is introduces but that requires more communication between lens and body.

0

For my Canon 6D I have to choose which lens correction data to download into the camera, for correcting the jpg data. I can download the same data into Photo Professional, for correcting the raw data. I depend on the availability of lens data. Canon only... :-(

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.