0
\$\begingroup\$

The Plan

So I am trying to take two pictures, one with a DSLR and one with an iPhone 7. Why an iPhone 7? That's just what I own currently I want the photos to be aligned so that they are exactly the same. The camera and iPhone will mounted on a tripod using a stereoscopic tripod mount like this, the iPhone using something like this.

The idea is to place the iPhone and the DSLR on the tripod and take the photo, then algorithmically align the photos. I do have a few concerns because I am somewhat new to photography...

The Gear

This is for a research project so I can use cameras on loan from my university. Here are the camera & lens combos I have available with their specs:

  • Canon 80D w/ a Canon 18-135mm EFS Lens
  • Canon 70D w/ a Tamron 18-200mm Lens
  • Canon 80D w/ a Tamron 18-200mm Lens

And it says online that the iPhone 7 has the following specs:

Resolution: 12MP

Sensor: Sony Exmor RS (1.22 µm, 1/3")

Focal Length: 4mm (28mm eff) 8mm (56mm eff)

Max Aperture: F/1.8

The Concerns

So my question is, what variables do I have here that I need to keep in mind. The first one is horizontal displacement seen here in a crude mockup using stock photos. This I think will not be too much of an issue. My one concern is that parallax will make the overlapping part not actually be the same exact photo and there will be some noticeable difference. That would be a problem.

The second problem is the one I have no idea where to start with: lens distortion. This is due to aperture and focal length, right? Theoretically, if I manually set the iPhone's focal length & aperture to be the same as the DSLR's(which is easily done using photography apps), will the pictures turn out to be exactly the same?

How should I start going about doing this? These both are big problems that I really want to know how to address most easily. I want to take lots of photos side by side and I'm trying to figure out a workflow/setup that will make aligning the photos as easy and accurate as possible. I'm just looking for guidance so I can know where to start, what to buy, and how to set things up somewhat professionally.

Feel free to ask followups if I left any important details out.

Edit:

  • This is for a ML project where I am trying to use DSLR images as ground truth to improve iPhone images. It's a recreation of this research. What they use is a linear transformation and cropping, which is very similar to @xiota's answer (You are a smart person, I'm glad I got your help :))
\$\endgroup\$
11
  • 2
    \$\begingroup\$ What do you mean by "exact same"? I mean, the obvious answer is "no, this is impossible, because one photo will be taken with a DSLR and the other with an iPhone 7". Can you better describe your criteria for "exact same"? \$\endgroup\$
    – mattdm
    Sep 17, 2018 at 0:13
  • \$\begingroup\$ @mattdm Yeah you're right, I'll include more info in an edit to the post, but basically, my criteria is that "lighting and picture quality excluded, everything in the photos must be in the exact same location on a pixel level" \$\endgroup\$
    – JoeVictor
    Sep 17, 2018 at 0:15
  • 2
    \$\begingroup\$ Setting aside whether or not you can match all the other variables, Can they truly be identical if they’re taken side-by-side? Wouldn’t the aperture of both lenses need to be in exactly the same position and not side-by-side.? This would mean you would have to take one photograph with the DSLR and then replace it on the tripod with the iPhone so that the aperture/lens position matched exactly where the DSLR aperture/lens was. \$\endgroup\$
    – Alaska Man
    Sep 17, 2018 at 0:22
  • \$\begingroup\$ @Alaskaman, that is my concern too. I was hoping for some clarifications on what variables affect the photos. I think lens position is fixable, but aperture will need to be manually set to be the same on both devices, which is doable. \$\endgroup\$
    – JoeVictor
    Sep 17, 2018 at 0:41
  • 3
    \$\begingroup\$ Even with the same camera, on a tripod, it can be difficult to get photos to be "exactly the same" "on a pixel level". It would be helpful to know the purpose of the project to have a better idea of the tolerance for differences. \$\endgroup\$
    – xiota
    Sep 17, 2018 at 1:26

2 Answers 2

7
\$\begingroup\$

Garbage-In, Garbage-Out

A big problem with using DSLR photos as "ground truth" for the improvement of cell-phone photos is the assumption that DSLR photos are necessarily better than cell phone photos.

  • It is possible to make a DSLR photo that closely matches a phone photo by closely matching the FOV and DOF.

  • The lenses you will be using have relatively slow F3.5-6.3 apertures, which will limit the background blur you can achieve at any given focal length.

  • With Canon cameras, I often have difficulty getting well-exposed shadows without blowing-out the highlights.

  • Different camera settings can give very different looks.

Another issue is that iPhones are considered to already produce very good images, so the amount of room for improvement isn't as much as with other phones.

Some Neural Net Training Ideas

These are just ideas. They may not actually work.

  • Train mini neural networks to improve specific aspects of an image. Noise, resolution, color... can each be trained in separate subnets, which are later combined.

  • The subnets don't necessarily need to output an image. An exposure-correcting subnet could return exposure correction settings. A color correction subnet could return adjustment curves.

  • For some of the subnets, it may not be necessary to train with images taken on different cameras.

    • To increase image resolution, use a high-res DSLR image along with a down-sampled version.

    • To work on DOF, use DSLR images of the same scene and location with different aperture settings (calculated to match that of the phone).

    • To correct exposure, use images captured on the same device, but with different exposure settings.

    • To reduce noise, use ISO 100 vs other ISO settings taken on the same device. (To correctly capture the phone's noise characteristics, probably best to use the phone. Exif data can be used to select the appropriately trained subnet for processing.)

  • Some features, such as color correction, may not need an exact match because the image will be preprocessed (eg, blurring). The match only needs to be good enough. Then pre-existing tools can do the rest (align_image_stack).

  • I'd also suggest training some subnets to improve flare and glare because they are particularly difficult to correct. Use images taken with and without lens hoods for training.


The following was written before I knew that the purpose of the images is to be used as neural network training data to improve phone photos. Although some of it has been revealed to be counter-productive, I am keeping it because it answers the question as I understood it to be asked.

Factors that Contribute to Image Differences

Obtaining two photos that are "exactly the same" "on a pixel level" with two different cameras and lenses is practically impossible for the following reasons:

  • Mismatched output dimensions makes producing pixel-exact images impossible.

  • Mismatched sensor ratios makes producing pixel-exact images impossible.

  • Other mismatched sensor characteristics, such as light sensitivity and color response.

  • Mismatched camera settings, such as demosaicing algorithms and color profiles.

  • Even with the same camera and lens on a tripod, it can be difficult to get photos to match with such precision. Even the slightest tap can cause images to be different "on a pixel level" because sensels are only about 5 microns. (Larger for full-frame cameras. Smaller for cell-phone cameras.)

  • There may be movement within the scene, such as leaves and branches swaying in the wind.

  • Any displacement of the nodal point would result in different images. The dual-camera tripod head will not work.

  • Lens field-of-view would need to match. (On DSLR, use phone focal length * crop factor.) Zoom on the DSLR lenses is adjusted manually. It would be difficult to consistently make pixel-precise adjustments manually.

  • "Equivalent" apertures of the lenses would need to match. (On the DSLR, use the the phone aperture * crop factor.) Often apertures move to preset levels within a margin of error. The exact values you need may not be available, or the margin of error may be greater than what you need to obtain pixel-level precision.

  • The different lenses are likely to have mismatched distortion characteristics, which may include barrel, pincushion, and mustache types. See also:

  • The different lenses may have mismatched chromatic aberration.

Approach that Replicates the Phone Images on the DSLR

To get images to match as closely as possible, you can try the following:

  1. Normalize the size and ratio of the images. Perhaps by using a center crop with the same pixel dimensions across all cameras.

  2. Calculate the crop factor for the normalized images.

  3. Take a photo with the iPhone.

  4. Use high-precision motors with a feedback loop to place the DSLR camera into position, with checks and adjustments against the iPhone image.

  5. Use high-precision motors with a feedback loop to set DSLR focal length and aperture, again checking and adjusting against the iPhone image.

  6. Take photo on DSLR with exposure settings to match iPhone.

  7. Adjust contrast, brightness, etc of DSLR image to match iPhone image.

  8. Align and correct the images with align_image_stack or similar software.

  9. Results may still not be pixel-level identical.

\$\endgroup\$
3
  • \$\begingroup\$ This is for a research project that tries to recreate the work of this paper. The tolerance is somewhat fuzzy, but again, the less differences the better. Thoughts? \$\endgroup\$
    – JoeVictor
    Sep 17, 2018 at 14:24
  • \$\begingroup\$ Would be interested to see your results, if you don't mind linking to it when you finish. \$\endgroup\$
    – xiota
    Sep 18, 2018 at 23:16
  • \$\begingroup\$ @QuantumHoneybees - I'd totally forgotten about writing this... How did your project go? \$\endgroup\$
    – xiota
    May 14, 2019 at 23:42
3
\$\begingroup\$

You can not.

You could overcome the displacement of the two lenses using either a prism or a mirror so you have the same position and split the beam of light.

You now then need to decide how to crop for the two sensors. You could probably need to strip the phone and camera from the lenses and use a lens before the light enters the prism so you avoid chromatic aberrations and distortions of the lenses differences, and then use some special optics to focus on the different sensor size and distances and sensitivities.

But even the same brand and model of cameras are made of different silicon plates. They could have the same structure, but they have molecular differences on the surfaces and micro dimensions.

Probably not even the same camera, on a tripod, on a studio controlled environment can take the exact same two pictures because of this quantum level noise. Ok, probably yes, but your example is not the case.


I do not understand the reason to take the same shot with two different cameras.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.