For a school project, I am capturing images from two cameras mounted side by side, one NIR and one RGB. Basically I want to make a simple multispectral camera so I need to align the two images on top of each other. The camera's are closely mounted but of course they have slightly different reference points so the images need alignment in post processing. Currently I am doing that using Photoshop's Edit→Auto-Align Layers function which works perfectly but I want an open source solution. I discovered the utility align_image_stack that seems to be want I need but when I run it on two images they are not very well aligned. I've tried setting the -c option higher (200) and -g option with no better luck. Does anyone have some pointers or maybe another tool I should try?

  • 2
    What are you capturing? That is, is your subject far away (like stars), or is your subject somewhat near (within the room)? – scottbb Oct 28 '19 at 20:05

Depending on what you're focusing on (how close the subjects are from the cameras), you're experiencing the unavoidable effects of parallax due to capturing images from two different perspectives.

In photography, perspective is the conceptual point of view of the camera — the actual position of the entrance pupil (apparent location of the iris or aperture) of the camera's lens. By having two separate cameras side-by-side, you're creating a stereoscopic imaging system. Normally, stereoscopic imaging systems use two identical cameras side-by-side, in order to create 3-dimensional image information, much like a human or animal's two eyes create 3D image space.

In your case, one of your systems "eyes" is an RGB sensor, and the other "eye" is a NIR sensor. Let's say the RGB camera is the left "eye", and the NIR is the right "eye". In that case, imagine closing or covering your right eye. The left eye will see RGB data from its perspective or point of view. When you switch eyes and only look through the right "eye", you can only see NIR data, and only from the perspective of where the right eye is.

If your subject is very far away (i.e., several orders of magnitude further away compared to the distance between the left and right "eyes" of the imaging system), then the parallax distortion is minimal, possibly even negligible or imperceptible. For instance, it doesn't matter if you look at the night sky with both eyes or just with a single eye — the planets and stars are far enough away, that the parallax error between the human eyes (~ 2.5" / 6.5 cm) will not show a difference between the left and right eyes.

However, if the subject is close enough that no matter how you overlay the two images that you can't get them to align correctly, then you are in the parallax error regime. Hold your phone about 10 cm / 2' from your face. Without moving your head, look at it with your left eye while closing the right. Then swap eyes, closing the left eye and look with your right. Keep swapping back and forth, while focusing on the phone. Notice how the background keeps jumping each time you swap your eyes. This is parallax error in the near regime. This is the error you are seeing when using side-by-side cameras.

There is nothing you can do to "fix" parallax error, without "fudging" or editing the image data. In essence, you can massage the data to make a closest fit, but your are literally lying about the data in order to fit the system. This comes down to the fact that the entrance pupils of the lens used for each image were at different points in space. Two objects cannot occupy the same space at the same time.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.