4

I have two cameras: one is a "visible" camera and the other one is a thermographic camera, which, as many of you may know, it is a camera that captures infrared radiation instead of visible light.

I have to superimpose the images captured by these two cameras. The problem are:

  • they are fixed at two different positions (parallax correction needed)
  • they have different intrinsic parameters of course

The problem of adjusting the parallax is that I need to retrieve automatically the distance of each object. I want to avoid to need the distance (there are countless limitations...). For this reason, my question is the following:

If I place, fix and calibrate two "visible" cameras instead of one (same model), so that both "visible" cameras and the infrared camera are coplanar and vertically aligned, I can extract the intermediate view from the two visible images, emulating a view virtually positioned at the same location of the infrared camera. Do I avoid in this way the need of the distance information?

I hope the question is clear. Thanks.

|improve this question
2

It depends on your scene geometry.

If the object that you are observing is mostly flat (or if you are far enough from the main object of interest), then you can estimate an homography based on the position of that plane in two images. This technique is used for example for panoramic mosaicing or for stabilization.

The homography can be determined from the locations of 4 points in both images. This can be done manually of automatically, see for example this student project http://cs.brown.edu/courses/cs129/results/final/jcmace/.

|improve this answer
  • Thank you! I flagged it as best answer, because it is a good starting point. – user3099605 Jun 12 '14 at 12:52
  • You're welcome. Unfortunately, it's hard to be more specific right now, but if you come later with example images I'll be glad to help. – sansuiso Jun 13 '14 at 8:12
1

If you have a pair of visible cameras above and below you will be able to generate a view corresponding to the location of the thermal camera. The most straightforward way is by generating a depth map using stereo matching, texture mapping the depth image using one of the original images and then re-rendering it.

This approach comes with some drawbacks, the accuracy of the depth map increases the further the two camera are apart, however so does the change that parts of the scene will only be visible from one (or neither) camera, meaning depth information is missing, leaving you with gaps in the reconstruction. How much of a problem this is depends on the complexity of the scene.

You may find a mechanical solution of mounting the visible and thermal cameras as close as possible will suffice, then it's just a problem of cropping one of the images to match the other's field of view.

|improve this answer
  • Thank you for you answer. Regarding the first solution, you mean that I need the depth map anyway, in order to know how much to "warp" each pixel, depending on the distance? About the second solution, I can't position the two cameras too close unfortunately (around 5 cm is the best it is possible to do). So the parallax error would be an issue. – user3099605 Jun 5 '14 at 8:14
  • @user3099605 You don't need a depth map, there are novel view synthesis methods that generate the extra view directly, but if you're looking to put together solutions using widely available software libraries then the depth map approach will be more straightforward. It's still a lot of effort to go through, the beam splitter approach mentioned in the other answer might work better. – Matt Grum Jun 5 '14 at 9:37
  • How is it possible to synthesize a virtual view without a depth map? – user3099605 Jun 5 '14 at 11:05
  • @user3099605 I don't know how it works but there have been papers on the subject, essentially it's a similar process but you the new view is rendered without ever explicitly building a depth map. – Matt Grum Jun 5 '14 at 12:28
1

In 3D stereo, the two eyes are generally a couple centimeters apart. However the cameras used are far too large to actually place side-by-side.

One common solution is to use a glass block with a half-silvered surface at 45 degrees through the block. (A "beam-splitter".) This redirects half the light at 90 degrees, where the second camera is positioned

enter image description here

This won't address the intrinsic parameters such as distortion and focal length of course, but that's a one-time computation. Once you've made a correction for that, you can apply it any time you use this system.

A couple drawbacks:

  • You'll lose a stop of exposure, and need to compensate for that
  • There's often a polarizing effect. This probably won't be a big deal when the two cameras are capturing different kinds of light, but when shooting stereo it can give a shimmer to the scene.
|improve this answer
  • Thank you for your answer! The idea is very interesting, I considered too a similar solution, although from a mechanical point of view it might be slightly more complicated. Do you know about any devices/experiments that use this approach? Aren't there distorsions caused by refraction or scattering? – user3099605 Jun 5 '14 at 8:14
  • Well, some (almost all really) 3d stereo rigs use some variation on this. Because the light enters and exits perpendicular to the glass, you won't get any more distortion than if you were shooting through a window. Looking around for ready-made solutions, the closest I could find was a teleprompter rig, which can be found for a couple hundred dollars – mhlester Jun 5 '14 at 14:28
  • Please keep in mind that the transmittance of the beam splitter for infrared and for visible light might be totally different. This might go up to the point where the splitter could reflect most of the infrared while letting most of the visible light through (similar to a dichroic prism). – Dubu Jun 11 '14 at 9:15
  • @Dubu, that's a great point. So one location would probably be better for one camera, and the other better for the other. – mhlester Jun 11 '14 at 15:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.