7
\$\begingroup\$

We are two high school students working on sending a high altitude balloon to 35 km altitude in order to capture panoramic video of Earth. We are exploring the option of using Hugin to stitch our frames. However, we are encountering some issues with the panorama stitching process.

We have three cameras arranged in a triangle (facing 60 degrees away from each other), horizontally. Each camera has a fisheye lens (GoPro Hero 1080), allowing it to see 165 degrees in the horizontal direction. They are fixed together, and do not move relative to each other.

Our plan thus far has been to take control points from a calibration image we capture and apply them to each set of three frames.

We have so far been experimenting with stitching together frames from two cameras. So that we get a good spread of control points over the images, we have been tilting the cameras so that the same landmark can be used for control points in different places in the image.

We have been using a landmark approximately 100m away. We have taken three sets of frames with the landmark (a tall building) in the bottom, middle and top of the frame.

With the building in the bottom, the two frames we used are at: https://i.sstatic.net/AJIk8.jpg https://i.sstatic.net/rirIV.jpg

With it in the middle, the frames we used were https://i.sstatic.net/uaPLD.jpg and https://i.sstatic.net/x165I.jpg

With it at the top, we used https://i.sstatic.net/MuARn.jpg and https://i.sstatic.net/8MGSw.jpg

We stitched each set of frames separately, then created a project file with control points from all three positions together. Using the frames with the landmark at the bottom, the control points from when the landmark was in the other positions don't match the clouds they are over (see https://i.sstatic.net/Ked2R.jpg). The result of this seems acceptable (https://i.sstatic.net/lvX4F.jpg). Using the same control points with the set of frames with the landmark in the middle, the control points match the features they are over better (https://i.sstatic.net/DfOCV.jpg) but the result is not very well stitched (imgur.com/pC30Svd - see the displacement of the van).

Could someone please explain why this is happening, or suggest a way we could apply the same control points to each of our video frames, given that the cameras are in a fixed position in relation to each other?

Improve this question
\$\endgroup\$

3 Answers 3

Reset to default
4
\$\begingroup\$

It seems that this is what the template feature is for.

How can I reuse a project as a template?
If you copy a .pto project to a different folder and open it with hugin, you will be prompted for the 'missing' images. You should delete any control points from this template project since they won't be relevant to the new photos.
Alternatively you can load your images as normal, then Apply template from the File menu, this will import image settings and parameters from a previous project.

http://wiki.panotools.org/Hugin_FAQ#How_can_I_reuse_a_project_as_a_template.3F

Before the template feature was available I think people would edit the .PTO file in a text editor to replace the images with new ones.

Improve this answer
\$\endgroup\$
1
\$\begingroup\$

You don't actually want to apply the same control points to each set of frames, you want to apply the same positions & geometry offsets. These are optimised from the control points you generate/create.

You may find you can use the command-line tools directly more effectively than loading images in Hugin, once you've done the initial photometric & geometric optimisations for one set of images.

This page has the approximate Hugin steps in command-line form.

Basically once you have the .pto file for one set of images, you can re-use it for your other images. You could do this very simply by putting each set of matching images in a folder, A.jpg, B.jpg, C.jpg, alongside the .pto project file. Then it's just a matter of doing the final step of generating the TIFFs and blending them together, on each of your folders of 3 images + project file.

If you know how to use a scripting language of some sort, this may be much faster than using the Hugin GUI. But you could easily use the same technique with Hugin itself, so long as the files are always named the same and you don't re-optimise/re-align the images after loading.

Improve this answer
\$\endgroup\$
0
\$\begingroup\$

The basic problem with trying to stitch images taken with such wide angle lenses is distortion. 160º in the horizontal direction is an equivalent angle of view (AoV) of about a 10mm lens on a 35mm camera. This is in fish-eye territory. Even with rectilinear lenses, stitched panoramas don't start lining up well without a lot of manipulation until around 50mm on a full frame camera, which is about a 40º horizontal AoV.

Improve this answer
\$\endgroup\$
5
  • 1
    \$\begingroup\$ Thanks for your reply. The problem we're having is not that we can't stitch the images we capture, but that we can't apply the same control points to a different set of images. Is there something you could suggest? \$\endgroup\$
    – XYZ791
    Commented Jun 25, 2013 at 11:54
  • \$\begingroup\$ Each set of images with that much distortion will need to be individually matched based on the content of those images. If the most critical parts that need to match up are in one part of the frames in one set of images and in another part of the frames in another, the control points will also need to be different. \$\endgroup\$
    – Michael C
    Commented Jun 25, 2013 at 22:08
  • 1
    \$\begingroup\$ @MichaelClark Why? The transformation to correct barrel distortion is the same regardless of image content, and the cameras are locked in position so there should be no problems in calibrating the rig once and then applying the same control points to all the resultant panoramas. I have no idea why you think "stitched panoramas don't start lining up well without a lot of manipulation until around 50mm" since it's common practice to shoot 360x180 degree panoramas fisheye lenses... \$\endgroup\$
    – Matt Grum
    Commented Jun 26, 2013 at 12:07
  • \$\begingroup\$ I've never used Hugin, but what panoramic stitching I've done seems to indicate that the stitching application is content aware and will attempt to line up distinct features more closely than areas that are less distinct, such as clouds or grass or walls painted a uniform color. In the examples included in the question, it is pretty obvious the control points are not over the same spots on the clouds in the two images in this one: imgur.com/wMKNtfB \$\endgroup\$
    – Michael C
    Commented Jun 26, 2013 at 13:29
  • \$\begingroup\$ And even a casual glance at the result imgur.com/ieKzmFD is fairly obvious the clouds are not aligned with each other where the two images merge. \$\endgroup\$
    – Michael C
    Commented Jun 26, 2013 at 13:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.