We are two high school students working on sending a high altitude balloon to 35 km altitude in order to capture panoramic video of Earth. We are exploring the option of using Hugin to stitch our frames. However, we are encountering some issues with the panorama stitching process.
We have three cameras arranged in a triangle (facing 60 degrees away from each other), horizontally. Each camera has a fisheye lens (GoPro Hero 1080), allowing it to see 165 degrees in the horizontal direction. They are fixed together, and do not move relative to each other.
Our plan thus far has been to take control points from a calibration image we capture and apply them to each set of three frames.
We have so far been experimenting with stitching together frames from two cameras. So that we get a good spread of control points over the images, we have been tilting the cameras so that the same landmark can be used for control points in different places in the image.
We have been using a landmark approximately 100m away. We have taken three sets of frames with the landmark (a tall building) in the bottom, middle and top of the frame.
We stitched each set of frames separately, then created a project file with control points from all three positions together. Using the frames with the landmark at the bottom, the control points from when the landmark was in the other positions don't match the clouds they are over (see https://i.stack.imgur.com/Ked2R.jpg). The result of this seems acceptable (https://i.stack.imgur.com/lvX4F.jpg). Using the same control points with the set of frames with the landmark in the middle, the control points match the features they are over better (https://i.stack.imgur.com/DfOCV.jpg) but the result is not very well stitched (imgur.com/pC30Svd - see the displacement of the van).
Could someone please explain why this is happening, or suggest a way we could apply the same control points to each of our video frames, given that the cameras are in a fixed position in relation to each other?