Panoramic camera and length measurements

Question

First of all, I don't have much experience with photography. If I ask something stupid, please accept my excuses.

I'm a self-taught amateur OpenGL programmer. In my job, we have a 360º-panoramic viewer that displays a panorama as a cube-map taken with a Ladybug camera ( composed by 6 sub-cameras, each one will produce one side of the cube ).

Now, I've been asked to check whether we could measure real-world lengths in the panorama. For example, if in a room we have a 3 meter long table, then measuring the table in the photo would tell us it's 3 meter long.

I can retrieve the mouse cursor position in model coordinates, but I can't relate them to real-world coordinates. Using my previous table example:

Table width goes from (0,0) to (3,0)    -> 3 meters
Model width goes from (0,0) to (0.356)  -> 0.356 unit-agnostic ( pixels? mm? )

I can get the FOV of the panorama viewer and information regarding the clipping planes and any transformations applied in the viewer (like rotations of the model).

Moreover, for every Ladybug shot (which translates in 6 pictures) I have the camer'as positional information:

Camera's Origin vector at the time of the shot (in UTM coords).
Camera's Direction vector at the time of the shot.
Camera's Up vector at the time of the shot (in UTM coords).
Camera's Roll, Pitch and Yaw values at the time of the shot.

I've been researching through this network and most likely I'll also need the camera's focal-length and sensor size, in order to retrieve the camera's distance to the model, but I'm still researching this topic.

So:

• Is it even possible to measure lengths in a picture/panorama?

• If it is possible, where can I find more information about this process? I'm not asking for a ready-to-go solution but for guidance in this topic, as I lack the proper background to relate one variable with another.

Answer 1

The pano image is a 2-D projection of a 3-D space. As such, you cannot, directly, calculate object sizes from the image. At least, you'll need the object's distance (depth) from the camera. Knowing that, and your FoV, you can estimate the object's dimension perpendicular to the line-of-sight.

Practically, since the projection through the lens is not perfect, images of straight lines in space do not result straight lines on the image plan (i.e., the transformation is not affine). Theoretically, you could use this knowledge to some extent, to get a rough size assessment.

---

Update:

You need some kind of a reference to your measurements. If you want to measure the size of an isolated object, then the distance from camera and the FoV are required. For example, if your FoV is 90 degrees, the image size is 1000 pixels and the object measures 100 pixels, then you can calculate the object's angle-of-view is Am. Once you know it, and the distance of Dm meters, the actual Sm size is:

Am = 90 * 100 / 1000 = 9 deg    -- Angle of View
(Sm / 2) / Dm = tan(Am / 2)     -- the trigonometric equation
Sm = 2 * Dm * tan(Am / 2)       -- object's size

As you can see, the one measurement that you have from your image is the Angle-of-View (Am) of the object you are measuring. It is easy to see that an object of size 1 meter, at 3 meters away has the same image as an object of 2 meters at 6 meters away. This is why the distance to camera is crucial.

_{Note that the size calculated is only the perpendicular size!. We can't measure the radial size unless more information is given.}

Your second option is using the known properties of a reference object. In your question, you mentioned a known size of a table, and a model size needs to be measured. If you know the relative positioning of the two with respect to the camera, i.e., how far is your model from the table (to make it simple, let's assume they are roughly on the same line-of-sight), then you can use that information to calculate the model's size.

For example, table is Ht = 3 meters wide, measuring St = 3 pixels. Model is Sm = 0.356 pixels, located dDm = 2 meters behind the table. Then, you can do:

At = 90 * 3 / 1000             -- table's angle of view
Dt = (Ht / 2) / tan(At / 2)    -- table's distance
Dm = Dt + dDm                  -- model's distance
Hm = Dm / Dt * Ht              -- model's size

If the model is not on the same LoS as the reference object, then dDm should be the radial difference in the distance from camera.

This is all very basic trigonometry, so if you are unfamiliar with the field, you can start your reading there.

I can imagine that in some research literature you will find that someone developed an esoteric method for assessing distance based on an object's "fuzziness" in the image, caused by the camera's limited Depth-of-Field (DoF), or something like that. Usually these cases work for a very specific and specialized equipment, that was precisely calibrated for taking the measurements.

Finally - if you can use two nearby cameras, or take two consecutive shots from nearby locations, you can generate a stereo-image which lets you estimate the object's distance from the object's images disparity (search for "generating a depth field from a stereo image").