8

While looking up 360° photos I found a lack of information about how the images actually work, I found some info suggesting that the images are stored as a regular jpeg but I was unable to find the finner details.

The questions I have are:

  • How are 360° photos stored? Do they have a special file format or are they projected onto a regular png/jpeg?
  • If they are projected onto a flat image will they lose quality in some areas where the spherical projection has warped the image?
  • Are there any non standard ways for storing and processing 360° photos which work differently to the common methods?
6

There are no finer details.

Let us start from the beginning.

A file

You have a camera with a lens and the file stores what was projected on the sensor.

You can change the lens, for example to a wide angle lens and the file just do the same. No black magic there.

You can then store an image taken with a fisheye lens. The file itself has no idea and stores the image exactly the same way.

A file format has no way to differentiate what is stored in it.


360 views

Ok. Now let's take a look at some types of "360" images.

A 360 image for start is one that contains information on what is around you. Behind you, in front, to the left, to the right.

The most primitive way to capture this information is by taking a photo of a metallic reflective sphere. This contains information about (almost) all your surroundings.

360 images from a spherical reflective ball

This is a "Probe Map" projection. But there are several others. Cubic projection, Spherical projection, Cylindrical projection.

What now has popularized these images is the ability to compute these transformations and unwrap them again.

So this simple Jpg with a distorted projection is projected into a 3D "solid" from the inside and then you see a section of it, and you can move around.

This solid can be a sphere or a cube, depending on the application. (Cube is the most common)

Cubic projection


Here is a link to a simple paper I wrote some years ago about the basics of different projections for environment maps.


The viewing

The trick is that we have now software that can interact with this projection, either on a 3D rendering software for a video or animation, or in real-time like a video game, a VR application, or a simple panorama on a website.

6
  • Thanks for the answer! with the solid, does it make a difference if you use cube or sphere? I'm guessing you cant see the corners on the cube? – Qwertie May 23 '17 at 4:50
  • Rendering a sphere takes more processing power that rendering a cube. A cube has just 6 faces, a "sphere" can have hundreds. But a cubic projection is different that a spherical one so they need to match. Ill post a link to some examples. – Rafael May 23 '17 at 4:54
  • Note that when using a spherical projection you run into a lot of inconvenient math at the poles. The pixels that belong at the poles get stretched out to the entire width of the image, for example. With cube faces, you have 6 different images (possibly combined into a single image), but no annoying mathematical issues. – user1118321 May 23 '17 at 5:13
  • Why to stick to something as mainstream as spherical projection. You can project on any surface. Analytical geometry can lead to wonderful distortion and serious headaches... – Crowley May 23 '17 at 11:18
  • Re, "A file format has no way to differentiate what is stored in it." What does that mean? Are you trying to say that 2D image file formats have no general way to tell what the 2D image represents? Many image, video, and audio file formats contain metadata that potentially tells quite a lot about what is stored in the file. But, maybe you are saying that there is no established standard for metadata that says that this 2D image is a certain type of projection of a full-sphere image onto a flat rectangle, etc. – Solomon Slow Feb 23 at 21:54
8

How are 360° photos stored? Do they have a special file format or are they projected onto a regular png/jpeg?

360x180 panos are stored a number of different ways, but most commonly as a single image in the usual visual file formats (TIFF, JPEG, PNG) in equirectangular projection. Equirectangular projection represents the sphere as a 2x1 rectangle, where the cartesian X-Y coordinates correspond to the longitude and latitude of the point on the sphere. As a result, there's a lot of warping around the poles.

When displayed interactively, that equirectangular is most commonly used as the basis for either HTML5 or Flash output; the older Apple QuicktimeVR format was another standard, which used a cubemap rather than an equirectangular projection as a basis. Cubemaps most common are done as a 3x2 grid of the faces, or as a cross layout (e.g., Blender skyboxes and environment maps).

The Google cardboard format is two stacked equirectangulars for 3-d; left-eye view over right.

If they are projected onto a flat image will they lose quality in some areas where the spherical projection has warped the image?

Depends on how you plan to view and rewarp to another projection.

Are there any non standard ways for storing and processing 360° photos which work differently to the common methods?

Tons. There is no standard. And remapping is kind of a sport among those who do 360x180s. Cartographers have figured out a lot of ways to represent a sphere on a flat surface. :) Just looking at the list of Flexify mappings should give a sense of how many possible mappings there are. Some common ones you'll see are stereographic little planets/tunnels. But sky's kind of the limit. I like using drostified quincuncial when I really wanna screw with peoples' heads.

Mathmap Quincuncial scrip with drostify set

See also: the tag.

1

A lot of these images that are made for commercial purposes, e.g., real estate walk throughs, are either cylindrical or spherical projections, and the method will depend on the requirements of the hosting environment. Many of the online hosting services have propriatary software, and have slightly different requirements. Whether a cylindrical or spherical projection image is chosen depends on the marketing budget for a particular property. A spherical projection image requires triple the number of individual (pre-stitched) images as a cylindrical projection image, and so costs more to produce. The answer mentioning other (non-typical) projections is spot-on. It would be only a matter of working out the mathematics. Virtually any 3d space would be possible once the conversion algorithm is determined. (A projection on a Klein bottle would be an example that might prove more difficult!) Any Euclidean 3d space should be rather straightforward.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.