Approximating raw image data from app-less iPhone6 by reverse processing using meta data?

Question

note: To preserve @MichaelClark's substantial answer below, I've volunteered to leave this question here and allow it to be closed, instead of just deleting it (see discussion below the answer). That means I'm volunteering to eat the downvotes, so please take a moment to consider it before casting one. Thanks!

---

Is it possible to reconstruct an approximation of the original raw sensor data from downloaded iPhone 6 images by interpreting the meta-data and mathematically undoing whatever was done in the phone?

I understand this is not the best way to proceed, but I would like to keep my phone app-less, and I believe at this time I can not access the raw data in the phone without a 3rd party app.

I generally use Python for everything I do, but if there is free software I could consider it. If the math is out there I can write my own script as well. I'm looking to understand what I'm doing more than finding a quick solution.

If I see something interesting or useful with this, I'll bring in a DSLR later.

What I'm planning to do:

I would like to look at relatively small color differences between two regions of an image, and see if the shift increases or decreases between one photo and another. For example, in each image I'll define two rectangles, then I'll just calculate r = R/(R+G+B) where R, G and B are each integrated within the rectangle. So r1 and r2 might be 0.31 and 0.32, and I'll say there's a shift of 0.01.

The images are fairly flat - white paper reflecting ambient light. I'll maintain the same distance and angle. (note: I'm only asking here about the data processing step so I'm only loosely describing this to better give an idea how I'll use the data, not how I'll interpret it.)

I'll do the same for the next photo, and see if the shift is larger or smaller. This is "science stuff" but not serious science stuff and I understand this is not the best way to proceed.

Answer 1

Nope. The reason is that all of the raw data is not represented in any single interpretation of that data used to produce a JPEG image. Much of the raw data has been discarded and can not be recovered.

When you open a raw image file with an application what you see on the screen is not the unfiltered, raw data. It is one interpretation of the raw data based on the default settings of the application doing the rendering. If the settings are changed, the application will go back and recompute the image using the raw data to produce another, different interpretation of the same raw data. But the full set of the raw data is not represented in either of the two interpretations. Only some of the information in the raw file will be used for each rendering.

(Yes, it is theoretically possible to produce an image with such a limited range of brightness and color that the information in the raw file is so uniform from one pixel to the next that it can all be represented in an 8-bit image. But who wants solid pure white or solid black photos? When taking a photo of a scene that contains areas of differing brightness, colors, and textures the total data in the raw file almost certainly exceeds the capability of an 8-bit display or 8-bit image format.)

If there is any change in brightness or color from one pixel to the next, the gradations will be much finer/smoother with a raw file than with a JPEG. 8-bit jpegs only allow for 256 discrete values for each color between totally dark and totally bright. A 12-bit raw file allows for 4,096 discrete values per color. A 14-bit raw file allows for 16,384.

If the values along a line of pixels increase by one for each successive pixel as we move from left to right of a 14 bit file then there would be 64 discrete values for a line 64 pixels long. But there would only be one change in value of one unit along the same line of 64 pixels with an 8-bit file. Now let's say we have another line of 64 pixels in which the values in the 14-bit file are more irregular. They go up and down as we move right to left, but they all stay within those same 64 discrete values as the first line. Once converted to 8-bits we wouldn't be able to tell the difference between the first line and the second. There are 64 possible values in the 14-bit raw file that all transform to the same value in the 8-bit jpeg. Now multiply those 64 possible values by every pixel in the image. That is how many possible different versions of a 14-bit raw file would be represented by the exact same 8-bit jpeg. If you only have a 1 MP image, that is 64 raised to the power of 1M (64^1,000,000)!

For more about what kind of information is contained in a raw file, please see: RAW files store 3 colors per pixel, or only one?

Answer 2

As Michael Clark has explained in detail, you cannot reconstruct the raw file because the mapping from the raw file to the JPEG is not one to one, there are many different RAW files that could have been processed into exactly the same JPEG file. However, this does not mean that all the possible raw files that are consistent with some given JPEG file are equally likely. It is possible to reconstruct the most likely raw file given a JPEG image, this requires doing computations that are quite similar to doing state of the art noise reduction computations where you want to eliminate the noise in a picture by calculation the most likely noise-free picture. Such methods are enormously computationally expensive (many hours or even days on a fast computer to process a single picture), but they usually do yield much better results than the usual ad hoc methods.

In case of JPEG, what you should try first is to simply decompress a JPEG better than using the decompression algorithm. A naive JPEG decompression does not yield the most likely image that upon JPEG decompression yielded the given JPEG file, obviously not, otherwise there would be no JPEG artifacts visible.