2

note: To preserve @MichaelClark's substantial answer below, I've volunteered to leave this question here and allow it to be closed, instead of just deleting it (see discussion below the answer). That means I'm volunteering to eat the downvotes, so please take a moment to consider it before casting one. Thanks!


Is it possible to reconstruct an approximation of the original raw sensor data from downloaded iPhone 6 images by interpreting the meta-data and mathematically undoing whatever was done in the phone?

I understand this is not the best way to proceed, but I would like to keep my phone app-less, and I believe at this time I can not access the raw data in the phone without a 3rd party app.

I generally use Python for everything I do, but if there is free software I could consider it. If the math is out there I can write my own script as well. I'm looking to understand what I'm doing more than finding a quick solution.

If I see something interesting or useful with this, I'll bring in a DSLR later.

What I'm planning to do:

I would like to look at relatively small color differences between two regions of an image, and see if the shift increases or decreases between one photo and another. For example, in each image I'll define two rectangles, then I'll just calculate r = R/(R+G+B) where R, G and B are each integrated within the rectangle. So r1 and r2 might be 0.31 and 0.32, and I'll say there's a shift of 0.01.

The images are fairly flat - white paper reflecting ambient light. I'll maintain the same distance and angle. (note: I'm only asking here about the data processing step so I'm only loosely describing this to better give an idea how I'll use the data, not how I'll interpret it.)

I'll do the same for the next photo, and see if the shift is larger or smaller. This is "science stuff" but not serious science stuff and I understand this is not the best way to proceed.

  • 1
    If you don't want to use a 3rd party app, and you know how to program, why not write your own app to get RAW images from the camera and do what you want with them yourself? – user1118321 Jan 31 '17 at 5:09
  • 1
    I've voted to close this question as unclear what you are asking because it is an xy problem. – Michael C Jan 31 '17 at 6:09
  • 3
    I'm voting to close this question as off-topic because it is about using a camera as a measuring device rather than about photography. – mattdm Jan 31 '17 at 9:17
  • 1
    Imagine a function y=sin(x) and say it is RAW data. Then imagine series a(n)=sin (npi)=0 and say it is JPEG. What you are tyring is interpolation of unknown function from a infinite set of zeros. First match: f(x)=0. Experienced guess: f(x)=asin(bx), b=pi, a="anything". – Crowley Jan 31 '17 at 10:01
  • 4
    @uhoh Works for me. :) I don't mean to be over-pedantic, but I really do want to keep this site's Q&A focused. If it's 90% engineering and photometrics, that scares away people interested in composition, lighting, storytelling, etc. That doesn't mean there can't be highly-technical questions; I'd just like to see that within the frame, as it were. We can discuss this more on meta.photo.stackexchange.com, if you like. (Note also that I'm not anyone special here; just a guy with strong opinions.) – mattdm Jan 31 '17 at 15:06
6

Nope. The reason is that all of the raw data is not represented in any single interpretation of that data used to produce a JPEG image. Much of the raw data has been discarded and can not be recovered.

When you open a raw image file with an application what you see on the screen is not the unfiltered, raw data. It is one interpretation of the raw data based on the default settings of the application doing the rendering. If the settings are changed, the application will go back and recompute the image using the raw data to produce another, different interpretation of the same raw data. But the full set of the raw data is not represented in either of the two interpretations. Only some of the information in the raw file will be used for each rendering.

(Yes, it is theoretically possible to produce an image with such a limited range of brightness and color that the information in the raw file is so uniform from one pixel to the next that it can all be represented in an 8-bit image. But who wants solid pure white or solid black photos? When taking a photo of a scene that contains areas of differing brightness, colors, and textures the total data in the raw file almost certainly exceeds the capability of an 8-bit display or 8-bit image format.)

If there is any change in brightness or color from one pixel to the next, the gradations will be much finer/smoother with a raw file than with a JPEG. 8-bit jpegs only allow for 256 discrete values for each color between totally dark and totally bright. A 12-bit raw file allows for 4,096 discrete values per color. A 14-bit raw file allows for 16,384.

If the values along a line of pixels increase by one for each successive pixel as we move from left to right of a 14 bit file then there would be 64 discrete values for a line 64 pixels long. But there would only be one change in value of one unit along the same line of 64 pixels with an 8-bit file. Now let's say we have another line of 64 pixels in which the values in the 14-bit file are more irregular. They go up and down as we move right to left, but they all stay within those same 64 discrete values as the first line. Once converted to 8-bits we wouldn't be able to tell the difference between the first line and the second. There are 64 possible values in the 14-bit raw file that all transform to the same value in the 8-bit jpeg. Now multiply those 64 possible values by every pixel in the image. That is how many possible different versions of a 14-bit raw file would be represented by the exact same 8-bit jpeg. If you only have a 1 MP image, that is 64 raised to the power of 1M (64^1,000,000)!

For more about what kind of information is contained in a raw file, please see: RAW files store 3 colors per pixel, or only one?

  • To double check, you understand I just mean to approximately undo some of the transforms and corrections by applying the inverse transforms in the right order, and that I am only looking for an approximate result. Of course it can not be done exactly, but is there enough metadata (or other data) to at least get a handle on things like gamma, sRGB, or similar? I would just implement in python so floating point limitations in software might be avoided. – uhoh Jan 31 '17 at 5:28
  • You'll never get more information out of a jpeg than what is in there. If you know the exact algorithm (good luck getting that out of Apple) that was used to transform the raw sensor data to the 8-bit jpeg, you're still unable to produce gradations smoother than the 256 discrete value limit imposed by 8-bit. That is severely limiting compared to 12 or 14-bit raw data. The R, G,& B multipliers and such are usually included in EXIF data from most dedicated cameras. What, if any of that information is included in the EXIF data of an iPhone photo I know not. – Michael C Jan 31 '17 at 5:38
  • Since I'm just going to look at the average color in a, say 200x200 pixel area, and look for changes of the order of a percent, I am looking to get very substantially less information than is there. I think your answer is very helpful for professional photographers and complex images, but I've written the question in such a way as to make it clear that I am asking something different. – uhoh Jan 31 '17 at 5:43
  • Even if you have everything included in the EXIF standard, you're still limited by the 8-bit depth of your data. If you try to undo gamma curves, for instance, you're not going to get a linear straight line response, you're going to get an irregular, chunky looking stairstep response. – Michael C Jan 31 '17 at 5:43
  • 1
    Close is probably better than delete. – Michael C Jan 31 '17 at 6:07
2

As Michael Clark has explained in detail, you cannot reconstruct the raw file because the mapping from the raw file to the JPEG is not one to one, there are many different RAW files that could have been processed into exactly the same JPEG file. However, this does not mean that all the possible raw files that are consistent with some given JPEG file are equally likely. It is possible to reconstruct the most likely raw file given a JPEG image, this requires doing computations that are quite similar to doing state of the art noise reduction computations where you want to eliminate the noise in a picture by calculation the most likely noise-free picture. Such methods are enormously computationally expensive (many hours or even days on a fast computer to process a single picture), but they usually do yield much better results than the usual ad hoc methods.

In case of JPEG, what you should try first is to simply decompress a JPEG better than using the decompression algorithm. A naive JPEG decompression does not yield the most likely image that upon JPEG decompression yielded the given JPEG file, obviously not, otherwise there would be no JPEG artifacts visible.

  • Thanks - I think you are focusing mainly on the decompression of spatial aspects. I'll be choosing areas, say 200x200 pixels, that are almost featureless, and trying to approximate the average R, G, and B levels that would have been in the raw within these small areas. I think I can appreciate the difficulty of what you are talking about, it sounds like a computationally expensive but relatively straighforward maximum likelihood problem. – uhoh Feb 1 '17 at 4:08
  • By limiting to small patches of nearly uniform color and luminance and ultimately going after only the most likely average sensor R, G, and B levels that would have been within them, any down-sampling of the chroma channels would not have a major impact. Today I'll try to "go deep" and read further. I'll start with the answer(s) to RAW files store 3 colors per pixel, or only one? linked by @MichaelClark. – uhoh Feb 1 '17 at 4:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.