What is the real world impact of the lossy compression used for NEF RAW files in some Nikon cameras?

Question

I intend on buying a DSLR in the near future and my eye has been stuck on the Nikon D5300. One of the things in photography that I feel uncomfortable with is compression, therefore I obviously care about a camera which can produce RAW files, so that I retain the most quality.

The Nikon D5300 can only yield 14-bit NEF files that use lossy compression. Now, I've read that this compression affects primarily highlights in the RAW image, but does it still make the NEF image quality superior to the JPEG image quality? Say I take a photo, using the D5300, with a JPEG+14-bit NEF (lossy compressed) output, then take the NEF file, convert it to a TIFF file and then compare its quality to the JPEG quality. Will the TIFF file, despite being produced from a compressed RAW file, still have much better image quality when carefully inspected?

Answer 1

Even with lossy compression the NEF file still contains a lot more information than a JPEG file.

There is considerably more data per pixel in the NEF file. Even if the 14 bits color depth is reduced somewhat by the compression, it's still way more than the 8 bits of a JPEG image.

You won't see much difference between the JPEG and the NEF in a direct comparison. The JPEG compression removes a lot of image information that isn't visible on a screen. The difference is clearly visible if you want to adjust the exposure in post production, though. Then you will see that the NEF file has a lot of information where the JPEG image starts to degrade.

Even when you compare lossy and non-lossy compression in NEF files, there is not much impact on the image quality. See http://www.diyphotography.net/12bit-vs-14bit-raw-and-compressed-vs-uncompressed-does-it-matter/

Answer 2

The particular lossy compression used here is nothing to worry about. Here's why.

Camera sensors are more or less linear devices, if you double the amount of light you double the signal produced by the sensor. Our eyes work logarithmically, so if you double the amount of light, it appears much less than twice as bright. Another example of this is if you display pure black and pure white on a computer screen, then find a shade of grey that looks right in the middle of those two colours it will only be around 18% of the brightness of pure white.

Our eyes are thus much more sensitive to slight changes in absolute brightness near the bottom of the scale and almost completely insensitive to small changes in absolute brightness near the white end. The sRGB/Adobe RGB colour spaces (amongst others) take advantage of this and store brightness on a nonlinear scale, using more bits (smaller steps) in the shadows where you're most likely to detect similar shades and fewer bits (larger steps) in the highlights where fine steps go unnoticed.

Nikon's lossy RAW compression maps a linear 14 bit space (that contains values directly from the linear sensor) to a non-linear 12 bit space similar to sRGB etc. in that is uses the bits where they make the most difference.

Technically you are losing data, but it's data that is not perceivable in most cases anyway.

Answer 3

If you do no editing, such as adjusting light curves, color balance, white point, sharpening, etc. prior to converting to TIFF or JPEG, then the only difference between the finished files will be whether you allowed the automated routines in the camera make the decisions (JPEG) or the preset/automated routines in your conversion software application make the decisions (RAW->TIFF). Whether the original RAW file is compressed or not will make no practical difference. Only when you are doing very precise editing to bring out very minor differences in the extreme highlights exposed very near to the saturation point will the difference between compressed and uncompressed NEF files be noticeable.

It's not so much that a RAW file (lossy compressed or otherwise) has that much better image quality than a TIFF or JPEG, it is that the RAW file contains more information than either of the others, and that additional information can be used to bring the image closer in terms of white balance, color, and dynamic range to the photographers intent. But to leverage that additional information into the final appearance of the image requires editing the image and adjusting light curves, white balance, etc.

The advantage of RAW over lossy formats isn't that what you see when you preview a RAW image on your monitor will look better than a JPEG or TIFF. This is because all of the data from the RAW image is not displayed. Your monitor is not capable of displaying anywhere near all of that data at once. What you see is an on-the-fly conversion to a format your monitor is capable of displaying, much like a JPEG would be.

The advantage of the RAW file is that all of the information captured when you created the image is still contained in the file, even if all of it is not currently being displayed. When you edit a RAW file, what you are doing is selecting and choosing exactly which parts of the data are displayed and which parts are not used. You are choosing the points at which subtle differences in brightness, color, etc. are emphasized and at which points fairly significant ranges of brightness, color, etc. are rendered the same.

Once that information is converted to another format, such as TIFF or JPEG, the unused information is discarded and can not be recovered from the converted file, because the converted file does not contain the unused information.

Answer 4

A camera is part of a system; So lets look at that system. I'm going to make generalisations and some assumptions - don't get upset if they're wrong.

Since you are looking at a D5300 it gives a good indication that your budget is relatively low. You're not really going to be using jpeg, so we're looking to see if there's enough value in the uncompressed raw format to make it worth the extra money (and wait.)

The kit lenses that come with the D5300 are better than those in most point & shoot and bridge cameras by a country mile, but in SLR terms images will suffer more from compromises that are needed to make your lens to a price than through lossy raw compression.

Software proficiency isn't going to be that good that you're going to notice differences between the two raw formats. In fact you'll probably lose more colour and detail information from inexperienced editing decisions or an uncalibrated screen than you will through the raw compression.

Even with a good software setup it's unlikely that your screen will show you the differences in detail unless you're pixel-peeping or you've made some extreme adjustments (and those yield more data loss in themselves than lossy compression.)

All in, at an entry level you aren't going to notice a problem with the lossy raw compression enough to make it worth while.

TL;DR

Get the cheaper body, put the money you save towards better lenses, a top notch screen for editing and some calibration. Most of the true limitations you'll hit are not going to be solved by having the modest amount of additional detail you might get from a lossless raw.

Answer 5

14-bit tiffs use more 1s and 0s per pixel than jpegs by a ratio of 16384 to 256.

Raw files store a little more data in the highlights and the shadows of an image, allowing you to better recover shadows or highlights.

When your camera takes a jpeg, it will perform edits on that picture, such as white balance and sharpening changes. If you were to edit one of these items on a computer you're performing a second edit on the picture, whereas when you make any edit to a raw picture, the raw picture will refer to the original data. This theoretically improves images quality; however the difference is only really noticeable when editing -- the image quality will degrade quicker when editing a jpeg.

However all these benefits come at the disadvantage of larger file sizes and a required editing and exporting process before you can upload or print an image.