Lossy but essential: is there a modern file format useful for modern camera files?

Question

I remember the early digital cameras that had anything resembling useful quality (megapixel) stored JPEG files that were noticeably lossy. As memory cards became more capable, this was addressed and manufacturers soon got the compression tuned just right to match the actual useful information from the sensor. After all, whose to say this sample should have been 145 rather than 146? The sampling is a little bit iffy anyway, and the pixels may line up a few nanometers to one side or the other on the scene, right? And a little softening is fine, as the optics are imperfect anyway and a sudden pixel of a different color must be noise anyway. So the JPEG was technically lossy but part of the overall capture pathway and was good enough to capture the meaningful information.

Now, we have 14-bit sensors and much greater depth and color than a normal JPEG file can represent. There is still some inherent softness, whether from DOF or diffraction, since we still have more pixels than image resolving power.

A comment someone made about shooting JPEG to save space got me thinking about how the situation has changed. Saving JPEG is shooting yourself in the foot now, because it is so much less than what the camera captures.

Is there a file format that is lossy but can be tuned to match the inherent information content of the image, that will offer a substantial space savings over RAW files? We can assume that the PC can perform significant processing on the image (not done in-camera) and do things like noise removal first. "soft" (e.g. out of focus background) should naturally give higher compression, without having to pre-identify regions of interest for different parameters. But given that Photoshop can automatically select "in focus" content, it's certainly not out of the question for the processing program to figure out how sharp each region is: that would be good for noise reduction algorithms too, for frequency separation and rejecting noise of higher frequency than expected content.

For burst shots, you could do inter-frame comparisons like video codecs can do. I think there is a new still format for the web based on a single video frame, right? Well, why not a bundle of 3 or 5 nearly identical frames?

JPEG 2000 never took off, and JPEG has more in its arsenal than the JFIF profile we use for common JPEG files. Then there's fancy stuff like Pixar's files.

Is there anything that I can teach people who find RAW files to be "too large" and keep only jpeg? What would be significanly smaller than RAW but (usefully) better than nieve JPEG?

Answer 1

Realistically this is a solved problem and manufacturers have been doing it in similar ways for some time.

Raw files from Canon and Nikon (and probably others too) use a JPEG-Wide variant that is similar to normal JPEG but capable of 16bpc. As it is essentially working on a monochrome image it only has to deal with 1 channel per pixel (and the conversion to RGB is processed later). This is the best place for savings in the workflow as less data in means less data out, plus any compression artefacts will be more likely to show up in the RGB image as a little chroma noise which we tune out quite well.

This means that the problem is not with the OP's correspondent's workflow but with their tooling. Essentially what they've done is bought a sports car and now refuse to drive it fast because of the cost of fuel.

So, if you want to convince them to use raw then you need to show that the value of the extra information in shadows and highlights is worth the extra time/space needed for dealing with the file formats that hold the information. However, if they think the speed is more important to them than ability to work in post then there will be little you can do to change their minds.

Answer 2

I'm not convinced a different file format is the best option. That is, if camera manufacturers could include a file format option that retained raw benefits at smaller file sizes, this would be an easy no-brainer: just switch to that format. However, the effort of doing this is on the user. Assuming there is a good file format choice then there's work in getting there:

• First, you need to ingest the raw file from the camera to the computer. Easy -- this is a solved problem with Lightroom or many other tools.
• Next, convert to this ideal file format. Likely it can be an automated conversion with some batch processing in Photoshop or some other tool. However, this still takes time to complete and while the CPU is chugging along you may not be able to use the machine for much. In other words, it's likely measurable time that you need to spend to complete this conversion -- in other words, time you or somebody else must pay for, in some way.
• Manage the old raw file and the new ideal-format file. In your colleague's case this means deleting the raw file and importing the ideal-format file to Lightroom (or whatever their process is so that they can add keywords, edit, etc).

Doing this for a few files or even a whole card's worth isn't a big deal. You've just got a few more simple steps added to your process -- 5 min? 15 min? No problem. But if you incorporate these steps into your workflow permanently take a look at the total cost of effort. Suddenly the total time to do this for, say, a month is an hour; a year is (for arguments sake) 12 hours. No doubt you will also eventually become annoyed with having to convert these files, and needing to pause on the work while the conversion is happening, and then you need to manage these two sets of files.

How much does that 12 hours/year cost? What is your rate (if you do this professionally) or what is your time worth if you do this for fun? Your colleague says he doesn't have enough disk space. A 1 TB external drive can be had for as little as $50. If 1 TB will handle 1 year of shooting, that means the cost of it spread out over the 12 hours is $4.17/hour. I suspect everybody's time is worth more than that.

Of course, an external drive also causes some additional work -- shuffling files over to it -- but I find this easy: monthly I can just move content over to it with a single drag-drop operation.

Answer 3

The best modern answer to this question would be the DNG file format.

Keep in mind that there are two major types of compression.

• Lossy - information is deliberately lost. This will always limit editing to one degree or another.
• Lossless - information is kept perfectly. This will be equivalent to the original raw file. But the file is smaller than the original.

The default for most programs that create DNG files is to compress the data using lossless compression. I would argue that in the modern world this is the best solution. It minimizes the file size using modern compression techniques AND it maximizes the editing potential of the image.

DNG also has the ability to compress an image using lossy compression. The benefit is that the file can be even smaller than one created via a lossless compression scheme. The downside is that information has been lost forever. This of course impacts editing of the image. DNG's lossy image compression is slightly better than regular jpeg.