3
\$\begingroup\$

I was at a photographer some time ago, shot some photos of my son and came back to view and select a range of images, some of which I wanted to print, and some of which I wanted and to receive by file transfer.

The photographer advised against only relying on digital raw files as they could deteriorate over time. Coming from a software background, this sounded strange for me, but I do not have any experience storing raw image data so I might be wrong. Aren't there any file integrity checks that could verify that the data has not deteriorated?

Is it true that raw files can deteriorate over time even if you have a redundant, reliable storage medium in which the file has not seemingly changed?

Improve this question
\$\endgroup\$
1

7 Answers 7

Reset to default
12
\$\begingroup\$

RAW files no more deteriorate over time than a DOC file would. As long as you store them correctly (with backups of course) the file will be fine.

It's true that the software that you use to process them will change but in general that is a positive. Files from older cameras often look much better in modern RAW processing software than at the time they were taken. Things have improved enormously there.

If the files that you have are from any reasonably common camera then you don't have much to worry about as far as compatibility. And there are some free apps that also support more or less everything and have source code available in the worst of cases.

Improve this answer
\$\endgroup\$
0
5
\$\begingroup\$

Yes, but in a different way than you imagine.

Raw files don't contain an image. They contain raw sensor data. This raw sensor data must be interpreted to create an image.

Software may change over time, so this interpretation of the raw data to create pixel values could change.

If you use the exact same software version for the next 20 years or so, this interpretation won't change. However, who uses the same RAW processing software for 20 years or so?

In contrast, the interpretation of JPG never changes.

Improve this answer
\$\endgroup\$
17
  • 3
    \$\begingroup\$ If the interpretation of raw sensor data is changed, it is likely to improve the results after interpretation. It makes absolutely no sense to assume that software evolution or modifications will lead to an inferior interpretation of the raw data and in any way be related to a 'deterioration' of the raw files. \$\endgroup\$
    – jarnbjo
    Aug 20, 2019 at 21:35
  • 1
    \$\begingroup\$ @jarnbjo Some folks might consider any change from what they decided they liked when they bought it to be less than ideal, even if the technical quality (color accuracy, etc.) has improved. \$\endgroup\$
    – Michael C
    Aug 20, 2019 at 23:32
  • 1
    \$\begingroup\$ @moot all the software to handle the RAW data improves. ... until it doesn't. Suppose I preferred the RAW processing of Apple Aperture. They stopped supporting and updating it years ago, and it won't load on macOS 10.15 (Catalina). That is decidedly not better. Features get removed, paywalled, or changed to require subscription when they didn't before. \$\endgroup\$
    – scottbb ♦
    Aug 23, 2019 at 19:23
  • 1
    \$\begingroup\$ @moot I don't have preferences for old software. I had a preference for a specific application that served my needs best, that became unsupported. Nevertheless, I also didn't imply that software isn't improving, so you shouldn't infer the implication. Just because software improves in general doesn't universally mean that all software will universally improve, nor does it mean that proprietary file formats can always be interpreted as originally done if the manufacturer stops producing the only software that can parse the format. \$\endgroup\$
    – scottbb ♦
    Aug 25, 2019 at 16:14
  • 2
    \$\begingroup\$ "Lytro’s ‘living pictures’ are now dead on the web" theverge.com/2017/12/6/16742314/… - not quite RAW, but imagine your RAW developer depends on a cloud service, which isn't unthinkable in todays world... \$\endgroup\$
    – ths
    Sep 24, 2019 at 14:20
4
\$\begingroup\$

Deterioration describes a gradual process - this is not what happens.

Files of all sorts can get corrupt - there might be a 'bit flip' (a 0 turns to a 1 or vice versa) during saving, or maybe the sector/NAND cell of your storage device gets faulty (this is called 'bit rot'). Also, it is easy for your operating system (OS) to accidentally delete files - it cannot delete prints, though.

While prints may mold or get bleached by the sun (which truly is a gradual process - they don't get visibly worse from one second to the other) -, a single bit error can wreck everything froma single pixel to the whole picture.

The digital solution is using backups (and, on a smaller scale, good software).

Different file formats are impacted on different scales by bit flips or bit rots, and there must be made a compromise in this instance: While formats that feature higher compression can pack the image in fewer bits -- if every millionth bit would typically flip, then binning half a million is probably worth it --, they typically are not able to handle faulty bits as well as TGAs or BMPs. I have no idea how well or badly RAWs fare in this comparison, but you could try using a HEX-editor and changing a single value in one of your RAWs (make a copy of it before doing so!) and see what happens.

Also, as noted by others, RAW formats (except DNG) are proprietary, so only the manufacturer knows exactly how to read them. It might well be that your CRW files cannot be read by software 50 years from now, as nobody cared to maintain the code to interpret them - or maybe they get interpreted in a very different way. Either way, for long term storage (i.e. not "until I have time to process them"), I would recommend to at least save an unprocessed JPEG (or something alike) alongside the RAW, as I guess that JPEGs will be able to be read for some more years.

Different file systems (FS) handle bit rot differently. NTFS, FAT, ext4, and most others just save your file, while inegrity checking FSs like btrfs, ZFS, ReFS, and some more also save the hash-value of every file. Without a backup, this does not prevent anything, but at least they tell you when a file got silently corrupted.

On the backup end, I would highly recommend a strategy that involves a snapshot-like structure, like differential or incremental backups. There are lots of good (and free) tools around for this purpose for all operating systems, for example Duplicati. I'm not affiliated, I just wanted to offer an example that I know is free and works pretty well.

On the hardware end, checking (and testing) your storage device's health regularly can help. Not every dying HDD tells you that it is about to fail in its SMART values, but if the SMART values look bad, I would definitely replace it ASAP. Do not use flash memory cards or sticks as backup or main storage - they are not designed for this purpose. Also note that not even DVDs stored under optimal conditions live forever.

Further reading: "What medium should be used for long term, high volume, data storage (archival)?" on superuser.com has some excellent answers when it comes to everything mentioned above.

My personal strategy involves most of the above: I use an integrity checking FS for both my main storage drive and the backup drives, so as soon as something goes bad, I will be noticed. I have my computer check the drives every now and then for errors on the hardware side of things just to be sure. And my backup strategy involves incremental backups, so in the case of a user error (the most likely thing to damage your files)[1] [2] [3], I can easily restore the state from a week ago.

On a side note: Backups are cheaper and easier on digital files: prints can burn, mold, soak through, bleach out,... While these are all things that a caring person can easily avoid, installing Duplicati and buying two extra disks (or using some old ones) seems to me a bit less troublesome. But I am a digital native and still I think that photos look better when printed (or developed in case of film) :-)

Improve this answer
\$\endgroup\$
3
\$\begingroup\$

Digital files do not "deteriorate" over time. However, the storage media that contain them can, which leads to file corruption. Raw files are typically several times the size of JPGs, so the likelihood of a raw file becoming corrupt is several times that of the corresponding JPG becoming corrupt.

Giving the photographer the benefit of the doubt, he may be using JPGs as backups to the raw files, whereas you would keep full "proper" backups. He may also be using the word "deteriorate" differently than you would, perhaps meaning "corrupt". More likely, however, he may have had an experience in which multiple raw files became corrupt, while the JPGs survived. Since his mental model of how technology works is different from yours, he describes the problem as "deterioration".

Even more likely, he may be trying upsell you on prints or books and the "raw" files aren't even really raws. Given that you received them by "file transfer" (via internet?), it's likely the files are heavily compressed and he is managing expectations to prevent you from complaining about the quality of the files he sent you.

Improve this answer
\$\endgroup\$
1
  • 1
    \$\begingroup\$ However, minor defects to an uncompressed(!!!) raw file could leave you with far more of the image recoverable than a JPEG with a couple bits flipped. \$\endgroup\$
    – rackandboneman
    Sep 21, 2019 at 16:16
1
\$\begingroup\$

Another way data deteriorates is because it is written to media that deteriorates. A typical examples is CDs and DVDs. This can cause files to be either corrupted, or be damaged. I'm not sure how much CRC (related) checks a RAW image contains, if not much, than there will be likely more deterioration (otherwise corrupted files, or at least warnings).

However, lately flash or hard disk storage is used, which do not deteriorate, at least not in a considerable amount of time.

Improve this answer
\$\endgroup\$
1
1
\$\begingroup\$

The problem with digital isn't storage, it's the cameras.

Data can't deteriorate and all the ways of storing data are vastly superior to storing film. Stop and think about a memory stick or disc saving compared to a container of film.

Digital cameras are what age.

Regular film cameras produce images with virtually infinite resolution. Whatever light was captured can be converted to pixels. So you can take regular film and make whatever resolution out of it - 4k, 8k, 1000k.

Digital cameras have a set resolution so they produces images that get "old." For a while now, cameras have been strong enough to create resolutions that are well beyond what we can see when the image is displayed normally. But still, the vast majority of information, of light, is lost.

That level of information is never used these days and may never be used. You can't see it with naked eye. The colors that are lost can't be displayed anyways. The digital cameras today are way out ahead of our displays and they may be past what we would ever need with human eyes.

Improve this answer
\$\endgroup\$
4
  • \$\begingroup\$ Film has a limited resolution too - IIRC the best non-technical/scientific films are about 150lp/mm (which is in the same ballpark as a non-antialiased 28MP APS-C sensor) \$\endgroup\$
    – rackandboneman
    Sep 21, 2019 at 16:19
  • \$\begingroup\$ @rackandboneman yeah i said virtually because it has a resolution point but it's not the same as digital. There's no resolution to physical film. That number is just what current sensors can generate. Tomorrow's sensors will get way more out of the same image. The entire way the image is captured will change. \$\endgroup\$
    – moot
    Sep 23, 2019 at 17:29
  • \$\begingroup\$ Re, "There is no resolution to physical film." People talked about the resolution (a.k.a., "resolving power") of optical systems long before there was any such thing as digital photography. The resolution of the system tells you how much detail it can capture. For film, it's about how many distinct "lines" it can record* before they all run together in a blur. Fine grained films can capture more lines/mm that coarse grained films. Larger negatives can record more total lines than smaller negs of the same kind of film. (* Google for "resolution test target.") \$\endgroup\$
    – Solomon Slow
    Feb 16, 2020 at 1:23
  • \$\begingroup\$ @SolomonSlow Yes there's definitely a type of resolution to film but future software and hardware may be able to get more out of film and it's still far better than current DSLRs. Look at how grains are randomly sized, shaped, and positioned. There's a ton of information that's not being used. It all gets physically processed, which physically blurs that info - the gaps between the grains and the shapes. Plus, we're just beginning to get our heads around how light works. Think about a quantum computer that knows the exact size and position of every grain \$\endgroup\$
    – moot
    Feb 16, 2020 at 21:06
1
\$\begingroup\$

Aren't there any file integrity checks that could verify that the data has not deteriorated?

Lightroom 5 and later will validate the integrity of raw files, but only in DNG format. The option is in the Library menu. For all the reasons mentioned about about file corruption, I run it regularly and do occasionally find a corrupt file, which can then be replaced with a backup copy.

There must be other options for checking file integrity for DNG and other raw formats, since it's just a checksum procedure, and you should be able to find them. This is just the one that I am familiar with.

Improve this answer
\$\endgroup\$
1
  • 1
    \$\begingroup\$ There must be other options for checking file integrity for DNG and other raw formats - since most RAW formats are proprietary, I highly doubt that. Saving the checksum from the original (i.e. the file on your SD-card) and re-checking that the file got properly copied/moved against that is the way to go, since this will (usually) report even a single flipping bit, while your file might still be a proper RAW if the values for pixel 1024,1024 flip. (Of course, this is a matter of probability.) \$\endgroup\$
    – flolilo
    Oct 13, 2019 at 15:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.