1

I was at a photographer some time ago, shot some photos of my son and came back to view and select a range of images, some of which I wanted to print, and some of which I wanted and to receive by file transfer.

The photographer advised against only relying on digital raw files as they could deteriorate over time. Coming from a software background, this sounded strange for me, but I do not have any experience storing raw image data so I might be wrong. Aren't there any file integrity checks that could verify that the data has not deteriorated?

Is it true that raw files can deteriorate over time even if you have a redundant, reliable storage medium in which the file has not seemingly changed?

10

RAW files no more deteriorate over time than a DOC file would. As long as you store them correctly (with backups of course) the file will be fine.

It's true that the software that you use to process them will change but in general that is a positive. Files from older cameras often look much better in modern RAW processing software than at the time they were taken. Things have improved enormously there.

If the files that you have are from any reasonably common camera then you don't have much to worry about as far as compatibility. And there are some free apps that also support more or less everything and have source code available in the worst of cases.

4

Yes, but in a different way than you imagine.

Raw files don't contain an image. They contain raw sensor data. This raw sensor data must be interpreted to create an image.

Software may change over time, so this interpretation of the raw data to create pixel values could change.

If you use the exact same software version for the next 20 years or so, this interpretation won't change. However, who uses the same RAW processing software for 20 years or so?

In contrast, the interpretation of JPG never changes.

  • 2
    If the interpretation of raw sensor data is changed, it is likely to improve the results after interpretation. It makes absolutely no sense to assume that software evolution or modifications will lead to an inferior interpretation of the raw data and in any way be related to a 'deterioration' of the raw files. – jarnbjo Aug 20 at 21:35
  • @jarnbjo Well, it's the question that mentions deterioration. My answer doesn't contain the word at all, saying instead that in a different way that the person asking the question imagines... – juhist Aug 20 at 22:30
  • @jarnbjo Some folks might consider any change from what they decided they liked when they bought it to be less than ideal, even if the technical quality (color accuracy, etc.) has improved. – Michael C Aug 20 at 23:32
  • I get what you're trying to say but this isn't correct. Data doesn't detoriorate and all the software to handle the RAW data improves. If you want to imagine anything, you should imagine RAW data improves. JPGs are processed by the software at the time so JPG actually do get old in some ways but they don't deteriorate. – moot Aug 23 at 14:36
  • 1
    Let's not ignore the fact that 50 years from now, the software of future may have dropped support for old cameras. There is only one JPG, but all kinds of RAW formats. – juhist Aug 26 at 15:48
3

Digital files do not "deteriorate" over time. However, the storage media that contain them can, which leads to file corruption. Raw files are typically several times the size of JPGs, so the likelihood of a raw file becoming corrupt is several times that of the corresponding JPG becoming corrupt.

Giving the photographer the benefit of the doubt, he may be using JPGs as backups to the raw files, whereas you would keep full "proper" backups. He may also be using the word "deteriorate" differently than you would, perhaps meaning "corrupt". More likely, however, he may have had an experience in which multiple raw files became corrupt, while the JPGs survived. Since his mental model of how technology works is different from yours, he describes the problem as "deterioration".

Even more likely, he may be trying upsell you on prints or books and the "raw" files aren't even really raws. Given that you received them by "file transfer" (via internet?), it's likely the files are heavily compressed and he is managing expectations to prevent you from complaining about the quality of the files he sent you.

  • However, minor defects to an uncompressed(!!!) raw file could leave you with far more of the image recoverable than a JPEG with a couple bits flipped. – rackandboneman 2 hours ago
1

Deterioration describes a gradual process - this is not what happens.

Files of all sorts can get corrupt - there might be a 'bit flip' (a 0 turns to a 1 or vice versa) during saving, or maybe the sector/NAND cell of your storage device gets faulty (this is called 'bit rot'). Also, it is easy for your operating system (OS) to accidentally delete files - it cannot delete prints, though.

The solution is using backups (and, on a smaller scale, good software).


Different file formats are impacted on different scales by bit flips or bit rots, and there must be made a compromise in this instance: While formats that feature higher compression can pack the image in fewer its - if every millionth bit would typically flip, then binning half a million is probably worth it - they typically are not able to handle faulty bits as well as TGAs or BMPs. I have no idea how well or badly RAWs fare in this comparison, but you could try using a HEX-editor and changing a single value in one of your RAWs (make a copy of it before doing so!) and see what happens.

Also, as noted by others, RAW formats (except DNG) are proprietary, so only the manufacturer knows exactly how to read them. It might well be that your CRW files cannot be read by software 50 years from now, as noone cared to maintain the code to interpret them - or maybe they get interpreted in a very different way. Either way, for long term storage (i.e. not "until I have time to process them"), I would recommend to at least save an unprocessed JPEG or something alike, as I guess that JPEGs will be able to be read for some more years.

Different file systems (FS) handle bit rot differently. NTFS, FAT, ext4, and most others just save your file, while inegrity checking FSs like btrfs, ZFS, ReFS, and some more also save the hash-value of every file. Without a backup, this does not prevent anything, but at least they tell you when a file got silently corrupted.

On the backup end, I would highly recommend a strategy that involves a snapshot-like structure, like differential or incremental backups. There are lots of good (and free) tools around for this purpose for all operating systems, for example Duplicati I'm not affiliated, I just wanted to offer an example that I know is free and works pretty well .

On the hardware end, checking (and testing) your storage device's health regularly can help. Not every dying HDD tells you that it is about to fail in its SMART values, but if the SMART values look bad, I would definitely replace it ASAP. Do not use flash memory cards or sticks as backup or main storage - they are not designed for this purpose. Also note that not even DVDs stored under optimal conditions live forever.


My personal strategy involves most of the above: I use a integrity checking FS for both my main storage drive and the backup drives, so as soon as something goes bad, I will be noticed. I have my computer check the drives every now and then for errors on the hardware side of things just to be sure. And my backup strategy involves incremental backups, so in the case of a user error (the most likely thing to damage your files), I can easily restore the state from a week ago.


On a side note: Backups are cheaper and easier on digital files - prints can burn, mold, soak through, bleach out,... - while these are all things that a caring person can easily avoid, installing Duplicati and buying two extra disks (or using some old ones) seems to me a bit less troublesome. But I am a digital native and still I think that photos look better when printed (or developed in case of film) :-)

0

Another way data deteriorates is because it is written to media that deteriorates. A typical examples is CDs and DVDs. This can cause files to be either corrupted, or be damaged. I'm not sure how much CRC (related) checks a RAW image contains, if not much, than there will be likely more deterioration (otherwise corrupted files, or at least warnings).

However, lately flash or hard disk storage is used, which do not deteriorate, at least not in a considerable amount of time.

0

The problem with digital isn't storage, it's the cameras.

Data can't deteriorate and all the ways of storing data are vastly superior to storing film. Stop and think about a memory stick or disc saving compared to a container of film.

Digital cameras are what age.

Regular film cameras produce images with virtually infinite resolution. Whatever light was captured can be converted to pixels. So you can take regular film and make whatever resolution out of it - 4k, 8k, 1000k.

Digital cameras have a set resolution so they produces images that get "old." For a while now, cameras have been strong enough to create resolutions that are well beyond what we can see when the image is displayed normally. But still, the vast majority of information, of light, is lost.

That level of information is never used these days and may never be used. You can't see it with naked eye. The colors that are lost can't be displayed anyways. The digital cameras today are way out ahead of our displays and they may be past what we would ever need with human eyes.

  • Film has a limited resolution too - IIRC the best non-technical/scientific films are about 150lp/mm (which is in the same ballpark as a non-antialiased 28MP APS-C sensor) – rackandboneman 2 hours ago
0

Aren't there any file integrity checks that could verify that the data has not deteriorated?

Lightroom 5 and later will validate the integrity of raw files, but only in DNG format. The option is in the Library menu. For all the reasons mentioned about about file corruption, I run it regularly and do occasionally find a corrupt file, which can then be replaced with a backup copy.

There must be other options for checking file integrity for DNG and other raw formats, since it's just a checksum procedure, and you should be able to find them. This is just the one that I am familiar with.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.