I'm not a professional photographer but I do value my photos very much. Sometimes when I look through them months or years after I've taken them, some of them just show up corrupt for no evident reason. I often get things such as gray areas, altered colours starting from a certain point on or even completely undisplayable photos. I use both Windows and Linux OS on my PCs.
Why does this happen and, most importantly, how to avoid this?
There are two main causes. The first is data degradation. Bits stored on magnetic media (such as your hard disk) can lose their magnetic orientation over time, corrupting the bit. In harsher conditions (high heat and humidity) the physical media itself can start to degrade. For solid state media such as an SSD, the mechanism is different but the outcome is similar.
The more common occurrence is silent data corruption, whereby an error occurs during the writing or reading of data. Though the image at the previous link is an extreme example, oftentimes a single flipped bit can corrupt an image.
There are two solutions to these issues, and assuming you rotate your backups, backups alone won't help. The more complicated and technical approach is to use a filesystem with built-in data integrity mechanisms (such as ZFS or btrfs), while a simpler approach would be to compute a hash for each file between backups to detect any changes.
Probably not worth the extra 1000% penalty in size... I'd say exactly the opposite - shoot RAW, store RAW, backup RAW. You keep the negatives, not the prints.
\$\endgroup\$
I'm not going to worry too much about how this happens, because hard disks will fail; just like anything else in this world, they aren't perfect. You can't get to a state where you're never going to lose a file.
However, you can get to a state where you never lose a photo - you do this by having multiple copies on separate hard drives, including at least one in a separate physical location. Then it didn't matter of one disk fails, because you can just get the photo from another location.
The answer to the why this happens depends on the type of drive, type of file, filesystem and operating system used, so it's impossible to give a complete answer to this question.
But regarding how to avoid this, the other answer touches on having backups - however, with file corruption like this, you need a way to identify when a particular copy of your file has been corrupted and avoid copying this to your backups. It's very easy to end up with 2 corrupted copies of a file. The technical answer is to use checksums and compare these to identify corrupted files.
The best way to avoid this these days, is to use an online storage service which will handle all redundancy, and file integrity for you.
their client automatically detects the change to the local copy of the photo and uploads the new version, which is corrupted. Actually, this is an interesting point, because unless the corruption is in a software layer the client won't actually be able to detect the corruption unless you re-write the file.
\$\endgroup\$
Photo files aren't special in this regard, any file on a computer can get corrupted. JPEGs are probably more likely to show up corruption in an obvious way than a lot of other files types though, so you may notice it more (see @pipe's comment on @alldayremix's answer).
Files can be corrupted in many ways, (non SSD) hard drives can be put too close to a magnet, CDs and DVDs can gradually rot, and electrical interference and bugs in firmware or software can corrupt files in transit. There are other reasons too, but I'd suggest asking on a more computer related stack if you want to go into more detail.
In terms of what can be done about it, the best answer I can come up with is calculating hashes and keeping multiple copies of the photos. To this end I wrote a free, open source, program to address this issue for myself: Archiverify.
It works by computing hashes, storing the hashes alongside the images, and comparing the files against the hashes when you run it. As long as you have more than one copy of the file it can automatically refresh the corrupted copy from the good one (assuming both don't get corrupted at once).
Alternatively, I'm sure that there are paid storage/backup services available that will do the hashing and checking for you periodically behind the scenes.
There are multiple reasons as stated above. It can just be natural wear over time that corrupts bits of files or physical degrading. If you are continually finding photos and other files severely corrupted it can be a sign of a computer virus. If you don't want your files to be corrupted the first two ways then invest in an SSD as it won't degrade as quickly and it 10 times faster. (You might also want to do a quick virus scan too.)
Why does this happen and, most importantly, how to avoid this?
You should ask this question on Security.SE.
To avoid corruption you should make backups.
In information technology, a backup, or the process of backing up, refers to the copying and archiving of computer data so it may be used to restore the original after a data loss event. The verb form is to back up in two words, whereas the noun is backup.
Writable digital optical media such as CD-R and DVD-R could be corrupted even by sunlight. To avoid this you might use something like double backup strategy. However, nowadays, we have such beautiful online methods to backup photos, like Instagram or other online services. For example, behind Google Drive there are large amount of data centers. Every one of your photos will be stored on the Google File System (GoogleFS or GFS) on servers hosted in these data centers. This filesystem was designed by Google to protect users from losing data. Your photos will be saved in three exemplars in three different geographic locations, to protect against data loss because of local disasters from lightning, tsunami, earthquake, etc.
Try to use multiple online services simultaneously (such as megaupload, dropbox, wuala, 4shared, etc.).
Thereafter, for the greatest peace-of-mind that none of your photos were modified or corrupted, you would use checksums to verify your data integrity.
A checksum is a small-size datum from a block of digital data for the purpose of detecting errors which may have been introduced during its transmission or storage.
The best way is to use cryptography for such task. Use the latest stable hash functions, such as SHA-256 or SHA-512. Older algortihms, such as MD5, are now considered insecure. A modern attacker could make forgery of your data with the same md5sum.
When you are done, you will have a distributed backup system which will not be easily broken, even by the most sophisticated and dedicated attackers (such as nation state -backed attacks).
Two DVD copy with identical datas inside:
1. Photo1.jpg
2. Photo2.jpg
3. Photo3.jpg
...
999. Photo999.jpg
1000. SHA256SUM
Where this SHA256SUM shall contain something like:
e5347dce99eb8cf694cf708d4a17d83abb3ec378241b5878c0abdab045859b24 Photo1.jpg
b497a12b608def869a0429d7e6bbbd112bd413256201647a5aff6773de3b7bd9 Photo2.jpg
b15b0d99bf8135286f444fc62bcf70278a89e60650252ab2bd3b6fffd40c4255 Photo3.jpg
...
209732fbdb499f0cad6fd3311b45185667bbb40e501106997d3ac2c49cb30a7e Photo999.jpg
The lines 209732fbdb499f0cad6fd3... are unique hashes of your photos. When one bit of your photo becomes corrupt, this hash will be changed to another, so you could test hashes against this list to be calm that your photos are keeping integrity.
Secondly, all of this photos and this SHA256SUM-hashes file are uploaded to e.g. dropbox and somewhere else.
Now, you have 5 copies of your photos. 2 off-line, 2 on-line, 1 on your current hard drive.
The scheme might be evolved to something more robust of course. Your imagination is your trump card.
I cannot rule out that the jpeg standards between your old camera and your latest image viewer have diverged somewhat, so the photos only look corrupted. Sometimes software complies only with a preliminary version of a standard. Try an older viewer (on an old computer).
