3

I stumbled across this article today which said:

It’s bad practice to delete images from your SD cards since it can lead to corrupted data. Instead, you want to format (or re-format) your cards between shoots or once they’re full.

Really? Is that true?

(In fairness I normally format my card as it's quicker, easier and I'm guaranteed to recover all possible space. I would never have considered that there is a chance of corrupted data.)

  • This is a question about digital storage media, not photography in a creative, artistic, or historical context. – xiota Aug 23 '18 at 11:39
4

It's not true.

  • Corrupted data is likely the result of not properly unmounting or ejecting cards prior to removing them from the camera or computer. Not following proper removal procedures can introduce inconsistencies to the file system. While formatting the card can "fix" these problems, so does scanning the card with chkdsk, fsck, or Disk Utility.

    While formatting may be useful as a last-resort troubleshooting step, it will not save a failing card. In cases where cards cannot be repaired by standard file system checks, but appear to resume working after formatting, a replacement card should be obtained as soon as possible. Getting a few extra shots on a card is not worth losing a day of shooting.

  • Formatting can cause other problems, associated with the changing the volume id or label. For instance, disk monitoring tools that automatically import images may no longer recognize the card.

  • Formatting effectively deletes all data from the card, which is undesirable if the card is used to store or transfer other files. (Both formatting and deletion mark sectors as unused. The difference is formatting does so indiscriminately by rewriting the file-system structures, while deletion does so only on selected files.)

  • Getting too cozy with the format option may result in accidentally formatting a card before its contents have been properly backed up. There is a reason the option tends to be buried in the menus.

    Similarly, getting too cozy with the delete function can result in accidentally deleting files. That is why computers have a "Trash Can" or "Recycle Bin". The delete function on cameras should be hidden away as well. Files should be deleted only after they have been copied and backed up, not while chimping.

Addressing some misunderstandings:

  • File fragmentation affects the performance of magnetic media. It does not affect the performance of flash memory.

  • File fragmentation does not normally increase the chance of file corruption. If corruption does occurs, it is the result of buggy firmware or software. You need to update your camera firmware or operating system, not format the card.

  • Formatting a card on a computer can cause problems if the wrong file system is selected. This might occur if the card is used as an installation or recovery drive or other non-standard, non-photographic uses. For instance, some people might partition a card with multiple file systems for various uses on a smart phone.

    However, the SD Card Association has already specified what format the card should use. (FAT16 vs FAT32 vs exFAT) If a card is never formatted, but used with the format it was purchased with, there should not be any issues with the file-system format itself.

  • Writing a disk image to an SD card is a great way to unnecessarily use up a card's write cycles. (Don't do it.) Typically, when formatting, only minimally necessary file-system structures are written to the card. The vast majority of the card is untouched because it will contain empty space after the format.

    Replicating a disk image will typically write to every block of the card, even if it is intended to be empty space. Worse, it will write whatever junk data was on the original drive from which the image was created. This will interfere with any later data recovery efforts, should they be necessary.

  • Not marking blocks as empty and available for reuse (discard/trim) is a problem with solid-state drives. On SD cards, the operation is handled automatically by the hardware and not needed.

  • Wear leveling is automatically handled by the hardware. It works regardless of whether the card is formatted or files are deleted. The difference between formatting and deleting all of the files is minimal.

    A few file system structures are stable and not frequently turned over. For instance, the volume id and label. These structures occupy so few blocks that a failing card would still fail even if they were available for use.

  • It is true that leaving half of the files in place will prevent those blocks from being used for other purposes. However, the most likely reason for not deleting all of the files is that you intend to keep them. If that is the case, formatting is not what you want to do.

  • Bad block detection and replacement on SD cards is done by the hardware while the card is in use. Detecting and marking bad sectors during formatting is how it was done on magnetic media decades ago. Nowadays, even formatting does not bother to detect bad sectors because it is too time consuming. Also, by the time such sectors are detectable, the drive or card is doomed. It's long past time to replace.

  • 2
    to be precise in camera by default during format data is not deleted. Just blocks are marked as free in to the allocation table. Only if you select "low level format" (Canon, not sure how it is for other brands) the information is overwritten. – Romeo Ninov Aug 23 '18 at 12:56
  • 1
    @RomeoNinov "to be precise in camera by default during format data is not deleted. Just blocks are marked as free in to the allocation table." Are you sure that's how it's implemented in cameras (or most of them)? For magnetic media, that's how it's done for speed reasons, but for flash storage media, it's easier and faster to issue the command to erase all flash blocks, and then just write a new blank allocation table, than it is to walk the table and write just the data required to zero each entry in the table. – scottbb Aug 23 '18 at 14:29
  • 2
    @scottbb, for flash media is the same. And the reason for this is the limit of write cycles of flash media. – Romeo Ninov Aug 23 '18 at 15:16
  • 1
    Formatting the card (to simply empty it) is a good option, so long as the camera being used does it. Use the camera option, instead of the PC. – WayneF Aug 23 '18 at 15:28
  • I've never accidentally formatted a card that had images I hadn't yet copied/backed up. I have accidentally deleted images trying to free space on a card by deleting only some of the images it contained. YMMV. – Michael C Aug 23 '18 at 18:31
0

There is a bit (pun intended) of truth to it. When some but not all images on a card are deleted the 'holes' between the images that remain are not all a uniform size because image file sizes are dependent upon the contents of the image. This is true of both raw and jpeg files. Depending on how well wear leveling is implemented in the card's memory controller, attempting to write a larger file over these spaces will fragment it, thus slowing down the write speed and very slightly increasing the chance of file corruption.

Formatting the card regularly using the camera can actually be beneficial.

It insures that the file system on the card is in the form needed by the camera. When a card is formatted in another device, it might lead to issues if the camera can't find files it expects to find on the card. Different cameras have different versions of storing image files to the memory card. Most, if not all, are DCIM compliant, but there is enough variation between them that using a card formatted in another camera or on your computer may lead to corrupted images or file directories.

It is possible to create a 'disk image' on another device that replicates the file system created by a camera when a card is formatted in the camera. Formatting in the other device and then copying the disk image should work just as well as in-camera formatting. For more about that, please see this answer to Canon 6D resetting folders on card format - a way to stop this? But the easiest way is to just do it in your camera.

You're also much more likely to extend the life of the card, as compared to never formatting and only erasing images in the card, by formatting regularly.

The way flash memory cards (and USB flash drives) work is that the memory controller on the card assigns different areas of the card for specific directory locations each time the card is formatted. They do this for what is known as load balancing/wear leveling.

Flash memory has a limited number of write cycles it can tolerate before it fails. The number of write cycles each bit on a flash memory card can handle before it fails is very large but it will eventually wear out. So the controller tries to ensure that each storage location of the entire card is written to roughly the same number of times over the life of the card. What this means is that even after you format the card the controller will continue using parts of the card that have not yet been written to until each storage location on the entire card has been written to before it will go back and begin using the locations that have already been used.

That's good for recovery because it means not much is overwritten until the entire card has been written to once. But then it's going to go back and use the very beginning again if that space is showing as empty. When you delete a file normally the space that file used is marked as empty but the state of each bit in that space is not altered. When you format a card pretty much all of the regular storage space on the card is marked as empty and available for use.

Formatting regularly helps the memory controller on the card do a better job of wear leveling. It also allows for better card performance by increasing the likelihood of sequential write operations. If you leave the same files on half the card and repeatedly write files to the other half, erase them, and rewrite other files then all of the wear is going to go to only half the card and the life expectancy of the card before half of it fails will be shorter!

Additionally, if a memory card's controller has detected bad blocks on the card it will remove those blocks from the listing of available blocks for the card. It does this each time the card is formatted.

Most memory cards have a bit more memory than their listed capacity. When a bad block is identified, the memory controller will no longer use the bad block and replace it with some of the 'spare' memory on the card. One of the common differences between top name brand memory cards and generic/no name memory cards is the number of bad blocks mapped out of the card's total memory before it leaves the factory. The greater the amount of memory that must be mapped out, the less reserve memory is available on the card for the controller to use as other blocks fail later on. All of the major brands (Lexar, SanDisk, Transcend, Kingston, etc.) get their components from the same handful of suppliers that actually manufacture the memory chips and controller chips. So do the off-brand names, but they usually buy the leftover components that may or may not have been good enough to pass the QC of the major brands' buyers. One of the thing the buyers look at is how much reserve memory remains on the chips after the bad blocks have been mapped out at the factory.

  • Michael, two small clarifications, the information is written to flash memory using blocks, not bytes or sectors so you should update/write entire block. And write cycles are something like 10000 to 100000 (IMHO not so large number) – Romeo Ninov Aug 23 '18 at 19:44
  • 1
    @RomeoNinov Not so large a number if one is using it as RAM. But it is what I consider a large number if each write cycle for, say, a 32GB memory card means I've taken another 32GB worth of photos. 32GB is over 1,000 22-24 MP raw image files per write cycle, which would mean 10,000,000 to 100,000,000 images that could be written to the card over its life cycle. Most of us replace cards much more frequently than that as faster and larger cards become available, or as newer cameras use different types of flash memory media. – Michael C Aug 24 '18 at 5:11
  • I don't think formatting affects card-implemented wear-leveling at all. The card just presents a block device — it doesn't know anything about the filesystem on top of that. – mattdm Aug 24 '18 at 12:57
  • @mattdm Sure it does if half the card always contains the same files that are never erased, thus all of the write/erase cycles are being carried out on the other half of the card. – Michael C Aug 24 '18 at 16:05
  • 1
    Maybe I'm misunderstanding what you're saying. Sure, if you leave half the card always with static data at rest, you're doubling the strain on the other half. But there's no difference between formatting and deleting all files. – mattdm Aug 24 '18 at 16:17

Not the answer you're looking for? Browse other questions tagged or ask your own question.