What are the pros and cons of deleting all of my archived RAW files?

Question

I've been shooting RAW+JPEG for years, but about the only time I use the RAW files is to play around with RAW processing techniques. For everything else, I use the JPEGs. I have several multi-TB hard drives that are running out of space. Instead of buying yet another hard drive, I'm considering running the following command:

# enable double asterisk for recursing directories
shopt -s globstar

# old command
# \rm -r **/*.raw **/*.arw **/*.crw **/*.cr2 **/*.raf **/*.nef **/*.pp3

# only delete files with an associated jpg
for i in **/*.jpg ; do
   f = "${i%.jpg}"
   for e in raw arw crw cr2 raf nef pp3 xml ; do
      rm "$f.$e" "$f.$e.pp3" "$f.$e.xml"
   done
done

Although I expect to feel an initial sense of loss, I likely won't miss them in the long run because I don't use them. I've already deleted some older raws, and so far the only noticeable repercussion has been increased disk space.

What are the pros and cons? Anything I'm missing? Perhaps this is something other packrats might consider trying?

Answer 1

Pros: You gain some space.

Cons: You won't ever be able to change your mind.

A 4TB drive is about $£€70 these days.
Buy one, stack all your old data on it & put it on a shelf. If the drive does die eventually, then at that point you'll probably regret not having a backup of it - but as you were going to throw them away beforehand, you ought to get 5 - 10 years of 'I wonder what happened to that pic... ahh, there it is' out of it, for something like the price of a month's TV/Internet access.

Personally, I'm not a fan of rm -r ... too many horror stories along the lines of "Oops, wrong path!". I'd be more inclined to find an alternative way to get them into the Trash if you're going to do it, then you've got an undo.

Answer 2

The biggest thing that is missing from a work flow of "Just blast it all away from the command line" is that of conscious control. And deleting your raw files to save space might leave you missing the forest for the trees...

• Are you actually deleting what you think you are deleting?
• Are you actually saving the files you really want to keep anyway?

First step to dealing with file-pack-rat-syndrome is to begin dealing with it before it happens on new files:

• Cull early, cull often.
• Use a modern image library management system, such as lightroom, and rate, flag, and keyword images early on. From here you can develop a workflow scheme that helps you clearly decide which raw files you wish to keep.
• Flag images with little to no value early on for deletion. [Ideally before you even bother keywording them.] If you have ten near duplicates of the same image with "slightly different lighting", then do you really want all ten of them?

Pick the best, trash the rest.

The important thing to remember is that images that you have no idea exist, or an easy way to find if you want them, have little to no value: Either ensure you establish a value in their storage. Keywording is your friend.

In my mind culling and maintaining a collection is an ongoing process. As new images are added, or skills, methods, or tastes change, then older images may become obsolete. Culling those of "Poor technical merit" is easy to do at time of ingest, and some artistic culling is also good to do at the time. But returning to the images for further review weeks, months, and years later is worth the effort to avoid an ever ballooning collection.

• If you build your personal collection to the point that you will never revisit 99.9% of the images in your life, then why are you hanging on to them and cluttering your drive? If a photo is not a "Good image" in your mind, or at least a "Good example of your past artistic views", then it is worth asking why is it being carted around as part of your collection?

As you put strong systems in place to keep from rapidly expanding your new images, you can begin addressing your old images in smaller bite sized chunks with confidence.

• Importing a new batch of images from the weekend? Process and cull one of your archive folders while you're at it.
• Have some spare time? Make a cup of tea or coffee, and sit down to run through old data to cull and keyword.

The important thing is to make deletion of files an active thought process, and massage random unsorted images into a usable collection of images, rather than gathering terabytes of meaningless data that holds little chance of ever actually being used at any point in the future.

Sorting and culling doesn't have to be a massively long process with minutes or hours of debating on each and every photo, it is perfectly fine to scroll through a folder from an afternoon of shooting and declare none of them are worth bothering with, but the active review of your collection is what improves it as a collection.

Avoid "Blind deletions" unless you're critically pressed for space, and avoid keeping even just the .jpegs "just because...". [

Blindly deleting all of your raw image data for the sake of storing thousands more .jpeg files that you'll never actually use or care about anyway hasn't actually put you any further ahead in the grand scheme of things.

The road out of file-pack-ratted-ness is not one of merely 'saving space', but one of addressing what the space you're using is actually being used for!

Answer 3

It's tempting to say that this is the equivalent, for a film photographer (like me) of destroying your negatives and keeping only the prints. This is something I would never do, because I can make more prints from the negatives, but without the negatives I'm pretty much stuck.

But that's wrong. If I destroy my negatives the only way I can make copies of pictures I have is by some horrible process of making photographs of the prints and printing from those. Those second-generation prints will not be as good as the first-generation ones, especially if I intentionally printed the first-generation print very dark or otherwise deliberately lost information that was in the negative. On the other hand, if you have a JPEG, you can make endless perfect copies of it, and if you are happy with that JPEG as an image then you never need to worry about the raw file. Even if you're not happy with it you can manipulate it to some extent.

So I think that no-one can answer this question but you: are you happy with the JPEGs? Do you think you will still be happy with them in ten years? Is there any possibility anyone else will care that you've destroyed these irreplaceable things once you are some famous dead photographer, or will it just make you more famous because of the mythical lost raw files? These are not questions anyone can answer for you, unfortunately.

I would think hard before doing it however.

---

How to do this on a Unixoid system

If you are making huge changes like this, then you typically want to do it in a very careful way. A good approach is to do a dry run which simply lists all the files you want to delete, and then do a real run which deletes them. You also need to do things in a way which is very robust against peculiar filenames. For the sake of argument let's say you want to:

• delete raw files under /myphotos;
• delete raw files with names like *.dng & *.DNG.

Then here is a process which should be reasonably safe.

1. Be in some safe working directory.
2. Make a list of the files you will delete: find /myphotos \( -name '*.dng' -o -name '*.DNG' \) -print > files-to-delete.
3. With a text editor look carefully through files-to-delete & make sure it is what you actually want to delete.
4. Delete the files (this step is irrevocable): find /myphotos \( -name '*.dng' -o -name '*.DNG' \) -print0 | xargs -0 rm. Here the -print0 and xargs -0 will ensure that files with odd characters in their names are handled properly.
5. Check you have done what you thought: run the find command from (3) again and there should be no output.

Note that I am not responsible for the safety of this: read the manuals & check carefully before making wholesale, irrevocable, changes like this.

Answer 4

My personal way of doing this:

I shoot RAW only - it is a bad habit, but since it gives me some peace of mind, I really do not care too much about changing it.

My library has three main folders (in descending file count): private (for holidays, pets, ...), professional (for work that is/was worth money), and projects (for larger, planned, [non-commercial] projects).

One day, I realized that I had roughly 500 GiB in RAWs on my hard drive. I looked through all photos and came to the conclusion that neither pet and holiday photos nor professional press photos need to be RAW - a decent JPEG will do for all of them (and is far more accessible).

Ever since then, I took Capture One (now: DigiKam) and went through all of the pictures. I delete whatever I do not like completely - and "develop" whatever is good enough to keep. Only pictures that in my opinion could be developed further artistically will be kept as RAW (without JPEG, typically). Whenever I feel the need to procrastinate, I try to go through a few of my photos.

---

If you are absolutely certain that you do not want any of the RAWs...go for it. Just nuke them - if there is a JPEG for every RAW and the JPEG is all you want, you will not lose a thing.

This raises a potential problem: What if a photo does not have a (working) JPEG? If you just delete all RAWs indiscriminately, there is a certain risk that you lose some important files.

Personally, this is why I do this on a picture-by-picture basis. It costs a lot of time, but then again, I get to appreciate almost forgotten treasures - and I can get rid of some of the worst photos.

Answer 5

No, don't just go deleting a huge amount of RAW files. If you haven't categorized the shots, purchase a large HDD and store the RAW files there. In the long term, you should consider categorizing your shots better, if not done already.

If you indeed have categorized the shots, I would keep the RAW files for those shots that you either:

1. find extremely valuable
2. find somewhat valuable and need some substantial processing done for the image apart from simple cropping

I find it hard to believe that you have that many shots that fulfill either of these two criteria. So, the space savings for shots not belonging to either category (1) or category (2) are going to be pretty significant.

In my case, I have found that quite many shots actually belong to category (2). The camera doesn't always choose the correct white balance and exposure. The exposure is probably the most typical adjustment I make for the RAW files.

Also, you should find out if your camera does lens corrections (vignetting, chromatic aberration, geometric distortion) automatically to JPG files. I don't know if this is the case for my camera, because I find slight exposure adjustments need to be done for about 50% of my shots, so I use RAW always and the JPG is there only for easily previewing the shots.

About 5% of my shots actually require so significant exposure adjustments that the JPG files are basically useless.