I have +120.000 images crowdsourced from a lot of phone and computer backups.

In that stack, I have real photos and screenshots that I want to keep, but also numbers of "junk images" (almost the half of total images), generally of small sizes, such as icons, logos, internal software bitmaps, grids, ad banners, low-quality memes, etc.

Is there a software that is able to recognize that "junk" images and to move them to another folder in order to review them before deleting?

(It's a preliminary step before managing duplicate images, failed ones, re-sorting the rest...)

  • These are your images, right? Crowdsource isn’t a term I’d apply to my own files and I can’t help but wonder if these are images taken through the course of your work on other’s devices.
    – OnBreak.
    Apr 10 '19 at 17:58
  • Junk images ? , you did not take photos of your junk did you?
    – Alaska Man
    Apr 10 '19 at 19:34

Since you already state that you will be going through images in at least two passes, I will not concern myself too much with false positives, since you will presumably catch them at a later time.

small sizes, such as icons, logos, internal software bitmaps, grids, ad banners, low-quality memes, etc.

You can use ExifTool to move files. You decide what qualifies as junk, then construct the appropriate command to throw them into folders.

Personally, I would move files based on extension first. Then sort by date and camera, since images from the same camera and date are likely to be related. Then I'd think about sorting whatever is left over by image size, but put it off till later because I can't remember the command right now.

  • File extension

    # Move files based on extension
    exiftool -Directory=delete -ext ico -ext gif -ext bmp -ext svg .
    # Easier to use native commands.
    # Following is for Linux or Mac OS.
    mkdir delete
    mv ./*.ico ./*.gif ./*.bmp ./*.svg delete/
  • Exif data

    # Move images that were taken with a digital camera
    exiftool -if '$model' -Directory=keep .
    # Can give particular cameras their own folders
    exiftool '-Directory<${Make;} ${Model;}' .
    # Can put images into folders based on date taken
    exiftool -d '%Y%m%d' '-Directory<${DateTimeOriginal}' .
    # Or both date and camera...
    exiftool -d '%Y%m%d' '-Directory<${DateTimeOriginal} - ${Make;} ${Model;}' .
  • File name – Similarly named files may be related. For instance, document_page_1.tif, document_page_2.tif, etc.

    # I don't know the relevant command.
    # I would need to look it up.
    # An ExifTool master is invited edit...
  • Resolution – Screenshots tend to have standard sizes. You should consider your computer-use history to determine the most relevant screen dimensions.

    # dump images into folders based on image dimensions
    exiftool '-Directory<$ImageSize' .

Note: Use of quotation marks in Windows will be slightly different. Instead of single quotes, switch to double quotes. There may also be other differences when running ExifTool in Windows.

  • if i remember correct you could move the files on windows also with native commands with the powershell at least but it is a little bit more complicated. described here: stackoverflow.com/questions/7151408/…
    – LuZel
    Apr 11 '19 at 7:26
  • @LuZel - There's also the cmd shell. I don't bother trying to figure out what the Windows commands will be because I'll probably get them wrong. It's been over a decade since Windows was my main OS. PowerShell didn't even exist back then.
    – xiota
    Apr 11 '19 at 7:32
  • i recommended the powershell over the cmd shell because it is more similar to the unix shell and has the same and more capabilities then the cmd shell (which is dying on windows by microsoft) and you can use shell commands. but the use of only the exiftool would probably be easier for most users.
    – LuZel
    Apr 11 '19 at 7:34

Generally speaking: No. No software can know what you consider or don't consider to be junk. Is every raster graphic below 128x128 pixel junk? If so, what about 129x129 px? Is every PNG junk? Is every picture with > 50% of the pixel area in white junk? Is every picture to be discarded that has no human face in it?

You can find solutions for all of these - but all can contain false-positives, as not a single one of the above criteria is causally linked with "junk" images. Even I as a fellow human cannot know what you would consider "junk" - I can certainly learn it (either by asking you several times or by creating a mental filter, e.g. with the questions I mentioned), but again, if there is not an absolute distinction (e.g. "all junk and only junk is PNG" or something like that), I might have some false-positives.

You can filter it out in waves: First filter below a certain size that would be useless, anyways. But after that...I would have to see the entire collection to know what to consider next.

  • OP states that this will be a multistage process. Although Q can be interpreted to ask for software that can read minds to determine what is and isn't junk, OP does not state that software must operate without user guidance or intervention.
    – xiota
    Apr 11 '19 at 7:55
  • @xiota it still holds true, though: not a single one of the criteria can sort out all junk and only junk at a 100% success rate. this won't be different if you use one or more.
    – flolilo
    Apr 11 '19 at 21:07
  • OP already factors in the possibility of false positives in the Q.
    – xiota
    Apr 11 '19 at 21:31

You are unlikely to find a viable "Single Pass" solution to a problem like this, and a solution will depend on what exactly you have to deal with in the collection.

How much care and attention you put into this process will depend on how critical any mistakenly culled images might be.

  • If it really doesn't matter if some get dropped, then you can move faster and with less care.
  • If you suspect there may be outlier images that ARE important, then you will want to carefully review images impacted at each step.

Culling images in stages is likely your best option by using a number of tools.

Easiest step may be to identify exceedingly small file sizes. sort the contents of a folder by file size, then drag everything below that threshold into a cull-folder.

You can repeat similar cullings based on file type.

After that you might be able to come up with some script or automation if you can still spot 'obvious' failures, such as images with nothing but text. [However programming and configuring such scripts is a bit beyond the scope of this Stack Exchange. But open source image recognition tools written in Python are potential options to explore.]

After you exhaust above options, you will come down to the nitty gritty culling, looking at each image and saying Yes or No to it.

For this stage I would suggest a tool like Lightroom:

  • Configure it to auto-advance after grading
  • Import a section of images to a collection
  • For each, either hit X to reject, or assign a flag/grade
  • When finished, use its tools to filter and move images to the desired location.

[Options such as training an AI might be viable as part of the scripting phase before resorting to something like Lightroom, but it will take time and computing power to pull it off.]

  • OP already states in Q that this will be a multistage process. Solutions offered, like programming in Python and training AI, are too cumbersome and hypothetical.
    – xiota
    Apr 10 '19 at 23:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.