There is a brand new version of Excire Foto (2.0) that adds a sophisticated and tuneable AI-based duplicate finding/weeding function.
I've been very happy with Excire Foto 1.3.4 for its AI-based keywording, and for its ability to sort through photos by similarity, or by faces. But it looks like version 2.0's dup finding is just what I needed!
You can set a slider to choose the degree of similarity required to call something a "duplicate." At the most strict setting, the images have to be identical, although not the same pixel dimensions. At the most loose setting, it can help you cull excess shots by including ones that differ subtly, so you can pick the best one.
Excire Foto 2.0 "Find Duplicates" dialogue.
As you can see, you can allow for sequences, which doesn't suit my needs, as I do a lot of time-lapses that have more than eight seconds between shots.
Once you've set the parameters, it displays progress while it works. My imagebase of several hundred thousand images (previously indexed by Excire Foto) took about ten minutes to return nearly 70,000 duplicates. (It's a long, sad story about trying to get my images out of Apple Aperture multiple times, after Apple abruptly stopped supporting it.)
When this process ends, you'll have your images grouped by similarity.
This is the listing of some 70,000 duplicate hits.
You can change how they are grouped and sorted, and you can see in the first row, a time-series, that several of these images are subtly different. Enter the "Flagging Assistant."
The "Flagging Assistant" allows you to automagically choose from among duplicates.
This allows you to select and flag images for review (or not!) and deletion, based upon criteria you can select.
Grouping by ascending photo count allowed me to put the sequences to the end, and focus on the true duplicates. Sorting by pixel dimensions made it easy to cull the second ones, which were of equal or lesser quality.
Grouping by ascending photo count shows the least number of duplicates first.
I've been looking long and hard for such a tool, and this is the best I've come across for the task! It's been out for not even a day yet, so I still have a lot of exploring and learning to do.
Getting the initial index built takes a long time for large collections. My ~300,000 images took about three days to index. If you are impatient, and if, like me, you have images in a file hierarchy, you can index individual sub-hierarchies if you'd rather. I just threw my entire terabyte
Pictures folder at it, to see how it would do!
Indexing takes time because it is doing much more than finding duplicates — it is using an AI-trained model to assign keywords, which is wonderful in its own right. That's why I originally bought it, and I'm delighted that this new version read my mind and made it simple to find duplicates!