Looking for a fuzzy image comparison tool (not online, but for harddrive search)

Question

I am looking for a programm, that compares screenshots from documentaries with a (very small) number (around 600) of source screenshots (from the footage they originate from). The differences between the source and the documentary screenhots my vary heavily, so I look for a tool that allows to assess rather weak levels of similarity.

I admit that I am not a coder, so would welcome a solution, as simple as possible. I could provide examples, if someone wants to help out (historiographical university project about footage from national socialism)

I am not looking for a duplicate finder.

Answer 1

I’ve successfully done something like this in the past for an entirely different reason. You’ll probably need someone who can code to package this up and loop through your drive, but the approach I took was:

Convert all images to smallish sizes such as 800x600 or even somewhat smaller.

Convert small images to Black and white with low dynamic range: 4, 2 or even 1 bit per pixel.

I used “ImageMagick” to do the above, but the tool doesn’t matter.

Now run a Fuzzy Hash against all the images.

I’ve used both “ssdeep” and “deeptoad” to produce match scores.

The concept behind it is that by first converting all images to small poor quality variants, you eliminate subtle changes you do not want to be factors. The smaller size also reduces fuzzy hashing processing loads substantially.

The fuzzy hashing scores attempt to evaluate degrees of match.

You may have to play with the parameters to find ones that work acceptably for you.

Note that the "ssdeep" user guide gives an example of comparing segments of videos to reference videos. This may be closer to what you want, but I' not sure.

Good luck!

Answer 2

While this may not fully answer the question (I don't know of such a program), what you're attempting to do is very related to video compression.

Video compression works by splitting the video into small blocks. Then, each block has an associated motion vector. A non-keyframe is calculated from a keyframe, with motion vectors applied, and with difference blocks specified for the differences that cannot be explained purely by motion vectors.

I think you could define a measure for the difference by comparing the discrete cosine transformation coefficients of the difference blocks, and the length and uniformity of the motion vectors.

So, for example:

• if the camera was shifted a little, most motion vectors have the same length and same direction
• if some subject has moved in the image, then only for that subject there are large motion vectors; most motion vectors are zero

What this doesn't take into account is exposure. You could try normalizing the exposure and then taking that into account in comparisons.

I'm sorry to say that what you're attempting to find may not be available (and software recommendations are often off-topic here, too), and building such a system by coding it from the scratch is very, very hard indeed.

Answer 3

The simplest way I've found to do this is to use the compare command from the ImageMagick tools. It has a way to vaguely compare images, for example, ignoring only a small number of different pixels between two images. See here: https://imagemagick.org/script/compare.php for more details.

I use it as a way to filter out similar images when capturing images for the purposes of surveillance. The exact command I use is below and it does a good job of getting rid of very similar images (the surveillance runs 24/7 generating hundreds, and occasionally thousands of images a day).

compare -verbose -metric AE -fuzz 25% <imagefile1> <imagefile2>

If the score from that command is less than a certain threshold (ie. not enough difference in the images) then the latest image gets deleted. You might need to use some trial and error in setting the correct fuzz and threshold levels. You might also want to experiment with the FUZZ metric as well - see the documentation link above for more details on the metric parameter.