I've been pulling my hair out at this task, which seems insurmountable. I have spent a lot of time consolidating all my past storage drives, and then doing it again, and now discover I have many duplicate copies. (You know that button that says save both? I clicked it too often)I'm not sure which of these duplicates to keep, because though the dimensions of the photos are the same, most of them have a small kb difference (i.e., 58kb vs 56kb). Does it matter which ones I discard? UGH! Feeling overwhelmed. Also, my logical brain wants to know why there is a difference in the saved files.
If the images actually are duplicates then the additional data is most likely to be metadata which has been added at some point. In other words the files have been modified, although the picture itself has not been touched.
Most likely an automated function in your photo viewing/management software has done it. For example facial recognition which is available in Windows Live Gallery, Picasa and others may have modified the metadata as a background task.
To see what modified them you could examine and compare the metadata between two images using something like
exiftool if your curiosity really has to be satisfied.
I'd say if you're unsure then keep the larger file in each case, although whatever was added can probably be recreated by the application if needed, so you shouldn't need to agonise too heavily over which one to keep or lose sleep if you get the wrong one.
Not to sound schoolmasterish, but, if you don't want this frustration to be repeated, then you need to be diligent about digital asset management. Keep originals, and keep edited photos worth keeping. If you use some kind of image management application for cataloguing digital images, pay attention to where it is saving files, and if it is saving copies when edits are made. If you have the same set of images backed up on several hard drives, then be consistent about which hard drive you use as a "working" hard drive for edits - the aim is to know where the latest/master/original version (or however you manage it) of any file is. That's my advice at least.
Just today, there is an article over on Lifehacker about applications for finding duplicate files in general, and duplicate images in particular: http://lifehacker.com/the-best-duplicate-file-finder-for-windows-1696492476
(Edit: I just read that Lifehacker article properly, and maybe it's not so useful, but you might get some pointers there anyway.)
You might be able to find some software there to help identify duplicate images, but what about photos that appear to be duplicates, but don't have bit-for-bit identical image data? In these cases, I agree with the suggestion in an earlier comment to retain the older file. Hopefully, 'Date Modified' is reliable here. Larger filesize does not necessarily mean better quality; what I would be trying to achieve here is to determine which file is the original (or closest to it).