Given a range of values, of varying lengths from 40,000 bytes in length to 100,000,000,000 bytes in length, the total number of combinations available to that range greatly exceeds the possible number of values represented by MD5, weighing in at a mere 128 bits of length. If you say 'MD5 can uniquely identify all files uniquely' then you have a logic error. I also advise people read mailing lists here titled 'file check' : MD5 is Exceedingly weak for collision conditions. Seeing I'm getting downvoted by people doing naïve approaches to file duplicate Identity, If you're going to rely entirely on a hash algorithm, for goodness sake, use something tougher like SHA256 or SHA512, at least you'll reduce the probability to a reasonable degree by having more bits checked. Seeks forward in those files until one differs.Use MD5s to find possible candidate duplicates, and then for each pair sharing an MD5 MD5 sums are not a reliable way to check for duplicates, they are only a way to check for differences.

Duplicate image deleter