duff the duplicate file finder can find recursively such files using duff -r <directory>
A file list (skipping one of the files to be kept) can be created using duff -re <directory>
> ../del.txt
After that while read file ; do rm "$file" ; done < ../del.txt will remove the files
Alternatively this can be done with rm $( duff -rPze <directory>
)
A duplicate file finder for Linux file system should understand links. duff has command line options for that, as -P do not follow symbolic links to directories (symbolic links to files are never followed)
-p do not follow hard links
-z does not compare files with 0 size
duff -pPrz <directory>
To improve speed most programs look first for the disk space and even for the first bytes to match and if everything is identical the time consuming checksum is calculated.