jcburley/duff

Duplicate File Finder

duff's Issues

Implement a more efficient data structure

See duffdriver.c:155 for this TODO (has_recurse_directory).

Change to getopt_long and add long options

Is this necessary?

Optimize counting number of entries by incrementing the count each time we add an entry

See duffdriver.c:344 for more (report_clusters).

join-duplicates.sh throws this error: "mktemp: too few X's in template `...'"

Probably need to quote the results of the dirname ... just its $file argument is now quoted; otherwise, spaces can lead to dirname thinking a template is being supplied, rather than the default.

Fix manpage formatting of sample commands

Detect duplicate file arguments on command line

Use a read buffer

See duffentry.c:352 (compare_entry_contents).

0.5 ChangeLog, CHANGES, etc. needs to be more thorough

Not all changes have been documented.

Implement i18n through gettext

Level of interest? Licensing?

"Stable" sort not necessarily stable

duffdriver.c's cmpentryp() uses the relationship between the left and right pointers to determine an ordering if nothing else differs. The idea here is to provide a "stable" sort, so the order of output (reporting), within a set of duplicates, is the same as the input ordering.

Whether this is even needed, I don't know. In any case, I'm concerned that qsort() might be moving those pointers around. To be safe, something intrinsic to the relative Entry objects should be used -- perhaps something as simple as a monotonically increasing counter so each Entry has a unique ID.

Add ALGORITHM section (or similar) to manpage

Check all malloc return values for NULL

Possibly tolerate out-of-memory conditions by backing off some aggressive optimizations? But at least don't blindly proceed assuming it always returns a non-NULL value.

Extend to support "internal" files?

This would mean allowing e.g. an email to be pulled apart and the individual attachments treated as (virtual) entries; ditto a tarball, a compressed file, a compressed tarball, etc.

Make this less suboptimal

See TODO in duffdriver.c:263 for details (process_path).

Compare contents directly for clusters of two in excess mode?

(enhancement request) Order by pathname (for excess mode)?

I rely on duff a lot and often find that the file that appears in the excess mode is not the copy of the file I wish to delete (one file has been sorted into the right subfolder, the other has not).

I try to manipulate the names of the subdirectories to force the order to be how I want, but I don't think it can be fully controlled.

How can I control which copy appears in excess mode? If there is no such control, how difficult would it be to introduce such a feature?

join-duplicates.sh needs some TLC

Document that the new version passes the -t (thorough) option, since that makes sense for a script that automatically deletes "duplicates". Install as chmod +x. Document it on the duff man page.

Detect duplicate directory subtrees on command line

Do smart stuff to make things go faster

Version 0.5 (RC1) does duplicate-entry detection via qsort for better performance; this should make the biggest single improvement reasonably possible against 0.4.

Change linear lists into hash tables or trees?

A "temporary" array of pointers to Entry structures is now used in 0.5 so qsort() can sort it for us. Not sure what else would be worth doing here...any ideas?

Make directory cache less stupid and slow

Add more examples to manpage