Skip to content

Deduplicate deletes all but one empty file (__init__.py, .gitkeep, py.typed) while reclaiming 0 bytes #151

Description

@matt-edmondson

What's wrong

All zero-length files have the same SHA-256, so FindDuplicates (FileDeduplicator/Deduplicator.cs:31-34) puts every empty file in the scanned tree into one group. DeleteDuplicates (Deduplicator.cs:39-99) then keeps the copy with the shortest filename and deletes the rest.

Deleting them frees nothing. But an empty file usually matters because it exists: Python __init__.py, .gitkeep placeholders, py.typed markers, sentinel and lock files.

Failure scenario

  1. The tree contains a/__init__.py, b/__init__.py and logs/.gitkeep, all empty.
  2. Run Deduplicate -p ~/src. It reports one group of 3 with "Space to reclaim: 0 B".
  3. Answer y. .gitkeep is kept because it has the shortest name, and both __init__.py files are deleted. Both Python packages stop importing.

That is irreversible damage for zero benefit.

DeduplicatorTests.EmptyFilesAreDuplicatesOfEachOther asserts that empty files form a group, so reporting them is deliberate. Deleting them is the problem.

Suggested fix / acceptance criteria

  • Leave zero-length groups out of DeleteDuplicates and the deletion plan. Scan and Stats can still show them, for example as "N empty files (not deduplicated)".
  • Optionally, add an opt-in --include-empty switch for anyone who really wants empty files collapsed.
  • Test: Deduplicate on two empty files plus one real duplicate pair deletes only the real duplicate and leaves both empty files in place.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions