Skip to content

code-index: unrepresentable Git paths vanish from the snapshot while status reports complete #2721

Description

@ScriptedAlchemy

Problem

Code-index capture drops a Git path that no logical path can carry (CapturedFileOutcomeV1::Unrepresentable, e.g. a literal backslash, a control character, or leading/trailing whitespace) from the roster. The only trace is a tracing::warn! (code_index_sources_skipped_unrepresentable_path). No snapshot row, disposition, coverage count or status field records it, so status still reports the index as complete and fresh while a tracked source file is missing. This is the "catch and downgrade to a log line" shape AGENTS.md forbids. It was introduced by #2429 (171aede), whose regression test backslash_paths_are_skipped_and_the_worktree_stays_fresh asserts the vanishing as the intended outcome.

Privacy-withheld files get a snapshot row (Ignored), but the withheld reason is also only logged (code_index_sources_withheld_by_privacy), so the same fix should carry it.

Reproduction (debug CLI from origin/master 9b79b48 plus a configuration-only change, code-index code unchanged; isolated HOME; one daemon under MemoryMax=6G)

$ git ls-files
docs/guide.rs
src/lib.rs
"src/odd\\name.rs"

$ tracedecay tool tracedecay_status --args '{"format":"json","wait_for":{"state":"fresh","timeout_ms":60000}}'
code_index_freshness.worktree: {"coverage": "complete", "staleness_state": "fresh", ...}  wait: {"outcome": "reached"}

$ tracedecay tool tracedecay_files --args '{"format":"json"}'
{"count":1,"files":[{"bytes":17,"path":"src/lib.rs","symbols":1}],"layout":"grouped"}
   (docs/guide.rs is excluded by the configured index.exclude.v1 `docs/**`; src/odd\name.rs is simply gone)

daemon log, the only record:
WARN ...git_tree_capture: code_index_sources_skipped_unrepresentable_path skipped=1 named=src/odd\name.rs

Why it was not fixed in the root-cause-swallowed-errors lane

  • The roster (CapturedFileRosterV1::push/finish) is in git_tree_capture.rs. The worktree capture that the fix(code-index): skip Git paths no logical path can carry #2429 test exercises (capture_authoritative_snapshot*) and active-generation reuse are in code_index_scheduler/reconcile.rs, which another root-cause lane owns. Status freshness is projected from the sealed generation in code_index_scheduler/registry.rs (dashboard_freshness_identity), which the layered-generations lane owns.
  • A snapshot row needs a logical path that passes validate_code_logical_path, which is exactly what these paths fail. "Reuse the omitted disposition" therefore needs a naming decision first.

Proposed design

  1. Domain: add SnapshotFileDispositionV1::Unrepresentable (an omitted disposition, never Present). Name the row with a deterministic, reversible escape of the raw Git path bytes, placed where no tracked Git path can collide. Git refuses .git path components, so a .git/unrepresentable/<hex of raw bytes> style name is collision-free. The alternative, percent-escaping in place, can collide with a real file literally named a%5Cb.rs.
  2. Capture: CapturedFileOutcomeV1::Unrepresentable becomes an omitted row through omitted_source_file, like Withheld. Delete unrepresentable_paths and its warn. The freshness witness needs no change: SourceContentManifestV1::for_snapshot keeps only Present rows, and the sweep already filters invalid candidates.
  3. Coverage: production/helpers.rs::coverage_summary counts the disposition (for example under files_excluded, or a new typed count). Status adds a typed omitted-sources projection with count and reason (unrepresentable path, privacy withheld) from the sealed snapshot in registry.rs. It must not demote CodeIndexFreshnessCoverageV1 to partial, because is_authoritative() requires Complete and wait_for: fresh would then never be reached for such a repository.
  4. Consumers with exhaustive matches: omitted_file_occurrence_id, dashboard code_reads.rs (RevisionPairFileDispositionV1, a contract regen), native_integration/analysis.rs, and search-eval candidate_output.rs.
  5. Test (fails on master): a Git fixture with src/odd\name.rs, mounted through CodeIndexSchedulerRegistryV1. Assert that the sealed snapshot holds the row with Unrepresentable, that the status omitted-sources projection reports 1 with that reason, and that the worktree stays fresh on an unchanged probe (the property fix(code-index): skip Git paths no logical path can carry #2429 protects).

Related, same area (code-read only, not runtime-verified)

  • capture_exact_git_tree_snapshot names blobs with entry.filepath.to_str_lossy(). Invalid UTF-8 path bytes become U+FFFD, which validate_code_logical_path accepts, so such a file is indexed under a name that does not exist, and two such names in one directory can collide.
  • code_index_scheduler/activation.rs: a failed deferred mount is logged with tracing::warn! and the route returns to idle. There is no typed state.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions