Skip to content

A symlink or submodule anywhere in the base tree makes shipgate diff refuse the whole repository — 4 of 12 public repositories, including CLAUDE.md -> AGENTS.md #688

Closed

Description

@pengfei-threemoonslab

Observed

Zero-noise harness over 12 public repositories with committed host configuration, 2026-09-11, main at 396de3b1, full clones. Four repositories failed on every one of 40 first-parent steps with the same shape:

repository refused at what it is
miantiao-me/bm.md CLAUDE.md a symlink to AGENTS.md — the common one-instruction-file convention
getsentry/sentry-mcp .claude/skills a symlinked skills directory
gtkx-org/gtkx website/public/tasks-forms.png a symlinked image nowhere near any host surface
awslabs/nx-plugin-for-aws .git-secrets/git-secrets a symlinked git hook helper
ConfigError: Git tree contains unsupported external binding at CLAUDE.md (mode 120000, type blob).

A third of the sample cannot run diff at all. Two of the four are refused because of a path the host reader would never open.

Why

_materialize_isolated_tree (cli/verify/git.py, around line 1786) walks git ls-tree -r -z of the base commit and raises on any entry whose mode is 120000 (symlink) or 160000 (submodule), regardless of path. archive_tree calls it for every diff, so one symlink anywhere in the tree — a PNG under website/ — refuses the entire comparison.

The refusal exists for a reason: a materialised symlink could point outside the scratch tree, and a submodule is content this checkout does not hold. Both are correct things to refuse at the path the reader would read. Refusing the repository because of an unrelated PNG is not that property; it is a repository-wide fail-closed on a per-path concern.

Proposed

  • Materialise only what the reader reads. Restrict the tree walk to the host-surface path set (the adapter exact paths and globs, plus their parent directories) instead of the whole tree. This also removes most of shipgate diff repacks the repository's entire reachable history to read one base tree: 27 s per call on a 26 MB repository, 1 s on a toy one #686's cost, since the tree pack shrinks to a handful of files.
  • A symlink at a host-surface path is an inventory issue, not a refusal. CLAUDE.md -> AGENTS.md is recorded as symlinked_instruction_file with the target path; if the target is inside the tree, read the target and attribute the grant to both paths; if it points outside, record the issue as blocking for that artifact only.
  • A submodule at a host-surface path is recorded as unsupported_submodule for that artifact only.
  • Fixtures: a repository with a symlinked PNG far from any host file (must produce rows normally), one with CLAUDE.md -> AGENTS.md (must produce the instruction grant once with the issue recorded), one with a symlink pointing outside the tree (blocking issue on that artifact, rows for everything else).

Acceptance

  • All four repositories above produce rows or a correct empty comparison on every step; none is refused.
  • A symlink pointing outside the tree at a host-surface path yields a blocking issue on that artifact and rows for the rest.
  • Negative control: the four fixtures fail on main at 396de3b1.
  • The harness re-run reports 0 repositories refused for this reason.

Found by the zero-noise harness built for the v1.0 roadmap. Parent #645. Related #683, #686, #660, #651.

Activity

  1. pengfei-threemoonslab commented on Oct 3, 2026

    @pengfei-threemoonslab
    ContributorAuthor

    A current released-1.2.0 replay still hits the repository-wide refusal described here.

    Source: govuk-forms/forms-deploy#2292
    Pinned base: c4efae4ceabe3a1338b0bb96a4c255b8b08a66a6
    Pinned head: 82f8e21d (full SHA in the replay receipt)
    Command: agents-shipgate diff --base c4efae4ceabe3a1338b0bb96a4c255b8b08a66a6

    Verbatim output:

    Cannot compare against c4efae4ceabe3a1338b0bb96a4c255b8b08a66a6: base_inventory_incomplete; head_inventory_incomplete
    

    Coverage JSON names infra/deployments/integration/account/tfvars as unreadable in both inventories and states that symbolic links and filesystem aliases are not followed. The result has comparison_status: incomparable, review: null, and no rows.

    The changed .claude/settings.json in this PR contains attribution metadata only, so this particular PR would not qualify for agent-capability outreach even after fixing the comparison. It is recorded here as a real 1.2.0 input-limit reproduction, not as a lost qualifying sample.

    Evidence is retained under research_notes/outreach-1.2.0-agent-capability-50-2026-10-02/cases/govuk-forms__forms-deploy__pr2292/ (replay.json, diff.json, diff.txt, raw-host.diff). No symlink was removed or candidate source modified to obtain a successful comparison.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions