Skip to content

Doc staleness via git history (context.drift) - #11

Merged
ranjan-mohanty merged 2 commits into
mainfrom
roadmap/doc-history
Nov 15, 2026
Merged

ranjan-mohanty merged 2 commits into
mainfrom
roadmap/doc-history

Conversation

@ranjan-mohanty

Copy link
Copy Markdown
Collaborator

Roadmap item: Doc staleness via git history (a doc untouched while the code it references churned).

What

New check context.drift (context category, weight 1) in src/checks/doc-history.ts. It complements context.fresh: that check catches references to paths that are gone, while this one catches docs describing code that changed underneath them.

  • Local git log only, no network. For each context doc (root and nested AGENTS.md/CLAUDE.md, Cursor rules, Copilot instructions), the check takes:
    • the doc's last commit, and
    • the existing paths the doc references, resolved like context.fresh does. Manifests, lockfiles, changelogs and other docs are excluded, because they change for unrelated reasons.
  • It then walks <doc commit>..HEAD -- <refs> (--no-merges, capped at 5000 commits) and counts commits that:
    • edited a file the doc names, or
    • added or removed files under a directory it describes. Ordinary edits inside src/ don't make an architecture section stale; new or deleted modules do.
  • A doc is drifting when it is ≥ 90 days old (relative to HEAD, so results are deterministic) and ≥ 25 such commits happened. Score = 1 − 0.5 × (share of drifting docs), so the minimum is 0.5.
  • Guards:
    • Symlinked docs (CLAUDE.md -> AGENTS.md, common in pydantic-ai) are judged by their target, because a symlink's own commit never changes.
    • A drifting doc with uncommitted edits is skipped.
    • Docs younger than 90 days skip the history walk, which keeps it cheap.
  • N/a outside git, in shallow clones (the summary says to use fetch-depth: 0) and for docs that reference no code. CI checkouts default to depth 1, so CI scores don't move unless you fetch full history.

Example finding (excalidraw):

   ! Docs keep up with the code they describe (git history) (75%) — 1 of 2 doc(s) untouched while the code they reference churned
       · CLAUDE.md unchanged for 537 days while 93 later commits edited the files it names or added/removed files in the directories it describes (4 of its 5 paths, e.g. `packages/excalidraw`, `excalidraw-app`, `packages`)

Tests

test/doc-history.test.ts (6 tests). It builds synthetic repos with backdated GIT_COMMITTER_DATE commits and covers:

  • flagged when the file churned after the doc
  • passes once the doc is updated
  • needs both age and churn
  • dir refs count only add/remove, not edits
  • a symlinked CLAUDE.md is reported once, under its target
  • a dirty doc is skipped
  • n/a for shallow clones (git clone --depth 1 file://…) and for prose-only docs

npm run check && npm run build pass (89 tests). Self-score is still 100/100.

Scores (before → after)

On the 14-repo comparison set, only vitest moved: context 87 → 88, overall 91 → 91. Its docs were recently edited, so it gains a passing check. Repos without context docs are unchanged.

Extra full-history clones (--filter=blob:none, read-only):

repo overall context
excalidraw 88 → 87 88 → 87 (CLAUDE.md last changed 537 days ago)
openai-agents-python 85 → 86 73 → 75
pydantic-ai (18 nested docs) 89 → 89 83 → 85
python-sdk 93 → 93 90 → 91
ollama 79 → 79 n/a (no code refs)

Cost: pydantic-ai (6k files, 5.6k commits, 39 docs) adds ~290 ms; vitest ~50 ms.

Decision for the owner

The 90-day / 25-commit thresholds (exported as MIN_AGE_DAYS / MIN_COMMITS) were tuned so that none of the actively maintained docs above trip them. Excalidraw's CLAUDE.md is the only hit. Tighter thresholds would catch more, at the risk of nagging healthy repos.

🤖 Generated with Claude Code

ranjan-mohanty and others added 2 commits November 15, 2026 00:22
A context doc is drifting when it hasn't been committed for 90+ days and
25+ later commits edited the files it names or added/removed files in
the directories it describes (manifests, lockfiles and docs excluded).
Local git log only; n/a outside git, in shallow clones and for docs
without code references. Symlinked docs are judged by their target and
docs with uncommitted edits are skipped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conflicts: src/checks/index.ts (docHistoryCheck registered alongside agentToolingChecks and runChecks), README.md (check table keeps every item and adds the 'not drifting' Context item; both the --run and doc-drift paragraphs kept; roadmap ticks doc staleness) and CHANGELOG.md (context.drift added to the single [Unreleased] Added list).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ranjan-mohanty
ranjan-mohanty merged commit 493c52f into main Nov 15, 2026
9 checks passed
@ranjan-mohanty
ranjan-mohanty deleted the roadmap/doc-history branch November 15, 2026 00:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant