Skip to content

fix(metadata-fs): scope the watcher's dotfile ignore to paths relative to the root (#7150) - #7208

Merged
os-help merged 2 commits into
mainfrom
claude/issue-7150-watcher-dot-root
Aug 10, 2026
Merged

fix(metadata-fs): scope the watcher's dotfile ignore to paths relative to the root (#7150)#7208
os-help merged 2 commits into
mainfrom
claude/issue-7150-watcher-dot-root

Conversation

@os-help

@os-help os-help commented Aug 10, 2026

Copy link
Copy Markdown
Collaborator

Fixes #7150

MetadataPlugin attaches the FileSystemRepository at <project>/.objectstack/metadata (REPO_SUBDIR). The watcher's ignored matcher was a bare dotfile regex, and chokidar applies that matcher to the watched root path itself, not only to entries discovered underneath it — so the .objectstack segment of the root matched and the entire watch was inert. MetadataManager.setRepository() subscribes to repo.watch({}) and invalidates the registry entry and the list() cache per event, so a live consumer was wired to a source that could never fire in the layout the product ships.

Premise verification

Re-verified on origin/main @ aeadbd61c (which already contains #7000 / PR #7152): ignored: [/(^|[\\/])\../] live at repository.ts:438, REPO_SUBDIR = '.objectstack/metadata' at packages/metadata/src/plugin.ts:57, joined at :389. Premise holds.

Re-ran the filer's A/B harness on chokidar 5.0.0 with the repository's own watch options (ignoreInitial: true, depth: 2, awaitWriteFinish, usePolling: true, interval: 1000), two identical trees differing only in whether the root sits under a dot-directory.

Legend for the getWatched dumps below: keys are printed relative to the watch root; (root) is the watch root itself and .. is chokidar's entry for its parent. (Spelled (root) rather than in angle brackets because GitHub's body sanitizer strips <+letter as an HTML tag and silently deleted it from the first revision of this body.)

CURRENT regex | plain root (control)
   getWatched : ["..","(root)","view"]
   events     : [["add","view/b.json"],["change","view/a.json"]]

CURRENT regex | DOT root (production layout)
   getWatched : []
   events     : []

The shape choice, and why (triage asked for this to be recorded)

Triage ruled this implementation-layer rather than decision-box, with the choice to be made by measurement. Both candidate shapes were measured on the same harness.

Shape A — function matcher scoped to the path relative to the root (chosen):

SHAPE A rel-matcher | DOT root
   getWatched : ["..","(root)","view"]
   events     : [["add","view/b.json"],["change","view/a.json"]]

Identical to the plain-root control. The .objectstack bookkeeping subtree never enters the poll set.

Shape B — drop the regex, rely on the parseItemPath guard (rejected):

SHAPE B no ignore | DOT root
   getWatched : ["..",".objectstack",".objectstack/.log","(root)","view"]
   events     : [["add","view/b.json"],["change",".objectstack/.log/main.jsonl"],["change","view/a.json"]]

Shape B does restore event delivery, but the body's premise that parseItemPath makes the matcher redundant does not survive measurement. parseItemPath rejects exactly one name, .objectstack — so other dot entries leak straight through it:

Shape B — events reaching handleFsChange, with parseItemPath verdict:
   add    .cache/x.json          parseItemPath => {"type":".cache","name":"x"}
   add    view/.scratch.json     parseItemPath => {"type":"view","name":".scratch"}

Both would be published as MetadataEvents with source: 'fs' and re-emitted through notifyWatchers, while scanHeads skips every dot entry on boot (entry.name.startsWith('.')). That leaves the boot scan and the watcher disagreeing about what the repository contains — a worse defect than the one being fixed. Shape B also puts .objectstack/.log/ in the poll set, so every one of the repository's own log appends wakes handleFsChange only to be discarded.

So the fix keeps the original intent verbatim and corrects only the frame of reference: judge the path relative to the root, so dot segments belonging to the root itself are never considered.

Note on the harness: an editor .swp artifact is a poor probe here — chokidar's built-in atomic option filters .*.sw[px] before any matcher runs, so it is suppressed under Shape B too and proves nothing. .cache/x.json and view/.scratch.json are the probes that actually separate the shapes.

Tests

Two new pins in packages/metadata-fs/test/watch-dot-root.test.ts, which are the two halves of the watcher's promise — a fix that merely widened the matcher would pass the first and fail the second:

  1. sees an external edit when the root is under a dot-directory — the production layout, asserting the event fires.
  2. still ignores dot entries UNDER the root, including its own bookkeeping — asserts .cache/x.json, view/.scratch.json and appends to .objectstack/.log/main.jsonl produce nothing, then carries a control write to a real item proving the watcher is alive.

no-root-on-attach.test.ts is untouched: its watcher-arming pin deliberately uses a non-dot root to measure arming (#7000), and it keeps measuring that.

End-to-end consumer proof in packages/metadata/src/metadata-repository-fs-dot-root.test.ts: a real MetadataManager + FileSystemRepository on a real dot-rooted temp dir, an out-of-process write, asserted at subscribe(). Cheap in the end — 1.1 s.

packages/metadata-fs test:  Test Files  4 passed (4)
packages/metadata-fs test:       Tests  30 passed (30)
packages/metadata test:  Test Files  30 passed (30)
packages/metadata test:       Tests  592 passed (592)

pnpm --filter @objectstack/metadata-fs typecheck clean (packages/metadata has no typecheck script). ESLint clean on all three touched files. node scripts/check-nul-bytes.mjs OK.

Reverse verification

Direction predicted before running: restoring the old regex must turn the positive cases red, and must leave case 2's toEqual([]) green — because that assertion is satisfied by a watcher that emits nothing at all. That is exactly what happened, which is why case 2 carries a control:

FAIL test/watch-dot-root.test.ts > sees an external edit when the root is under a dot-directory
AssertionError: expected [] to have a length of 1 but got +0
 ❯ test/watch-dot-root.test.ts:108   the positive assertion

FAIL test/watch-dot-root.test.ts > still ignores dot entries UNDER the root, including its own bookkeeping
AssertionError: expected [] to have a length of 1 but got +0
 ❯ test/watch-dot-root.test.ts:150   the CONTROL, not the toEqual([]) at :138

 Tests  2 failed | 28 passed (30)

The consumer test was reverse-verified separately, and required rebuilding metadata-fs's dist from the reverted source first — packages/metadata consumes the built artifact, so reverting only src left it green against a stale dist (recording it because it is the trap that makes this kind of check silently meaningless). Against the correctly reverted build it burned its full 15 s deadline and failed:

× an out-of-process write reaches subscribe() in the dot-rooted layout 15486ms
  → expected [] to have a length of 1 but got +0

Scope note

The claim comment declared the file surface as packages/metadata-fs/src/repository.ts + its tests + .changeset/. This PR additionally adds one test-only file under packages/metadata/src/, which is the end-to-end assertion the dispatch asked for; no non-test file outside the declared surface is touched.

Changeset: patch on @objectstack/metadata-fs (user-visible — external edits are now seen).


Generated by Claude Code

…e to the root (#7150)

`MetadataPlugin` attaches the FileSystemRepository at
`<project>/.objectstack/metadata` (REPO_SUBDIR). The watcher's `ignored`
matcher was a bare dotfile regex, and chokidar applies that matcher to the
watched root path itself, not only to entries discovered underneath it — so
the `.objectstack` segment of the root matched and the entire watch was
inert. Measured on chokidar 5 with the repository's own watch options, two
identical trees differing only in whether the root sits under a dot-directory:

  plain root   getWatched: ['<root>', 'view']   events: add + change
  dot root     getWatched: []                   events: none

The matcher is now a function evaluated against the path RELATIVE to the
watch root, so dot segments belonging to the root itself are never
considered while dotfiles under the root — including the repository's own
`.objectstack/` bookkeeping subtree — stay ignored as before.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016R9de1FqP7NvwKvqXi92Gh
@vercel

vercel Bot commented Aug 10, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectstack Ignored Ignored Aug 10, 2026 3:56am

Request Review

@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/metadata-fs.

1 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/concepts/metadata-lifecycle.mdx (via @objectstack/metadata-fs)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

@github-actions

Copy link
Copy Markdown
Contributor

⛔ merge queue 构建失败 — 先分诊,再决定要不要重排

队列构建 31351599963 红了。队列跑的是全量套件(PR 侧 CI 只跑 affected 子集),
所以失败的测试可能在本 PR 没碰过的包里 —— 那不是重排能修的。每次盲目重排都会让排在后面的所有 PR 重建一轮。

失败的 job(日志抽取,best effort):

  • Test Core (3/3) — 失败步骤: Run this shard's tests

    �[41m�[1m FAIL �[22m�[49m test/fs-behavior.test.ts�[2m > �[22mFileSystemRepository — on-disk semantics�[2m > �[22mchokidar: external file change emits an update event
    

历史信号:

  • 本 PR 过去 24h 无队列失败记录(首次)。
  • 过去 24h 队列共有 6 个失败构建(不含本次)。

分诊清单:

  1. 失败测试在本 PR 改动的包里 → 真回归,修 PR。
  2. 失败测试与本 PR 无关 → 在其他 PR 的同类评论里搜同名测试;出现过 ⇒ flaky 实锤,开 issue 修/隔离那条测试。修好前重排只会再烧一轮全队列。
  3. 两者都不是 → 可能与同组 PR 语义冲突;等前面的 PR 落地或失败出队后再重排一次即可,不要连续重排。

Generated by Claude Code · merge-queue-triage workflow (#4859)

@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Aug 10, 2026

os-help commented Aug 10, 2026

Copy link
Copy Markdown
Collaborator Author

Merge-queue ejection triage (lane PM, session session_016R9de1FqP7NvwKvqXi92Gh) — per the triage bot's checklist above, before any re-queue.

Signature: test/fs-behavior.test.ts > FileSystemRepository — on-disk semantics > chokidar: external file change emits an update event — a pre-existing test in this PR's own package.

Verdict: load-timing flake, not a regression from this diff. Three readings:

  1. Green control on the identical SHA: c47cc79e0 passed this very test in PR-side CI (Test Core shards all success, personally verified at accept time) — same code, same test, green there, red only in the queue's full-suite run.
  2. The matcher change cannot reach this test's semantics (read-only verification): the failing test's root is mkdtemp(os.tmpdir() + 'objectstack-fsbeh-') — a plain, dot-free path. Old regex and new relative matcher agree byte-for-byte on every path this test produces; the behavioral delta of this PR exists only under dot-rooted watch roots.
  3. Mechanism fits load: the test rides usePolling: 1000ms + awaitWriteFinish stability windows; the queue runs the full suite with heavier parallel load than PR CI. First occurrence of this signature in 24h per the bot's history line.

Action: re-queueing once (re-arm auto-merge). Standing rule honored: same signature on a second ejection ⇒ escalate + file the flaky-test card, no further re-queues — each blind re-queue rebuilds every PR behind this one.

One forward-looking note: this PR's own new tests add more watcher-timing surface (4s/8s waits) to the same suite; if queue-load flakes recur in this family, the fix direction is deadline-widening or event-promise waits in the pre-existing test, filed as its own card, not silent re-queues.


Generated by Claude Code

@os-help
os-help added this pull request to the merge queue Aug 10, 2026
@github-actions

Copy link
Copy Markdown
Contributor

⛔ merge queue 构建失败 — 先分诊,再决定要不要重排

队列构建 31352802812 红了。队列跑的是全量套件(PR 侧 CI 只跑 affected 子集),
所以失败的测试可能在本 PR 没碰过的包里 —— 那不是重排能修的。每次盲目重排都会让排在后面的所有 PR 重建一轮。

失败的 job(日志抽取,best effort):

  • Test Core (3/3) — 失败步骤: Run this shard's tests

    �[41m�[1m FAIL �[22m�[49m test/watch-dot-root.test.ts�[2m > �[22mFileSystemRepository watcher — dot-rooted watch root (#7150)�[2m > �[22msees an external edit when the root is under a dot-directory
    

历史信号:

  • ⚠️ 本 PR 过去 24h 已在队列失败 1 次(不含本次)。 内容未变而反复失败 ⇒ 高度怀疑 flaky 测试或与同组 PR 的语义冲突,重排不解决。
  • 过去 24h 队列共有 7 个失败构建(不含本次)。

分诊清单:

  1. 失败测试在本 PR 改动的包里 → 真回归,修 PR。
  2. 失败测试与本 PR 无关 → 在其他 PR 的同类评论里搜同名测试;出现过 ⇒ flaky 实锤,开 issue 修/隔离那条测试。修好前重排只会再烧一轮全队列。
  3. 两者都不是 → 可能与同组 PR 语义冲突;等前面的 PR 落地或失败出队后再重排一次即可,不要连续重排。

Generated by Claude Code · merge-queue-triage workflow (#4859)

@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Aug 10, 2026
…l-suite queue load (#7150)

Test-only. No production code changes, no new cases, no assertion changes.

PR #7208 was ejected from the merge queue twice, on a different watcher case
each time and on the identical SHA `c47cc79e0`:

  1. run 31351599963 — `test/fs-behavior.test.ts > chokidar: external file
     change emits an update event` (pre-existing test, 3s event deadline).
  2. run 31352802812 — `test/watch-dot-root.test.ts > sees an external edit
     when the root is under a dot-directory` (this PR's own test, 8s deadline).

Same family both times. The queue runs the FULL suite while PR-side CI runs
only the affected subset, and `FileSystemRepository`'s watcher rides
`usePolling: 1000ms` plus an `awaitWriteFinish` stability window — wall-clock
timers that stretch under a saturated runner while nothing the assertions look
at changes. Both cases were green on that same SHA in PR CI and locally, so
this is load-timing, not semantics. The lane PM's ejection triage named
deadline-widening in the pre-existing test as the sanctioned direction.

Every positive event wait in the family now goes through a named
`EVENT_WAIT_MS`, raced against the event promise as before, so a healthy run
still costs about one poll interval and the widened number is only paid on the
way to a failure:

  - `metadata-fs/test/fs-behavior.test.ts`         3s  -> 20s (case cap 10s -> 45s)
  - `metadata-fs/test/watch-dot-root.test.ts`      8s  -> 20s (case cap 30s -> 60s, both cases)
  - `metadata-fs/test/no-root-on-attach.test.ts`   8s  -> 20s (case cap 20s -> 45s)
  - `metadata/src/metadata-repository-fs-dot-root.test.ts` 15s -> 25s (case cap 40s -> 60s)

The last two are beyond the two ejected files and are disclosed as such: both
carry the identical event-wait shape against the same watcher in the same
shard, so they were the next candidates to eject rather than anything the
diff's semantics touch.

Case caps were raised alongside each deadline. A cap that no longer clears
`EVENT_WAIT_MS` plus setup kills the case on the vitest timeout before its own
deadline is reached, which reports as a timeout instead of as the missing
event — reintroducing the flake from the other side.

`watch-dot-root.test.ts`'s 4s quiet window is unchanged and marked never to be
shortened: a too-short quiet window cannot fail, it can only produce a false
pass on `toEqual([])`. Its liveness control is a positive assertion and was
widened with the rest.

Reverse verification (direction predicted first): setting `EVENT_WAIT_MS = 1`
must turn every positive wait red while leaving the quiet-window assertion
untouched. Observed exactly that — 4 failed / 26 passed, and the negative case
failed at its CONTROL, not at its `toEqual([])`:

  × chokidar: external file change emits an update event               319ms
  × arms the watcher on the first write, so external edits are still detected  429ms
  × sees an external edit when the root is under a dot-directory       433ms
  × still ignores dot entries UNDER the root, including its own bookkeeping  4425ms
    AssertionError: expected [] to have a length of 1 but got +0

Restored, both suites green: metadata-fs 4 files / 30 tests, metadata 30 files
/ 592 tests. `typecheck` clean, ESLint clean on all four files,
`check-nul-bytes` OK.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016R9de1FqP7NvwKvqXi92Gh

os-help commented Aug 10, 2026

Copy link
Copy Markdown
Collaborator Author

Patch round accepted — re-queueing (third and final attempt under the standing rule). Session session_016R9de1FqP7NvwKvqXi92Gh.

3a9f5b880: test-only deadline hardening across the watcher-timing family — every positive event wait now rides a named EVENT_WAIT_MS (3–8s → 20s, case caps raised in step so a cap can't masquerade as the same flake from the other side), the 4s quiet window deliberately untouched and documented as never-to-shorten (it can only false-pass, not flake). Two additional same-shape test files hardened beyond the two ejection signatures (no-root-on-attach.test.ts carried the tightest cap in the package — the next candidate to eject), disclosed in the patch report. Zero production changes, 4 test files, +117/−11.

Verification: PR CI re-converged at 3a9f5b880 — all jobs success including Test Core (3/3), the shard both ejections happened on. Reverse-verified: EVENT_WAIT_MS=1 turns exactly the four positive waits red (control-position failures, quiet window unaffected).

Honest caveat from the patch report, endorsed: local green + PR-CI green is what the previous SHA also had — queue load can only be tested in the queue. This lowers ejection probability rather than proving it away. If this attempt ejects again on a watcher-timing signature, the standing rule fires: no fourth attempt — the flaky-family card gets filed and the PR escalates to the maintainer.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/m tests tooling

Projects

None yet

2 participants