Skip to content

Honor context cancellation while enumerating file storage - #301

Merged
jonyoder merged 2 commits into
mainfrom
fix/enumerate-honors-cancellation
Aug 25, 2026
Merged

Honor context cancellation while enumerating file storage#301
jonyoder merged 2 commits into
mainfrom
fix/enumerate-honors-cancellation

Conversation

@jonyoder

Copy link
Copy Markdown
Collaborator

Description

enumerate() in the file storage server only checked its context on a channel send. A cancelled walk that was traversing subtrees producing no matching items therefore never reached a check: it ran the walk to completion and returned an empty list with a nil error, so a caller that had already given up was told the store held nothing. On a large tree it also waited out the whole walk to hear it.

The reader loop did not select on ctx.Done() at all, so a caller could sit for the full walkTimeout (five minutes by default) after asking to stop.

Two checks, and both are needed:

  • ctx.Err() at the top of the WalkDir callback bounds the work: it stops the walker reading directories after the reader has returned. On the oversized stores where this matters, that wasted I/O is the reason to cancel in the first place.
  • case <-ctx.Done() in the reader bounds the wait, and is the only one that can. WalkDir invokes the callback per directory entry, so a filesystem that has stopped answering ReadDir blocks inside WalkDir and the callback never runs again.

A cancellation is now a returned error, so Enumerate and EnumeratePrefix no longer log one at Error. A caller that cancelled got what it asked for, and reporting a fault every time a process shuts down mid-listing is a false operational signal.

Found by an unresolved RoboRev review on the prefix-enumeration branch; the finding survived to main.

Testing

Two new cases in FileEnumerationSuite:

  • TestEnumerateCancelledWithNoMatchesReportsCancellation is the regression test. A cancelled context and a prefix matching nothing means no send is ever attempted, which is exactly the case the old send-side check could not see. Neutering both new checks makes it fail with obtained = nil against context canceled, reproducing the original bug precisely.
  • TestEnumerateCancelledDoesNotReturnPartialResults pins that cancellation wins over buffered items, so a caller never gets a partial listing that reads as complete. Its comment states plainly that it passes on the pre-fix code too, since the pre-existing send-side check already covered it; it is there to hold that behaviour, not as coverage for this change.

On coverage, stated rather than glossed: the callback check has no test of its own, and cannot easily have one. Once the context is cancelled the reader's ctx.Done() case is ready on the first pass through the select, so it decides the returned value regardless; the callback check only changes how much of the tree gets read. Isolating that would need either an injected walk counter or a timing assertion, and a deliberately timing-sensitive test is not worth it here. Both code comments say so at the point a reader would wonder.

go test ./pkg/rsstorage/servers/file/ passes, including leaktest on both new cases.

go test ./pkg/rsstorage/... fails in servers/postgres and internal/integration_test on my host. Those are the Docker integration suites (failed to connect to user=admin database=postgres: lookup postgres: no such host) and I confirmed they fail identically with the change stashed, so they pre-date it.

Risks

Enumerate and EnumeratePrefix can now return context.Canceled where they previously returned an empty list and nil. Callers that ignore the error and read the slice see the same empty slice as before. Callers that check the error now see the cancellation they caused, which is the point.

jonyoder and others added 2 commits August 24, 2026 17:12
enumerate() only checked the context on a channel send, so a cancelled walk
that was traversing subtrees producing no matching items ran to completion
and returned an empty list with a NIL error. A caller that had already given
up was told the store held nothing, and on a large tree it waited out the
whole walk to hear it. The reader loop did not select on ctx.Done() at all,
so a caller could also sit for the full walkTimeout -- five minutes by
default -- after asking to stop.

Check ctx.Err() at the top of the WalkDir callback, and add a ctx.Done() case
to the reader. Both are needed: the callback covers the ordinary walk, and
the reader covers a filesystem that has stopped answering ReadDir, where
WalkDir blocks internally and the callback never runs again.

A cancellation is now a returned error, so Enumerate and EnumeratePrefix no
longer log one at Error. A caller that cancelled got what it asked for, and
reporting a fault on every shutdown mid-listing is a false signal.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
RoboRev on the previous commit: enumerate returns ctx.Err(), which is
DeadlineExceeded for a context.WithTimeout and Canceled only for a
context.WithCancel. Guarding the log on Canceled alone left every timed-out
listing reporting a storage fault, which is the noise the guard was added to
remove.

Name the condition instead of repeating it at both call sites, and cover both
forms plus the walk timeout that must still be reported.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jonyoder

Copy link
Copy Markdown
Collaborator Author

Follow-up in 9051cb5, from RoboRev on the first commit: enumerate returns ctx.Err(), which is DeadlineExceeded for a context.WithTimeout and Canceled only for a context.WithCancel. My log guard matched Canceled alone, so a caller that set a deadline still got "Error enumerating storage" — exactly the noise the guard was meant to remove. Both forms are now covered by a named isContextEnded helper rather than the condition being repeated at the two call sites.

TestEnumerateDoesNotLogAnEndedContextAsAFault covers the deadline and cancel cases, and also asserts that a walk timeout is still logged, since that one is not the caller's doing. Mutation tested in both directions: narrowing the helper back to Canceled fails the deadline case, and widening it to always suppress fails the walk-timeout case.

@jonyoder
jonyoder merged commit d8473c1 into main Aug 25, 2026
3 checks passed
@jonyoder
jonyoder deleted the fix/enumerate-honors-cancellation branch August 25, 2026 10:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant