Honor context cancellation while enumerating file storage - #301
Conversation
enumerate() only checked the context on a channel send, so a cancelled walk that was traversing subtrees producing no matching items ran to completion and returned an empty list with a NIL error. A caller that had already given up was told the store held nothing, and on a large tree it waited out the whole walk to hear it. The reader loop did not select on ctx.Done() at all, so a caller could also sit for the full walkTimeout -- five minutes by default -- after asking to stop. Check ctx.Err() at the top of the WalkDir callback, and add a ctx.Done() case to the reader. Both are needed: the callback covers the ordinary walk, and the reader covers a filesystem that has stopped answering ReadDir, where WalkDir blocks internally and the callback never runs again. A cancellation is now a returned error, so Enumerate and EnumeratePrefix no longer log one at Error. A caller that cancelled got what it asked for, and reporting a fault on every shutdown mid-listing is a false signal. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
RoboRev on the previous commit: enumerate returns ctx.Err(), which is DeadlineExceeded for a context.WithTimeout and Canceled only for a context.WithCancel. Guarding the log on Canceled alone left every timed-out listing reporting a storage fault, which is the noise the guard was added to remove. Name the condition instead of repeating it at both call sites, and cover both forms plus the walk timeout that must still be reported. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Follow-up in 9051cb5, from RoboRev on the first commit:
|
Description
enumerate()in the file storage server only checked its context on a channel send. A cancelled walk that was traversing subtrees producing no matching items therefore never reached a check: it ran the walk to completion and returned an empty list with a nil error, so a caller that had already given up was told the store held nothing. On a large tree it also waited out the whole walk to hear it.The reader loop did not select on
ctx.Done()at all, so a caller could sit for the fullwalkTimeout(five minutes by default) after asking to stop.Two checks, and both are needed:
ctx.Err()at the top of theWalkDircallback bounds the work: it stops the walker reading directories after the reader has returned. On the oversized stores where this matters, that wasted I/O is the reason to cancel in the first place.case <-ctx.Done()in the reader bounds the wait, and is the only one that can.WalkDirinvokes the callback per directory entry, so a filesystem that has stopped answeringReadDirblocks insideWalkDirand the callback never runs again.A cancellation is now a returned error, so
EnumerateandEnumeratePrefixno longer log one atError. A caller that cancelled got what it asked for, and reporting a fault every time a process shuts down mid-listing is a false operational signal.Found by an unresolved RoboRev review on the
prefix-enumerationbranch; the finding survived tomain.Testing
Two new cases in
FileEnumerationSuite:TestEnumerateCancelledWithNoMatchesReportsCancellationis the regression test. A cancelled context and a prefix matching nothing means no send is ever attempted, which is exactly the case the old send-side check could not see. Neutering both new checks makes it fail withobtained = nilagainstcontext canceled, reproducing the original bug precisely.TestEnumerateCancelledDoesNotReturnPartialResultspins that cancellation wins over buffered items, so a caller never gets a partial listing that reads as complete. Its comment states plainly that it passes on the pre-fix code too, since the pre-existing send-side check already covered it; it is there to hold that behaviour, not as coverage for this change.On coverage, stated rather than glossed: the callback check has no test of its own, and cannot easily have one. Once the context is cancelled the reader's
ctx.Done()case is ready on the first pass through the select, so it decides the returned value regardless; the callback check only changes how much of the tree gets read. Isolating that would need either an injected walk counter or a timing assertion, and a deliberately timing-sensitive test is not worth it here. Both code comments say so at the point a reader would wonder.go test ./pkg/rsstorage/servers/file/passes, includingleakteston both new cases.go test ./pkg/rsstorage/...fails inservers/postgresandinternal/integration_teston my host. Those are the Docker integration suites (failed to connect to user=admin database=postgres: lookup postgres: no such host) and I confirmed they fail identically with the change stashed, so they pre-date it.Risks
EnumerateandEnumeratePrefixcan now returncontext.Canceledwhere they previously returned an empty list and nil. Callers that ignore the error and read the slice see the same empty slice as before. Callers that check the error now see the cancellation they caused, which is the point.