Skip to content

fix(shared): serialize TokenBucket.waitAndConsume to prevent token over-issue (closes #209)#213

Merged
williamzujkowski merged 1 commit into
mainfrom
fix/rate-limiter-concurrency
Jun 23, 2026
Merged

fix(shared): serialize TokenBucket.waitAndConsume to prevent token over-issue (closes #209)#213
williamzujkowski merged 1 commit into
mainfrom
fix/rate-limiter-concurrency

Conversation

@williamzujkowski

Copy link
Copy Markdown
Collaborator

Completes #209 (the rate-limiter half; the retry.ts half is in #212).

Bug

TokenBucket.waitAndConsume computed a single waitMs, slept, then did this.tokens -= count unconditionally. N concurrent waiters that found the bucket empty all scheduled the same wait, woke together, and each subtracted from a bucket that refilled enough for only one → tokens goes negative and the limiter silently exceeds its rate. It's used by the annotator and the CourtListener client; today's call sites are sequential so it's latent, but it's a real correctness bug in a shared primitive and the existing test only exercised a single waiter.

Fix

  • Serialize waiters via a FIFO promise chain (mutex): each waiter awaits the prior one, then loops — refill(), and only tokens -= count after re-checking tokens >= count. Concurrent waiters can never over-issue, and ordering is preserved.
  • Add a guard: waitAndConsume(count > capacity) throws RangeError instead of waiting for a deficit that can never close.

Tests

  • concurrent waiters served FIFO across separate refill intervals without driving tokens negative;
  • capacity guard throws.
  • Existing tryConsume/refill/waitAndConsume tests unchanged. shared 27 pass; annotator 76 / build 8/8 unaffected.

🤖 Generated with Claude Code

waitAndConsume computed a single wait then unconditionally did
`this.tokens -= count` after sleeping. N concurrent waiters that found
the bucket empty all scheduled the same wait, woke together, and each
subtracted from a bucket that refilled enough for only one — driving
`tokens` negative and letting the limiter exceed its rate (the exact
failure it exists to prevent). Used by the annotator / CourtListener
client.

Serialize waiters via a FIFO promise chain so they consume one at a
time, each looping until enough tokens have actually refilled and only
subtracting after a re-check (never negative). Add a capacity guard:
waitAndConsume(count > capacity) now throws instead of waiting forever.

New tests: concurrent waiters are served FIFO without over-issuing, and
the capacity guard throws. shared 27 pass; annotator/fetcher and build
unaffected.

Closes #209

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@williamzujkowski williamzujkowski requested a review from a team as a code owner June 23, 2026 05:04
@williamzujkowski williamzujkowski merged commit 7f92425 into main Jun 23, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant