Skip to content

fix(toolkit-lib): clean up the loser of noOlderThan()'s timer/resolver race - #1870

Open
Adityaj0 wants to merge 1 commit into
aws:mainfrom
Adityaj0:fix/gc-background-refresh-timer-leak
Open

fix(toolkit-lib): clean up the loser of noOlderThan()'s timer/resolver race#1870
Adityaj0 wants to merge 1 commit into
aws:mainfrom
Adityaj0:fix/gc-background-refresh-timer-leak

Conversation

@Adityaj0

Copy link
Copy Markdown
Contributor

fixes #1869

Reason for this change

BackgroundStackRefresh.noOlderThan() races a "wait for the background refresh" promise against a "reject after ms" setTimeout via Promise.race(), but never cleans up whichever side loses:

return Promise.race([
  new Promise(resolve => this.queuedPromises.push(resolve)),
  new Promise((_, reject) => setTimeout(() => reject(new ToolkitError(...)), ms)),
]);
  • Refresh wins (the common case): the losing setTimeout handle is never captured, so it's never cleared -- it keeps the event loop alive and its reject closure retained until it eventually fires on its own.
  • Timeout wins: the resolve callback already pushed into queuedPromises for the losing branch is never removed. Only justRefreshedStacks() ever drains that array, so the stale resolver sits there forever -- under cdk gc's polling loop with many concurrent timed-out noOlderThan() calls, queuedPromises grows without bound.

Description of changes

Restructured noOlderThan() around a single Promise executor instead of Promise.race() over two independently-constructed promises:

  • onRefresh clears the timeout handle before resolving.
  • The timeout callback removes its own onRefresh entry from queuedPromises (via indexOf/splice) before rejecting.

Whichever side wins, the other side's state is always cleaned up.

Testing

Added three tests to test/api/garbage-collection/garbage-collection.test.ts's existing BackgroundStackRefresh describe block:

  • noOlderThan() clears its timeout handle once the refresh lands (asserts via jest.getTimerCount(), following the pattern already used elsewhere in this file for the stop() test).
  • A single timed-out call leaves no dangling entry in queuedPromises.
  • 25 concurrent timed-out calls don't accumulate entries in queuedPromises either.

All existing BackgroundStackRefresh tests (6) and the new ones (3) pass; full garbage-collection.test.ts suite (32 tests) passes.

Checklist

  • Unit tests added/updated
  • Integration tests added/updated (not applicable — no new AWS resource types or cross-service interactions)
  • No manual edits to generated files

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

…r race

fixes aws#1869

BackgroundStackRefresh.noOlderThan() raced a "wait for refresh" promise
against a "reject after ms" setTimeout via Promise.race(), without
cleaning up whichever side lost:

- When the refresh landed first (the common case under cdk gc's polling
  loop), the losing setTimeout handle was never captured or cleared,
  keeping the event loop alive and its reject closure retained until it
  eventually fired on its own.
- When the timeout won instead, the resolve callback already pushed into
  queuedPromises for the losing branch was never removed. Only
  justRefreshedStacks() drains that array, so the stale resolver sat
  there forever -- under many concurrent timed-out noOlderThan() calls,
  queuedPromises grows without bound.

Restructure noOlderThan() around a single Promise executor that clears
the timeout on a successful refresh, and removes its own entry from
queuedPromises on a timeout, so the loser is always cleaned up regardless
of which side wins.

Added three regression tests: the timeout handle is cleared once a
refresh lands, a single timed-out call leaves no dangling entry in
queuedPromises, and 25 concurrent timed-out calls don't accumulate
entries either.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.10%. Comparing base (7ff50e7) to head (b60cff9).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1870   +/-   ##
=======================================
  Coverage   91.10%   91.10%           
=======================================
  Files          80       80           
  Lines       12205    12205           
  Branches     1742     1742           
=======================================
  Hits        11119    11119           
  Misses       1050     1050           
  Partials       36       36           
Flag Coverage Δ
suite.unit 91.10% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BackgroundStackRefresh.noOlderThan() leaks a timer (and a queued resolver on timeout)

2 participants