refactor(toolkit-lib): batch asset-active lookups in cdk gc with a multi-pattern search - #1861
Open
Adityaj0 wants to merge 2 commits into
Open
refactor(toolkit-lib): batch asset-active lookups in cdk gc with a multi-pattern search#1861Adityaj0 wants to merge 2 commits into
Adityaj0 wants to merge 2 commits into
Conversation
…pattern search fixes aws#1860 ActiveAssetCache.contains() checked each candidate asset against every remembered stack template with a linear per-stack String.includes() scan. Since cdk gc calls this once per asset (batches of up to 1000, potentially tens of thousands of assets in a long-lived account), the total cost was O(assets x stacks x avg template size) -- the search work was fully re-done from scratch for every single asset. Add containsAny(), which builds a single Aho-Corasick multi-pattern index over a batch of candidate asset identifiers and scans each stack template exactly once, regardless of how many candidates are being checked. This brings the cost down to O(stacks x avg template size + sum of candidate lengths) per batch. garbageCollectEcr/garbageCollectS3 now call it once per batch instead of calling contains() once per asset. Benchmarked against a synthetic large account (500 stacks x ~20KB templates, 50,000 orphaned assets in batches of 1000): 13.3s -> 5.3s (~2.5x), with results cross-checked to be identical to the old implementation (see stack-refresh.test.ts's randomized-workload test) -- no change in which assets are considered active, so no risk of a false negative causing a live asset to be deleted. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adityaj0
requested a deployment
to
integ-approval
August 20, 2026 10:05 — with
GitHub Actions
Waiting
aws-cdk-automation
enabled auto-merge
August 20, 2026 10:05
CI's eslint (no-bitwise) rejected the LCG's & 0x7fffffff mask. Replace it with Math.imul for a 32-bit-safe multiply and a modulo-based non-negative fold, avoiding both bitwise operators and the double- precision overflow the raw multiplication would otherwise hit. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
auto-merge was automatically disabled
August 23, 2026 03:13
Head branch was pushed to by a user without write access
Adityaj0
requested a deployment
to
integ-approval
August 23, 2026 03:13 — with
GitHub Actions
Waiting
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1861 +/- ##
==========================================
+ Coverage 90.93% 91.13% +0.19%
==========================================
Files 80 80
Lines 12250 12205 -45
Branches 1756 1744 -12
==========================================
- Hits 11140 11123 -17
+ Misses 1073 1046 -27
+ Partials 37 36 -1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fixes #1860
Reason for this change
cdk gc'sActiveAssetCache.contains()checked whether an asset is still referenced by any stack via a linear per-stackString.includes()scan, called once per asset:garbageCollectEcr/garbageCollectS3call this once per asset in every batch (batch size 1000), so for an account withSstacks andAaccumulated orphaned assets, the lookup phase alone costsO(A x S x avg template size)— every single asset re-scans every stack template from scratch. In a long-lived account with hundreds of stacks and tens of thousands of orphaned S3 objects / ECR images, this dominatescdk gcruntime.Description of changes
ActiveAssetCache.containsAny(assets: string[]), which builds a single Aho-Corasick multi-pattern search index over a batch of candidate asset identifiers and scans each remembered stack template exactly once, regardless of how many candidates are in the batch. This brings the cost down toO(stacks x avg template size + sum of candidate lengths)per batch.garbageCollectEcr/garbageCollectS3now callcontainsAny()once per batch (of up to 1000 assets/tags) instead of callingcontains()once per asset.contains(asset)as a thin wrapper aroundcontainsAny([asset])for any other/future single-item callers.Testing
test/api/garbage-collection/stack-refresh.test.ts, including:contains/containsAnybehavior across multiple stackscontainsAny()against the naive per-candidatecontains()for exact agreement (no false negatives, no false positives)garbage-collection.test.tssuite passes unchanged (40/40 tests).Metrics
Synthetic benchmark modeling a large, long-lived account (500 stacks x ~20KB templates, 50,000 orphaned assets processed in batches of 1000, matching
cdk gc's real batch size):Results were identical between old and new implementations across all benchmarked scales (3,000 / 20,000 / 50,000 assets) — verified via direct set comparison, not just count.
Checklist
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license