Repository navigation
feat(aws/ladder): family-aware stable-usage estimate for the EC2 Instance SP base layer #731
Description
Follow-up to #31 (stable-usage estimator, merged in #730).
GetUsageBaseline estimates StableUSDPerHour from the region's EC2 on-demand total across all instance families (the series built in recommendations/ondemand_series.go). An EC2 Instance Savings Plan is bound to one family, so a flat total whose family mix migrates (for example m5 to m7) still passes every gate and can strand a family-bound commitment.
Build a per-family (or per-family-group) stable-usage estimate, so the base layer can be sized by each family's own durable level. Open points: a per-family Cost Explorer series (cost and call volume), how the platform's builder splits Stable across families today (weights from stored recommendations), and whether the gate thresholds from #730 carry over per family.
Also worth checking while here: how often Cost Explorer lag leaves the last complete day partial, since the estimator's 3-day minimum turns a partial trailing day into a low or nil estimate (safe, but the base layer can be intermittently absent), and the weekend-dip case (a weekday/weekend ratio below 0.5 returns nil on every run).
Activity
cristim commentedon Nov 12, 2026
Plan (reviewed: 3 adversarial passes, same reviewer, verdict PLAN OK)
Challenge: on origin/main f1ffe43, the base layer is still flat.
providers/aws/ladder/baseline.go:81fetches one region-wide daily series, andbaseline.go:307-315setsStableUSDPerHourviaestimateStableUSDPerHour(providers/aws/ladder/stable_usage.go:74-89). Not family-aware, so proceeding.Change:
providers/aws/recommendations: new per-account fetch of daily on-demand cost grouped byINSTANCE_TYPE(oneGetCostAndUsage, DAILY, same SERVICE/PURCHASE_TYPE/REGION/LINKED_ACCOUNT filter). Family = part before the first..NoInstanceTypeskipped by named constant. Days CE returns are zero-filled for families absent that day. Unverified against live CE.providers/aws/ladder: optional family source wired only from the factory. The flat path is unchanged when it is nil.min(Flat, sum_f min(series_f) * (1 - buffer)), or nil when Flat is nil.Tests: (a) never exceeds the sum of per-family minimums; (b) no family data falls back to flat, and nil flat stays nil; (c) single family equals flat exactly. Mutation probes: min to max in the per-family minimum, and removing the flat clamp. Each must fail a test.
Commits: C1 fetch and adapter; C2 estimator and wiring. Both reference #731 but do not close it: the builder split across families and the per-family gate carry-over remain open.
feat(aws): fetch on-demand spend per EC2 family for the stable base
feat(aws/ladder): family-aware stable estimate for the SP base layer
cristim commentedon Nov 12, 2026
Landed on main (fast-forward), Refs #731, issue left open:
Decisions recorded (Blockers rule, owner go 2026-11-12 decide-now list): a family fetch error falls back to the flat estimate with a WARNING log; one or zero families keep the flat value.
Not covered by this change, so the issue stays open: the builder split across families, per-family gate carry-over, partial trailing day and weekend-dip acceptance points. Unverified risks: the DAILY GroupBy INSTANCE_TYPE shape on GetCostAndUsage has not been checked against live Cost Explorer; it adds one Cost Explorer call per baseline run; a single volatile family still falls back to flat, which is less conservative than dropping that family.
cristim commentedon Nov 13, 2026
Plan: per-family gate carry-over (open point b of #731)
Chosen point: (b). A family that fails the #730 single-series gate must contribute nothing to the family sum.
Challenge (origin/main 0b5bd15):
providers/aws/ladder/stable_usage.go:120addsslices.Min(series)*(1-f)for every family with no gate. A volatile family (stable_usage.go:61-63fails: p10 < 0.5*median) or a series shorter thanminStableSeriesDaysstill adds its minimum. Not already handled.Change:
stable_usage.gofamilyStableUSDPerHour: per family, computestableQuantiles; ifstableGatePassesfails, add nothing; otherwise addslices.Min(series)*(1-f). Validation stays before the gate. nil when total <= 0 or flat nil (unchanged). Update the doc comment (lines 94-101).stable_usage_family_test.go: rewrite fixtures that would now fail the gate (NeverExceedsSumOfFamilyMinimums: uniform 0-100 becomes 50+rng*20; CappedByFlat and NilFlatStaysNil: length 3 becomes length 14+).Tests (new): volatile family dropped, estimate equals steady family alone; 13-day family contributes nothing; all families failing the gate gives nil; p10 == 0.5*median exactly passes (pins >=).
Direction: dropping a non-negative term can only lower the sum, so the change never raises the estimate.
Mutation probe: bypass the gate check; the volatile-family test must fail. Restore from backup, verify with cmp.
Commit: one commit,
Refs #731(issue stays open). Still open after this: (a) builder split across families, (c) partial trailing day, (d) weekend-dip.fix(aws/ladder): count only gate-passing families in the stable estimate
cristim commentedon Nov 13, 2026
Landed on main as 86cff2d (fast-forward, Refs #731, issue left open).
Decision: point (b), per-family gate carry-over. A family that fails the #730 single-series gate (n >= 14, median > 0, p10 >= 0.5 * median) now contributes nothing to the family sum, instead of its minimum. Dropping a non-negative term can only lower the estimate, so this is the conservative direction. Validation still runs first, so bad family series still fail loud. The "one family" path is unchanged.
Tests (
providers/aws/ladder/stable_usage_family_test.go):TestFamilyStable_VolatileFamilyContributesNothing,TestFamilyStable_ShortFamilyContributesNothing,TestFamilyStable_AllFamiliesFailGateReturnsNil: fail on the pre-fix code (9.9, 12.6 and 4.5 respectively), pass after.TestFamilyStable_GateBoundaryPassespins>=; it passes both before and after. Mutation to>fails it.NeverExceedsSumOfFamilyMinimums,CappedByFlatandNilFlatStaysNilnow pass the gate, so they still test what they claim.Verification:
GOWORK=off go test -count=1 ./...exit 0;go vet,gofmt -l, golangci-lint: clean (0 issues). Two adversarial review passes, same reviewer, no findings on either pass.Still open on #731:
cristim commentedon Nov 13, 2026
Point (a) blocked: no split exists to weight, and stored recs are not an input to the base layer.
Challenge on origin/main (fetched this turn):
providers/aws/ladder/stable_usage.go:103-133(familyStableUSDPerHour) returnsmin(flat, sum over families of min(series) * (1 - f)). It is a single scalar sum of per-family minimums. There is no equal split and no per-family allocation of a base amount to weight.pkg/ladder/allocate.go:806-838(computeBaseGap) routes onestableLeftvalue to the base layer. No code assigns that value to families.providers/aws/ladder/baseline.go:301-353(baselineStats,familyStable) takes only the CE daily series and the flat estimate. Nothing inproviders/aws/ladderorpkg/ladderreads stored recs duringGetUsageBaseline.SavingsPlanDetails.InstanceFamilyandHourlyCommitment(pkg/common/types.go, EC2Instance SP recs). That is a CE-sized commitment from a past run, not a usage weight, and it is absent for non-SP recs.Decision (Blockers rule): stop, no code. Wiring stored recs into the baseline path plus a new per-family base allocator is a design change, not the smallest self-contained slice, so it needs the owner's call. Options for the owner:
HourlyCommitmentper family as weights. Needs the baseline to read stored recs, a new input and a staleness rule.Still open on #731: (a) as above, (c) partial trailing day, (d) weekend-dip. Issue stays open. Worktree
~/.claude/worktrees/go-731-builder(branchfeat/731-builder, from origin/main) is left in place, empty of commits.cristim commentedon Nov 13, 2026
Decision on the builder point (a) (Blockers rule, claude-go-cli-mcp, 2026-11-13): option (1). Split the stable base across families in proportion to each family's stable minimum, computed from the daily series the baseline already reads. No stored recommendations are needed, and the single-family and no-data cases keep the current behaviour. Option (2) would make the baseline read stored recommendations and is not the smallest slice. Implementation to follow in a separate slice; this issue stays open.
cristim commentedon Nov 13, 2026
Point (a) on hold: plan review found the slice is dead data, so nothing is landed.
Plan reviewed by one non-author reviewer (sonnet, high). Facts and ordering checked out on origin/main 1aa9d7b:
computeBaseGapstill reads one scalar (pkg/ladder/allocate.go:811), the family minimums are available from the daily series (providers/aws/ladder/stable_usage.go,baseline.gofamilyStable), andproviders/awsbuilds against the published pkg pin underGOWORK=off, so a newpkg/ladderfield needs the two-commit pin bump.Blocker: a per-family share carried on
UsageBaselinehas no consumer.computeBaseGapignores it, so sizing output does not change. The split only has an effect once the allocator emits per-family base rows, which needs a family dimension inpkg/ladderallocations and purchase rows. That is a design change larger than one slice, so I did not land the field or the pin bump.Also found:
base*c/totalcan exceedbaseby 1 ulp when one family carries the whole total. The implementation would clamp withmin(share, base).Options for the owner: (1) take the allocator family dimension as the next slice, scoped in its own plan, and land the split with it; (2) accept the carrier-only slice now (field, pin bump, provider split) as pre-staging. The reviewer and I lean to (1).
Worktree
~/.claude/worktrees/go-731-split(branchfeat/731-split, from origin/main 1aa9d7b) holds the uncommitted draft and is kept. Issue stays open.cristim commentedon Nov 13, 2026
Decision on the blocker (Blockers rule, claude-go-cli-mcp, 2026-11-13): option (1). The allocator family dimension is the next slice, with its own plan and review, and the split lands together with it. The uncommitted draft in ~/.claude/worktrees/go-731-split stays for reference and is not pushed. The option (2) pre-staging is not taken, since a carrier field with no consumer would be dead data. Issue stays open.
feat(ladder): carry an instance family through allocations, tranches …
feat(ladder): split the EC2 Instance SP base gap across instance fami…
feat(ladder): net in-flight base tranches per instance family
fix(build): pin the azure and gcp modules to the pkg with the new ter…
cristim commentedon Nov 13, 2026
Closed by the family dimension plus the split, landed on main (fast-forward, no PR):
CI: green on b5ff9a6 (runs 43485501121, 43485501385) and on dde2ecc (43492081261, 43492081212). Two adversarial reviewer passes plus a confirmation pass on the pkg diff (opus), and two passes on the provider diff (tree 534aac1c), findings fixed before each push. Plan went through several review rounds (final rounds clean).
Decisions recorded under the Blockers rule (owner can override): a family fetch error or no family data now leaves no stable base (WARNING logged, gap routes to flex) instead of falling back to the flat estimate, because a family-bound commitment must never be sized without a family; a single family now gets min(its own minimum, flat) and its own gate; each family row takes one MaxActionsPerRun slot; existing base on a total-basis baseline withholds the family base (AWS baselines are residual so it never fires there).
Evidence is fixtures and loopback Cost Explorer responses only. Live Cost Explorer (DAILY, GroupBy INSTANCE_TYPE) is unverified. Until the platform sets InFlightTranche.Family, in-flight netting is by share, not per-family headroom (total still bounded).
Still open, tracked in #746: (c) partial trailing day, (d) weekend dip, per-family existing commitment, and the platform store/executor work (do not bump pkg/providers/aws in the platform before it reads Tranche.Family; PurchaseLayer has no cross-check of the family).