test(observability): preserve #345 UA wire-capture checks as in-tree diagnostics + runbook - #894
Open
scottschreckengaust wants to merge 5 commits into
Open
scottschreckengaust wants to merge 5 commits into
scottschreckengaust wants to merge 5 commits into
Conversation
…diagnostics PR #345 (#319) shipped unit tests for the UA helpers, but two wire-capture verification scripts written during that work were never committed. They verify what the unit tests structurally cannot: the SDK-native `app/uksb-…#{APP_ID}` segment (injected by botocore/SDK v3 from AWS_SDK_UA_APP_ID, not by our code) actually reaches the wire alongside our helper-supplied `md/…#{component}` segment. CloudTrail can't be used here (DynamoDB data events are blocked), so a wire capture is the only observation point — the regression guard for the recurring "dropping solution UA on a new AWS client (#319)" hazard. Relocate both to their proper homes (fixing a duplicate import in the TS file and the import paths for the new locations): - agent/scripts/diagnostics/ua_wire_check.py (boto3 / before-send capture) - cdk/scripts/ua-wire-check.ts (SDK v3 / finalizeRequest capture) Add a row to cdk/scripts/README.md and a runbook at docs/verification/319-ua-wire-runbook.md (how to run each tier, expected output, and the AWS_SDK_UA_APP_ID='' opt-out check). Both import the real helpers (no mirror → no drift) and live in scripts/ dirs outside the lint/type-check/dead-code scopes by design — manual, credentialed diagnostics, not CI tests. Closes #893 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… failure) `build (agentcore)` failed on "Fail build on mutation": the agent quality step runs ruff over the whole `agent/` package (not just `src/`), and RUF100 auto-removed the `# noqa: E402` / `# noqa: ANN001` comments on ua_wire_check.py as unnecessary (those rules aren't enabled in the agent ruff config), leaving the tree dirty. Remove them so the committed file is already ruff-clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The `319-` prefix read as if the runbook were scoped to that issue; it's general reference material for verifying solution User-Agent attribution on the wire. Rename to docs/verification/ua-wire-check-runbook.md (matching the script names) and update the three in-repo references (both script headers + cdk/scripts/README.md). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
scottschreckengaust
marked this pull request as ready for review
September 14, 2026 21:22
scottschreckengaust
requested review from
a team and
backgroundagents
as code owners
September 14, 2026 21:22
scottschreckengaust
enabled auto-merge
September 15, 2026 18:21
4 tasks
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Preserve the two UA wire-capture verification scripts from #345 (#319) as in-tree diagnostics, and document how to run them.
Area
tooling/test— verification scripts + runbook (no runtime change)Related
Closes #893. Follow-up to merged #345 (native
AWS_SDK_UA_APP_IDattribution, #319). These scripts existed only as uncommitted files in thefeat-319-ua-appidworktree.Why (the gap unit tests can't cover)
Attributed AWS calls carry two
User-Agentsegments:The shipped unit tests assert the helper returns the right
md/…string, but theapp/…segment is injected by the SDK/botocore itself — a helper unit test structurally cannot observe it. CloudTrail is unavailable here (DynamoDB data events are blocked), so a wire capture is the only end-to-end observation point. This is the regression guard for the recurring hazard AGENTS.md calls out: "Dropping solution UA on a new AWS client (#319)."Changes
agent/scripts/diagnostics/ua_wire_check.pybefore-sendhook prints the assembled UA (STS/DDB/S3/SecretsManager); imports the realagent/src/ua.pycdk/scripts/ua-wire-check.tsfinalizeRequestmiddleware prints the UA (Lambda/DDB/S3/SecretsManager); imports the realsrc/handlers/shared/ua.tscdk/scripts/README.mddocs/verification/ua-wire-check-runbook.mdAWS_SDK_UA_APP_ID=''opt-out checkFixes applied while relocating:
@aws-sdk/client-lambdaimport in the TS script (it wouldn't compile as-is).../src/handlers/shared/uafor TS;../../srconsys.pathfor Python).Scope / safety
uamodules).scripts/dirs that are outside thecdk/agentlint, type-check, and dead-code scopes (which coversrc/test) — matching the existingcdk/scripts/generate-*.tsandagent/scripts/diagnostics/*.pyhelper scripts. No CI gate analyzes them.docs/verification/is not Starlight-mirrored, so nodocs:syncregeneration.Verification
python -m py_compile agent/scripts/diagnostics/ua_wire_check.py→ OK.client_config/SOLUTION_ID/COMPONENT(agent/src/ua.py),abcaUserAgent/SOLUTION_ID/COMPONENT_ENV(cdk/src/handlers/shared/ua.ts); CDK default componentapi, agent componentagent.Acknowledgment
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the project license.
🤖 Generated with Claude Code