Skip to content

fix(export): make generated Strands runtimes deployable - #2154

Open
aidandaly24 wants to merge 11 commits into
aws:refactorfrom
aidandaly24:fix/export-harness-generated-runtime
Open

fix(export): make generated Strands runtimes deployable#2154
aidandaly24 wants to merge 11 commits into
aws:refactorfrom
aidandaly24:fix/export-harness-generated-runtime

Conversation

@aidandaly24

@aidandaly24 aidandaly24 commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Summary

  • align generated Strands Python agents with the released strands-agents 1.54.x APIs and provider extras
  • replace the stateful execution-limit hook with native per-invocation turn/token limits and request-scoped cancellation
  • defer credential-backed MCP identity lookup until transport startup, and generate collision-resistant provider/function names
  • preserve provider parameters and memory retrieval tuning while emitting export notes for fields that cannot be reproduced
  • strictly validate service harness ARNs while preserving service-provided network configuration
  • surface unknown or incomplete service union members in EXPORT_NOTES.md instead of silently dropping them
  • add --vpc-id to project export harness: export rewrites a containerUri harness as a Dockerfile build so the agent code can be layered in, which makes CodeBuild's VPC id mandatory where the source harness never needed one
  • keep additionalParams restricted to the lite_llm provider, and drop it with an export note on the --arn path rather than writing a spec the pinned CDK rejects at synth

Verification

Source gate

  • bun install --frozen-lockfile
  • bun test src: 2,597 pass, 0 fail
  • bun run typecheck
  • bun run lint:check
  • bun run format:check
  • bun run build
  • bun run secrets:check

Generated Python matrix

Generated real exports and ran uv sync, compileall, and import main for:

  • Bedrock with skills, sliding-window truncation, execution limits, timeout, and credential-backed remote MCP
  • OpenAI Responses
  • Gemini
  • LiteLLM
  • Bedrock Mantle Responses

All five resolved strands-agents 1.54.0 and imported successfully. OpenAI/Mantle resolved openai 2.54.0, Gemini resolved google-genai 2.20.0, and LiteLLM resolved litellm 1.96.0. A generated-module probe invoked the Bedrock entrypoint twice and verified fresh cancellation signals plus identical per-invocation limits (LIMITS_OK).

Live AWS

Account 603141041947, region us-east-1, profile deploy:

  • deployed source harness ExportHarnessFix0831_source-jBo3nhtmm2 to READY and invoked it
  • exported by local name and by service ARN (with ambient us-west-2, proving ARN-region precedence)
  • built and deployed runtimes:
    • ExportHarnessFix0831_exported_agent-hLU1ha80fd
    • ExportHarnessFix0831_exported_arn_agent-ItzBMG47U0
  • invoked each runtime twice in the same session; all four calls returned HTTP 200, end_turn, and the requested marker
  • deployed credential-backed MCP runtime ExportHarnessFix0831_mcp_runtime-6ZSyRO2yME
  • confirmed module import succeeds without request identity context
  • after creating the referenced disposable API-key provider, invoked the runtime successfully with MCP_CONTEXT_OK

The current refactor deploy path consumes project credentials as references rather than provisioning them, so the exporter now states that the named provider must exist instead of claiming deploy creates it.

Follow-up after review

AlexanderRichey's review asked why the EC2 client existed; the DescribeSubnets lookup was
removed in b3fbb3dd. Reviewing that removal surfaced that dropping it left the underlying problem
unaddressed, and that a separate guard had been deleted:

  • --vpc-id (48cae1e6). The Runtime NetworkConfiguration API genuinely has no vpcId — the
    review comment is correct — but CodeBuild's CreateProject requires one, and export is what turns
    a no-build containerUri harness into a build by emitting FROM <containerUri>. Neither source of
    a harness can supply it: the service's VpcConfig has no such field, and a local containerUri
    harness is never built, so its schema rightly does not demand one. The flag lets the caller supply
    it, and export now fails before writing anything when it is needed and absent. No AWS lookup is
    involved: getHarness remains the only request on the --arn path.
  • additionalParams guard (b178ac65). Removing the lite_llm-only refinement moved a clean
    failure at project add harness to a raw zod dump at project build, since the pinned CDK
    re-parses harness.json at synth and still rejects the field on other providers. Restored, with
    the --arn mapper dropping it under an export note so a harness authored outside this CLI still
    exports.
  • f82282b9 reverts formatting residue the EC2 removal left in two test files, so both now match
    the base byte for byte.
  • 60f499a1 aligns the flag with repo conventions: reuse NetworkConfigSchema.shape.vpcId rather
    than restating its regex, use the <placeholder> remedy form (the previous vpc-xxxxxxxx example
    could not satisfy the flag's own pattern), and document the flag in README.md.
  • e504774c follows a mutation pass over every test this PR adds — each one had the behaviour it
    covers deliberately broken to confirm it fails. One duplicated assertion was removed, and two gaps
    closed: the --vpc-id failure test now asserts nothing was written, and the ARN test now pins the
    12-digit account group (loosening \d{12} to \d+ previously passed).

Verified by generating a real project with a VPC + containerUri harness and parsing the resulting
agentcore.json with AgentEnvSpecSchema from @aws/agentcore-cdk@0.1.0-alpha.45, the version the
generated project pins: the pre-fix output is rejected with
networkConfig.vpcId is required for Container builds in VPC mode, and the --vpc-id output is
accepted. This path was not re-run against live AWS — it needs a VPC container harness, which the
matrix above does not cover.

Cleanup

  • stack AgentCore-ExportHarnessFix0831-default: deleted
  • matching AgentCore runtimes: none
  • matching AgentCore harnesses: none
  • disposable API-key credential provider: deleted

@github-actions github-actions Bot added the size/xl PR size: XL label Aug 31, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added agentcore-harness-reviewing AgentCore Harness review in progress claude-security-reviewing Claude Code /security-review in progress labels Aug 31, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Aug 31, 2026
@codecov-commenter

codecov-commenter commented Aug 31, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 87.42857% with 22 lines in your changes missing coverage. Please review.
✅ Project coverage is 97.19%. Comparing base (d304147) to head (e504774).
⚠️ Report is 3 commits behind head on refactor.

Files with missing lines Patch % Lines
src/handlers/project/export/serviceHarness.ts 78.00% 22 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##           refactor    #2154      +/-   ##
============================================
- Coverage     97.22%   97.19%   -0.04%     
============================================
  Files           507      507              
  Lines         33809    33943     +134     
============================================
+ Hits          32872    32992     +120     
- Misses          937      951      +14     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@agentcore-devx-automation agentcore-devx-automation Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AgentCore Harness Review

Verdict: Looks good

Nice, tightly-scoped fix. I walked through the template updates, the exporter changes, and the new EC2-backed VPC lookup and didn't find anything that needs to change before merge. A few observations, none of which are blockers:

  • HarnessClient.resolveVpcIdFromSubnets is wired cleanly through AwsClients/CoreClient and the ec2 client is cached like the others; the new tests cover the multi-VPC / no-VPC / happy paths without mocking anything below the SDK boundary. 👍
  • The switch to a strict ARN regex in parseHarnessArn is the right call, and the behavior change of always trusting the ARN's region (rather than falling back to CLI region) is now consistent between harnessIdFromArn and regionFromHarnessArn. The test in harness.test.ts (arn:aws:lambda:...) and the partition test in serviceHarness.test.ts (arn:aws-cn:...) cover the important edges.
  • The main.py template's limits = { … } or None idiom is intentional: when hasExecutionLimits is true solely because timeoutSeconds is set, both {{#if}} branches inside the dict literal are stripped, so it evaluates to None. As long as agent.stream_async(..., limits=None, ...) is accepted by strands-agents 1.54 (per the verification notes in the PR description, it is), this is fine.
  • serviceHarness.ts now surfaces $unknown union members as export notes instead of silently dropping them (skills, memory, environment, environment artifact, filesystem configs). Good coverage in serviceHarness.test.ts.
  • Header credential names / python function names now hash their inputs, so X-Api-Key vs X_Api_Key no longer collide — nice; that's exercised by the new "keeps normalized header names distinct" test.
  • readStrandsVersion regex was updated to tolerate the new strands-agents[extras] form.
  • No new features here that would need telemetry instrumentation; existing plumbing is untouched.

LGTM to merge.

@agentcore-devx-automation agentcore-devx-automation Bot removed the agentcore-harness-reviewing AgentCore Harness review in progress label Aug 31, 2026
@aidandaly24
aidandaly24 force-pushed the fix/export-harness-generated-runtime branch from ed3b929 to bee4e65 Compare August 31, 2026 21:55
@github-actions github-actions Bot added size/xl PR size: XL and removed size/xl PR size: XL labels Aug 31, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added the claude-security-reviewing Claude Code /security-review in progress label Aug 31, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Aug 31, 2026
Comment thread src/core/types.tsx Outdated
// results to. CloudWatch is a distinct service from the AgentCore data plane,
// so it gets its own client/factory rather than reusing `data`.
logs(config: ClientConfig): CloudWatchLogsClient;
ec2(config: ClientConfig): EC2Client;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need an EC2 client?

Comment thread src/core/harness.tsx Outdated
.send(new GetHarnessCommand({ harnessId: id }));
}

async resolveVpcIdFromSubnets(subnetIds: string[], options: CoreOptions): Promise<string> {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread src/handlers/project/export/harness.ts Outdated
spec.networkConfig &&
!spec.networkConfig.vpcId
) {
spec.networkConfig.vpcId = await config.core.harness.resolveVpcIdFromSubnets(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure this need to be in scope. The main thing is generating the code that the user can then use. The user can also configure VPC stuff on their own.

@github-actions github-actions Bot added size/l PR size: L and removed size/xl PR size: XL labels Sep 1, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added the claude-security-reviewing Claude Code /security-review in progress label Sep 1, 2026
@github-actions github-actions Bot added size/l PR size: L and removed size/l PR size: L labels Sep 1, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Sep 1, 2026
The pinned @aws/agentcore-cdk rejects additionalParams on every provider but
lite_llm, and re-parses harness.json at synth. Dropping the CLI refinement moved
that failure from `project add harness` to `project build`, where it surfaces as
a raw zod dump — reachable via `project create --additional-params`, whose
provider defaults to bedrock.

Restore the refinement, and drop the field with an export note on the --arn path
instead of hard-failing, since a harness authored outside this CLI can carry it.
Export layers the generated agent into the harness's image by writing a
`FROM <containerUri>` Dockerfile, which turns a no-build harness into a CodeBuild
build. CodeBuild's CreateProject needs an explicit vpcId and cannot infer one
from subnets, so the exported project failed at `project build` with a raw zod
dump from CDK synth.

Neither source of a harness carries a vpcId — the service's VpcConfig has no such
field, and a local containerUri harness is never built so its schema rightly does
not demand one. Export is what creates the requirement, so add --vpc-id and fail
before writing anything when it is needed and absent.

No AWS lookup is involved: getHarness remains the only request on the --arn path.
Commit 8570ab2 added an `ec2:` stub key and a fake client, which pushed one
object past prettier's width and shifted a blank line. b3fbb3d removed the EC2
code but left the reflowed formatting, so the PR still showed two core test files
as changed with no bug behind them. Both files now match the base byte for byte.
Three follow-ups from auditing the new commits against surrounding code:

- reuse NetworkConfigSchema.shape.vpcId for the flag instead of restating its
  regex, matching how every other validated flag reuses a projectSchemas schema
  (BuildTypeSchema, ProtocolModeSchema, NetworkModeSchema); the inline regex was
  the only one in src/handlers
- the remedy said `--vpc-id vpc-xxxxxxxx`, which VPC_ID_PATTERN rejects because
  x is not a hex digit, so copy-pasting it produced a second error; use the
  <placeholder> form the other free-form remedies use
- document the flag in README, which enumerates this command's flags in prose
- cover both new mapper branches in export.test.ts, which owns
  mapHarnessToExportPlan branch coverage and already tests the sibling throw
Mutation-tested every test this PR adds by breaking the behaviour each one
claims to cover and re-running it. Three results worth acting on:

- the lite_llm half of "notes additionalParams..." asserted what the
  pre-existing "maps openai and litellm model configs" already asserts with the
  same fixture value; both fail on the same mutation, so it was pure duplication
- "requires --vpc-id..." claimed in its comment to fail before writing anything
  but only asserted rejection, so relocating the throw after the write would have
  kept it green; now snapshots agentcore.json and checks the agent dir is absent
- the ARN test never pinned the 12-digit account group; loosening \d{12} to \d+
  passed. It now fails, verified by applying that mutation.

Everything else detected its mutation and stays as is.
@github-actions github-actions Bot added size/xl PR size: XL and removed size/l PR size: L labels Sep 1, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added the claude-security-reviewing Claude Code /security-review in progress label Sep 1, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Sep 1, 2026
@github-actions github-actions Bot added size/xl PR size: XL and removed size/xl PR size: XL labels Sep 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/xl PR size: XL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants