docs(aws-healthomics): parameter types, run failure patterns, and the ECR Public fallback - #184
Open
hmkim wants to merge 1 commit into
Open
docs(aws-healthomics): parameter types, run failure patterns, and the ECR Public fallback#184hmkim wants to merge 1 commit into
hmkim wants to merge 1 commit into
Conversation
…, and the ECR Public fallback Four gaps found while running a migrated workflow end to end, each of which cost a failed run to discover. Parameter types. A struct passed as a JSON-encoded string is rejected by the engine, not by StartRun, so it costs the same minutes as a real defect before reporting, and the message names the struct rather than the mistake. The parameterTemplate from GetAHOWorkflow cannot resolve this either: it carries only description and optional, with no type information, so the type has to be read from the workflow's input block. Adds a type table and the failing and working forms side by side. Run failure patterns. Adds a table mapping observed error messages to the causes worth checking first, framed as starting points rather than diagnoses since the same message has several possible causes. Records two properties that shape how to read any run failure: the message names where execution stopped rather than what is misconfigured, and input errors arrive after PENDING. ECR Public fallback. Docker Hub pull-through caches require a Secrets Manager credential, and the call is rejected outright without one. Where credentials are unavailable, many images a legacy workflow names on Docker Hub are also on ECR Public, which needs none. Documents that path, including the repository name differing between the registries, and flags it as URI replacement rather than a registry map entry, since it substitutes one image for another. Packaging. Records that a bare .wdl/.nf/.cwl file behind a path or S3 URI is now packaged automatically, that this does not extend to a workflow with imports, and that inline content and .zip files are left alone. Also corrects the failure procedure, which assumed every run failure is a workflow defect and directed a new workflow version unconditionally. A run that failed on its role, output location or inputs needs neither. Depends on two MCP server tools that are not yet released: ValidateAHORunReadiness (awslabs/mcp#4486) and WaitForAHORun (awslabs/mcp#4493).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
Four gaps found while taking a workflow through migration and running it end to end. Each one cost a failed run to discover, and none of them were workflow defects.
Parameter types
running-a-workflow.mdhad no guidance on how parameter values map to WDL types. Passing a struct as a JSON-encoded string produces:Two things make this expensive. Inputs are validated by the engine rather than by
StartRun, so it arrives minutes in, afterPENDING— the same cost as a real defect. And the message names the struct, not the mistake, so the natural next move is to go looking at the struct definition.parameterTemplatefromGetAHOWorkflowdoes not resolve it either: it carries onlydescriptionandoptional, with no type information. The type has to be read from the workflow'sinput {}block, which is now stated.Adds a type table and the failing and working forms side by side.
Run failure patterns
troubleshooting.mdcovered creation failures and VPC connectivity, and routed everything else toDiagnoseAHORunFailure. Adds a table mapping observed messages to what to check first.Deliberately framed as starting points rather than diagnoses —
S3 access deniedalone can be role policy, bucket policy or KMS, and presenting one as the answer would send an agent down a single track. Each row is ordered by likelihood and the section says to fall back to the logs.Also records two properties that shape how to read any run failure:
PENDING.ECR Public fallback
ecr-pull-through-cache.mdalready documents creating the Docker Hub secret, but not what happens without one:Where credentials are unavailable, many images a legacy workflow names on Docker Hub are also published on ECR Public, which needs none. I hit this with
ubuntu:20.04and reached it asecr-public/ubuntu/ubuntu:20.04.Two things the new section is careful about:
It also notes that tags and contents are not guaranteed to match across registries.
Packaging
Records that a bare
.wdl/.nf/.cwlfile behind a path or S3 URI is packaged automatically (awslabs/mcp#4485), and the limit of that: only the named file goes in, so a workflow with imports still has to be packaged. Also notespath_to_mainis unnecessary for the single-entry archive, and that inline content and.zipfiles are left untouched.Correction to the failure procedure
The existing steps assumed every run failure is a workflow defect:
Three of the four failures in my session were configuration, not definition — role, bucket region, bucket permissions. A new workflow version is the wrong move for those, and an agent following this literally would create versions that change nothing. Now conditional on having actually modified the definition.
Verification
All seven cross-document anchors checked against the heading set. Section spacing matches the surrounding files. Content limited to behaviour observed against the service in
ap-northeast-2andus-east-1.Related
Follows #179. Companion to the three MCP server PRs: #4485, #4486, #4493.