Repository navigation
feat(telemetry): add the telemetry-3 conformance handler - #882
Conversation
Requirement telemetry-3 checks that an OperationOverride result transform
exports only the keys it returns. The new handler runs step fetch-user,
which returns {userId, email, token}. The override returns {userId} alone.
The content section also disables input and output, so the record carries
no input and no output key.
TelemetryConformanceConfig gains a telemetryConfig(logGroupName, content)
overload, a durableConfig(telemetryConfig) overload, the telemetry-3
settings builder resultSelectTelemetryConfig, and the selectUserId
transform. The handler static initializer reads the environment, so the
testable pieces live in the config class, as the existing tests require.
The template maps Telemetry3ResultSelect to telemetry-3. Four tests cover
the transform, the shared settings, the disabled input/output, and the
override entry.
This comment has been minimized.
This comment has been minimized.
|
/ai review |
This comment has been minimized.
This comment has been minimized.
ParidelPooya
left a comment
There was a problem hiding this comment.
Looks good. The handler matches telemetry-3. applyResultOverride parses the checkpoint into a Map before it calls the transform, so selectUserId is right. CI ran telemetry-3 against cb3753e and it passed.
A few small things:
TelemetryConformanceConfig.java:13-19: the class Javadoc still says every handler uses the same plugin settings. telemetry-3 now uses a different content section. Suggest: every handler shares the exporter, emit mode, sampling rate and operation detail, and the content section differs per requirement.- The PR body ("Draft until") and the workflow comment at
telemetry-conformance-tests.yml:48say "before leaving draft", but the PR is not a draft. Suggest rewording it to "before this PR merges". README.md:23: the new row is 4 characters shorter than the other rows.README.md:36: the layout comment forTelemetryConformanceConfigTest.javadoes not mention the new telemetry-3 tests.
Review 5832826567 raised three documentation items. 1. The TelemetryConformanceConfig class Javadoc said every handler uses the same plugin settings. telemetry-3 uses a different content section. The Javadoc now says the handlers share the exporter, emit mode, sampling rate and operation detail, and the content section differs per requirement. 2. The workflow re-pin TODO said "before this PR leaves draft". The PR is not a draft. It now says "before this PR merges". 3. README: the telemetry-3 coverage row is padded to the table width, and the TelemetryConformanceConfigTest layout comment mentions the telemetry-3 tests.
|
Thanks for the review. All three items are addressed in 419e4e7.
The Java change is Javadoc only. Module build: 11 tests, 0 failures. |
This comment has been minimized.
This comment has been minimized.
Conformance PR #183 merged as cf13ef27. The workflow now pins that commit and the TODO to re-pin is removed.
This comment has been minimized.
This comment has been minimized.
Codex AI reviewNo actionable findings. The prior design-rationale finding is resolved. Residual test risk: this was a static review; the reported test results and updated external workflow pin were not independently verified. Reviewed commit |
|
Thanks. All three documentation items were fixed in 419e4e7 (Javadoc, workflow TODO wording, README row width and layout comment). Since then: e445605 re-pins the workflow to the #183 merge commit Telemetry e2e on the merged head: run 42714830848, telemetry-1, -2, -3 passed. |
ParidelPooya
left a comment
There was a problem hiding this comment.
Thanks. The class Javadoc, the README and the pin comment are fixed. The suite is pinned to the #183 merge commit cf13ef2, and telemetry-3 passes on it. Approving.
Summary
Adds the Java handler for conformance requirement telemetry-3 (per-operation result selection). The requirement is defined in aws/aws-durable-execution-conformance-tests#183 and was raised in aws/aws-durable-execution-conformance-tests#182.
Step
fetch-userreturns{userId, email, token}. AnOperationOverride.withResult("fetch-user", selectUserId)transform returns{userId}alone. The content section setsinput(false)andoutput(false). So the exported record has noinput, nooutput, andoperationsByName.fetch-user.result == {"userId": "u-123"}. The execution result still holds all three keys, which the suite's coreExpectedResultcheck verifies.Design and alternatives
Problem: Verify that telemetry exports only
userIdfrom a step result while the execution still returnsuserId,email, andtoken.Coverage gap: The existing telemetry scenarios do not exercise per-operation result selection.
Alternatives evaluated during review:
userIdfrom the step. This changes the execution result. It also avoids testing whether the telemetry transform selects the field correctly.Chosen approach: Keep a dedicated handler with one step and one result transform. Disable execution input and output in its telemetry configuration. Return the complete step result unchanged. This preserves the existing scenarios without adding scenario-selection logic.
Removals: None. This adds conformance coverage using the existing SDK API.
Java configuration structure: Copying configuration into the new handler would avoid adding overloads. It would duplicate the exporter, emit mode, sampling rate, operation detail, and plugin registration.
The retained overloads share that setup and accept the configuration that differs. The existing entry points retain the settings used by telemetry-1 and telemetry-2. The telemetry settings remain independently testable without initializing the handler's environment-dependent static fields.
So retain the package-private configuration methods. No SDK public API change is needed.
Changes
Telemetry3ResultSelect.java: new handler.TelemetryConformanceConfig.java:telemetryConfig(logGroupName, content)anddurableConfig(telemetryConfig)overloads;resultSelectTelemetryConfig(logGroupName)andselectUserId(Object)for telemetry-3. The handler's static initializer reads the environment, so the testable pieces live here, following the existing module convention.template.yaml:Telemetry3ResultSelect→TestDescription: [telemetry-3].TelemetryConformanceConfigTest.java: four new tests (transform output, shared settings kept, input/output disabled, override entry targetsfetch-userand applies the transform).README.md,pom.xmldescription..github/workflows/telemetry-conformance-tests.yml: pinned to the [Docs]: Add missing javadoc #183 merge commit on main (cf13ef2).Verification
JAVA_SDK_VERSION=3.1.1-SNAPSHOT mvn -B packageinconformance-tests-telemetry/(aftermvn -Dmaven.test.skip=true installat the root): BUILD SUCCESS, 11 tests, 0 failures.d4aca515. Re-run after the re-pin: Run 42714830848 atf8ac60b7, pinned tocf13ef27, 3 passed, 0 failed.