Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions .github/workflows/telemetry-conformance-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,8 @@ jobs:
id-token: write
# The pinned SHA also selects the runner and telemetry-extension revision the shared
# workflow installs, so it must be reachable from a branch. It is the merge commit of
# aws-durable-execution-conformance-tests PR #150 on that repository's main branch.
uses: aws/aws-durable-execution-conformance-tests/.github/workflows/telemetry-suite.yml@0eaf7341c0c7ff937e6a4bf6bda1682ff07e7502
# aws-durable-execution-conformance-tests PR #179 on that repository's main branch.
uses: aws/aws-durable-execution-conformance-tests/.github/workflows/telemetry-suite.yml@ca2cd02e1fb0dce34c9d81e361c1556a7e3707ee
with:
language: java
resource_prefix: java
Expand Down Expand Up @@ -107,7 +107,9 @@ jobs:
echo "Using Java SDK version $JAVA_SDK_VERSION"
# The runner's sam build runs in a later step. Only GITHUB_ENV carries the version to it.
echo "JAVA_SDK_VERSION=$JAVA_SDK_VERSION" >> "$GITHUB_ENV"
mvn -B -q -DskipTests package \
# No -DskipTests: the module is outside the SDK reactor, so this is the only place its
# unit tests run. The package phase runs them before the deploy artifact is built.
mvn -B -q package \
-Ddurable.sdk.version="$JAVA_SDK_VERSION" \
--file "$EXAMPLES_DIR/pom.xml"
secrets:
Expand Down
12 changes: 9 additions & 3 deletions conformance-tests-telemetry/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ template that maps it to the requirement ID.
| Suite | Requirement | What it pins down |
| ----------- | ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
| `telemetry` | `telemetry-1` | One successful single-step execution emits one `SUCCEEDED` record with `input`, `output`, ARN-derived identity fields, and `operationsByName`. |
| `telemetry` | `telemetry-2` | One failed execution emits one `FAILED` record with an `error` on the execution and on `operationsByName.charge`. |

## Layout

Expand All @@ -27,10 +28,14 @@ pom.xml # standalone Maven project (not a reac
template.yaml # telemetry suite; declares TelemetryLogGroupName and creates the group
src/main/java/software/amazon/lambda/durable/conformance/telemetry/
Telemetry1Success.java # single step "increment"; plugin + CloudWatchLogsExporter
Telemetry2Failure.java # step "charge" fails; handler throws "payment failed" with the step error as cause
TelemetryConformanceConfig.java # shared log-group guard and plugin config; both handlers call it
src/test/java/software/amazon/lambda/durable/conformance/telemetry/
TelemetryConformanceConfigTest.java # guard rejects null/blank; telemetry settings match the runner
```

The handler uses the real SDK API only. It constructs no record and calls no exporter method. A missing or wrong record
is the plugin's defect. That defect is the signal the suite measures.
The handlers use the real SDK API only. They construct no record and call no exporter method. A missing or wrong
record is the plugin's defect. That defect is the signal the suite measures.

The handler configuration is: one `CloudWatchLogsExporter`, emit mode `ON_COMPLETE`, sampling rate 1, operation detail
`TOP_LEVEL`, and content input and output enabled.
Expand All @@ -56,6 +61,7 @@ mvn -B -q -Dmaven.test.skip=true install
export JAVA_SDK_VERSION=$(mvn -q help:evaluate -Dexpression=project.version -DforceStdout)

cd conformance-tests-telemetry
mvn -B -q package # runs the unit tests; the module is outside the reactor, so the root build does not
sam build
```

Expand All @@ -74,7 +80,7 @@ durable-execution-conformance \

## CI

[`.github/workflows/telemetry-conformance-tests.yml:1–118`](../.github/workflows/telemetry-conformance-tests.yml#L1-L118)
[`.github/workflows/telemetry-conformance-tests.yml:1–120`](../.github/workflows/telemetry-conformance-tests.yml#L1-L120)
runs on pull requests and pushes to `main` that touch the SDK, telemetry
exporter, this module, or the root `pom.xml`. It also supports manual dispatch.

Expand Down
20 changes: 19 additions & 1 deletion conformance-tests-telemetry/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
<name>AWS Lambda Durable Execution SDK Telemetry Conformance Tests</name>
<description>
Java SDK handlers and SAM template for the Durable Execution Telemetry conformance
suite (telemetry-1).
suite (telemetry-1, telemetry-2).
</description>

<properties>
Expand All @@ -33,6 +33,9 @@
<durable.sdk.version>${env.JAVA_SDK_VERSION}</durable.sdk.version>
<aws.sdk.version>2.56.0</aws.sdk.version>
<aws.lambda.java.core.version>1.4.1</aws.lambda.java.core.version>
<!-- Standalone pom: test tool versions are repeated here from the SDK root pom. -->
<junit.version>6.1.3</junit.version>
<surefire.version>3.6.1</surefire.version>
</properties>

<dependencies>
Expand All @@ -56,6 +59,14 @@
<artifactId>aws-lambda-java-core</artifactId>
<version>${aws.lambda.java.core.version}</version>
</dependency>

<!-- Test dependencies -->
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
<version>${junit.version}</version>
<scope>test</scope>
</dependency>
</dependencies>

<build>
Expand All @@ -66,6 +77,13 @@
<version>3.15.0</version>
</plugin>

<!-- Runs the unit tests in src/test. The telemetry conformance workflow runs them before packaging. -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${surefire.version}</version>
</plugin>

<!-- Shaded (uber) JAR so the Lambda deploys as a single artifact. -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,6 @@
import software.amazon.lambda.durable.DurableConfig;
import software.amazon.lambda.durable.DurableContext;
import software.amazon.lambda.durable.DurableHandler;
import software.amazon.lambda.durable.telemetry.ContentConfig;
import software.amazon.lambda.durable.telemetry.DurableExecutionTelemetry;
import software.amazon.lambda.durable.telemetry.DurableExecutionTelemetryConfig;
import software.amazon.lambda.durable.telemetry.exporters.CloudWatchLogsExporter;

/**
* Single successful step scenario for requirement telemetry-1.
Expand All @@ -18,34 +14,13 @@
*/
public final class Telemetry1Success extends DurableHandler<Telemetry1Success.Input, Telemetry1Success.Output> {

// The deployed function must export to the log group the runner reads.
// A missing variable would produce a function that exports nowhere.
// So class initialization fails when the variable is unset.
private static final String LOG_GROUP_NAME = System.getenv("TELEMETRY_LOG_GROUP_NAME");

static {
if (LOG_GROUP_NAME == null || LOG_GROUP_NAME.isBlank()) {
throw new IllegalStateException("TELEMETRY_LOG_GROUP_NAME must be set");
}
}
// Class initialization fails when the variable is unset. See TelemetryConformanceConfig.requireLogGroupName.
private static final String LOG_GROUP_NAME =
TelemetryConformanceConfig.requireLogGroupName(System.getenv(TelemetryConformanceConfig.LOG_GROUP_ENV));

@Override
protected DurableConfig createConfiguration() {
return DurableConfig.builder()
.withPlugins(DurableExecutionTelemetry.durableExecutionTelemetry(
DurableExecutionTelemetryConfig.builder()
.addExporter(CloudWatchLogsExporter.builder()
.logGroupName(LOG_GROUP_NAME)
.build())
.emitMode(DurableExecutionTelemetryConfig.EmitMode.ON_COMPLETE)
.samplingRate(1.0)
.operationDetail(DurableExecutionTelemetryConfig.OperationDetail.TOP_LEVEL)
.content(ContentConfig.builder()
.input(true)
.output(true)
.build())
.build()))
.build();
return TelemetryConformanceConfig.durableConfig(LOG_GROUP_NAME);
}

@Override
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
// SPDX-License-Identifier: Apache-2.0
package software.amazon.lambda.durable.conformance.telemetry;

import software.amazon.lambda.durable.DurableConfig;
import software.amazon.lambda.durable.DurableContext;
import software.amazon.lambda.durable.DurableHandler;
import software.amazon.lambda.durable.config.StepConfig;
import software.amazon.lambda.durable.retry.RetryStrategies;

/**
* Failed single-step scenario for requirement telemetry-2.
*
* <p>The step {@code charge} throws {@code card declined}. The retry strategy declines, so the step fails on its first
* attempt. The handler catches the step error and throws {@code payment failed} with the step error as its cause. So
* the execution fails and the exported record carries an {@code error} on the execution and on
* {@code operationsByName.charge}. Error names and cause depth differ per SDK by design, so telemetry-2 asserts only
* that each error is an object. It pins the messages {@code card declined} and {@code payment failed} through the core
* execution-result and history sections. Do not change them.
*
* <p>The handler configures the telemetry plugin exactly as {@link Telemetry1Success}. It constructs no record and
* calls no exporter method. The plugin builds and exports the record.
*/
public final class Telemetry2Failure extends DurableHandler<Telemetry2Failure.Input, Void> {

// Class initialization fails when the variable is unset. See TelemetryConformanceConfig.requireLogGroupName.
private static final String LOG_GROUP_NAME =
TelemetryConformanceConfig.requireLogGroupName(System.getenv(TelemetryConformanceConfig.LOG_GROUP_ENV));

@Override
protected DurableConfig createConfiguration() {
return TelemetryConformanceConfig.durableConfig(LOG_GROUP_NAME);
Comment thread
hln33 marked this conversation as resolved.
Resolved
}

@Override
public Void handleRequest(Input input, DurableContext context) {
try {
context.step(
"charge",
Void.class,
step -> {
throw new RuntimeException("card declined");
},
StepConfig.builder()
.retryStrategy(RetryStrategies.Presets.NO_RETRY)
.build());
} catch (RuntimeException stepError) {
// The SDK rebuilds the step's original exception from the checkpoint and rethrows it, so the
// step surfaces as RuntimeException("card declined"). StepFailedException is only the fallback
// when that rebuild fails. Catching StepFailedException would let "card declined" escape.
throw new RuntimeException("payment failed", stepError);
}
return null;
}

/** Event payload: {"value": 41}. */
public static final class Input {
public int value;

public Input() {}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
// SPDX-License-Identifier: Apache-2.0
package software.amazon.lambda.durable.conformance.telemetry;

import software.amazon.lambda.durable.DurableConfig;
import software.amazon.lambda.durable.telemetry.ContentConfig;
import software.amazon.lambda.durable.telemetry.DurableExecutionTelemetry;
import software.amazon.lambda.durable.telemetry.DurableExecutionTelemetryConfig;
import software.amazon.lambda.durable.telemetry.exporters.CloudWatchLogsExporter;

/**
* Shared plugin setup for the telemetry conformance handlers.
*
* <p>Every handler in this module exports to the same log group with the same plugin settings. The runner reads that
* log group and compares the record to the requirement. A handler with different settings would produce a record the
* runner cannot match. So the settings live here once and each handler calls these methods.
*/
final class TelemetryConformanceConfig {

/** Environment variable that names the log group the runner reads. */
static final String LOG_GROUP_ENV = "TELEMETRY_LOG_GROUP_NAME";

private TelemetryConformanceConfig() {}

/**
* Validates the configured log group name.
*
* <p>The deployed function must export to the log group the runner reads. A missing name would produce a function
* that exports nowhere, and the runner would time out instead of reporting a configuration error. So this method
* rejects a null or blank name, and the caller fails class initialization.
*
* @param logGroupName the value of {@link #LOG_GROUP_ENV}, possibly null
* @return the same value when it is set and not blank
* @throws IllegalStateException when the value is null or blank
*/
static String requireLogGroupName(String logGroupName) {
if (logGroupName == null || logGroupName.isBlank()) {
throw new IllegalStateException(LOG_GROUP_ENV + " must be set");
}
return logGroupName;
}

/**
* Builds the telemetry plugin settings: one CloudWatch Logs exporter for the given log group, emit on completion,
* sample every execution, top-level operation detail, input and output included.
*
* <p>This method constructs no client. The exporter creates its CloudWatch Logs client on first export, and the
* settings object is a plain value. So a unit test can call it with no credentials and no network.
*
* @param logGroupName the log group the exporter writes to
* @return the plugin settings every handler in this module uses
*/
static DurableExecutionTelemetryConfig telemetryConfig(String logGroupName) {
return DurableExecutionTelemetryConfig.builder()
.addExporter(CloudWatchLogsExporter.builder()
.logGroupName(logGroupName)
.build())
.emitMode(DurableExecutionTelemetryConfig.EmitMode.ON_COMPLETE)
.samplingRate(1.0)
.operationDetail(DurableExecutionTelemetryConfig.OperationDetail.TOP_LEVEL)
.content(ContentConfig.builder().input(true).output(true).build())
.build();
}

/**
* Builds the handler configuration with the telemetry plugin from {@link #telemetryConfig(String)}.
*
* <p>{@code DurableConfig.build()} constructs the default Lambda client, and that constructor sends one warm-up
* request. So this method is for the deployed handler only. Tests check {@link #telemetryConfig(String)} instead.
*
* @param logGroupName the log group the exporter writes to
* @return the configuration a telemetry conformance handler returns from {@code createConfiguration()}
*/
static DurableConfig durableConfig(String logGroupName) {
return DurableConfig.builder()
.withPlugins(DurableExecutionTelemetry.durableExecutionTelemetry(telemetryConfig(logGroupName)))
.build();
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
// SPDX-License-Identifier: Apache-2.0
package software.amazon.lambda.durable.conformance.telemetry;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertInstanceOf;
import static org.junit.jupiter.api.Assertions.assertSame;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static software.amazon.lambda.durable.telemetry.DurableExecutionTelemetryConfig.EmitMode.ON_COMPLETE;
import static software.amazon.lambda.durable.telemetry.DurableExecutionTelemetryConfig.OperationDetail.TOP_LEVEL;

import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.NullSource;
import org.junit.jupiter.params.provider.ValueSource;
import software.amazon.lambda.durable.telemetry.exporters.CloudWatchLogsExporter;

/**
* Tests the shared handler setup through the helper with explicit values.
*
* <p>The handlers read the log group name in a static initializer, so the guard is tested here, not through the
* environment. {@code durableConfig} builds a {@code DurableConfig}, whose constructor creates the default Lambda
* client and sends a warm-up request. So these tests check {@code telemetryConfig}, which constructs no client.
*/
class TelemetryConformanceConfigTest {

private static final String LOG_GROUP = "/aws/lambda/durable-execution-telemetry";

@ParameterizedTest
@NullSource
@ValueSource(strings = {"", " ", "\t"})
void requireLogGroupNameRejectsMissingValue(String value) {
var e = assertThrows(IllegalStateException.class, () -> TelemetryConformanceConfig.requireLogGroupName(value));
assertEquals("TELEMETRY_LOG_GROUP_NAME must be set", e.getMessage());
}

@Test
void requireLogGroupNameReturnsSetValue() {
assertSame(LOG_GROUP, TelemetryConformanceConfig.requireLogGroupName(LOG_GROUP));
}

@Test
void telemetryConfigUsesOneCloudWatchLogsExporter() {
var config = TelemetryConformanceConfig.telemetryConfig(LOG_GROUP);
assertEquals(1, config.exporters().size());
// The exporter exposes no log group accessor, so the name is not asserted here. The handler passes its
// static LOG_GROUP_NAME straight through, and the deployed run matches no record if the name is wrong.
assertInstanceOf(CloudWatchLogsExporter.class, config.exporters().get(0));
}

@Test
void telemetryConfigMatchesTheRunnerExpectations() {
var config = TelemetryConformanceConfig.telemetryConfig(LOG_GROUP);
// The runner reads one record per execution with its input and output. These four settings produce that.
assertEquals(ON_COMPLETE, config.emitMode());
assertEquals(1.0, config.samplingRate());
assertEquals(TOP_LEVEL, config.operationDetail());
assertTrue(config.content().includeInput());
assertTrue(config.content().includeOutput());
}
}
11 changes: 11 additions & 0 deletions conformance-tests-telemetry/template.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -43,3 +43,14 @@ Resources:
CodeUri: .
Handler: software.amazon.lambda.durable.conformance.telemetry.Telemetry1Success
Role: !Ref LambdaExecutionRoleArn

Telemetry2Failure:
Type: AWS::Serverless::Function
DependsOn: TelemetryLogGroup
TestingMetadata:
TestDescription:
- telemetry-2
Properties:
CodeUri: .
Handler: software.amazon.lambda.durable.conformance.telemetry.Telemetry2Failure
Role: !Ref LambdaExecutionRoleArn
Loading