Add decoded span assert rules by PerfectSlayer · Pull Request #12088 · DataDog/dd-trace-java

PerfectSlayer · 2026-07-28T09:15:53Z

What Does This Do

This PR introduces decoded span assert rules.

EDIT: This PR unit tests need test agent JSON  span deserialization first. I’ll introduce an intermediate PR to the stack to add it. It is still good to review in the meantime.

Motivation

The goal is to provide capabilities for smoke tests to evaluate trace and span structures and values.

Additional Notes

This is mainly inspired from my prior work with instrumentation tests but decoded spans are not like DDSpan. They don’t hold the same values / types so new rules needed to be put in place for smoke tests.

Contributor Checklist

Format the title according to the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any other useful labels
Avoid using close, fix, or any linking keywords when referencing an issue
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, migration, or deletion
Update public documentation with any new configuration flags or behaviors
Once approved, use merge queue to merge the PR

Jira ticket: APMLP-1247

datadog-datadog-prod-us1-2 · 2026-07-28T09:18:31Z

Bits has a CI fix ready

🟢 Investigated · 🟢 Fix prepared · ⚪ Validation skipped · 🟠 Ready

SmokeMatcherTest imports the nonexistent datadog.smoketest.backend.TestAgentTraceDecoder, so the smoke-test compile job cannot resolve its synthetic trace fixtures. The tests now create lightweight in-test DecodedTrace and DecodedSpan fixtures, preserving the matcher coverage without depending on an unavailable backend decoder.

Commit fix to this PR

View in Datadog | Reviewed commit 5134699 · Any feedback? Reach out in #deveng-pr-agent

datadog-datadog-prod-us1-2

⚠️ Autotest was unable to complete this review. View session

Please try again by commenting @autotest review.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 513469965b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

dd-octo-sts · 2026-07-28T09:40:07Z

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	13.97 s	13.93 s	[-0.4%; +0.9%] (no difference)
startup:insecure-bank:tracing:Agent	12.93 s	13.02 s	[-1.5%; +0.2%] (no difference)
startup:petclinic:appsec:Agent	16.95 s	16.01 s	[+1.5%; +10.3%] (significantly worse)
startup:petclinic:iast:Agent	16.88 s	16.94 s	[-1.2%; +0.5%] (no difference)
startup:petclinic:profiling:Agent	16.59 s	16.97 s	[-3.7%; -0.9%] (maybe better)
startup:petclinic:sca:Agent	16.86 s	16.95 s	[-1.5%; +0.5%] (no difference)
startup:petclinic:tracing:Agent	16.06 s	16.15 s	[-1.7%; +0.6%] (no difference)

Commit: 85833735 · CI Pipeline · Benchmarking Platform UI

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

PerfectSlayer · 2026-07-28T16:44:03Z

@autotest review

datadog-datadog-prod-us1-2

Unordered trace matching can reject a valid trace when a reusable childOfPrevious matcher first probes a non-matching candidate: assertSpan mutates the matcher to that candidate's parent ID, so later candidates are checked against stale state. Keep the resolved parent matcher local instead of mutating the configured matcher.

_{Open Bits AI session}

_{🤖 Datadog Autotest · Commit d7f5363 · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest}

datadog-datadog-prod-us1-2

More details

The decoder and assertion paths do not show a confirmed behavioral regression or diff-only production hazard. Adversarial cases were reviewed for malformed JSON, unsigned IDs, missing fields, duplicate/unordered trace matching, ancestry ordering, partial trace chunks, and nested meta_struct values; execution was blocked before tests ran because the checkout requires JDK 25 and the sandbox has only JDK 8/11/17/21.

Was this helpful? React 👍 or 👎

_{Open Bits AI session}

_{🤖 Datadog Autotest · Commit 798798a · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest}

PerfectSlayer · 2026-07-29T06:00:55Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 798798ab44

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

PerfectSlayer · 2026-07-29T11:38:17Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1004c0a40d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-07-29T11:44:44Z

+  @Json(name = "meta_struct")
+  Map<String, Object> metaStruct;


Normalize numeric meta_struct values across decoders

When a meta_struct entry contains numeric leaves, Moshi's generic Object adapter materializes them as Double, while the existing msgpack decoder returns Long for integers and Float for floating-point values in SpanV04.convertValueToObject (SpanV04.java:204-207). Consequently, an exact metaStruct(..., is(expected)) matcher or a predicate casting a numeric leaf can pass with the in-process backend and fail with the dd-apm-test-agent JSON backend; decode these nested numbers into the same representation before exposing the shared DecodedSpan model.

Useful? React with 👍 / 👎.

datadog-datadog-prod-us1-2

More details

The decoded-span assertion layer matches the intended behavior for positional, sorted, ancestry-based, unordered, parent-link, default-constraint, and meta_struct checks. No diff-only regression was confirmed; focused Gradle execution was attempted but the sandbox lacks JDK 25 and cannot download it, so validation is limited to source analysis and existing test coverage.

Was this helpful? React 👍 or 👎

_{Open Bits AI session}

_{🤖 Datadog Autotest · Commit 1004c0a · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest}

PerfectSlayer · 2026-07-29T12:40:54Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 85833735e9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-07-29T12:48:17Z

+
+  private SpanMatcher() {
+    this.serviceMatcher = validates(s -> s != null && !s.isEmpty());
+    this.typeMatcher = isNull();


Accept empty wire types as absent

When an untyped span is decoded from v0.5 or v1 input, getType() returns "", not null: TraceMapperV0_5.writeDictionaryEncoded serializes null strings as empty strings, and SpanV1.unpack initializes an omitted type to "". Consequently the default span() matcher rejects ordinary untyped spans from these backends, even though the same span passes when decoded from the JSON fixture used by this commit's tests. Normalize absent types or make the default matcher accept both null and empty strings so assertions behave consistently across supported decoded formats.

Useful? React with 👍 / 👎.

PerfectSlayer added the type: feature Enhancements and improvements label Jul 28, 2026

PerfectSlayer requested a review from a team as a code owner July 28, 2026 09:15

PerfectSlayer added the comp: testing Testing label Jul 28, 2026

PerfectSlayer requested a review from manuel-alvarez-alvarez July 28, 2026 09:15

PerfectSlayer mentioned this pull request Jul 28, 2026

Improving smoke test capabilities #12051

Draft

datadog-datadog-prod-us1-2 Bot reviewed Jul 28, 2026

View reviewed changes

chatgpt-codex-connector Bot reviewed Jul 28, 2026

View reviewed changes

Comment thread dd-smoke-tests/src/main/java/datadog/smoketest/trace/SmokeTraceAssertions.java Outdated

Comment thread dd-smoke-tests/src/main/java/datadog/smoketest/trace/SpanMatcher.java Outdated

Comment thread dd-smoke-tests/src/main/java/datadog/smoketest/trace/SmokeTraceAssertions.java

This comment has been minimized.

Sign in to view

PerfectSlayer marked this pull request as draft July 28, 2026 09:59

PerfectSlayer force-pushed the bbujon/smoke-tests-decoded-span-assertions branch from 5134699 to d7f5363 Compare July 28, 2026 16:42

datadog-datadog-prod-us1-2 Bot reviewed Jul 28, 2026

View reviewed changes

Comment thread dd-smoke-tests/src/main/java/datadog/smoketest/trace/SpanMatcher.java Outdated

PerfectSlayer force-pushed the bbujon/smoke-tests-decoded-span-assertions branch from d7f5363 to 798798a Compare July 28, 2026 17:31

PerfectSlayer marked this pull request as ready for review July 28, 2026 17:34

datadog-datadog-prod-us1-2 Bot reviewed Jul 28, 2026

View reviewed changes

chatgpt-codex-connector Bot reviewed Jul 29, 2026

View reviewed changes

PerfectSlayer force-pushed the bbujon/smoke-tests-decoded-span-assertions branch from 798798a to 1004c0a Compare July 29, 2026 10:00

chatgpt-codex-connector Bot reviewed Jul 29, 2026

View reviewed changes

PerfectSlayer changed the base branch from bbujon/smoke-tests-decoded-spans to bbujon/smoke-tests-test-agent-decoding July 29, 2026 12:04

datadog-datadog-prod-us1-2 Bot reviewed Jul 29, 2026

View reviewed changes

PerfectSlayer force-pushed the bbujon/smoke-tests-decoded-span-assertions branch from 1004c0a to 8583373 Compare July 29, 2026 12:40

PerfectSlayer force-pushed the bbujon/smoke-tests-test-agent-decoding branch from fa90299 to 8ff8af2 Compare July 29, 2026 12:40

chatgpt-codex-connector Bot reviewed Jul 29, 2026

View reviewed changes

feat(testing): Add decoded span assert rules

d100b5a

PerfectSlayer force-pushed the bbujon/smoke-tests-test-agent-decoding branch from 8ff8af2 to 3e09e7c Compare July 29, 2026 13:02

PerfectSlayer force-pushed the bbujon/smoke-tests-decoded-span-assertions branch from 8583373 to d100b5a Compare July 29, 2026 13:03

Uh oh!

Conversation

PerfectSlayer commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

datadog-datadog-prod-us1-2 Bot commented Jul 28, 2026 • edited by datadog-prod-us1-6 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bits has a CI fix ready

Uh oh!

datadog-datadog-prod-us1-2 Bot left a comment

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

dd-octo-sts Bot commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🟢 Java Benchmark SLOs — All performance SLOs passed

Uh oh!

PerfectSlayer commented Jul 28, 2026

Uh oh!

datadog-datadog-prod-us1-2 Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

datadog-datadog-prod-us1-2 Bot left a comment

Choose a reason for hiding this comment

Uh oh!

PerfectSlayer commented Jul 29, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

PerfectSlayer commented Jul 29, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

datadog-datadog-prod-us1-2 Bot left a comment

Choose a reason for hiding this comment

Uh oh!

PerfectSlayer commented Jul 29, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

PerfectSlayer commented Jul 28, 2026 •

edited

Loading

datadog-datadog-prod-us1-2 Bot commented Jul 28, 2026 •

edited by datadog-prod-us1-6 Bot

Loading

dd-octo-sts Bot commented Jul 28, 2026 •

edited

Loading