Fail instrumentation tests on logged stack overflows by ygree · Pull Request #13142 · DataDog/dd-trace-java

ygree · 2026-10-31T00:02:25Z

What Does This Do

Fail Spock and JUnit instrumentation tests when JUL or Logback logs a StackOverflowError during a test. Report the failure during cleanup with the original error as its cause.

Motivation

The Vert.x regression introduced by #13067 logged background stack overflows (fixed in #13128) while its response and trace assertions still passed. Both PR and merge-queue checks accepted those runs.

Make these errors visible as test failures instead of relying on them to cause missing traces or task timeouts.

Additional Notes

Capture the first error without formatting it on the thread whose stack is exhausted; remove logging hooks after each test.
Does not cover unlogged errors, other logging backends, or hangs before cleanup.

As expected, the build fails now. The fix works, though, and now detects the overlooked issue:

> Task :dd-java-agent:instrumentation:vertx:vertx-rx-3.5:forkedTest
VertxRxCircuitBreakerHttpServerForkedTest > test blocking of request for request body variant #variant > test blocking of request for request body variant urlencoded FAILED
    java.lang.AssertionError at LoggedStackOverflowCheck.java:53
        Caused by: java.lang.StackOverflowError at LinePerThreadBufferingOutputStream.java:243

Rebase once Fix unresolved write promises in Netty blocking handler #13067 merged, the test should then pass

Contributor Checklist

Format the title according to the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any other useful labels
Avoid using close, fix, or any linking keywords when referencing an issue
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, migration, or deletion
Update public documentation with any new configuration flags or behaviors
Once approved, use merge queue to merge the PR

Jira ticket: [PROJ-IDENT]

dd-octo-sts · 2026-10-31T00:25:17Z

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	13.93 s	13.95 s	[-0.7%; +0.4%] (no difference)
startup:insecure-bank:tracing:Agent	12.90 s	13.01 s	[-1.5%; -0.1%] (maybe better)
startup:petclinic:appsec:Agent	16.60 s	16.88 s	[-5.9%; +2.6%] (no difference)
startup:petclinic:iast:Agent	16.45 s	16.99 s	[-7.5%; +1.1%] (no difference)
startup:petclinic:profiling:Agent	16.52 s	16.81 s	[-2.8%; -0.6%] (maybe better)
startup:petclinic:sca:Agent	17.04 s	16.93 s	[-0.3%; +1.6%] (no difference)
startup:petclinic:tracing:Agent	16.08 s	16.18 s	[-1.6%; +0.4%] (no difference)

Commit: 26cceef1 · CI Pipeline · Benchmarking Platform UI

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

chatgpt-codex-connector · 2026-11-02T19:33:25Z

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-11-02T19:39:30.346036Z	`26cceef`	Draft marked ready
🔒 Security Review	✅ Completed 2026-11-02T19:37:17.459147Z	`26cceef`	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

datadog-datadog-prod-us1-2

Existing R2DBC teardown overrides bypass the new checks and leak logging hooks. Spring Boot logging resets also remove the Logback filter, allowing later stack overflows to pass undetected.

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit 26cceef}

datadog-datadog-prod-us1-2 · 2026-11-02T19:38:18Z

+      loggedStackOverflow.check();
+    } finally {
+      if (loggedStackOverflow != null) {
+        loggedStackOverflow.close();


Guarantee logging-hook cleanup when subclasses override tearDown

Existing R2DBC subclasses override tearDown() without calling super, so JUnit skips this base cleanup while inherited init() still installs the logging hooks. Those tests never check captured stack overflows and retain another handler/filter after each invocation. Put hook checking and detachment in dedicated lifecycle callbacks that cannot be bypassed by these overrides, or update all affected subclasses.

React: 👍 useful · 👎 wrong · 😕 unclear · 👀 right, but not acting on it

What each reaction means

👍 Useful: correct, and you're fixing it or will.

👎 Wrong: the finding is incorrect, for example the bug can't happen or the code was misread.

😕 Unclear: you can't tell what it means or why it matters.

👀 Right, not acting: correct, but intended, out of scope for this PR, a known risk, or too minor.

_{🤖 Bits Code Review} _{· Open Bits AI session}

datadog-datadog-prod-us1-2 · 2026-11-02T19:38:19Z

+      ch.qos.logback.classic.LoggerContext context = (ch.qos.logback.classic.LoggerContext) factory;
+      LogbackCapture filter = new LogbackCapture(check);
+      filter.start();
+      context.addTurboFilter(filter);


Preserve Logback capture across context resets

SpringBootApplicationTest starts Spring applications inside its feature methods, after the harness installs this filter. Spring Boot logging initialization resets LoggerContext, which clears its turbo filters. Subsequent Logback stack overflows therefore escape detection and can still pass cleanup. Add reset-resistant listener handling that reinstalls and restarts the filter, and remove both listener and filter during close.

React: 👍 useful · 👎 wrong · 😕 unclear · 👀 right, but not acting on it

What each reaction means

👍 Useful: correct, and you're fixing it or will.

👎 Wrong: the finding is incorrect, for example the bug can't happen or the code was misread.

😕 Unclear: you can't tell what it means or why it matters.

👀 Right, not acting: correct, but intended, out of scope for this PR, a known risk, or too minor.

_{🤖 Bits Code Review} _{· Open Bits AI session}

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 26cceef1d1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-11-02T19:39:26Z

+      ch.qos.logback.classic.LoggerContext context = (ch.qos.logback.classic.LoggerContext) factory;
+      LogbackCapture filter = new LogbackCapture(check);
+      filter.start();
+      context.addTurboFilter(filter);


Preserve capture across Logback context resets

When a test initializes or reconfigures Logback after the base setup() has run, LoggerContext.reset() clears the turbo-filter list, so the filter installed here silently disappears for the remainder of the test. This occurs in instrumentation suites that start an application from inside a feature—for example, SpringBootApplicationTest calls SpringApplication.run(...) after InstrumentationSpecification.setup()—and a subsequently logged StackOverflowError can still pass unnoticed. Make the capture reset-resistant or reinstall it when the context resets.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-11-02T19:39:26Z


    configureLoggingLevels()

+    loggedStackOverflow = new LoggedStackOverflowCheck()


Start capture before suite fixtures run

Because the capture is created only in per-iteration setup() and closed after every cleanup, setupSpec() and cleanupSpec() execute with no logging hook. Many instrumentation specifications start and stop asynchronous servers, clients, or containers in those suite fixtures; if a framework catches and logs a StackOverflowError from that work without propagating it, the entire suite can still pass. Keep a suite-level capture active across the fixture windows and report any retained error as a suite failure.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-11-02T19:39:26Z

+        String format,
+        Object[] params,
+        Throwable throwable) {
+      check.capture(throwable);


Ignore Logback events disabled by the effective level

When code passes a StackOverflowError to a disabled Logback call, such as logger.debug("caught", error) while the logger is configured at WARN, this filter still records it and fails the test even though no event is logged. Logback invokes turbo filters before applying the logger's effective-level check, and this filter captures unconditionally before returning NEUTRAL; only capture events that would remain enabled after the neutral decision.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-11-02T19:39:26Z

+        firstError.compareAndSet(null, (StackOverflowError) error);
+        return;
+      }
+      error = error.getCause();


Inspect suppressed exceptions for stack overflows

When a framework aggregates failures by attaching the original StackOverflowError as a suppressed exception, the logged wrapper renders that stack overflow but this traversal follows only getCause(), so cleanup does not fail. This is common for multi-failure and resource-cleanup wrappers; traverse a bounded, cycle-safe throwable graph that includes getSuppressed() rather than only the cause chain.

Useful? React with 👍 / 👎.

Fail instrumentation tests on logged stack overflows

4cf71e8

ygree self-assigned this Oct 31, 2026

ygree added comp: testing Testing tag: no release notes Changes to exclude from release notes labels Oct 31, 2026

ygree mentioned this pull request Oct 31, 2026

Fix Vert.x request blocking timeouts #13128

Merged

This comment has been minimized.

Sign in to view

ygree added the tag: ai generated Largely based on code generated by an AI or LLM label Oct 31, 2026

Merge branch 'master' into ygree/fail-logged-stack-overflows

26cceef

ygree marked this pull request as ready for review November 2, 2026 19:33

ygree requested review from a team as code owners November 2, 2026 19:33

ygree requested review from AlexeyKuznetsov-DD and jbachorik and removed request for a team November 2, 2026 19:33

datadog-datadog-prod-us1-2 Bot reviewed Nov 2, 2026

View reviewed changes

chatgpt-codex-connector Bot reviewed Nov 2, 2026

View reviewed changes

ygree marked this pull request as draft November 2, 2026 21:16

Navigation Menu

Repository navigation

Fail instrumentation tests on logged stack overflows - #13142

Fail instrumentation tests on logged stack overflows#13142
ygree wants to merge 2 commits into
masterfrom
ygree/fail-logged-stack-overflows

ygree commented Oct 31, 2026 •

edited

Loading

Uh oh!

This comment has been minimized.

dd-octo-sts Bot commented Oct 31, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector Bot commented Nov 2, 2026 •

edited

Loading

Uh oh!

datadog-datadog-prod-us1-2 Bot left a comment

Uh oh!

datadog-datadog-prod-us1-2 Bot Nov 2, 2026

Uh oh!

datadog-datadog-prod-us1-2 Bot Nov 2, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

chatgpt-codex-connector Bot Nov 2, 2026

Uh oh!

chatgpt-codex-connector Bot Nov 2, 2026

Uh oh!

chatgpt-codex-connector Bot Nov 2, 2026

Uh oh!

chatgpt-codex-connector Bot Nov 2, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant


		configureLoggingLevels()

		loggedStackOverflow = new LoggedStackOverflowCheck()

Conversation

ygree commented Oct 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

This comment has been minimized.

dd-octo-sts Bot commented Oct 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🟢 Java Benchmark SLOs — All performance SLOs passed

Uh oh!

chatgpt-codex-connector Bot commented Nov 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codex Review Summary

Uh oh!

datadog-datadog-prod-us1-2 Bot left a comment

Choose a reason for hiding this comment

Uh oh!

datadog-datadog-prod-us1-2 Bot Nov 2, 2026

Choose a reason for hiding this comment

Uh oh!

datadog-datadog-prod-us1-2 Bot Nov 2, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Nov 2, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Nov 2, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Nov 2, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Nov 2, 2026

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

ygree commented Oct 31, 2026 •

edited

Loading

dd-octo-sts Bot commented Oct 31, 2026 •

edited

Loading

chatgpt-codex-connector Bot commented Nov 2, 2026 •

edited

Loading