Skip to content

Reference-chains design and architecture docs - #802

Merged
jbachorik merged 7 commits into
mainfrom
jb/rc-8-docs
Oct 21, 2026
Merged

jbachorik merged 7 commits into
mainfrom
jb/rc-8-docs

Conversation

@jbachorik

@jbachorik jbachorik commented Sep 17, 2026 •

Copy link
Copy Markdown
Collaborator

What does this PR do?

Adds four reference-chain documents:

  • A collection summary with limits and result semantics.
  • A design record with alternatives and trade-offs.
  • An implementation reference for the current code.
  • A scheduling reference for signals, cost controls, urgency, and restart behavior.

Updates the documentation index. The PR changes documentation only.

Why is this change needed?

The reference-chain implementation was developed across the PR series through #800. PR #801 added tests and reproduction tooling. This PR documents the current implementation for maintainers and operators.

The documents distinguish adaptive targets from hard limits. They also describe best-effort behavior, abandonment, incomplete results, and historical observations across passes.

How was this change tested?

  • ./.claude/commands/build-and-summarize spotlessApply
  • git diff --check
  • Internal Markdown link validation.
  • Source-identifier and constant checks against the current implementation.
  • STE-style prose checks, including a 25-word sentence limit.

For Datadog employees

  • This PR does not change customer data handling.
  • Jira: PROF-15341

@dd-octo-sts

dd-octo-sts Bot commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

Scan-Build Report

User:runner@runnervmlun5p
Working Directory:/home/runner/work/java-profiler/java-profiler/ddprof-lib/src/test/make
Command Line:make -j4 all
Clang Version:Ubuntu clang version 18.1.3 (1ubuntu1)
Date:Thu Sep 17 20:07:38 2026

Bug Summary

Bug TypeQuantityDisplay?
All Bugs4
C++ move semantics
Use-after-move1
Logic error
Dereference of null pointer1
Result of operation is garbage or undefined1
Unused code
Dead increment1

Reports

Bug Group Bug Type ▾ File Function/Method Line Path Length
Unused codeDead incrementreferenceChains.cppcollectStaticFieldAnchorsForRotation36771
Logic errorDereference of null pointerfaultInjection.cppcrashNow242
Logic errorResult of operation is garbage or undefinedlivenessTracker.cppsecondsToOOM138324
C++ move semanticsUse-after-movereferenceChains.cppbuildCanaryChainEvent608368

@datadog-datadog-prod-us1-2

datadog-datadog-prod-us1-2 Bot commented Sep 17, 2026 •

Copy link
Copy Markdown

Pipelines

✨ Unblock PR with BitsAI

❌ Errors

Your PR has failed checks. Please review the issues below and take necessary action before merging.

🚦 2 Pipeline jobs failed

DataDog/java-profiler | integration-test-x64-musl: [openj9, 25] — 🔧 Needs a code fix, caused by this PR

View more details · View in GitLab

DataDog/java-profiler | post-pr-comment — 🔧 Needs a code fix, caused by this PR

View more details · View in GitLab

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: ae5a910 | Docs | View more details | Give us feedback!

@dd-octo-sts

dd-octo-sts Bot commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

CI Test Results

Run: #39485982421 | Commit: b26add4 | Duration: 18m 8s (longest job)

✅ All 32 test jobs passed

Status Overview

JDK glibc-aarch64/debug glibc-amd64/debug musl-aarch64/debug musl-amd64/debug
8 - ✅ - -
8-ibm - ✅ - -
8-j9 ✅ ✅ - -
8-librca - - ✅ ✅
8-orcl - ✅ - -
11 - ✅ - -
11-j9 ✅ ✅ - -
11-librca - - ✅ ✅
17 ✅ ✅ - -
17-graal ✅ ✅ - -
17-j9 ✅ ✅ - -
17-librca - - ✅ ✅
21 ✅ ✅ - -
21-graal ✅ ✅ - -
21-librca - - ✅ ✅
25 ✅ ✅ - -
25-graal ✅ ✅ - -
25-librca - - ✅ ✅

Legend: ✅ passed | ❌ failed | ⚪ skipped | 🚫 cancelled

Summary: Total: 32 | Passed: 32 | Failed: 0


Updated: 2026-10-20 20:06:24 UTC

@dd-octo-sts

dd-octo-sts Bot commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

❌ 39 passed, 1 failed out of 40 expected configurations

Test Matrix

Platform JDK 8 JDK 11 JDK 17 JDK 21 JDK 25
glibc-x64-hotspot ✅ ✅ ✅ ✅ ✅
glibc-x64-openj9 ✅ ✅ ✅ ✅ ✅
glibc-arm64-hotspot ✅ ✅ ✅ ✅ ✅
glibc-arm64-openj9 ✅ ✅ ✅ ✅ ✅
musl-x64-hotspot ✅ ✅ ✅ ✅ ✅
musl-x64-openj9 ✅ ✅ ✅ ✅ ❌
musl-arm64-hotspot ✅ ✅ ✅ ✅ ✅
musl-arm64-openj9 ✅ ✅ ✅ ✅ ✅

Failure Details

musl-x64-openj9-jdk25

Profiler-only:

"  Skipped (tracer not expected in this configuration)"
""
"[7/8] Checking for unexpected events..."
"  ✓ No unexpected events found"
""
"[8/8] Scenario-specific validation (ddprof_only)..."
"  Validating profiler-only scenario..."
"  ✓ Profiler-only scenario checks passed"
""
"=== Validation Summary ==="
"ExecutionSample:            0.0 events (datadog.ExecutionSample)"
"Stack traces:               0.0 samples"
"Thread diversity:           0.0 threads"
"Allocation samples:         0 events (jdk.ObjectAllocationSample)"
"ThreadAllocationStatistics: 45 events"
""
"VALIDATION_FAILED: One or more checks did not pass"

Script executed successfully: 425 commands
VALIDATION_FAILED: Validation checks did not pass

Links

@jbachorik
jbachorik force-pushed the jb/rc-8-docs branch 2 times, most recently from 6c5ba81 to 58262d1 Compare September 22, 2026 15:55
@dd-octo-sts

dd-octo-sts Bot commented Sep 22, 2026

Copy link
Copy Markdown
Contributor

Reliability & Chaos Results

❌ 29 failure(s) detected Pipeline: https://gitlab.ddbuild.io/DataDog/java-profiler/-/pipelines/139232204

❌ profiler gmalloc aarch64Xjit
Could not fetch ddprof 1.51.0-jb_rc-8-docs-SNAPSHOT jar
❌ profiler tcmalloc aarch64Xjit
Could not fetch ddprof 1.51.0-jb_rc-8-docs-SNAPSHOT jar
❌ profiler tcmalloc amd64Xjit
Could not fetch ddprof 1.51.0-jb_rc-8-docs-SNAPSHOT jar
❌ profiler tracer gmalloc aarch64Xjit
Could not fetch ddprof 1.51.0-jb_rc-8-docs-SNAPSHOT jar
❌ profiler tracer jemalloc amd64Xjit
Could not fetch ddprof 1.51.0-jb_rc-8-docs-SNAPSHOT jar
❌ profiler tracer tcmalloc amd64Xjit
Could not fetch ddprof 1.51.0-jb_rc-8-docs-SNAPSHOT jar
❌ chaos: profiler gmalloc aarch64 21 0 3 temXchaos
chaos.jar unavailable
❌ chaos: profiler gmalloc aarch64 25 0 3 temXchaos
chaos.jar unavailable
❌ chaos: profiler gmalloc amd64 21 0 3 temXchaos
chaos.jar unavailable
❌ chaos: profiler gmalloc amd64 25 0 3 temXchaos
ddprof jar unavailable (Maven snapshot download failed)
❌ chaos: profiler jemalloc aarch64 21 0 3 temXchaos
chaos.jar unavailable
❌ chaos: profiler jemalloc aarch64 25 0 3 temXchaos
ddprof jar unavailable (Maven snapshot download failed)
❌ chaos: profiler jemalloc amd64 21 0 3 temXchaos
chaos.jar unavailable
❌ chaos: profiler jemalloc amd64 25 0 3 temXchaos
chaos.jar unavailable
❌ chaos: profiler tcmalloc aarch64 21 0 3 temXchaos
chaos.jar unavailable
❌ chaos: profiler tcmalloc aarch64 25 0 3 temXchaos
chaos.jar unavailable
❌ chaos: profiler tcmalloc amd64 21 0 3 temXchaos
ddprof jar unavailable (Maven snapshot download failed)
❌ chaos: profiler tcmalloc amd64 25 0 3 temXchaos
chaos.jar unavailable
❌ chaos: profiler tracer gmalloc aarch64 21 0 3 temXchaos
chaos.jar unavailable
❌ chaos: profiler tracer gmalloc aarch64 25 0 3 temXchaos
ddprof jar unavailable (Maven snapshot download failed)
❌ chaos: profiler tracer gmalloc amd64 21 0 3 temXchaos
chaos.jar unavailable
❌ chaos: profiler tracer gmalloc amd64 25 0 3 temXchaos
chaos.jar unavailable
❌ chaos: profiler tracer jemalloc aarch64 21 0 3 temXchaos
ddprof jar unavailable (Maven snapshot download failed)
❌ chaos: profiler tracer jemalloc aarch64 25 0 3 temXchaos
ddprof jar unavailable (Maven snapshot download failed)
❌ chaos: profiler tracer jemalloc amd64 21 0 3 temXchaos
chaos.jar unavailable
❌ chaos: profiler tracer jemalloc amd64 25 0 3 temXchaos
chaos.jar unavailable
❌ chaos: profiler tracer tcmalloc aarch64 21 0 3 temXchaos
chaos.jar unavailable
❌ chaos: profiler tracer tcmalloc aarch64 25 0 3 temXchaos
chaos.jar unavailable
❌ chaos: profiler tracer tcmalloc amd64 21 0 3 temXchaos
chaos.jar unavailable

@jbachorik
jbachorik force-pushed the jb/rc-8-docs branch 2 times, most recently from 9931385 to 19b9d3f Compare September 23, 2026 16:07
@jbachorik
jbachorik removed this pull request from stack #803 October 12, 2026 08:02
@jbachorik
jbachorik added this pull request to stack #859 October 14, 2026 08:16
Base automatically changed from jb/rc-7-tooling to main October 20, 2026 18:33
@jbachorik
jbachorik marked this pull request as ready for review October 20, 2026 19:33
@jbachorik
jbachorik requested a review from a team as a code owner October 20, 2026 19:33
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-20T19:35:47.600998Z 9e9e02c Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9e9e02c7bd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread doc/ReferenceChainsCollectionSummary.md Outdated

@datadog-datadog-prod-us1-2 datadog-datadog-prod-us1-2 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: FAIL

The documented enablement option cannot produce reference-chain events without generation tracking. The implementation references also misstate deadline sharing and omit container-memory pressure as an OOM trigger.

Open Bits AI session

🤖 Bits Code Review · Commit 9e9e02c

Comment thread doc/ReferenceChainsCollectionSummary.md Outdated
Comment thread doc/architecture/LiveHeapReferenceChainsImplementation.md Outdated
Comment thread doc/architecture/ReferenceChainsSignalsExplained.md Outdated
@jbachorik
jbachorik merged commit 172a4b6 into main Oct 21, 2026
113 of 116 checks passed
@jbachorik
jbachorik deleted the jb/rc-8-docs branch October 21, 2026 10:16
@github-actions github-actions Bot added this to the 1.52.0 milestone Oct 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant