Skip to content

test: add X.509 end-to-end verification - #941

Merged
jbeckwith-oai merged 2 commits into
mainfrom
codex/x509-java-v2-e2e
Aug 27, 2026
Merged

test: add X.509 end-to-end verification#941
jbeckwith-oai merged 2 commits into
mainfrom
codex/x509-java-v2-e2e

Conversation

@jbeckwith-oai

Copy link
Copy Markdown
Contributor

Summary

  • extend X509TransportTest with deterministic loopback TLS coverage for one fixed client alias on both exact authorities, SNI, hostname and trust failures, bearer separation, raw issuer wire bytes, cleanup, and sanitized diagnostics
  • add a test-source-only, explicitly opt-in live issuer-to-GET /v1/models verification that reads credentials only from environment variables and proves client-alias selection independently on both TLS handshakes
  • add a manual, protected-environment workflow with minimal permissions, immutable action SHAs, explicit NOT RUN/PASSED evidence, and no credential artifacts
  • document hosted/local setup, observable evidence, diagnosis, rotation, and safe teardown

This validates the existing raw X509Transport contract. It does not add production authentication, public API, generated service, provider-specific, Realtime, Castiron, retry, refresh, cache, or lifecycle behavior.

Verification

  • focused X.509 selection: 12 tests, 11 passed and the credential-dependent live test skipped
  • ./gradlew :openai-java-client-okhttp:test
  • ./scripts/lint
  • ./scripts/build --no-configuration-cache
  • ./scripts/detect-breaking-changes origin/main
  • python3 -m unittest discover -s scripts/castiron -p 'test_custom_code*.py'
  • workflow YAML, embedded Bash syntax, permission shape, action SHA pinning, and git diff --check
  • terminal security diff review: complete scoped coverage, no reportable findings
  • adversarial review: two consecutive clean rounds from independent correctness/security and architecture/scope reviewers

Live verification status

NOT RUN locally: the required enrolled certificate and protected environment values were intentionally unavailable. The workflow remains manual, main-only, and gated by the x509-live-smoke environment; a skipped live test is not reported as production evidence.

Canonical context:

@jbeckwith-oai
jbeckwith-oai requested a review from a team as a code owner August 27, 2026 00:54
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Aug 27, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-08-27T01:58:18.744986Z 14a1524 New commits
🔒 Security Review Completed 2026-08-27T01:59:10.813165Z 14a1524 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@openai-sdks

openai-sdks Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

OkTest Summary

237/237 SDK tests passed in 14.834s for Java SDK PR #941.

Test results — 42 files
Test Result Time
tests/chat-completions-complex-body.test.ts ✅ Passed 419ms
tests/chat-completions-create.test.ts ✅ Passed 415ms
tests/chat-completions-stream.test.ts ✅ Passed 283ms
tests/files-content-binary.test.ts ✅ Passed 445ms
tests/files-create-multipart.test.ts ✅ Passed 256ms
tests/files-list-pagination.test.ts ✅ Passed 249ms
tests/initialize-config.test.ts ✅ Passed 245ms
tests/instance-isolation.test.ts ✅ Passed 204ms
tests/models-list.test.ts ✅ Passed 199ms
tests/responses-background-lifecycle.test.ts ✅ Passed 228ms
tests/responses-body-method-errors.test.ts ✅ Passed 554ms
tests/responses-cancel-timeout.test.ts ✅ Passed 232ms
tests/responses-cancel.test.ts ✅ Passed 259ms
tests/responses-compact-retries.test.ts ✅ Passed 306ms
tests/responses-compact.test.ts ✅ Passed 284ms
tests/responses-create-advanced-stream.test.ts ✅ Passed 539ms
tests/responses-create-advanced.test.ts ✅ Passed 1.315s
tests/responses-create-disconnect.test.ts ✅ Passed 1.072s
tests/responses-create-errors.test.ts ✅ Passed 349ms
tests/responses-create-malformed-api-responses.test.ts ✅ Passed 186ms
tests/responses-create-retries.test.ts ✅ Passed 283ms
tests/responses-create-stream-failures.test.ts ✅ Passed 165ms
tests/responses-create-stream-timeout.test.ts ✅ Passed 191ms
tests/responses-create-stream-wire.test.ts ✅ Passed 5.391s
tests/responses-create-stream.test.ts ✅ Passed 89ms
tests/responses-create-terminal-states.test.ts ✅ Passed 406ms
tests/responses-create-timeout.test.ts ✅ Passed 201ms
tests/responses-create.test.ts ✅ Passed 834ms
tests/responses-delete.test.ts ✅ Passed 235ms
tests/responses-input-items-errors.test.ts ✅ Passed 277ms
tests/responses-input-items-list.test.ts ✅ Passed 237ms
tests/responses-input-items-options.test.ts ✅ Passed 148ms
tests/responses-input-tokens-count-timeout.test.ts ✅ Passed 232ms
tests/responses-input-tokens-count.test.ts ✅ Passed 334ms
tests/responses-malformed-inputs.test.ts ✅ Passed 4.384s
tests/responses-not-found-errors.test.ts ✅ Passed 376ms
tests/responses-parse.test.ts ✅ Passed 666ms
tests/responses-retrieve-retries.test.ts ✅ Passed 225ms
tests/responses-retrieve.test.ts ✅ Passed 299ms
tests/responses-stored-method-errors.test.ts ✅ Passed 902ms
tests/retry-behavior.test.ts ✅ Passed 3.469s
tests/sdk-error-shape.test.ts ✅ Passed 396ms

View OkTest run #33031688819

SDK merge (54224ed7ec06) · head (14a1524a0e70) · base (35324bf54adf) · OkTest (2b1bdfd25e98)

@github-actions

github-actions Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Castiron custom code

✅ No new custom-code files detected.

53 mixed files remain; 0 existing customizations changed.

Compared 35324bf54adf14a1524a0e70. Generated baselines verified.

53 existing customizations unchanged
  • openai-java-core/src/main/kotlin/com/openai/models/audio/AudioResponseFormat.kt
  • openai-java-core/src/main/kotlin/com/openai/models/chat/completions/ChatCompletionCreateParams.kt
  • openai-java-core/src/main/kotlin/com/openai/models/chat/completions/ChatCompletionMessageFunctionToolCall.kt
  • openai-java-core/src/main/kotlin/com/openai/models/chat/completions/ChatCompletionToolMessageParam.kt
  • openai-java-core/src/main/kotlin/com/openai/models/embeddings/Embedding.kt
  • openai-java-core/src/main/kotlin/com/openai/models/embeddings/EmbeddingCreateParams.kt
  • openai-java-core/src/main/kotlin/com/openai/models/responses/ResponseCreateParams.kt
  • openai-java-core/src/main/kotlin/com/openai/models/responses/ResponseFunctionToolCall.kt
  • openai-java-core/src/main/kotlin/com/openai/models/responses/ResponseFunctionWebSearch.kt
  • openai-java-core/src/main/kotlin/com/openai/models/responses/ResponseInputItem.kt
  • openai-java-core/src/main/kotlin/com/openai/models/responses/ResponseTextConfig.kt
  • openai-java-core/src/main/kotlin/com/openai/models/videos/Video.kt
  • openai-java-core/src/main/kotlin/com/openai/services/async/BetaServiceAsync.kt
  • openai-java-core/src/main/kotlin/com/openai/services/async/BetaServiceAsyncImpl.kt
  • openai-java-core/src/main/kotlin/com/openai/services/async/ResponseServiceAsync.kt
  • openai-java-core/src/main/kotlin/com/openai/services/async/ResponseServiceAsyncImpl.kt
  • openai-java-core/src/main/kotlin/com/openai/services/async/WebhookServiceAsync.kt
  • openai-java-core/src/main/kotlin/com/openai/services/async/WebhookServiceAsyncImpl.kt
  • openai-java-core/src/main/kotlin/com/openai/services/async/audio/TranscriptionServiceAsyncImpl.kt
  • openai-java-core/src/main/kotlin/com/openai/services/async/chat/ChatCompletionServiceAsync.kt
  • openai-java-core/src/main/kotlin/com/openai/services/async/finetuning/checkpoints/PermissionServiceAsyncImpl.kt
  • openai-java-core/src/main/kotlin/com/openai/services/blocking/BetaService.kt
  • openai-java-core/src/main/kotlin/com/openai/services/blocking/BetaServiceImpl.kt
  • openai-java-core/src/main/kotlin/com/openai/services/blocking/ResponseService.kt
  • openai-java-core/src/main/kotlin/com/openai/services/blocking/ResponseServiceImpl.kt
  • openai-java-core/src/main/kotlin/com/openai/services/blocking/WebhookService.kt
  • openai-java-core/src/main/kotlin/com/openai/services/blocking/WebhookServiceImpl.kt
  • openai-java-core/src/main/kotlin/com/openai/services/blocking/audio/TranscriptionServiceImpl.kt
  • openai-java-core/src/main/kotlin/com/openai/services/blocking/chat/ChatCompletionService.kt
  • openai-java-core/src/main/kotlin/com/openai/services/blocking/finetuning/checkpoints/PermissionServiceImpl.kt
  • openai-java-core/src/test/kotlin/com/openai/models/beta/responses/BetaResponsesServerEventTest.kt
  • openai-java-core/src/test/kotlin/com/openai/models/responses/ResponsesServerEventTest.kt
  • openai-java-core/src/test/kotlin/com/openai/services/async/CompletionServiceAsyncTest.kt
  • openai-java-core/src/test/kotlin/com/openai/services/async/ImageServiceAsyncTest.kt
  • openai-java-core/src/test/kotlin/com/openai/services/async/ResponseServiceAsyncTest.kt
  • openai-java-core/src/test/kotlin/com/openai/services/async/WebhookServiceAsyncTest.kt
  • openai-java-core/src/test/kotlin/com/openai/services/async/audio/TranscriptionServiceAsyncTest.kt
  • openai-java-core/src/test/kotlin/com/openai/services/async/beta/ResponseServiceAsyncTest.kt
  • openai-java-core/src/test/kotlin/com/openai/services/async/beta/ThreadServiceAsyncTest.kt
  • openai-java-core/src/test/kotlin/com/openai/services/async/beta/threads/RunServiceAsyncTest.kt

13 more in the full report.

A changed generated baseline means this report cannot reliably identify which handwritten lines changed.

Inspect the custom-code diff

Download the exact patch produced by this run (requires repository access):

gh run download 33031710813 --repo openai/openai-java \
  --name castiron-custom-code-33031710813-1 --dir /tmp/castiron-custom-code-33031710813-1
git apply --stat /tmp/castiron-custom-code-33031710813-1/custom-code.patch
cat /tmp/castiron-custom-code-33031710813-1/custom-code.patch

Or reproduce it from an SDK checkout containing the vendored reporter:

git fetch --no-tags origin 35324bf54adf68ea1853f7684bdbdf29c9fd329c 14a1524a0e70270ca966bdb6f5da8439157483d7
python3 scripts/castiron/custom_code_report.py report \
  --base 35324bf54adf68ea1853f7684bdbdf29c9fd329c \
  --head 14a1524a0e70270ca966bdb6f5da8439157483d7 --fetch --require-head-hash --public \
  --out /tmp/castiron-custom-code-14a1524a0e70
cat /tmp/castiron-custom-code-14a1524a0e70/custom-code.patch

This is the current full custom patch for mixed files, not an attribution of only the handwritten lines changed by this PR.

Full report and patch

@HAYDEN-OAI HAYDEN-OAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The end-to-end TLS verification and default-deny live gate look solid, but the protected credential-bearing workflow bypasses the repository's established Gradle cache trust boundary. Please isolate its Gradle User Home before exposing the enrolled client certificate and private-key credentials.

Comment thread .github/workflows/x509-live-smoke.yml

@HAYDEN-OAI HAYDEN-OAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed the protected live-verification workflow and confirmed the previous credential-boundary finding is fully addressed. The workflow creates a unique private per-run Gradle User Home, rejects preexisting paths and symlinks, disables shared Gradle caching before any protected credentials are attached, and matches the established release isolation policy. The manual main-only environment gate, fixed-origin issuer/API mTLS checks, certificate alias selection, sensitive-data redaction, and deterministic transport verification remain intact. Approved.

@jbeckwith-oai
jbeckwith-oai added this pull request to the merge queue Aug 27, 2026
Merged via the queue into main with commit ed2a307 Aug 27, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants