diff --git a/AGENTS.md b/AGENTS.md index 96fc23c..ad901b8 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -2,13 +2,13 @@ ## Project -Threadline is a full-Rust bridge between VS Code Copilot BYOK Custom Endpoint requests and the Codex backend WebSocket protocol. +Threadline is a full-Rust bridge between VSCode Copilot BYOK Custom Endpoint requests and the Codex backend WebSocket protocol. -Threadline is inspired by lessons learned from ChatMock experiments, but it must not copy, port, or reuse ChatMock source code. Implementations, module boundaries, names, tests, and comments must be original to this repository. +Threadline is informed by lessons from ChatMock experiments, but it must not copy, port, translate, or preserve ChatMock source code, internal names, comments, tests, or implementation structure. ## Primary goals -* Provide a stable `/v1/responses` endpoint for VS Code BYOK. +* Provide a stable `/v1/responses` endpoint for VSCode BYOK. * Bridge HTTP/SSE requests to Codex backend WebSocket sessions. * Preserve `previous_response_id` continuity through retained WebSocket sessions. * Keep retained upstream WebSockets alive with a pump-based Ping/Pong design. @@ -20,29 +20,37 @@ Threadline is inspired by lessons learned from ChatMock experiments, but it must Do not turn Threadline into a general-purpose OpenAI-compatible proxy. -Avoid adding compatibility for unrelated providers, historical ChatMock behavior, prompt-file injection, or Python ChatMock behavior unless explicitly requested. +Do not add unrelated providers, historical ChatMock behavior, prompt-file injection, or Python ChatMock behavior unless explicitly requested. -Do not implement `/v1/chat/completions` unless it is needed for VS Code BYOK compatibility. `/v1/responses` is the primary API. +Do not implement `/v1/chat/completions` unless it is needed for VSCode BYOK compatibility. `/v1/responses` is the primary API. -## Source independence rule +## Rule priority -Do not copy code from ChatMock. +Root `AGENTS.md` contains always-on rules. -Do not preserve ChatMock-specific module names, function names, comments, test names, or internal terminology unless the term describes a public protocol concept. +Detailed examples and expanded guidance live in `docs/agent/`. + +If a root rule conflicts with a detail document, follow the root rule and update the detail document later. + +## Read-on-demand docs + +Read `docs/agent/protocol.md` before changing WebSocket, retained session, registry, internal tool, job, or error behavior. -Allowed: +Read `docs/agent/architecture.md` before changing module boundaries or doing large refactors. -* Reusing design lessons learned from prior experiments. -* Reimplementing behavior from scratch. -* Using public protocol names such as `response.create`, `previous_response_id`, `session-id`, and `thread-id`. +Read `docs/agent/conventions.md` before editing names, comments, test names, tracing, logs, commits, PR text, or public wording. -Not allowed: +Read `docs/agent/workflow.md` before final validation, CI, CodeQL, local-only notes, or development summaries. -* Copying ChatMock functions or structs. -* Translating ChatMock code line-by-line. -* Keeping ChatMock-only names such as `chatmock_*` for Threadline features. +## Source independence + +Do not copy, port, or translate ChatMock code. + +Do not preserve ChatMock-specific module names, function names, comments, test names, or internal terminology unless the term describes a public protocol concept. -Use `threadline_*` for internal tools and Threadline-specific concepts. +Allowed: reusing design lessons, reimplementing behavior from scratch, and using public protocol names such as `response.create`, `previous_response_id`, `session-id`, and `thread-id`. + +Use `threadline_*` for Threadline-owned internal tools and Threadline-specific concepts. ## Architecture principles @@ -63,155 +71,51 @@ Suggested module boundaries: Keep protocol types separate from transport code. -## Naming conventions - -Use Rust naming conventions: - -* Modules: `snake_case` -* Functions: `snake_case` -* Variables: `snake_case` -* Types: `PascalCase` -* Enum variants: `PascalCase` -* Constants: `SCREAMING_SNAKE_CASE` +New code should fit an existing responsibility or introduce a clearly named module with one durable purpose. -Prefer precise names over short names. +## Naming and terminology -Good: +Use normal Rust naming conventions: `snake_case` for modules/functions/variables, `PascalCase` for types/enum variants, and `SCREAMING_SNAKE_CASE` for constants. -* `RetainedSessionRegistry` -* `UpstreamWebSocketPump` -* `ResponseMarker` -* `ThreadlineJobManager` -* `PendingInternalToolOutput` -* `send_followup_tool_outputs` +Prefer precise names over short or phase-based names. -Avoid: - -* `Thing` -* `Manager2` -* `handle_stuff` -* `phase1_handler` -* `test_new_flow` -* `chatmock_*` - -## Public terminology +Avoid names such as `Thing`, `Manager2`, `handle_stuff`, `phase1_handler`, `test_new_flow`, or `chatmock_*`. Use these terms consistently: * `upstream`: Codex backend WebSocket side. -* `downstream`: VS Code BYOK HTTP/SSE client side. -* `response marker`: a `previous_response_id` / completed response id used for continuation. +* `downstream`: VSCode BYOK HTTP/SSE client side. +* `response marker`: a `previous_response_id` or completed response id used for continuation. * `retained session`: a stored upstream WebSocket plus session metadata. * `internal tool`: a Threadline-handled tool hidden from downstream clients. * `job`: a long-running local or subprocess task managed by Threadline. * `pump`: the task that continuously reads/writes an upstream WebSocket and handles Ping/Pong. -## Comments +## Comments and tests Comments should explain durable design intent, protocol quirks, or safety constraints. -Do not write comments that only describe temporary implementation phases, local experiments, or orchestration history. - -Allowed: +Do not write comments that only describe temporary implementation phases, local experiments, orchestration history, or model conversations. -```rust -// The pump must keep reading while the session is idle so server Ping frames receive Pong responses. -``` +Do not include phase labels, local machine details, personal paths, transcript details, or “Codex told me to...” style text in comments, tests, public docs, commit messages, or PR text. -Not allowed: - -```rust -// Phase 2 fix from the ChatMock experiment. -``` - -Not allowed: - -```rust -// Local test workaround from today's debugging. -``` - -Do not include: - -* Phase labels such as `Phase 1`, `Phase 2`, `rust-test`, or `temporary ChatMock fix`. -* Local machine details. -* Personal paths. -* Chat transcript details. -* Debugging history that will not matter to future maintainers. -* Model conversation artifacts. -* “Codex told me to...” style comments. - -If historical context is useful, write it as a general protocol/design reason. - -## Test naming +If historical context is useful, rewrite it as a general protocol or design reason. Test names must describe behavior, not implementation phase or local context. -Good: - -```rust -retained_session_reconnects_after_idle_close_before_first_event -websocket_pump_replies_to_server_ping_while_idle -internal_tool_outputs_are_sent_after_intermediate_response_completes -job_manager_returns_incremental_output_after_offset -``` - -Bad: - -```rust -phase_3_reconnect_test -chatmock_regression_test -test_from_logs_0605 -rust_test_branch_case -``` - -## Tracing and logs - -Use structured tracing fields. +## Tracing, logs, and errors -Good: +Use structured tracing fields and stable, grep-friendly log event names. -```rust -tracing::debug!( - response_id = %response_id, - session_id = %session_id, - "retained_session_acquired" -); -``` - -Avoid putting secrets, raw tokens, refresh tokens, cookies, or full authorization headers in logs. +Never log secrets, raw tokens, refresh tokens, cookies, or full authorization headers. Raw upstream `error` events may be logged at debug/trace level only after confirming they do not contain secrets. -Log event names should be stable and grep-friendly: - -* `ws_pump_started` -* `ws_pump_ping_received` -* `ws_pump_pong_sent` -* `upstream_event_received` -* `internal_tool_detected` -* `internal_tool_followup_sent` -* `final_response_completed` -* `retained_session_acquired` -* `retained_session_released` -* `reconnect_continuation_attempt` -* `reconnect_continuation_failed` -* `upstream_error_event` - -## Error handling - Prefer typed errors internally. -Public HTTP/SSE errors should be stable and VS Code compatible. - -Use clear error codes for expected states: +Public HTTP/SSE errors should be stable and VSCode compatible. -* `previous_response_not_found` -* `retained_session_conflict` -* `retained_session_capacity_exceeded` -* `upstream_websocket_connect_failed` -* `upstream_websocket_closed` -* `internal_tool_failed` -* `job_not_found` +Use clear error codes for expected states. Do not panic for protocol errors, malformed client input, missing markers, closed sockets, or upstream errors. @@ -221,14 +125,7 @@ All live upstream WebSockets must be pump-based. Do not directly hold and use `WebSocketStream` from route handlers. -The pump must: - -* Run while the session is retained. -* Read frames even when no HTTP request is currently waiting. -* Reply to server Ping frames with Pong. -* Forward Text/Binary frames into an inbound queue. -* Accept outbound Text/Ping/Close commands through a channel. -* Mark close/error metadata when the socket closes. +The pump must run while the session is retained, read idle frames, reply to Ping with Pong, forward Text/Binary frames into an inbound queue, accept outbound commands through a channel, and mark close/error metadata. A retained WebSocket that is idle must still be alive enough to answer Ping/Pong. @@ -236,17 +133,7 @@ A retained WebSocket that is idle must still be alive enough to answer Ping/Pong The retained session registry maps completed response markers to upstream session state. -A registry entry should store: - -* response marker -* upstream WebSocket handle -* session id -* thread id -* window generation -* turn state -* in-use flag -* close/recoverable state -* last-used timestamp +A registry entry should preserve the response marker, upstream handle, session id, thread id, window generation, turn state, in-use flag, close/recoverable state, and last-used timestamp. If a retained socket is closed after a completed response, preserve recoverable metadata when possible. @@ -256,16 +143,9 @@ Do not delete a response marker merely because a socket close was observed after Threadline internal tools must use the `threadline_*` prefix. -Internal tool calls must never be forwarded downstream to VS Code. - -When an upstream response emits a Threadline internal tool call: +Internal tool calls must never be forwarded downstream to VSCode. -1. Execute the internal tool locally. -2. Store the output as pending. -3. Wait for the intermediate response to complete. -4. Send a follow-up `response.create` with `function_call_output`. -5. Continue reading the follow-up response. -6. Forward only the final assistant output downstream. +When an upstream response emits a Threadline internal tool call, execute it locally, store the output as pending, wait for the intermediate response to complete, send a follow-up `response.create` with `function_call_output`, continue reading the follow-up response, and forward only the final assistant output downstream. Do not send follow-up tool outputs before the intermediate response completes. @@ -275,56 +155,23 @@ Do not treat an intermediate response completion as the final completion. Long-running work should be represented as jobs. -A job should be started quickly and return a `job_id`. +A job should start quickly and return a `job_id`. Use polling or result retrieval for later status. -Internal job tools: - -* `threadline_start_job` -* `threadline_poll_job` -* `threadline_read_job_output` -* `threadline_get_job_result` -* `threadline_cancel_job` - -Job completion must not automatically push a new upstream response. - -Job completion should update stored job state only. - -## Security +Internal job tools should use the `threadline_*` prefix. -Never log secrets. +Job completion must update stored job state only and must not automatically push a new upstream response. -Never commit local credentials, cookies, refresh tokens, access tokens, or account identifiers. +## Security and local-only notes -Use `.gitignore` for local state directories. +Never commit local credentials, cookies, refresh tokens, access tokens, account identifiers, or production credentials. -Recommended ignored paths: +Use `.gitignore` for local state such as `.threadline/`, `*.local.json`, `*.local.toml`, and `*.log`. -```gitignore -.threadline/ -*.local.json -*.local.toml -*.log -``` +Local orchestration notes must not be committed unless generalized into durable documentation. -Do not store production credentials in test fixtures. - -## Local-only notes - -Local orchestration notes must not be committed unless they are generalized into durable documentation. - -Use local-only files such as: - -```txt -.threadline/notes.md -.threadline/debug-log.md -.threadline/orchestration.md -``` - -These files should be ignored by git. - -Do not copy local-only context into source comments, test names, public docs, or commit messages. +Do not copy local-only context into source comments, test names, public docs, commit messages, or test fixtures. ## Commit and PR guidance @@ -332,32 +179,15 @@ Keep commits focused. Commit messages should describe behavior, not orchestration phase. -Good: - -```txt -Add pump-based upstream websocket transport -Preserve recoverable retained sessions after idle close -Add internal job tools for long-running tasks -``` - -Bad: - -```txt -Phase 2 fixes -Apply Codex suggestions -Fix bug from local log -Port ChatMock behavior -``` +Avoid temporary branch names, phase labels, local debugging notes, model-conversation artifacts, or ChatMock porting language in commits and PRs. ## Validation Before considering a change complete, run the relevant checks: -```sh -cargo fmt -cargo clippy --all-targets --all-features -cargo test -``` +* `cargo fmt` +* `cargo clippy --all-targets --all-features` +* `cargo test` If a check cannot be run, record why in the final development summary, not in source comments. @@ -365,23 +195,24 @@ If a check cannot be run, record why in the final development summary, not in so Keep GitHub Actions workflow names stable and descriptive. -Use CodeQL for Rust security scanning. Prefer manual build mode so analysis sees the same crate graph that `cargo build` uses. +Use CodeQL for Rust security scanning, preferably with manual build mode so analysis sees the same crate graph that `cargo build` uses. Do not add temporary branch names, local phase labels, or orchestration notes to workflow names, job names, or step names. ## Development summary format -When reporting changes, use: +When reporting changes, use this shape: -```txt Changed: -- ... + +* ... Validation: -- ... + +* ... Risks: -- ... -``` -Do not include private local paths, temporary phase labels, or transcript-only context in code or tests. +* ... + +Do not include private local paths, temporary phase labels, or transcript-only context in code, tests, or summaries. diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..bf7d7cb --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,9 @@ +# CLAUDE.md + +@AGENTS.md + +## Claude Code + +- Follow the project instructions in AGENTS.md. +- When making non-trivial changes, explain the plan before editing. +- Prefer small, reviewable commits. diff --git a/Cargo.lock b/Cargo.lock new file mode 100644 index 0000000..b3c4b9f --- /dev/null +++ b/Cargo.lock @@ -0,0 +1,2531 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 4 + +[[package]] +name = "aes" +version = "0.8.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b169f7a6d4742236a0a00c541b845991d0ac43e546831af1249753ab4c3aa3a0" +dependencies = [ + "cfg-if", + "cipher", + "cpufeatures", +] + +[[package]] +name = "aho-corasick" +version = "1.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ddd31a130427c27518df266943a5308ed92d4b226cc639f5a8f1002816174301" +dependencies = [ + "memchr", +] + +[[package]] +name = "anstream" +version = "1.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "824a212faf96e9acacdbd09febd34438f8f711fb84e09a8916013cd7815ca28d" +dependencies = [ + "anstyle", + "anstyle-parse", + "anstyle-query", + "anstyle-wincon", + "colorchoice", + "is_terminal_polyfill", + "utf8parse", +] + +[[package]] +name = "anstyle" +version = "1.0.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "940b3a0ca603d1eade50a4846a2afffd5ef57a9feac2c0e2ec2e14f9ead76000" + +[[package]] +name = "anstyle-parse" +version = "1.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "52ce7f38b242319f7cabaa6813055467063ecdc9d355bbb4ce0c68908cd8130e" +dependencies = [ + "utf8parse", +] + +[[package]] +name = "anstyle-query" +version = "1.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "40c48f72fd53cd289104fc64099abca73db4166ad86ea0b4341abe65af83dadc" +dependencies = [ + "windows-sys 0.61.2", +] + +[[package]] +name = "anstyle-wincon" +version = "3.0.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "291e6a250ff86cd4a820112fb8898808a366d8f9f58ce16d1f538353ad55747d" +dependencies = [ + "anstyle", + "once_cell_polyfill", + "windows-sys 0.61.2", +] + +[[package]] +name = "anyhow" +version = "1.0.102" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7f202df86484c868dbad7eaa557ef785d5c66295e41b460ef922eca0723b842c" + +[[package]] +name = "async-broadcast" +version = "0.7.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "435a87a52755b8f27fcf321ac4f04b2802e337c8c4872923137471ec39c37532" +dependencies = [ + "event-listener", + "event-listener-strategy", + "futures-core", + "pin-project-lite", +] + +[[package]] +name = "async-channel" +version = "2.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "924ed96dd52d1b75e9c1a3e6275715fd320f5f9439fb5a4a11fa51f4221158d2" +dependencies = [ + "concurrent-queue", + "event-listener-strategy", + "futures-core", + "pin-project-lite", +] + +[[package]] +name = "async-executor" +version = "1.14.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c96bf972d85afc50bf5ab8fe2d54d1586b4e0b46c97c50a0c9e71e2f7bcd812a" +dependencies = [ + "async-task", + "concurrent-queue", + "fastrand", + "futures-lite", + "pin-project-lite", + "slab", +] + +[[package]] +name = "async-fs" +version = "2.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8034a681df4aed8b8edbd7fbe472401ecf009251c8b40556b304567052e294c5" +dependencies = [ + "async-lock", + "blocking", + "futures-lite", +] + +[[package]] +name = "async-io" +version = "2.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "456b8a8feb6f42d237746d4b3e9a178494627745c3c56c6ea55d92ba50d026fc" +dependencies = [ + "autocfg", + "cfg-if", + "concurrent-queue", + "futures-io", + "futures-lite", + "parking", + "polling", + "rustix", + "slab", + "windows-sys 0.61.2", +] + +[[package]] +name = "async-lock" +version = "3.4.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "290f7f2596bd5b78a9fec8088ccd89180d7f9f55b94b0576823bbbdc72ee8311" +dependencies = [ + "event-listener", + "event-listener-strategy", + "pin-project-lite", +] + +[[package]] +name = "async-process" +version = "2.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fc50921ec0055cdd8a16de48773bfeec5c972598674347252c0399676be7da75" +dependencies = [ + "async-channel", + "async-io", + "async-lock", + "async-signal", + "async-task", + "blocking", + "cfg-if", + "event-listener", + "futures-lite", + "rustix", +] + +[[package]] +name = "async-recursion" +version = "1.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3b43422f69d8ff38f95f1b2bb76517c91589a924d1559a0e935d7c8ce0274c11" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "async-signal" +version = "0.2.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "52b5aaafa020cf5053a01f2a60e8ff5dccf550f0f77ec54a4e47285ac2bab485" +dependencies = [ + "async-io", + "async-lock", + "atomic-waker", + "cfg-if", + "futures-core", + "futures-io", + "rustix", + "signal-hook-registry", + "slab", + "windows-sys 0.61.2", +] + +[[package]] +name = "async-task" +version = "4.7.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8b75356056920673b02621b35afd0f7dda9306d03c79a30f5c56c44cf256e3de" + +[[package]] +name = "async-trait" +version = "0.1.89" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9035ad2d096bed7955a320ee7e2230574d28fd3c3a0f186cbea1ff3c7eed5dbb" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "atomic-waker" +version = "1.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1505bd5d3d116872e7271a6d4e16d81d0c8570876c8de68093a09ac269d8aac0" + +[[package]] +name = "autocfg" +version = "1.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53" + +[[package]] +name = "axum" +version = "0.8.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "31b698c5f9a010f6573133b09e0de5408834d0c82f8d7475a89fc1867a71cd90" +dependencies = [ + "axum-core", + "axum-macros", + "bytes", + "form_urlencoded", + "futures-util", + "http", + "http-body", + "http-body-util", + "hyper", + "hyper-util", + "itoa", + "matchit", + "memchr", + "mime", + "percent-encoding", + "pin-project-lite", + "serde_core", + "serde_json", + "serde_path_to_error", + "serde_urlencoded", + "sync_wrapper", + "tokio", + "tower", + "tower-layer", + "tower-service", + "tracing", +] + +[[package]] +name = "axum-core" +version = "0.5.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "08c78f31d7b1291f7ee735c1c6780ccde7785daae9a9206026862dab7d8792d1" +dependencies = [ + "bytes", + "futures-core", + "http", + "http-body", + "http-body-util", + "mime", + "pin-project-lite", + "sync_wrapper", + "tower-layer", + "tower-service", + "tracing", +] + +[[package]] +name = "axum-macros" +version = "0.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7aa268c23bfbbd2c4363b9cd302a4f504fb2a9dfe7e3451d66f35dd392e20aca" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "bitflags" +version = "2.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "84d7ced0ae9557296835c32bf1b1e02b44c746701f898460fb000d7eaa84f00a" + +[[package]] +name = "block-buffer" +version = "0.10.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3078c7629b62d3f0439517fa394996acacc5cbc91c5a20d8c658e77abd503a71" +dependencies = [ + "generic-array", +] + +[[package]] +name = "block-padding" +version = "0.3.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a8894febbff9f758034a5b8e12d87918f56dfc64a8e1fe757d65e29041538d93" +dependencies = [ + "generic-array", +] + +[[package]] +name = "blocking" +version = "1.6.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e83f8d02be6967315521be875afa792a316e28d57b5a2d401897e2a7921b7f21" +dependencies = [ + "async-channel", + "async-task", + "futures-io", + "futures-lite", + "piper", +] + +[[package]] +name = "bumpalo" +version = "3.20.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "72f5acc6cb2ba439de613abc23857ec3d78374d8ed5ac84e9d11336e87da8649" + +[[package]] +name = "byteorder" +version = "1.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1fd0f2584146f6f2ef48085050886acf353beff7305ebd1ae69500e27c67f64b" + +[[package]] +name = "bytes" +version = "1.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1e748733b7cbc798e1434b6ac524f0c1ff2ab456fe201501e6497c8417a4fc33" + +[[package]] +name = "cbc" +version = "0.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "26b52a9543ae338f279b96b0b9fed9c8093744685043739079ce85cd58f289a6" +dependencies = [ + "cipher", +] + +[[package]] +name = "cc" +version = "1.2.63" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "556e016178bb5662a08681bbe0f00f8e17631781a4dfc8c45e466e4b185ec27f" +dependencies = [ + "find-msvc-tools", + "shlex", +] + +[[package]] +name = "cfg-if" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801" + +[[package]] +name = "cfg_aliases" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "613afe47fcd5fac7ccf1db93babcb082c5994d996f20b8b159f2ad1658eb5724" + +[[package]] +name = "cipher" +version = "0.4.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "773f3b9af64447d2ce9850330c473515014aa235e6a783b02db81ff39e4a3dad" +dependencies = [ + "crypto-common", + "inout", +] + +[[package]] +name = "clap" +version = "4.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1ddb117e43bbf7dacf0a4190fef4d345b9bad68dfc649cb349e7d17d28428e51" +dependencies = [ + "clap_builder", + "clap_derive", +] + +[[package]] +name = "clap_builder" +version = "4.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "714a53001bf66416adb0e2ef5ac857140e7dc3a0c48fb28b2f10762fc4b5069f" +dependencies = [ + "anstream", + "anstyle", + "clap_lex", + "strsim", +] + +[[package]] +name = "clap_derive" +version = "4.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f2ce8604710f6733aa641a2b3731eaa1e8b3d9973d5e3565da11800813f997a9" +dependencies = [ + "heck", + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "clap_lex" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c8d4a3bb8b1e0c1050499d1815f5ab16d04f0959b233085fb31653fbfc9d98f9" + +[[package]] +name = "colorchoice" +version = "1.0.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570" + +[[package]] +name = "concurrent-queue" +version = "2.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4ca0197aee26d1ae37445ee532fefce43251d24cc7c166799f4d46817f1d3973" +dependencies = [ + "crossbeam-utils", +] + +[[package]] +name = "core-foundation" +version = "0.9.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "91e195e091a93c46f7102ec7818a2aa394e1e1771c3ab4825963fa03e45afb8f" +dependencies = [ + "core-foundation-sys", + "libc", +] + +[[package]] +name = "core-foundation" +version = "0.10.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b2a6cd9ae233e7f62ba4e9353e81a88df7fc8a5987b8d445b4d90c879bd156f6" +dependencies = [ + "core-foundation-sys", + "libc", +] + +[[package]] +name = "core-foundation-sys" +version = "0.8.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "773648b94d0e5d620f64f280777445740e61fe701025087ec8b57f45c791888b" + +[[package]] +name = "cpufeatures" +version = "0.2.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "59ed5838eebb26a2bb2e58f6d5b5316989ae9d08bab10e0e6d103e656d1b0280" +dependencies = [ + "libc", +] + +[[package]] +name = "crossbeam-utils" +version = "0.8.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d0a5c400df2834b80a4c3327b3aad3a4c4cd4de0629063962b03235697506a28" + +[[package]] +name = "crypto-common" +version = "0.1.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "78c8292055d1c1df0cce5d180393dc8cce0abec0a7102adb6c7b1eef6016d60a" +dependencies = [ + "generic-array", + "typenum", +] + +[[package]] +name = "data-encoding" +version = "2.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a4ae5f15dda3c708c0ade84bfee31ccab44a3da4f88015ed22f63732abe300c8" + +[[package]] +name = "dbus" +version = "0.9.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b942602992bb7acfd1f51c49811c58a610ef9181b6e66f3e519d79b540a3bf73" +dependencies = [ + "libc", + "libdbus-sys", + "windows-sys 0.61.2", +] + +[[package]] +name = "dbus-secret-service" +version = "4.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "708b509edf7889e53d7efb0ffadd994cc6c2345ccb62f55cfd6b0682165e4fa6" +dependencies = [ + "aes", + "block-padding", + "cbc", + "dbus", + "fastrand", + "hkdf", + "num", + "once_cell", + "sha2", + "zeroize", +] + +[[package]] +name = "digest" +version = "0.10.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292" +dependencies = [ + "block-buffer", + "crypto-common", + "subtle", +] + +[[package]] +name = "endi" +version = "1.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "66b7e2430c6dff6a955451e2cfc438f09cea1965a9d6f87f7e3b90decc014099" + +[[package]] +name = "enumflags2" +version = "0.7.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1027f7680c853e056ebcec683615fb6fbbc07dbaa13b4d5d9442b146ded4ecef" +dependencies = [ + "enumflags2_derive", + "serde", +] + +[[package]] +name = "enumflags2_derive" +version = "0.7.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "67c78a4d8fdf9953a5c9d458f9efe940fd97a0cab0941c075a813ac594733827" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "equivalent" +version = "1.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f" + +[[package]] +name = "errno" +version = "0.3.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb" +dependencies = [ + "libc", + "windows-sys 0.61.2", +] + +[[package]] +name = "event-listener" +version = "5.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e13b66accf52311f30a0db42147dadea9850cb48cd070028831ae5f5d4b856ab" +dependencies = [ + "concurrent-queue", + "parking", + "pin-project-lite", +] + +[[package]] +name = "event-listener-strategy" +version = "0.5.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8be9f3dfaaffdae2972880079a491a1a8bb7cbed0b8dd7a347f668b4150a3b93" +dependencies = [ + "event-listener", + "pin-project-lite", +] + +[[package]] +name = "fastrand" +version = "2.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9f1f227452a390804cdb637b74a86990f2a7d7ba4b7d5693aac9b4dd6defd8d6" + +[[package]] +name = "find-msvc-tools" +version = "0.1.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5baebc0774151f905a1a2cc41989300b1e6fbb29aff0ceffa1064fdd3088d582" + +[[package]] +name = "foldhash" +version = "0.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d9c4f5dac5e15c24eb999c26181a6ca40b39fe946cbe4c263c7209467bc83af2" + +[[package]] +name = "foreign-types" +version = "0.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f6f339eb8adc052cd2ca78910fda869aefa38d22d5cb648e6485e4d3fc06f3b1" +dependencies = [ + "foreign-types-shared", +] + +[[package]] +name = "foreign-types-shared" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "00b0228411908ca8685dba7fc2cdd70ec9990a6e753e89b6ac91a84c40fbaf4b" + +[[package]] +name = "form_urlencoded" +version = "1.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cb4cb245038516f5f85277875cdaa4f7d2c9a0fa0468de06ed190163b1581fcf" +dependencies = [ + "percent-encoding", +] + +[[package]] +name = "futures-channel" +version = "0.3.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "07bbe89c50d7a535e539b8c17bc0b49bdb77747034daa8087407d655f3f7cc1d" +dependencies = [ + "futures-core", +] + +[[package]] +name = "futures-core" +version = "0.3.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7e3450815272ef58cec6d564423f6e755e25379b217b0bc688e295ba24df6b1d" + +[[package]] +name = "futures-io" +version = "0.3.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cecba35d7ad927e23624b22ad55235f2239cfa44fd10428eecbeba6d6a717718" + +[[package]] +name = "futures-lite" +version = "2.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f78e10609fe0e0b3f4157ffab1876319b5b0db102a2c60dc4626306dc46b44ad" +dependencies = [ + "fastrand", + "futures-core", + "futures-io", + "parking", + "pin-project-lite", +] + +[[package]] +name = "futures-macro" +version = "0.3.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e835b70203e41293343137df5c0664546da5745f82ec9b84d40be8336958447b" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "futures-sink" +version = "0.3.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c39754e157331b013978ec91992bde1ac089843443c49cbc7f46150b0fad0893" + +[[package]] +name = "futures-task" +version = "0.3.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "037711b3d59c33004d3856fbdc83b99d4ff37a24768fa1be9ce3538a1cde4393" + +[[package]] +name = "futures-util" +version = "0.3.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "389ca41296e6190b48053de0321d02a77f32f8a5d2461dd38762c0593805c6d6" +dependencies = [ + "futures-core", + "futures-io", + "futures-macro", + "futures-sink", + "futures-task", + "memchr", + "pin-project-lite", + "slab", +] + +[[package]] +name = "generic-array" +version = "0.14.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "85649ca51fd72272d7821adaf274ad91c288277713d9c18820d8499a7ff69e9a" +dependencies = [ + "typenum", + "version_check", +] + +[[package]] +name = "getrandom" +version = "0.2.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ff2abc00be7fca6ebc474524697ae276ad847ad0a6b3faa4bcb027e9a4614ad0" +dependencies = [ + "cfg-if", + "libc", + "wasi", +] + +[[package]] +name = "getrandom" +version = "0.4.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0de51e6874e94e7bf76d726fc5d13ba782deca734ff60d5bb2fb2607c7406555" +dependencies = [ + "cfg-if", + "libc", + "r-efi", + "wasip2", + "wasip3", +] + +[[package]] +name = "hashbrown" +version = "0.15.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9229cfe53dfd69f0609a49f65461bd93001ea1ef889cd5529dd176593f5338a1" +dependencies = [ + "foldhash", +] + +[[package]] +name = "hashbrown" +version = "0.17.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a" + +[[package]] +name = "heck" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea" + +[[package]] +name = "hermit-abi" +version = "0.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fc0fef456e4baa96da950455cd02c081ca953b141298e41db3fc7e36b1da849c" + +[[package]] +name = "hex" +version = "0.4.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7f24254aa9a54b5c858eaee2f5bccdb46aaf0e486a595ed5fd8f86ba55232a70" + +[[package]] +name = "hkdf" +version = "0.12.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7b5f8eb2ad728638ea2c7d47a21db23b7b58a72ed6a38256b8a1849f15fbbdf7" +dependencies = [ + "hmac", +] + +[[package]] +name = "hmac" +version = "0.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6c49c37c09c17a53d937dfbb742eb3a961d65a994e6bcdcf37e7399d0cc8ab5e" +dependencies = [ + "digest", +] + +[[package]] +name = "http" +version = "1.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8be7462df143984c4598a256ef469b251d7d7f9e271135073e78fc535414f3d0" +dependencies = [ + "bytes", + "itoa", +] + +[[package]] +name = "http-body" +version = "1.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1efedce1fb8e6913f23e0c92de8e62cd5b772a67e7b3946df930a62566c93184" +dependencies = [ + "bytes", + "http", +] + +[[package]] +name = "http-body-util" +version = "0.1.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b021d93e26becf5dc7e1b75b1bed1fd93124b374ceb73f43d4d4eafec896a64a" +dependencies = [ + "bytes", + "futures-core", + "http", + "http-body", + "pin-project-lite", +] + +[[package]] +name = "httparse" +version = "1.10.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6dbf3de79e51f3d586ab4cb9d5c3e2c14aa28ed23d180cf89b4df0454a69cc87" + +[[package]] +name = "httpdate" +version = "1.0.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "df3b46402a9d5adb4c86a0cf463f42e19994e3ee891101b1841f30a545cb49a9" + +[[package]] +name = "hyper" +version = "1.10.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "55281c53a1894c864990125767da440a4e630446785086f52523b20033b74498" +dependencies = [ + "atomic-waker", + "bytes", + "futures-channel", + "futures-core", + "http", + "http-body", + "httparse", + "httpdate", + "itoa", + "pin-project-lite", + "smallvec", + "tokio", +] + +[[package]] +name = "hyper-util" +version = "0.1.20" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "96547c2556ec9d12fb1578c4eaf448b04993e7fb79cbaad930a656880a6bdfa0" +dependencies = [ + "bytes", + "http", + "http-body", + "hyper", + "pin-project-lite", + "tokio", + "tower-service", +] + +[[package]] +name = "id-arena" +version = "2.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3d3067d79b975e8844ca9eb072e16b31c3c1c36928edf9c6789548c524d0d954" + +[[package]] +name = "indexmap" +version = "2.14.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d466e9454f08e4a911e14806c24e16fba1b4c121d1ea474396f396069cf949d9" +dependencies = [ + "equivalent", + "hashbrown 0.17.1", + "serde", + "serde_core", +] + +[[package]] +name = "inout" +version = "0.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "879f10e63c20629ecabbb64a8010319738c66a5cd0c29b02d63d272b03751d01" +dependencies = [ + "block-padding", + "generic-array", +] + +[[package]] +name = "is_terminal_polyfill" +version = "1.70.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a6cb138bb79a146c1bd460005623e142ef0181e3d0219cb493e02f7d08a35695" + +[[package]] +name = "itoa" +version = "1.0.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682" + +[[package]] +name = "js-sys" +version = "0.3.99" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "142bc4740e452c1e57ade0cbc129f139c9093e354346f0872ef985f4f5cf5f11" +dependencies = [ + "cfg-if", + "futures-util", + "once_cell", + "wasm-bindgen", +] + +[[package]] +name = "keyring" +version = "3.6.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "eebcc3aff044e5944a8fbaf69eb277d11986064cba30c468730e8b9909fb551c" +dependencies = [ + "byteorder", + "dbus-secret-service", + "linux-keyutils", + "log", + "secret-service", + "security-framework 2.11.1", + "security-framework 3.7.0", + "windows-sys 0.60.2", + "zbus", + "zeroize", +] + +[[package]] +name = "lazy_static" +version = "1.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bbd2bcb4c963f2ddae06a2efc7e9f3591312473c50c6685e1f298068316e66fe" + +[[package]] +name = "leb128fmt" +version = "0.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "09edd9e8b54e49e587e4f6295a7d29c3ea94d469cb40ab8ca70b288248a81db2" + +[[package]] +name = "libc" +version = "0.2.186" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66" + +[[package]] +name = "libdbus-sys" +version = "0.2.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "328c4789d42200f1eeec05bd86c9c13c7f091d2ba9a6ea35acdf51f31bc0f043" +dependencies = [ + "pkg-config", +] + +[[package]] +name = "linux-keyutils" +version = "0.2.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "83270a18e9f90d0707c41e9f35efada77b64c0e6f3f1810e71c8368a864d5590" +dependencies = [ + "bitflags", + "libc", +] + +[[package]] +name = "linux-raw-sys" +version = "0.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53" + +[[package]] +name = "log" +version = "0.4.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "953f07c43838f8e6f9758cab68bf5bed85465e7587ebe0b823f1bcd81978ad3a" + +[[package]] +name = "matchers" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d1525a2a28c7f4fa0fc98bb91ae755d1e2d1505079e05539e35bc876b5d65ae9" +dependencies = [ + "regex-automata", +] + +[[package]] +name = "matchit" +version = "0.8.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "47e1ffaa40ddd1f3ed91f717a33c8c0ee23fff369e3aa8772b9605cc1d22f4c3" + +[[package]] +name = "memchr" +version = "2.8.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6b947ae49db0d222b1dbc6b113ce7248a3fc3a6ca21b696717bfc000ba4484d8" + +[[package]] +name = "memoffset" +version = "0.9.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "488016bfae457b036d996092f6cb448677611ce4449e970ceaf42695203f218a" +dependencies = [ + "autocfg", +] + +[[package]] +name = "mime" +version = "0.3.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6877bb514081ee2a7ff5ef9de3281f14a4dd4bceac4c09388074a6b5df8a139a" + +[[package]] +name = "mio" +version = "1.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "02bd0af71c67b473010cbbc60715ee815645a4dc942899111f494b4b737d6fda" +dependencies = [ + "libc", + "wasi", + "windows-sys 0.61.2", +] + +[[package]] +name = "native-tls" +version = "0.2.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "465500e14ea162429d264d44189adc38b199b62b1c21eea9f69e4b73cb03bbf2" +dependencies = [ + "libc", + "log", + "openssl", + "openssl-probe", + "openssl-sys", + "schannel", + "security-framework 3.7.0", + "security-framework-sys", + "tempfile", +] + +[[package]] +name = "nix" +version = "0.29.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "71e2746dc3a24dd78b3cfcb7be93368c6de9963d30f43a6a73998a9cf4b17b46" +dependencies = [ + "bitflags", + "cfg-if", + "cfg_aliases", + "libc", + "memoffset", +] + +[[package]] +name = "nu-ansi-term" +version = "0.50.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7957b9740744892f114936ab4a57b3f487491bbeafaf8083688b16841a4240e5" +dependencies = [ + "windows-sys 0.61.2", +] + +[[package]] +name = "num" +version = "0.4.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "35bd024e8b2ff75562e5f34e7f4905839deb4b22955ef5e73d2fea1b9813cb23" +dependencies = [ + "num-bigint", + "num-complex", + "num-integer", + "num-iter", + "num-rational", + "num-traits", +] + +[[package]] +name = "num-bigint" +version = "0.4.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a5e44f723f1133c9deac646763579fdb3ac745e418f2a7af9cd0c431da1f20b9" +dependencies = [ + "num-integer", + "num-traits", +] + +[[package]] +name = "num-complex" +version = "0.4.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "73f88a1307638156682bada9d7604135552957b7818057dcef22705b4d509495" +dependencies = [ + "num-traits", +] + +[[package]] +name = "num-integer" +version = "0.1.46" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7969661fd2958a5cb096e56c8e1ad0444ac2bbcd0061bd28660485a44879858f" +dependencies = [ + "num-traits", +] + +[[package]] +name = "num-iter" +version = "0.1.45" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1429034a0490724d0075ebb2bc9e875d6503c3cf69e235a8941aa757d83ef5bf" +dependencies = [ + "autocfg", + "num-integer", + "num-traits", +] + +[[package]] +name = "num-rational" +version = "0.4.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f83d14da390562dca69fc84082e73e548e1ad308d24accdedd2720017cb37824" +dependencies = [ + "num-bigint", + "num-integer", + "num-traits", +] + +[[package]] +name = "num-traits" +version = "0.2.19" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "071dfc062690e90b734c0b2273ce72ad0ffa95f0c74596bc250dcfd960262841" +dependencies = [ + "autocfg", +] + +[[package]] +name = "once_cell" +version = "1.21.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50" + +[[package]] +name = "once_cell_polyfill" +version = "1.70.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "384b8ab6d37215f3c5301a95a4accb5d64aa607f1fcb26a11b5303878451b4fe" + +[[package]] +name = "openssl" +version = "0.10.80" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a45fa2aa886c42762255da344f0a0d313e254066c46aad76f300c3d3da62d967" +dependencies = [ + "bitflags", + "cfg-if", + "foreign-types", + "libc", + "openssl-macros", + "openssl-sys", +] + +[[package]] +name = "openssl-macros" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a948666b637a0f465e8564c73e89d4dde00d72d4d473cc972f390fc3dcee7d9c" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "openssl-probe" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7c87def4c32ab89d880effc9e097653c8da5d6ef28e6b539d313baaacfbafcbe" + +[[package]] +name = "openssl-sys" +version = "0.9.116" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f28a22dc7140cda5f096e5e7724a6962ca81a7f8bfd2979f9b18c11af56318c4" +dependencies = [ + "cc", + "libc", + "pkg-config", + "vcpkg", +] + +[[package]] +name = "ordered-stream" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9aa2b01e1d916879f73a53d01d1d6cee68adbb31d6d9177a8cfce093cced1d50" +dependencies = [ + "futures-core", + "pin-project-lite", +] + +[[package]] +name = "parking" +version = "2.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f38d5652c16fde515bb1ecef450ab0f6a219d619a7274976324d5e377f7dceba" + +[[package]] +name = "percent-encoding" +version = "2.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220" + +[[package]] +name = "pin-project-lite" +version = "0.2.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd" + +[[package]] +name = "piper" +version = "0.2.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c835479a4443ded371d6c535cbfd8d31ad92c5d23ae9770a61bc155e4992a3c1" +dependencies = [ + "atomic-waker", + "fastrand", + "futures-io", +] + +[[package]] +name = "pkg-config" +version = "0.3.33" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "19f132c84eca552bf34cab8ec81f1c1dcc229b811638f9d283dceabe58c5569e" + +[[package]] +name = "polling" +version = "3.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5d0e4f59085d47d8241c88ead0f274e8a0cb551f3625263c05eb8dd897c34218" +dependencies = [ + "cfg-if", + "concurrent-queue", + "hermit-abi", + "pin-project-lite", + "rustix", + "windows-sys 0.61.2", +] + +[[package]] +name = "ppv-lite86" +version = "0.2.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "85eae3c4ed2f50dcfe72643da4befc30deadb458a9b590d720cde2f2b1e97da9" +dependencies = [ + "zerocopy", +] + +[[package]] +name = "prettyplease" +version = "0.2.37" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "479ca8adacdd7ce8f1fb39ce9ecccbfe93a3f1344b3d0d97f20bc0196208f62b" +dependencies = [ + "proc-macro2", + "syn", +] + +[[package]] +name = "proc-macro-crate" +version = "3.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e67ba7e9b2b56446f1d419b1d807906278ffa1a658a8a5d8a39dcb1f5a78614f" +dependencies = [ + "toml_edit", +] + +[[package]] +name = "proc-macro2" +version = "1.0.106" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8fd00f0bb2e90d81d1044c2b32617f68fcb9fa3bb7640c23e9c748e53fb30934" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "quote" +version = "1.0.45" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "41f2619966050689382d2b44f664f4bc593e129785a36d6ee376ddf37259b924" +dependencies = [ + "proc-macro2", +] + +[[package]] +name = "r-efi" +version = "6.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f8dcc9c7d52a811697d2151c701e0d08956f92b0e24136cf4cf27b57a6a0d9bf" + +[[package]] +name = "rand" +version = "0.8.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5ca0ecfa931c29007047d1bc58e623ab12e5590e8c7cc53200d5202b69266d8a" +dependencies = [ + "libc", + "rand_chacha", + "rand_core", +] + +[[package]] +name = "rand_chacha" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e6c10a63a0fa32252be49d21e7709d4d4baf8d231c2dbce1eaa8141b9b127d88" +dependencies = [ + "ppv-lite86", + "rand_core", +] + +[[package]] +name = "rand_core" +version = "0.6.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ec0be4795e2f6a28069bec0b5ff3e2ac9bafc99e6a9a7dc3547996c5c816922c" +dependencies = [ + "getrandom 0.2.17", +] + +[[package]] +name = "regex-automata" +version = "0.4.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6e1dd4122fc1595e8162618945476892eefca7b88c52820e74af6262213cae8f" +dependencies = [ + "aho-corasick", + "memchr", + "regex-syntax", +] + +[[package]] +name = "regex-syntax" +version = "0.8.10" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dc897dd8d9e8bd1ed8cdad82b5966c3e0ecae09fb1907d58efaa013543185d0a" + +[[package]] +name = "rustix" +version = "1.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190" +dependencies = [ + "bitflags", + "errno", + "libc", + "linux-raw-sys", + "windows-sys 0.61.2", +] + +[[package]] +name = "rustversion" +version = "1.0.22" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b39cdef0fa800fc44525c84ccb54a029961a8215f9619753635a9c0d2538d46d" + +[[package]] +name = "ryu" +version = "1.0.23" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9774ba4a74de5f7b1c1451ed6cd5285a32eddb5cccb8cc655a4e50009e06477f" + +[[package]] +name = "schannel" +version = "0.1.29" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "91c1b7e4904c873ef0710c1f407dde2e6287de2bebc1bbbf7d430bb7cbffd939" +dependencies = [ + "windows-sys 0.61.2", +] + +[[package]] +name = "secret-service" +version = "4.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e4d35ad99a181be0a60ffcbe85d680d98f87bdc4d7644ade319b87076b9dbfd4" +dependencies = [ + "aes", + "cbc", + "futures-util", + "generic-array", + "hkdf", + "num", + "once_cell", + "rand", + "serde", + "sha2", + "zbus", +] + +[[package]] +name = "security-framework" +version = "2.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "897b2245f0b511c87893af39b033e5ca9cce68824c4d7e7630b5a1d339658d02" +dependencies = [ + "bitflags", + "core-foundation 0.9.4", + "core-foundation-sys", + "libc", + "security-framework-sys", +] + +[[package]] +name = "security-framework" +version = "3.7.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b7f4bc775c73d9a02cde8bf7b2ec4c9d12743edf609006c7facc23998404cd1d" +dependencies = [ + "bitflags", + "core-foundation 0.10.1", + "core-foundation-sys", + "libc", + "security-framework-sys", +] + +[[package]] +name = "security-framework-sys" +version = "2.17.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6ce2691df843ecc5d231c0b14ece2acc3efb62c0a398c7e1d875f3983ce020e3" +dependencies = [ + "core-foundation-sys", + "libc", +] + +[[package]] +name = "semver" +version = "1.0.28" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8a7852d02fc848982e0c167ef163aaff9cd91dc640ba85e263cb1ce46fae51cd" + +[[package]] +name = "serde" +version = "1.0.228" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9a8e94ea7f378bd32cbbd37198a4a91436180c5bb472411e48b5ec2e2124ae9e" +dependencies = [ + "serde_core", + "serde_derive", +] + +[[package]] +name = "serde_core" +version = "1.0.228" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "41d385c7d4ca58e59fc732af25c3983b67ac852c1a25000afe1175de458b67ad" +dependencies = [ + "serde_derive", +] + +[[package]] +name = "serde_derive" +version = "1.0.228" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d540f220d3187173da220f885ab66608367b6574e925011a9353e4badda91d79" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "serde_json" +version = "1.0.150" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e8014e44b4736ed0538adeecded0fce2a272f22dc9578a7eb6b2d9993c74cfb9" +dependencies = [ + "itoa", + "memchr", + "serde", + "serde_core", + "zmij", +] + +[[package]] +name = "serde_path_to_error" +version = "0.1.20" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "10a9ff822e371bb5403e391ecd83e182e0e77ba7f6fe0160b795797109d1b457" +dependencies = [ + "itoa", + "serde", + "serde_core", +] + +[[package]] +name = "serde_repr" +version = "0.1.20" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "175ee3e80ae9982737ca543e96133087cbd9a485eecc3bc4de9c1a37b47ea59c" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "serde_urlencoded" +version = "0.7.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d3491c14715ca2294c4d6a88f15e84739788c1d030eed8c110436aafdaa2f3fd" +dependencies = [ + "form_urlencoded", + "itoa", + "ryu", + "serde", +] + +[[package]] +name = "sha1" +version = "0.10.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e3bf829a2d51ab4a5ddf1352d8470c140cadc8301b2ae1789db023f01cedd6ba" +dependencies = [ + "cfg-if", + "cpufeatures", + "digest", +] + +[[package]] +name = "sha2" +version = "0.10.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a7507d819769d01a365ab707794a4084392c824f54a7a6a7862f8c3d0892b283" +dependencies = [ + "cfg-if", + "cpufeatures", + "digest", +] + +[[package]] +name = "sharded-slab" +version = "0.1.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f40ca3c46823713e0d4209592e8d6e826aa57e928f09752619fc696c499637f6" +dependencies = [ + "lazy_static", +] + +[[package]] +name = "shlex" +version = "2.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba" + +[[package]] +name = "signal-hook-registry" +version = "1.4.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c4db69cba1110affc0e9f7bcd48bbf87b3f4fc7c61fc9155afd4c469eb3d6c1b" +dependencies = [ + "errno", + "libc", +] + +[[package]] +name = "slab" +version = "0.4.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0c790de23124f9ab44544d7ac05d60440adc586479ce501c1d6d7da3cd8c9cf5" + +[[package]] +name = "smallvec" +version = "1.15.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "67b1b7a3b5fe4f1376887184045fcf45c69e92af734b7aaddc05fb777b6fbd03" + +[[package]] +name = "socket2" +version = "0.6.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "52d1cfed4120b4d927bf7c0f86d2087a4a7d6027c906d9f9d525a80573b9be51" +dependencies = [ + "libc", + "windows-sys 0.61.2", +] + +[[package]] +name = "static_assertions" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a2eb9349b6444b326872e140eb1cf5e7c522154d69e7a0ffb0fb81c06b37543f" + +[[package]] +name = "strsim" +version = "0.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7da8b5736845d9f2fcb837ea5d9e2628564b3b043a70948a3f0b778838c5fb4f" + +[[package]] +name = "subtle" +version = "2.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "13c2bddecc57b384dee18652358fb23172facb8a2c51ccc10d74c157bdea3292" + +[[package]] +name = "syn" +version = "2.0.117" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e665b8803e7b1d2a727f4023456bbbbe74da67099c585258af0ad9c5013b9b99" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "sync_wrapper" +version = "1.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0bf256ce5efdfa370213c1dabab5935a12e49f2c58d15e9eac2870d3b4f27263" + +[[package]] +name = "tempfile" +version = "3.27.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd" +dependencies = [ + "fastrand", + "getrandom 0.4.2", + "once_cell", + "rustix", + "windows-sys 0.61.2", +] + +[[package]] +name = "thiserror" +version = "1.0.69" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6aaf5339b578ea85b50e080feb250a3e8ae8cfcdff9a461c9ec2904bc923f52" +dependencies = [ + "thiserror-impl 1.0.69", +] + +[[package]] +name = "thiserror" +version = "2.0.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4288b5bcbc7920c07a1149a35cf9590a2aa808e0bc1eafaade0b80947865fbc4" +dependencies = [ + "thiserror-impl 2.0.18", +] + +[[package]] +name = "thiserror-impl" +version = "1.0.69" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4fee6c4efc90059e10f81e6d42c60a18f76588c3d74cb83a0b242a2b6c7504c1" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "thiserror-impl" +version = "2.0.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ebc4ee7f67670e9b64d05fa4253e753e016c6c95ff35b89b7941d6b856dec1d5" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "thread_local" +version = "1.1.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f60246a4944f24f6e018aa17cdeffb7818b76356965d03b07d6a9886e8962185" +dependencies = [ + "cfg-if", +] + +[[package]] +name = "threadline" +version = "0.1.0" +dependencies = [ + "axum", + "clap", + "futures-util", + "keyring", + "serde", + "serde_json", + "sha2", + "tempfile", + "thiserror 2.0.18", + "tokio", + "tokio-tungstenite", + "tower", + "tracing", + "tracing-subscriber", + "uuid", +] + +[[package]] +name = "tokio" +version = "1.52.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8fc7f01b389ac15039e4dc9531aa973a135d7a4135281b12d7c1bc79fd57fffe" +dependencies = [ + "bytes", + "libc", + "mio", + "pin-project-lite", + "socket2", + "tokio-macros", + "windows-sys 0.61.2", +] + +[[package]] +name = "tokio-macros" +version = "2.7.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "385a6cb71ab9ab790c5fe8d67f1645e6c450a7ce006a33de03daa956cf70a496" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "tokio-native-tls" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bbae76ab933c85776efabc971569dd6119c580d8f5d448769dec1764bf796ef2" +dependencies = [ + "native-tls", + "tokio", +] + +[[package]] +name = "tokio-tungstenite" +version = "0.24.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "edc5f74e248dc973e0dbb7b74c7e0d6fcc301c694ff50049504004ef4d0cdcd9" +dependencies = [ + "futures-util", + "log", + "native-tls", + "tokio", + "tokio-native-tls", + "tungstenite", +] + +[[package]] +name = "toml_datetime" +version = "1.1.1+spec-1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3165f65f62e28e0115a00b2ebdd37eb6f3b641855f9d636d3cd4103767159ad7" +dependencies = [ + "serde_core", +] + +[[package]] +name = "toml_edit" +version = "0.25.12+spec-1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d2153edc6955a6c354fad8f5efd38b6a8769bdccf9fe50f8e1329f81b0baa5d7" +dependencies = [ + "indexmap", + "toml_datetime", + "toml_parser", + "winnow", +] + +[[package]] +name = "toml_parser" +version = "1.1.2+spec-1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a2abe9b86193656635d2411dc43050282ca48aa31c2451210f4202550afb7526" +dependencies = [ + "winnow", +] + +[[package]] +name = "tower" +version = "0.5.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ebe5ef63511595f1344e2d5cfa636d973292adc0eec1f0ad45fae9f0851ab1d4" +dependencies = [ + "futures-core", + "futures-util", + "pin-project-lite", + "sync_wrapper", + "tokio", + "tower-layer", + "tower-service", + "tracing", +] + +[[package]] +name = "tower-layer" +version = "0.3.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "121c2a6cda46980bb0fcd1647ffaf6cd3fc79a013de288782836f6df9c48780e" + +[[package]] +name = "tower-service" +version = "0.3.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8df9b6e13f2d32c91b9bd719c00d1958837bc7dec474d94952798cc8e69eeec3" + +[[package]] +name = "tracing" +version = "0.1.44" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "63e71662fa4b2a2c3a26f570f037eb95bb1f85397f3cd8076caed2f026a6d100" +dependencies = [ + "log", + "pin-project-lite", + "tracing-attributes", + "tracing-core", +] + +[[package]] +name = "tracing-attributes" +version = "0.1.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7490cfa5ec963746568740651ac6781f701c9c5ea257c58e057f3ba8cf69e8da" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "tracing-core" +version = "0.1.36" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "db97caf9d906fbde555dd62fa95ddba9eecfd14cb388e4f491a66d74cd5fb79a" +dependencies = [ + "once_cell", + "valuable", +] + +[[package]] +name = "tracing-log" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ee855f1f400bd0e5c02d150ae5de3840039a3f54b025156404e34c23c03f47c3" +dependencies = [ + "log", + "once_cell", + "tracing-core", +] + +[[package]] +name = "tracing-subscriber" +version = "0.3.23" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cb7f578e5945fb242538965c2d0b04418d38ec25c79d160cd279bf0731c8d319" +dependencies = [ + "matchers", + "nu-ansi-term", + "once_cell", + "regex-automata", + "sharded-slab", + "smallvec", + "thread_local", + "tracing", + "tracing-core", + "tracing-log", +] + +[[package]] +name = "tungstenite" +version = "0.24.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "18e5b8366ee7a95b16d32197d0b2604b43a0be89dc5fac9f8e96ccafbaedda8a" +dependencies = [ + "byteorder", + "bytes", + "data-encoding", + "http", + "httparse", + "log", + "native-tls", + "rand", + "sha1", + "thiserror 1.0.69", + "utf-8", +] + +[[package]] +name = "typenum" +version = "1.20.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6f5e870be6c3b371b77fe0ee0bafb859fa4964b4404c27de1d380043c4dda20" + +[[package]] +name = "uds_windows" +version = "1.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f2f6fb2847f6742cd76af783a2a2c49e9375d0a111c7bef6f71cd9e738c72d6e" +dependencies = [ + "memoffset", + "tempfile", + "windows-sys 0.61.2", +] + +[[package]] +name = "unicode-ident" +version = "1.0.24" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75" + +[[package]] +name = "unicode-xid" +version = "0.2.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ebc1c04c71510c7f702b52b7c350734c9ff1295c464a03335b00bb84fc54f853" + +[[package]] +name = "utf-8" +version = "0.7.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "09cc8ee72d2a9becf2f2febe0205bbed8fc6615b7cb429ad062dc7b7ddd036a9" + +[[package]] +name = "utf8parse" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "06abde3611657adf66d383f00b093d7faecc7fa57071cce2578660c9f1010821" + +[[package]] +name = "uuid" +version = "1.23.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d258b83ceec21034727ecee8c382cfa6c3e133699b0742c64571814fb420c9f7" +dependencies = [ + "getrandom 0.4.2", + "js-sys", + "serde_core", + "wasm-bindgen", +] + +[[package]] +name = "valuable" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ba73ea9cf16a25df0c8caa16c51acb937d5712a8429db78a3ee29d5dcacd3a65" + +[[package]] +name = "vcpkg" +version = "0.2.15" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "accd4ea62f7bb7a82fe23066fb0957d48ef677f6eeb8215f372f52e48bb32426" + +[[package]] +name = "version_check" +version = "0.9.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a" + +[[package]] +name = "wasi" +version = "0.11.1+wasi-snapshot-preview1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ccf3ec651a847eb01de73ccad15eb7d99f80485de043efb2f370cd654f4ea44b" + +[[package]] +name = "wasip2" +version = "1.0.3+wasi-0.2.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "20064672db26d7cdc89c7798c48a0fdfac8213434a1186e5ef29fd560ae223d6" +dependencies = [ + "wit-bindgen 0.57.1", +] + +[[package]] +name = "wasip3" +version = "0.4.0+wasi-0.3.0-rc-2026-01-06" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5428f8bf88ea5ddc08faddef2ac4a67e390b88186c703ce6dbd955e1c145aca5" +dependencies = [ + "wit-bindgen 0.51.0", +] + +[[package]] +name = "wasm-bindgen" +version = "0.2.122" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3ed04576f974d2b2fba0f38c51dbc5518011e38c36bf1143164be765528fd409" +dependencies = [ + "cfg-if", + "once_cell", + "rustversion", + "wasm-bindgen-macro", + "wasm-bindgen-shared", +] + +[[package]] +name = "wasm-bindgen-macro" +version = "0.2.122" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "916151b09da36bd82f6615cbf3a419e2f0ba23a03c6160e8e92eb6bd4aa1dec6" +dependencies = [ + "quote", + "wasm-bindgen-macro-support", +] + +[[package]] +name = "wasm-bindgen-macro-support" +version = "0.2.122" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "299047362ccbfce148b67ab7e73349f77748e00c8296f9542adfad2ad82c5c5e" +dependencies = [ + "bumpalo", + "proc-macro2", + "quote", + "syn", + "wasm-bindgen-shared", +] + +[[package]] +name = "wasm-bindgen-shared" +version = "0.2.122" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9a929b2c61f11ba3e9bc35b50c1f25cb38e0e892c0c231ae2b8cf78d5dad4437" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "wasm-encoder" +version = "0.244.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "990065f2fe63003fe337b932cfb5e3b80e0b4d0f5ff650e6985b1048f62c8319" +dependencies = [ + "leb128fmt", + "wasmparser", +] + +[[package]] +name = "wasm-metadata" +version = "0.244.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bb0e353e6a2fbdc176932bbaab493762eb1255a7900fe0fea1a2f96c296cc909" +dependencies = [ + "anyhow", + "indexmap", + "wasm-encoder", + "wasmparser", +] + +[[package]] +name = "wasmparser" +version = "0.244.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "47b807c72e1bac69382b3a6fb3dbe8ea4c0ed87ff5629b8685ae6b9a611028fe" +dependencies = [ + "bitflags", + "hashbrown 0.15.5", + "indexmap", + "semver", +] + +[[package]] +name = "windows-link" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5" + +[[package]] +name = "windows-sys" +version = "0.52.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "282be5f36a8ce781fad8c8ae18fa3f9beff57ec1b52cb3de0789201425d9a33d" +dependencies = [ + "windows-targets 0.52.6", +] + +[[package]] +name = "windows-sys" +version = "0.59.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1e38bc4d79ed67fd075bcc251a1c39b32a1776bbe92e5bef1f0bf1f8c531853b" +dependencies = [ + "windows-targets 0.52.6", +] + +[[package]] +name = "windows-sys" +version = "0.60.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f2f500e4d28234f72040990ec9d39e3a6b950f9f22d3dba18416c35882612bcb" +dependencies = [ + "windows-targets 0.53.5", +] + +[[package]] +name = "windows-sys" +version = "0.61.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc" +dependencies = [ + "windows-link", +] + +[[package]] +name = "windows-targets" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9b724f72796e036ab90c1021d4780d4d3d648aca59e491e6b98e725b84e99973" +dependencies = [ + "windows_aarch64_gnullvm 0.52.6", + "windows_aarch64_msvc 0.52.6", + "windows_i686_gnu 0.52.6", + "windows_i686_gnullvm 0.52.6", + "windows_i686_msvc 0.52.6", + "windows_x86_64_gnu 0.52.6", + "windows_x86_64_gnullvm 0.52.6", + "windows_x86_64_msvc 0.52.6", +] + +[[package]] +name = "windows-targets" +version = "0.53.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4945f9f551b88e0d65f3db0bc25c33b8acea4d9e41163edf90dcd0b19f9069f3" +dependencies = [ + "windows-link", + "windows_aarch64_gnullvm 0.53.1", + "windows_aarch64_msvc 0.53.1", + "windows_i686_gnu 0.53.1", + "windows_i686_gnullvm 0.53.1", + "windows_i686_msvc 0.53.1", + "windows_x86_64_gnu 0.53.1", + "windows_x86_64_gnullvm 0.53.1", + "windows_x86_64_msvc 0.53.1", +] + +[[package]] +name = "windows_aarch64_gnullvm" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "32a4622180e7a0ec044bb555404c800bc9fd9ec262ec147edd5989ccd0c02cd3" + +[[package]] +name = "windows_aarch64_gnullvm" +version = "0.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a9d8416fa8b42f5c947f8482c43e7d89e73a173cead56d044f6a56104a6d1b53" + +[[package]] +name = "windows_aarch64_msvc" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "09ec2a7bb152e2252b53fa7803150007879548bc709c039df7627cabbd05d469" + +[[package]] +name = "windows_aarch64_msvc" +version = "0.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b9d782e804c2f632e395708e99a94275910eb9100b2114651e04744e9b125006" + +[[package]] +name = "windows_i686_gnu" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8e9b5ad5ab802e97eb8e295ac6720e509ee4c243f69d781394014ebfe8bbfa0b" + +[[package]] +name = "windows_i686_gnu" +version = "0.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "960e6da069d81e09becb0ca57a65220ddff016ff2d6af6a223cf372a506593a3" + +[[package]] +name = "windows_i686_gnullvm" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0eee52d38c090b3caa76c563b86c3a4bd71ef1a819287c19d586d7334ae8ed66" + +[[package]] +name = "windows_i686_gnullvm" +version = "0.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fa7359d10048f68ab8b09fa71c3daccfb0e9b559aed648a8f95469c27057180c" + +[[package]] +name = "windows_i686_msvc" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "240948bc05c5e7c6dabba28bf89d89ffce3e303022809e73deaefe4f6ec56c66" + +[[package]] +name = "windows_i686_msvc" +version = "0.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1e7ac75179f18232fe9c285163565a57ef8d3c89254a30685b57d83a38d326c2" + +[[package]] +name = "windows_x86_64_gnu" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "147a5c80aabfbf0c7d901cb5895d1de30ef2907eb21fbbab29ca94c5b08b1a78" + +[[package]] +name = "windows_x86_64_gnu" +version = "0.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9c3842cdd74a865a8066ab39c8a7a473c0778a3f29370b5fd6b4b9aa7df4a499" + +[[package]] +name = "windows_x86_64_gnullvm" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "24d5b23dc417412679681396f2b49f3de8c1473deb516bd34410872eff51ed0d" + +[[package]] +name = "windows_x86_64_gnullvm" +version = "0.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0ffa179e2d07eee8ad8f57493436566c7cc30ac536a3379fdf008f47f6bb7ae1" + +[[package]] +name = "windows_x86_64_msvc" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "589f6da84c646204747d1270a2a5661ea66ed1cced2631d546fdfb155959f9ec" + +[[package]] +name = "windows_x86_64_msvc" +version = "0.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d6bbff5f0aada427a1e5a6da5f1f98158182f26556f345ac9e04d36d0ebed650" + +[[package]] +name = "winnow" +version = "1.0.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0592e1c9d151f854e6fd382574c3a0855250e1d9b2f99d9281c6e6391af352f1" +dependencies = [ + "memchr", +] + +[[package]] +name = "wit-bindgen" +version = "0.51.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d7249219f66ced02969388cf2bb044a09756a083d0fab1e566056b04d9fbcaa5" +dependencies = [ + "wit-bindgen-rust-macro", +] + +[[package]] +name = "wit-bindgen" +version = "0.57.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1ebf944e87a7c253233ad6766e082e3cd714b5d03812acc24c318f549614536e" + +[[package]] +name = "wit-bindgen-core" +version = "0.51.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ea61de684c3ea68cb082b7a88508a8b27fcc8b797d738bfc99a82facf1d752dc" +dependencies = [ + "anyhow", + "heck", + "wit-parser", +] + +[[package]] +name = "wit-bindgen-rust" +version = "0.51.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b7c566e0f4b284dd6561c786d9cb0142da491f46a9fbed79ea69cdad5db17f21" +dependencies = [ + "anyhow", + "heck", + "indexmap", + "prettyplease", + "syn", + "wasm-metadata", + "wit-bindgen-core", + "wit-component", +] + +[[package]] +name = "wit-bindgen-rust-macro" +version = "0.51.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0c0f9bfd77e6a48eccf51359e3ae77140a7f50b1e2ebfe62422d8afdaffab17a" +dependencies = [ + "anyhow", + "prettyplease", + "proc-macro2", + "quote", + "syn", + "wit-bindgen-core", + "wit-bindgen-rust", +] + +[[package]] +name = "wit-component" +version = "0.244.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9d66ea20e9553b30172b5e831994e35fbde2d165325bec84fc43dbf6f4eb9cb2" +dependencies = [ + "anyhow", + "bitflags", + "indexmap", + "log", + "serde", + "serde_derive", + "serde_json", + "wasm-encoder", + "wasm-metadata", + "wasmparser", + "wit-parser", +] + +[[package]] +name = "wit-parser" +version = "0.244.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ecc8ac4bc1dc3381b7f59c34f00b67e18f910c2c0f50015669dde7def656a736" +dependencies = [ + "anyhow", + "id-arena", + "indexmap", + "log", + "semver", + "serde", + "serde_derive", + "serde_json", + "unicode-xid", + "wasmparser", +] + +[[package]] +name = "xdg-home" +version = "1.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ec1cdab258fb55c0da61328dc52c8764709b249011b2cad0454c72f0bf10a1f6" +dependencies = [ + "libc", + "windows-sys 0.59.0", +] + +[[package]] +name = "zbus" +version = "4.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bb97012beadd29e654708a0fdb4c84bc046f537aecfde2c3ee0a9e4b4d48c725" +dependencies = [ + "async-broadcast", + "async-executor", + "async-fs", + "async-io", + "async-lock", + "async-process", + "async-recursion", + "async-task", + "async-trait", + "blocking", + "enumflags2", + "event-listener", + "futures-core", + "futures-sink", + "futures-util", + "hex", + "nix", + "ordered-stream", + "rand", + "serde", + "serde_repr", + "sha1", + "static_assertions", + "tracing", + "uds_windows", + "windows-sys 0.52.0", + "xdg-home", + "zbus_macros", + "zbus_names", + "zvariant", +] + +[[package]] +name = "zbus_macros" +version = "4.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "267db9407081e90bbfa46d841d3cbc60f59c0351838c4bc65199ecd79ab1983e" +dependencies = [ + "proc-macro-crate", + "proc-macro2", + "quote", + "syn", + "zvariant_utils", +] + +[[package]] +name = "zbus_names" +version = "3.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4b9b1fef7d021261cc16cba64c351d291b715febe0fa10dc3a443ac5a5022e6c" +dependencies = [ + "serde", + "static_assertions", + "zvariant", +] + +[[package]] +name = "zerocopy" +version = "0.8.50" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3b065d4f0e55f82fae73202e189638116a87c55ab6b8e6c2721e13dd9d854ad1" +dependencies = [ + "zerocopy-derive", +] + +[[package]] +name = "zerocopy-derive" +version = "0.8.50" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0b631b19d36a892ab55420c92dbc83ccd79274f25be714855d3074aa71cab639" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "zeroize" +version = "1.8.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b97154e67e32c85465826e8bcc1c59429aaaf107c1e4a9e53c8d8ccd5eff88d0" +dependencies = [ + "zeroize_derive", +] + +[[package]] +name = "zeroize_derive" +version = "1.4.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "85a5b4158499876c763cb03bc4e49185d3cccbabb15b33c627f7884f43db852e" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "zmij" +version = "1.0.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b8848ee67ecc8aedbaf3e4122217aff892639231befc6a1b58d29fff4c2cabaa" + +[[package]] +name = "zvariant" +version = "4.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2084290ab9a1c471c38fc524945837734fbf124487e105daec2bb57fd48c81fe" +dependencies = [ + "endi", + "enumflags2", + "serde", + "static_assertions", + "zvariant_derive", +] + +[[package]] +name = "zvariant_derive" +version = "4.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "73e2ba546bda683a90652bac4a279bc146adad1386f25379cf73200d2002c449" +dependencies = [ + "proc-macro-crate", + "proc-macro2", + "quote", + "syn", + "zvariant_utils", +] + +[[package]] +name = "zvariant_utils" +version = "2.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c51bcff7cc3dbb5055396bcf774748c3dab426b4b8659046963523cee4808340" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] diff --git a/Cargo.toml b/Cargo.toml new file mode 100644 index 0000000..83da8d0 --- /dev/null +++ b/Cargo.toml @@ -0,0 +1,35 @@ +[package] +name = "threadline" +version = "0.1.0" +edition = "2024" + +[dependencies] +axum = { version = "0.8", features = ["macros"] } +clap = { version = "4.5", features = ["derive", "env"] } +futures-util = "0.3" +keyring = { version = "3.6", default-features = false, features = ["crypto-rust"] } +serde = { version = "1", features = ["derive"] } +serde_json = "1" +sha2 = "0.10" +thiserror = "2" +tokio = { version = "1", features = ["io-util", "macros", "net", "rt-multi-thread", "sync", "time"] } +tokio-tungstenite = { version = "0.24", features = ["connect", "native-tls"] } +tracing = "0.1" +tracing-subscriber = { version = "0.3", features = ["env-filter", "fmt"] } +uuid = { version = "1", features = ["serde", "v4", "v7"] } + +[target.'cfg(target_os = "linux")'.dependencies] +keyring = { version = "3.6", default-features = false, features = ["linux-native-async-persistent"] } + +[target.'cfg(target_os = "macos")'.dependencies] +keyring = { version = "3.6", default-features = false, features = ["apple-native"] } + +[target.'cfg(target_os = "windows")'.dependencies] +keyring = { version = "3.6", default-features = false, features = ["windows-native"] } + +[target.'cfg(any(target_os = "freebsd", target_os = "openbsd"))'.dependencies] +keyring = { version = "3.6", default-features = false, features = ["sync-secret-service"] } + +[dev-dependencies] +tempfile = "3" +tower = { version = "0.5", features = ["util"] } \ No newline at end of file diff --git a/README.md b/README.md index afa1cee..edde6f4 100644 --- a/README.md +++ b/README.md @@ -1,2 +1,214 @@ # Threadline -A stateful Codex/WebSocket bridge for OpenAI-compatible clients. + +Threadline is a Rust service that bridges VSCode Copilot BYOK Responses API traffic to the Codex backend WebSocket protocol. + +Threadline is a BYOK `/v1/responses` bridge. It is not native VS Code Copilot and it does not have native editor, terminal, or extension-host tool integration. + +The current implementation exposes these HTTP endpoints: + +- `GET /health` +- `GET /v1/models` +- `POST /v1/responses` + +Threadline currently supports two route profiles: + +- Main: retained-session `/v1/responses` routing for primary assistant turns. +- Utility: stateless one-shot `/v1/responses` routing for utility turns. Utility does not retain upstream sessions, does not use `previous_response_id`, does not keep `context_management`, and does not execute Threadline internal tools or jobs. + +## Expected bridge UX + +When the `/v1/responses` bridge is implemented, the downstream experience will remain close to VSCode BYOK behavior, but it will not be identical to native Copilot UX. + +- Threadline forwards assistant output over `/v1/responses` and SSE, but native VS Code editor and terminal tool integration remains outside Threadline. +- Threadline-owned `threadline_*` internal tools are executed locally and hidden from downstream clients. +- Intermediate completions that exist only to carry internal tool work are consumed by Threadline and used for follow-up requests. Downstream clients should only see the final assistant-facing turn. +- Long-running work is represented as jobs. Job state and job output are read back through job APIs, and incremental output is read by offset rather than pushed as a native editor or terminal stream. + +## Observability and job output + +Threadline keeps internal tool execution and job state observable without exposing Threadline-only tool calls to downstream clients. + +- Job tools return stable identifiers and status so follow-up requests can poll, read buffered output, and fetch final results. +- Successful job starts return immediately and may include a short hint telling the caller to continue independent work before polling or reading output. +- Buffered job output is available before job completion, so partial output can be read incrementally instead of waiting for a single final payload. +- Output reads are append-oriented and offset-based. Repeated reads should pass the returned `next_offset` so later checks continue from the last consumed byte range. +- The output buffer is finite. When older bytes are dropped, `truncated_before` advances and any stored offset older than that value must be treated as no longer recoverable. +- UI rendering cadence still depends on how often the client or follow-up turn reads job output. Threadline improves partial output availability, but it does not promise native Copilot-identical live rendering cadence. +- Threadline exposes job tools and buffered output only. It does not provide native VS Code terminal, editor, or extension-host tool streaming. + +## Non-goals + +Threadline is not a general OpenAI-compatible proxy. + +It does not implement unrelated providers or historical compatibility layers. + +## Configuration + +Threadline reads configuration from CLI flags or environment variables. + +| Flag | Environment variable | Stable default | Description | +| --- | --- | --- | --- | +| `--host` | `THREADLINE_HOST` | `127.0.0.1` | Listen address for the downstream HTTP server that accepts local `/v1/responses` requests. | +| `--port` | `THREADLINE_PORT` | `8100` | Listen port for the downstream HTTP server. | +| `--utility-port` | `THREADLINE_UTILITY_PORT` | None | Optional port for a second utility listener in the same process. | +| `--profile` | `THREADLINE_PROFILE` | `main` | Route profile for this listener. Use `main` for retained-session routes and `utility` for stateless utility-only model aliases. | +| `--codex-client-version` | `THREADLINE_CODEX_CLIENT_VERSION` | `0.136.0` | Codex client version Threadline sends to the upstream backend for compatibility. | +| `--retained-session-capacity` | `THREADLINE_RETAINED_SESSION_CAPACITY` | `64` | Maximum number of retained sessions kept available for response continuation. | +| `--jobs-enabled` | `THREADLINE_JOBS_ENABLED` | `false` | Enables local job execution support for long-running work. | +| `--job-output-buffer-limit-bytes` | `THREADLINE_JOB_OUTPUT_BUFFER_LIMIT_BYTES` | `32768` | Maximum in-memory buffered job output before older output is dropped. | +| `--job-retention-ttl-secs` | `THREADLINE_JOB_RETENTION_TTL_SECS` | `300` | How long completed job metadata and buffered output remain available after completion. | +| `--job-allowed-commands` | `THREADLINE_JOB_ALLOWED_COMMANDS` | None | comma-separated exact program names allowed for jobs. Threadline compares `command[0]` against each configured entry exactly, without normalizing wrappers, paths, or aliases. | +| `--log-level` | `THREADLINE_LOG_LEVEL` | `info` | Threadline log verbosity. Supported Rust tracing levels include `error`, `warn`, `info`, `debug`, and `trace`. | + +Threadline does not accept an arbitrary model override through CLI flags or environment variables. + +## Main And Utility Startup + +The recommended startup path is one Threadline process with two listener ports: + +```bash +threadline --port 8100 --jobs-enabled --utility-port 8101 +``` + +This starts the default Main listener on port `8100` and a second stateless Utility listener in the same process on port `8101`. + +The Utility listener remains stateless because the Utility route profile always uses a fresh one-shot upstream connection and never registers or retains upstream session state. + +fallback/debug mode still supports two separate Threadline processes with profile-specific ports: + +```bash +threadline --port 8100 --jobs-enabled +threadline --port 8101 --profile utility +``` + +Use the two-process form when startup isolation or process-by-process debugging is more useful than the one-process convenience path. + +`--retained-session-capacity 0` is optional hardening for a Main listener that should avoid retained continuation state. It is not the mechanism that makes Utility stateless. Utility is stateless because the Utility route profile always uses a fresh one-shot upstream connection and never registers or retains upstream session state. + +`--utility-port` starts a second stateless Utility listener in the same process. Keep Main and Utility on separate endpoint base URLs, even in one-process mode, so clients still target `http://127.0.0.1:8100/v1` for Main and `http://127.0.0.1:8101/v1` for Utility. + +## Supported Model Aliases + +These are the visible model ids that Threadline advertises from `/v1/models`. + +Main profile aliases: + +- `threadline-main-gpt-5.6-sol` +- `threadline-main-gpt-5.6-terra` +- `threadline-main-gpt-5.6-luna` +- `threadline-main-gpt-5.5` +- `threadline-main-gpt-5.4` + +Utility profile aliases: + +- `threadline-utility-gpt-5.4-mini` +- `threadline-utility-gpt-5.3-codex-spark` + +The `gpt-5.6-sol`, `gpt-5.6-terra`, and `gpt-5.6-luna` entries are next models. Threadline currently covers local advertisement, validation, and `model`-field rewriting for those ids. Live upstream behavior remains unverified until upstream release makes direct testing possible. + +These visible ids are aliases for VS Code selection and routing. The upstream model ids sent to Codex remain `gpt-*` ids such as `gpt-5.6-sol`, `gpt-5.6-terra`, `gpt-5.6-luna`, `gpt-5.5`, `gpt-5.4`, `gpt-5.4-mini`, and `gpt-5.3-codex-spark`. + +For Main compatibility, Threadline still accepts direct `gpt-*` ids on the Main profile even though `/v1/models` advertises only the `threadline-main-*` aliases. + +Persistent CoT with `reasoning.context=all_turns` does not currently support the raw compatibility ids `gpt-5.6-sol`, `gpt-5.6-terra`, `gpt-5.6-luna`, `gpt-5.5`, and `gpt-5.4`; revisit that later rather than enabling it now. For now, keep `github.copilot.chat.responsesApi.persistentCoT.enabled=false` in VS Code; the current default is already `false`. When supported all-turn reasoning is needed, use the advertised Threadline aliases rather than the raw compatibility ids. + +## VS Code Custom Endpoint Setup + +Use distinct visible ids and distinct profile-specific URLs so VS Code can keep Main and Utility models separate under `customendpoint/{id}`. + +```json +{ + "chat.customEndpoints": [ + { + "uri": "http://127.0.0.1:8100/v1", + "models": [ + { + "id": "threadline-main-gpt-5.6-sol", + "name": "Threadline Main GPT-5.6 Sol" + }, + { + "id": "threadline-main-gpt-5.6-terra", + "name": "Threadline Main GPT-5.6 Terra" + }, + { + "id": "threadline-main-gpt-5.6-luna", + "name": "Threadline Main GPT-5.6 Luna" + }, + { + "id": "threadline-main-gpt-5.5", + "name": "Threadline Main GPT-5.5" + }, + { + "id": "threadline-main-gpt-5.4", + "name": "Threadline Main GPT-5.4" + } + ] + }, + { + "uri": "http://127.0.0.1:8101/v1", + "models": [ + { + "id": "threadline-utility-gpt-5.4-mini", + "name": "Threadline Utility GPT-5.4 Mini", + "supportsReasoningEffort": true + }, + { + "id": "threadline-utility-gpt-5.3-codex-spark", + "name": "Threadline Utility GPT-5.3 Codex Spark" + } + ] + } + ], + "chat.utilityModel": "customendpoint/threadline-utility-gpt-5.4-mini", + "chat.utilitySmallModel": "customendpoint/threadline-utility-gpt-5.4-mini" +} +``` + +The visible ids in this JSON are aliases only. VS Code uses `customendpoint/threadline-main-gpt-5.5` and `customendpoint/threadline-utility-gpt-5.4-mini` as local model selectors, while Threadline rewrites the upstream `model` field to the matching `gpt-*` id. + +Utility preserves `reasoning.effort` by default when the client sends it. The `supportsReasoningEffort` model setting only controls whether VS Code shows the effort picker for that visible model id. + +Utility remains stateless even when the Main listener enables retained sessions or jobs. Utility does not retain upstream sessions, does not register continuation markers, and does not execute Threadline internal tools or Threadline jobs. + +Running `threadline` without a subcommand starts the server. `threadline login` is informational only and prints guidance to sign in with Codex Desktop or Codex CLI. + +## Login And Credential Discovery + +Before Threadline can authenticate to Codex, sign in with Codex Desktop or Codex CLI. Running `threadline login` only prints that guidance. + +Threadline does not acquire, store, delete, or inspect credentials. It relies on Codex-managed authentication sources that are already present on the machine. + +At runtime, Threadline uses only the Codex-managed sources it already supports: + +1. The Codex OS credential-manager entry, read through the existing compatibility path. +2. Supported `auth.json` compatibility roots. + +For Codex interoperability, Threadline can read the same OS credential-manager entry Codex uses: service `Codex Auth` with an account derived from `CODEX_HOME`. Normal Threadline command output does not print that derived account value. Threadline treats the Codex entry as a compatibility input only: it can read those credentials at runtime, but it does not write, rewrite, or delete them. + +`CODEX_HOME` affects two runtime compatibility paths: + +- It selects the Codex home directory used to derive the Codex keyring account for read-only interoperability. +- It is also one of the file fallback roots for `auth.json` discovery. + +If `CODEX_HOME` is unset, Threadline skips the Codex keyring lookup and continues with the remaining supported sources. + +When runtime auth checks the OS credential manager, keyring service failures are not always terminal. If the Codex keyring lookup cannot be used at runtime, Threadline may fall through to the supported `auth.json` compatibility roots. + +The file fallback search keeps existing compatibility behavior and checks these roots in order: + +1. `CHATGPT_LOCAL_HOME` +2. `CODEX_HOME` +3. The default per-user `.chatgpt-local` directory +4. The default per-user `.codex` directory + +`auth.json` file fallbacks are read for compatibility only. Threadline does not write, rewrite, or delete them. + +## Local validation + +Run these commands from the Threadline directory: + +```bash +cargo fmt --all --check +cargo clippy --all-targets --all-features -- -D warnings +cargo test --locked --all-targets --all-features +``` diff --git a/docs/agent/architecture.md b/docs/agent/architecture.md new file mode 100644 index 0000000..c0457a0 --- /dev/null +++ b/docs/agent/architecture.md @@ -0,0 +1,243 @@ +# Agent Architecture + +This document expands Threadline architecture guidance for module boundaries, ownership, dependency direction, and large refactors. + +Root `AGENTS.md` contains the always-on rules. If this file conflicts with root `AGENTS.md`, follow root `AGENTS.md`. + +## Scope + +Use this file before adding or moving modules, changing module responsibilities, changing transport ownership, changing request normalization or protocol type boundaries, introducing shared state/background tasks/IO abstractions, or doing a large refactor. + +For protocol sequencing, also read `docs/agent/protocol.md`. + +For names, comments, tests, logs, commits, and PR wording, also read `docs/agent/conventions.md`. + +## Architecture goals + +Threadline should remain focused, maintainable, and specific to VSCode BYOK `/v1/responses` bridging. + +The architecture should support: + +* HTTP/SSE downstream handling +* Codex backend WebSocket upstream handling +* retained WebSocket sessions and `previous_response_id` continuity +* pump-based Ping/Pong handling while idle +* nested subagent execution without idle timeout +* long-running work through jobs +* safe cleanup and recovery paths + +Do not expand Threadline into a general-purpose OpenAI-compatible proxy or unrelated provider framework. + +Threadline may reuse lessons from previous experiments, but module boundaries, names, comments, tests, and implementation structure must be original to Threadline. Do not preserve ChatMock-specific architecture names or layering. + +## Module principles + +Prefer small modules with one durable responsibility. + +Keep these boundaries clear: + +* protocol types separate from transport code +* request normalization separate from transport code +* downstream SSE translation separate from upstream WebSocket frame handling +* long-lived transport ownership out of route handlers +* local jobs separate from upstream protocol flow unless an internal tool connects them + +Avoid catch-all modules such as `util`, `misc`, `common`, or `manager` unless the module has a narrow documented purpose. + +## Suggested module boundaries + +| Module | Responsibility | +| --- | --- | +| `http` | Axum routes, HTTP wrappers, and public HTTP/SSE error conversion. It must not drive Codex WebSocket IO. | +| `responses` | `/v1/responses` normalization, continuation decisions, lifecycle coordination, downstream SSE translation, and internal tool filtering. | +| `codex_ws` | Codex backend WebSocket connection setup plus upstream protocol message serialization/deserialization. | +| `ws_pump` | Continuous upstream socket IO, outbound commands, Ping/Pong, inbound forwarding, and close/error metadata. All live upstream WebSockets must be pump-owned. | +| `registry` | Completed response marker mapping, retained session metadata, in-use tracking, close/recoverable state, lookup, cleanup, and capacity policy. | +| `jobs` | Local long-running job start/state/poll/output/cancel/cleanup. Jobs must not push upstream responses when they complete. | +| `tools` | `threadline_*` internal tool definitions, validation, local execution, output shaping, and job-tool dispatch. | +| `auth` | ChatGPT/Codex credential loading, refresh, and safe credential access. Secret handling should be centralized here. | +| `config` | CLI flags, environment configuration, defaults, validation, and typed config values. It should not perform network IO. | +| `errors` | Typed internal errors, stable public error codes, safe public payloads, and boundary conversions. | + +## Dependency direction + +Prefer this dependency direction: + +```txt +http -> responses -> registry/tools/jobs/codex_ws/errors +http -> config/errors +tools -> jobs/errors +codex_ws -> auth/config/errors +codex_ws -> ws_pump -> errors +registry -> errors +``` + +This is a guide, not a rigid graph. Avoid dependency cycles. If a cycle appears, narrow the boundary or move shared protocol/model types into a small purpose-built module. + +## Transport ownership + +Long-lived upstream WebSocket transport belongs to `ws_pump`. + +The pump owns the socket. Other modules communicate with it through handles, channels, or narrow methods. + +Do not pass raw `WebSocketStream` into route handlers or high-level response orchestration. + +Do not let downstream request lifetime determine whether the upstream socket can answer Ping/Pong. + +A retained upstream socket may outlive a downstream HTTP request. + +## Protocol type separation + +Protocol types should describe protocol shape; transport code should move frames and handle socket mechanics. + +Good separation: + +* message structs and event enums describe upstream protocol shape +* connector code sends and receives protocol messages +* pump code moves frames and handles Ping/Pong +* response code decides what downstream clients should see + +Avoid mixing axum route logic with upstream event parsing, raw WebSocket handling with response normalization, registry mutation with low-level socket reads, or job output storage with SSE formatting. + +## Request lifecycle shape + +A new downstream `/v1/responses` request should generally flow: + +1. `http` receives the request. +2. `responses` normalizes it. +3. `codex_ws` prepares upstream protocol state. +4. `ws_pump` owns live upstream IO. +5. `responses` translates relevant upstream events into downstream SSE. +6. `registry` records continuation metadata after a continuable completion. +7. `http` completes the downstream response. + +A continuation request should generally flow: + +1. `http` receives `previous_response_id`. +2. `responses` resolves the marker through `registry`. +3. Open retained sessions continue through the retained pump. +4. Closed but recoverable sessions use explicit recovery logic. +5. Missing or unrecoverable markers produce stable public errors. +6. The marker is preserved or updated according to protocol rules. + +## Internal tools and jobs + +Internal tools are Threadline-owned behavior requested through `threadline_*` tool calls. + +Keep detection, local execution, pending output storage, follow-up `response.create`, and downstream filtering as separate concerns. + +Do not embed one-off internal tool behavior inside SSE formatting or route handlers. + +Internal tools that start or inspect long-running work should call `jobs` rather than managing job state directly. + +Jobs are local Threadline state for long-running work. + +A job should start quickly, not require the original downstream HTTP request to stay open, have explicit state and retrievable output, support polling/result/output/cancel/cleanup, and update local job state only when it completes. + +A later internal tool call or downstream-triggered request may retrieve job state or output. Do not design jobs as hidden background upstream response senders. + +## Registry architecture + +The registry is the authority for retained response markers. + +Registry entries should be updated deliberately and should preserve enough state to continue, reject, or recover deterministically. + +Do not scatter marker ownership across unrelated modules. + +Do not let the pump silently delete registry entries. + +Do not let socket close handling erase continuation metadata without explicit registry logic. + +Registry cleanup should be policy-driven, such as TTL, capacity, or explicit invalidation. + +## Error, config, and auth architecture + +Use typed errors internally and convert them into stable public errors at HTTP, SSE, internal tool, and job boundaries. + +Public errors must not expose secrets, local paths, credential details, account identifiers, or unstable debug strings. + +Configuration should be typed and validated early. Runtime modules should receive typed config values rather than repeatedly reading environment variables. + +Authentication should be isolated. Connection code may need credentials, but unrelated modules should not parse, store, or log credential material. + +Do not store production credential material in fixtures, tests, or logs. + +## Concurrency architecture + +Treat retained sessions as shared mutable protocol state. + +Use clear ownership and synchronization. + +Avoid holding locks across network IO when possible. + +Prefer message passing for pump IO. + +Ensure cleanup paths release in-use flags and do not orphan pumps, jobs, or registry entries when downstream requests fail or are cancelled. + +## Testing architecture + +Place tests near the behavior they verify when possible. + +Behavioral tests should focus on durable module contracts. + +Add or update tests when changing response marker handling, retained session lifecycle, pump Ping/Pong behavior, idle socket handling, recovery after socket close, registry conflicts, internal tool sequencing, job lifecycle, public error conversion, or SSE translation. + +Test names must describe behavior, not implementation phases or local debugging history. + +## Adding or moving modules + +Before adding or moving a module, check: + +* What single responsibility does this module own? +* Why does the current module not fit? +* What public types or functions does it expose? +* Which modules may depend on it? +* Does it introduce a dependency cycle? +* Does it own state or IO? +* Does it need tests? +* Does it preserve Threadline source independence? +* Does it keep protocol types separate from transport code? + +Do not add a module just to park temporary code. + +A refactor should not change behavior unless the behavior change is explicit and tested. + +## Shared types and abstraction + +Shared types should have a clear home: protocol message types near `codex_ws`, public error types near `errors`, registry state near `registry`, job state near `jobs`, and config types near `config`. + +Avoid a broad `types` module unless it has a narrow documented purpose. + +Do not introduce traits, generic providers, plugin systems, or broad compatibility layers unless there is an immediate Threadline need. + +A useful abstraction should reduce duplication now, preserve protocol clarity, have a small interface, be easy to test, and not hide ordering or ownership rules. + +## Refactor safety + +During large refactors: + +* preserve current behavior unless explicitly changing it +* keep changes focused +* avoid mixing formatting-only noise with logic changes +* keep commit and PR wording behavior-oriented +* run relevant validation +* call out risks honestly + +Do not include phase labels, local debugging history, transcript-only context, or ChatMock porting language in code or docs. + +## Architecture checklist + +Before finalizing an architecture change, check: + +* Each changed module has one clear responsibility. +* Protocol types are separate from transport code. +* Route handlers do not own raw long-lived WebSockets. +* Live upstream sockets are pump-owned. +* Request normalization and SSE translation are separate from raw frame handling. +* Retained session state is owned by the registry. +* Jobs are local state and not hidden upstream push mechanisms. +* Internal tools are hidden from downstream clients. +* Dependency cycles are avoided. +* Secrets are isolated in auth/config boundaries. +* Public errors are stable and safe. +* Tests, names, comments, and logs match the changed behavior. diff --git a/docs/agent/conventions.md b/docs/agent/conventions.md new file mode 100644 index 0000000..f29cb5d --- /dev/null +++ b/docs/agent/conventions.md @@ -0,0 +1,204 @@ +# Agent Conventions + +This document expands naming, wording, comments, tests, tracing, logs, commits, and PR conventions for Threadline. + +Root `AGENTS.md` contains the always-on rules. If this file conflicts with root `AGENTS.md`, follow root `AGENTS.md`. + +## Scope + +Use this file before editing names, comments, TODO comments, test names or fixtures, tracing event names, log fields, error wording, commit messages, PR text, or public documentation wording. + +## Naming conventions + +Use normal Rust naming conventions: + +| Item | Convention | +| --- | --- | +| Modules, functions, variables | `snake_case` | +| Types and enum variants | `PascalCase` | +| Constants | `SCREAMING_SNAKE_CASE` | + +Prefer precise names that describe durable behavior and protocol concepts. + +Good examples: + +```rust +RetainedSessionRegistry +UpstreamWebSocketPump +ResponseMarker +ThreadlineJobManager +PendingInternalToolOutput +send_followup_tool_outputs +``` + +Avoid vague, phase-based, or project-derived names such as `Thing`, `Manager2`, `handle_stuff`, `phase_handler`, `test_new_flow`, or `legacy_cache`. + +Threadline may reuse public protocol concepts and design lessons, but it must not preserve ChatMock-specific internal names. Use `threadline_*` for Threadline-owned internal tools and Threadline-specific concepts. + +Allowed public protocol names include `response.create`, `previous_response_id`, `session-id`, `thread-id`, and `function_call_output`. Do not rename public protocol concepts just to make them look original. + +## Public terminology + +Use these terms consistently: + +| Term | Meaning | +| --- | --- | +| `upstream` | The Codex backend WebSocket side. | +| `downstream` | The VSCode BYOK HTTP/SSE client side. | +| `response marker` | A completed response id or `previous_response_id` used for continuation. | +| `retained session` | A stored upstream WebSocket plus session metadata managed by the registry. | +| `internal tool` | A `threadline_*` tool call executed locally and hidden from downstream clients. | +| `job` | A long-running local or subprocess task managed by Threadline. | +| `pump` | The task that continuously reads and writes an upstream WebSocket and handles Ping/Pong. | + +## Comments + +Comments should explain durable design intent, protocol quirks, or safety constraints. + +Good comments explain why behavior is required: + +```rust +// The pump must keep reading while idle so server Ping frames receive Pong responses. +``` + +Bad comments describe temporary history, orchestration, or local debugging. + +Do not include phase labels, local machine details, personal paths, chat transcript details, short-lived debugging history, model conversation artifacts, branch-specific notes, or vague TODO comments. + +If historical context is useful, rewrite it as a general protocol or design reason. + +Use TODO comments only when they include a durable reason and a concrete removal condition: + +```rust +// TODO: Remove this compatibility branch once VSCode no longer sends empty tool output arrays. +``` + +Do not use TODO comments to store local planning notes. + +## Tests + +Test names must describe behavior, not implementation phase or local context. + +Prefer: + +```txt +__ +``` + +Examples: + +```rust +retained_session_reconnects_after_idle_close_before_first_event +websocket_pump_replies_to_server_ping_while_idle +internal_tool_outputs_are_sent_after_intermediate_response_completes +registry_preserves_marker_after_recoverable_idle_close +``` + +Avoid names tied to dates, branches, previous projects, local logs, or development phases. + +Tests should assert stable Threadline behavior. Avoid assertions that depend on local paths, local timestamps, temporary branch names, exact debug strings unless public, model conversation text, or ChatMock-specific implementation details. + +## Tracing and logs + +Use structured tracing fields when useful: + +```rust +tracing::debug!( + response_id = %response_id, + session_id = %session_id, + "retained_session_acquired" +); +``` + +Useful fields include `response_id`, `previous_response_id`, `session_id`, `thread_id`, `job_id`, `tool_name`, `marker`, `generation`, `close_code`, and `recoverable`. + +Never log secrets, including access tokens, refresh tokens, cookies, full authorization headers, account identifiers, credential file contents, production credentials, or raw request bodies that may contain credentials. + +Raw upstream `error` events may be logged at debug or trace level only after confirming they do not contain secrets. + +## Stable log event names + +Log event names should be stable and grep-friendly. + +Prefer event names such as: + +```txt +ws_pump_started +ws_pump_ping_received +ws_pump_pong_sent +upstream_event_received +internal_tool_detected +internal_tool_followup_sent +final_response_completed +retained_session_acquired +retained_session_released +reconnect_continuation_attempt +reconnect_continuation_failed +upstream_error_event +``` + +New event names should be lowercase, `snake_case`, behavior-oriented, stable across refactors, and free of branch names, dates, and phase labels. + +## Error wording + +Public error wording should be stable, clear, and VSCode-compatible. + +Prefer typed internal errors and stable public error codes. + +Expected public error codes include: + +```txt +previous_response_not_found +retained_session_conflict +retained_session_capacity_exceeded +upstream_websocket_connect_failed +upstream_websocket_closed +internal_tool_failed +job_not_found +``` + +Do not expose local paths, tokens, cookies, private account information, or unstable debug strings in public errors. + +Do not panic for protocol errors, malformed client input, missing markers, closed sockets, or upstream errors. + +## Commits and PRs + +Keep commits focused and behavior-oriented. + +Good commit subjects: + +```txt +Add pump-based upstream websocket transport +Preserve recoverable retained sessions after idle close +Add internal job tools for long-running tasks +``` + +Avoid vague, phase-based, transcript-derived, or ChatMock-porting language. + +Prefer imperative subject lines. PR titles should describe user-visible or maintainer-visible behavior. + +PR descriptions may mention design motivation, validation, and risks, but must not include transcript-only context, private local paths, local debugging logs, phase labels, model-generation wording, or ChatMock porting language. + +## Public docs wording + +Public docs should describe Threadline directly. + +Allowed: + +```txt +Threadline bridges VSCode BYOK `/v1/responses` requests to retained Codex backend WebSocket sessions. +``` + +Avoid describing Threadline as a ChatMock port. It is acceptable to say Threadline is inspired by lessons from prior experiments when relevant, but do not imply code lineage. + +## Review checklist + +Before finalizing naming, comments, tests, logs, commits, or PR text, check: + +* The wording describes durable behavior. +* ChatMock-specific private names and implementation structure are absent. +* Local paths, dates, branches, and transcript-only context are absent. +* Public protocol terms are preserved where they are actually public terms. +* Logs are structured and free of secrets. +* Test names are behavior-oriented. +* Commit and PR messages are focused on behavior. diff --git a/docs/agent/protocol.md b/docs/agent/protocol.md new file mode 100644 index 0000000..2ebc640 --- /dev/null +++ b/docs/agent/protocol.md @@ -0,0 +1,418 @@ +# Agent Protocol Rules + +This document expands Threadline protocol rules for `/v1/responses`, upstream Codex WebSocket sessions, retained sessions, internal tools, jobs, and protocol-facing errors. + +Root `AGENTS.md` contains the always-on rules. If this file conflicts with root `AGENTS.md`, follow root `AGENTS.md`. + +## Scope + +Use this file before changing `/v1/responses` handling, SSE translation, Codex backend WebSocket connection behavior, WebSocket pump or Ping/Pong behavior, retained session registry behavior, `previous_response_id` continuation, internal `threadline_*` tool execution, job tools, public protocol errors, reconnect, or recovery logic. + +## Protocol boundaries + +`/v1/responses` is the primary API. + +Threadline is a VSCode BYOK to Codex ABI translator for `/v1/responses`, bridging VSCode BYOK requests to Codex backend WebSocket sessions while keeping the implementation focused. + +Do not turn Threadline into a general-purpose OpenAI-compatible proxy. + +Do not add `/v1/chat/completions` unless it is required for VSCode BYOK compatibility. + +Do not add unrelated provider compatibility unless explicitly requested. + +Use `downstream` for the VSCode BYOK HTTP/SSE client side and `upstream` for the Codex backend WebSocket side. + +Downstream clients must not see Threadline-only internal tool calls. + +## Core invariants + +These invariants should remain true across refactors: + +* Live upstream WebSockets are owned by a pump, not by route handlers. +* Retained sessions keep enough state to continue from a completed response marker. +* Idle retained WebSockets keep reading so Ping frames receive Pong responses. +* A completed response marker is not deleted merely because an idle socket later closes. +* Internal `threadline_*` tool calls are executed locally and hidden from downstream clients. +* Intermediate completions for internal tool calls are not final downstream completions. +* Long-running work is represented as jobs, not long blocking tool calls. +* Job completion updates stored state only and does not push a new upstream response by itself. +* Classified auxiliary summary requests are a narrow exception to retained continuation markers and must stay outside retained session registry semantics. +* Public errors are stable and safe to expose. +* Secrets are never logged. + +## `/v1/responses` handling + +Normalize downstream `/v1/responses` requests before sending protocol messages upstream. + +Keep request normalization separate from transport code. + +Keep SSE translation separate from upstream WebSocket frame handling. + +As a narrow compatibility normalization for the Threadline `/v1/responses` bridge, visible assistant text may be accumulated from `response.output_text.delta`, `response.output_text.done`, assistant `response.output_item.done` messages, and final `response.completed` output. + +If Threadline has not already forwarded equivalent visible assistant text downstream, it may emit a synthetic downstream `response.output_text.delta` immediately before forwarding the terminal downstream event that carries the final visible text. + +If earlier visible text was streamed but the final completed assistant message would otherwise be missing or incomplete, Threadline may backfill the final completed assistant message from the accumulated visible assistant text. + +When forwarding the final downstream `response.completed`, Threadline may sanitize `response.completed.response.output` to remove Threadline-internal `threadline_*` function calls while preserving downstream-consumable output, including preserved compaction, context, or other marker-like items that remain part of the completed downstream output. + +Preserved completed `type: "compaction"` items are part of the ordinary downstream protocol contract when Threadline forwards or retains them in `response.completed.response.output`. + +Treat preserved compaction items as opaque state markers. Threadline may use stable structural facts such as item counts, `type`, `id`, item position, and field presence for routing or diagnostics, but it must not decrypt, summarize, normalize, or log `encrypted_content`. + +For ordinary downstream requests, a successful downstream stream may end with `response.completed` only when Threadline has already produced downstream-observable output for that request. + +For this ordinary-request success rule, downstream-observable output is limited to downstream-visible assistant text, forwarded external non-`threadline_*` tool calls, `image_generation_call.result`, and concrete downstream-consumable compaction, context, or state-marker output when Threadline explicitly forwards that output downstream or retains it in the completed downstream output. + +The `threadline_no_observable_output` guard exists to prevent empty or effectively invisible ordinary-request successes. It does not reject a response merely because the only observable output is preserved compaction or another preserved downstream-consumable state marker. + +Server-side `context_management` compaction remains distinct from client-side auxiliary summary behavior. In the current v1 bridge, Threadline strips downstream `context_management` before every upstream `response.create`. Do not treat preserved compaction, context, or marker-like output as summary-only behavior merely because downstream `context_management` fields were present. + +Client-side auxiliary summary remains a separate VS Code behavior used for summary-only prompt shapes. It does not replace retained-session continuation markers, and the v1 bridge does not claim or prove server-side `context_management` passthrough support. + +Internal `threadline_*` tool events, intermediate completions that only finish internal-tool work, and other marker-like payloads that Threadline neither forwards downstream nor retains in the completed downstream output do not themselves satisfy the downstream-observable-output requirement. + +`image_generation_call.result` remains a valid successful non-text output and may satisfy the downstream-observable-output requirement even when no visible assistant text is present. + +If an ordinary request reaches a terminal state without downstream-observable output, including terminal states with only internal `threadline_*` items, only hidden intermediate completions, only compaction-only items that Threadline neither forwards downstream nor retains in the completed downstream output, or other non-observable marker-like payloads, Threadline must end the stream as `response.failed` with a stable `threadline_no_observable_output` failure instead of an empty success. + +Auxiliary summary and transient auxiliary behavior remain narrow exceptions to the ordinary no-observable-output failure rule. + +When a downstream request includes `previous_response_id`, use it as a continuation marker. + +Ordinary downstream requests that include `previous_response_id` keep retained continuation semantics and must not silently start unrelated fresh sessions. + +`response.completed.id` is the continuation-safe marker for later `previous_response_id` requests. + +If a later turn fails upstream, do not reinterpret that failed turn as a new continuation marker. + +A response marker may refer to a retained session that is open, closed but recoverable, missing, or unrecoverable. Handle each state explicitly. + +Do not assume that a missing or closed socket means the response marker should be forgotten. + +A classified auxiliary summary request is a narrow exception to that rule. This request class is identified by summary-only prompt fingerprints carried in `input`, using the observed summary instruction item shape for auxiliary summarization, and it may also carry a downstream `previous_response_id` as client context. + +Classify this request type by its summary-only auxiliary behavior, not by `context_management` fields alone. + +When a request is classified as an auxiliary summary request, do not acquire a retained marker, do not forward its downstream `previous_response_id` upstream, do not consume retained registry capacity, and do not register the completed summary response id as a continuation marker. + +After terminal completion, failure, or cancellation of an auxiliary summary request, clean up any transient auxiliary state associated with that request. + +## WebSocket pump ownership + +All live upstream WebSockets must be pump-based. + +Route handlers must not directly hold and use `WebSocketStream`. + +The pump owns continuous socket IO. Other code communicates with the pump through channels or clearly defined handles. + +The pump must support reading upstream frames, writing outbound upstream messages, replying to server Ping frames with Pong, forwarding Text/Binary frames into an inbound queue, accepting outbound Text/Ping/Close commands, recording close/error metadata, and running while a session is retained. + +## Idle sessions + +A retained session may be idle from the downstream perspective while still needing active upstream IO. + +The pump must keep reading while idle. + +Do not pause the read loop just because no HTTP request is waiting. + +Do not rely on a future downstream request to read pending Ping frames. + +If the upstream sends Ping while the retained session is idle, the pump must reply with Pong. + +## Pump close behavior + +When the upstream WebSocket closes, record close metadata. + +Close metadata should distinguish normal close, protocol error, transport error, recoverable idle close, and unrecoverable close if known. + +Do not discard the response marker merely because the socket closed after a successful `response.completed`. + +If continuation is possible through stored metadata, preserve that metadata. + +If continuation is not possible, keep enough information to produce a clear public error. + +## Registry purpose and contents + +The retained session registry maps completed response markers to upstream session state. + +A response marker is the lookup key for later continuation. + +The registry should store enough state to continue, reject, or recover a request deterministically. + +A registry entry should store the response marker, upstream WebSocket pump handle, session id, thread id, window generation, turn state, in-use flag, close state, recoverable state, and last-used timestamp. + +Store only what is needed for correct continuation, diagnostics, and safe cleanup. + +Do not store secrets in registry entries. + +## Registry lifecycle and conflicts + +Create or update registry entries when an upstream response reaches a completed state that can be continued. + +Do not create retained registry entries for classified auxiliary summary requests or for their completed summary response ids. + +Do not register upstream failed response ids as continuation markers. + +A failed response id may still be emitted downstream for diagnostics when the upstream payload provides one. + +Mark entries in use while a downstream request is actively continuing through them. + +Release the in-use flag when the request finishes, fails, or is cancelled. + +Update last-used timestamps when entries are used. + +Evict entries only through explicit capacity, TTL, or cleanup policy. + +Do not remove a marker as a side effect of observing a post-completion idle close. + +If a marker is already in use and the new request cannot safely share it, return a stable conflict error. + +A conflict should not corrupt the registry entry. + +## Continuation and recovery + +When continuing from `previous_response_id`, first resolve the marker in the registry. + +If the session is open and usable, continue through the retained pump. + +An open retained upstream is a best-effort continuation condition, not a guarantee that upstream still recognizes the marker. + +For ordinary downstream requests that include `previous_response_id`, forward that marker upstream only when the same marker still has an open retained upstream being continued. If no open retained upstream exists for that marker, do not send a known-stale marker upstream just because the downstream request supplied it. + +If a later upstream turn ends with a recoverable `response.failed`, preserve any earlier completed marker that still identifies the retained session. + +If the socket is closed but recoverable metadata exists, attempt recovery or reconnect according to the current protocol implementation for recoverable metadata cases other than ordinary downstream `previous_response_id` continuation where that marker no longer has an open retained upstream. + +If the first upstream send for that continued turn fails before any upstream event is observed, or if the retained upstream closes before the first upstream event arrives, surface the stable downstream `previous_response_not_found` replay signal instead of reconnecting and resending the same marker. + +If recovery fails, return a stable error and keep enough diagnostic information for logs. + +If the marker is unknown, return `previous_response_not_found`. + +Do not silently start an unrelated fresh session for a continuation marker. + +## Internal tool boundary + +Threadline internal tools must use the `threadline_*` prefix. + +Internal tool calls must never be forwarded downstream to VSCode. + +Internal tool names should be treated as Threadline implementation details unless explicitly documented as public. + +Do not let downstream clients invoke arbitrary local tools. + +## Internal tool lifecycle + +When an upstream response emits a Threadline internal tool call, preserve this order: detect the internal tool, execute it locally, store output as pending, keep reading upstream, wait for the intermediate response to complete, send a follow-up `response.create` with `function_call_output`, continue reading the follow-up response, and forward only the final assistant output downstream. + +Do not send follow-up tool outputs before the intermediate response completes. + +Do not treat the intermediate response completion as the final downstream completion. + +Internal `threadline_*` tool events and intermediate completions that only finish internal-tool work are consumed inside Threadline, stay hidden downstream, and are not final downstream completions. + +Visible-text normalization is final-only and applies only to the downstream-visible assistant result after internal-tool follow-up has completed. + +Do not expose internal tool call details downstream unless explicitly required for diagnostics and safe to expose. + +## Pending internal tool output and failure + +Pending internal tool output should be associated with the response or turn that requested it. + +Pending output must not be lost if the intermediate response completes normally. + +Pending output must not be sent twice. + +If local tool execution fails, convert the failure into the expected protocol-level tool output or a stable internal tool error. + +Internal tool failures should be handled without panics. + +Return stable public errors when the failure affects the downstream request. + +Log enough structured metadata to debug the failure without logging secrets. + +Use `internal_tool_failed` for expected public error states involving internal tool execution failure. + +## Job model and tools + +Long-running work should be represented as jobs. + +A job should start quickly, return a `job_id`, and continue asynchronously in local Threadline state. + +Successful `threadline_start_job` calls should return immediately. The current implementation returns a short `next_action_hint` alongside the initial `starting` status to reinforce that the job keeps running in the background. + +Use polling or result retrieval for later status. + +Poll or read output at natural checkpoints when status is actually needed. Avoid tight polling loops when other useful work can continue independently. + +Do not block a single tool call or HTTP request for work that should continue independently. + +Jobs are local Threadline state unless explicitly connected to upstream protocol flow. + +Internal job tools should use the `threadline_*` prefix. + +Expected job tools include `threadline_start_job`, `threadline_poll_job`, `threadline_read_job_output`, `threadline_get_job_result`, and `threadline_cancel_job`. + +Use `threadline_get_job_result` after a terminal status is observed, or before making final claims that depend on success, failure, or cancellation. + +These tools are internal and must not be forwarded downstream as normal model-visible tool calls. + +## Job lifecycle + +A job should have explicit state. In the current implementation and tests, the exposed status strings are `starting`, `running`, `completed`, `failed`, and `cancelled`. + +A job should store enough metadata for polling, result retrieval, incremental output, cancellation, and cleanup. + +A job must not require the original downstream HTTP request to stay open. + +Cancellation should be best effort. A cancelled job should move to a stable cancelled or failed state and should not corrupt stored output. + +Unknown job ids should return `job_not_found`. + +## Job completion and output + +Job completion must not automatically push a new upstream response. + +Job completion should update stored job state only. + +A later internal tool call or downstream-triggered request may retrieve job status or output. + +Do not invent a background upstream response just because a local job completed. + +Long job output should be retrievable incrementally through offsets or cursors. + +`threadline_read_job_output` returns a finite buffered view of job output, including `items`, `next_offset`, and `truncated_before`. + +Callers should pass the returned `next_offset` back on the next incremental read. + +If `truncated_before` is greater than a caller's stored offset, older output has already been dropped from the finite buffer and the next read should resume from `truncated_before`. + +Buffered output may be available before job completion, but final claims that depend on the terminal outcome should be confirmed with `threadline_get_job_result`. + +Do not return unbounded logs in a single response. + +Do not expose local paths, credentials, environment secrets, or private machine details through job output. + +## Error handling + +Prefer typed errors internally. + +Public HTTP/SSE errors should be stable and VSCode compatible. + +Use clear error codes for expected states. + +Do not panic for protocol errors, malformed client input, missing markers, closed sockets, upstream errors, internal tool failures, unknown job ids, or registry conflicts. + +Panic only for impossible internal invariants where continuing would be unsafe. + +## Public error codes and safety + +Use stable error codes for expected states, including `previous_response_not_found`, `retained_session_conflict`, `retained_session_capacity_exceeded`, `upstream_websocket_connect_failed`, `upstream_websocket_closed`, `internal_tool_failed`, and `job_not_found`. + +Add new public error codes only when callers can act on them or logs need stable categorization. + +Do not expose implementation-only error strings as public contracts. + +Public errors must not include tokens, cookies, authorization headers, credential paths, full upstream request bodies, account identifiers, private local machine paths, or transcript-only debugging context. + +Prefer concise user-facing messages plus structured internal logs. + +## Terminal downstream events and SSE + +Raw upstream `error` events may contain sensitive or unstable information. + +Log them only at debug or trace level after confirming they do not contain secrets. + +Upstream `response.failed` and `response.incomplete` are separate downstream terminal paths from raw upstream `error` events. + +Downstream `[DONE]` is only an optional trailer after a terminal downstream event. `[DONE]` alone is never the success signal. + +When Threadline receives an upstream `response.failed`, forward it downstream as terminal SSE `event: response.failed`. + +When Threadline receives an upstream `response.incomplete`, forward it downstream as terminal SSE `event: response.incomplete`. + +Terminal downstream `response.failed` and `response.incomplete` payloads should keep stable, safe Responses-style fields appropriate to the terminal status. + +For `response.failed`, use top-level `type` set to `response.failed`, `response.status` set to `failed`, and `response.error.code` plus `response.error.message` populated from stable public error wording. + +Include `response.id` when the upstream failure payload provides one. + +For `response.incomplete`, preserve safe status-specific fields and do not expose unstable upstream-only internals. + +After emitting a terminal downstream `response.failed` or `response.incomplete` event, Threadline may terminate the stream with downstream `[DONE]`. + +Successful downstream streams should terminate with `response.completed` only when Threadline has already forwarded or preserved concrete downstream-observable output for that request, such as downstream-visible assistant text, forwarded external non-`threadline_*` tool calls, `image_generation_call.result`, or other downstream-consumable output that Threadline explicitly forwards downstream or retains in the completed downstream output. + +A non-empty final `response.completed.response.output` is not by itself the success criterion, and external non-`threadline_*` tool-call-only responses remain valid successful completions when that forwarded tool output is the downstream-observable result. + +Emitting a failed `response.id` downstream does not make that id continuation-safe. Only previously completed markers remain valid for later `previous_response_id` requests. + +If a prior completed marker exists and the upstream `response.failed` is recoverable, preserve that earlier marker for later resume or retry. + +If SSE has already started and upstream later reports `previous_response_not_found`, classify that terminal downstream outcome as `previous_response_not_found` while preserving the prior completed marker or releasing it recoverably. Do not terminal-remove the marker solely because that late not-found was observed. + +If an upstream error must be forwarded downstream, normalize it into a stable public error shape. + +Do not blindly forward raw upstream errors as public API responses. + +Keep raw upstream `error` handling and malformed protocol handling separate from the `response.failed` terminal path unless the protocol implementation is intentionally changed. + +Downstream SSE should represent the final client-facing response stream. + +Internal tool calls, internal-tool intermediate completions, and assistantless intermediate terminal states should not appear as final assistant output. + +If an upstream sequence contains an internal tool call followed by a follow-up response, downstream should observe the final assistant-facing result, not the internal orchestration. + +Keep SSE event names and payloads stable for VSCode compatibility. + +## Ordering rules + +Preserve protocol ordering. + +In particular, do not send internal tool output before the intermediate response completes, mark downstream final completion on an intermediate completion, release a retained session before all required upstream events are processed, delete a marker before continuation/recovery decisions are complete, or push job completion upstream without an explicit request path. + +Ordering bugs are likely to create hard-to-debug continuation failures. + +## Concurrency and logging + +Treat retained sessions as shared mutable protocol state. + +Protect registry entries from concurrent incompatible use. + +Avoid holding locks across network IO when possible. + +Avoid route-handler ownership of long-lived transport state. + +Prefer message passing for pump IO. + +Ensure cleanup paths release in-use flags and do not orphan jobs or pumps. + +Use structured tracing for protocol events. + +Useful fields include `response_id`, `previous_response_id`, `session_id`, `thread_id`, `job_id`, `tool_name`, `marker`, `generation`, `recoverable`, and `close_code`. + +For compaction-sensitive diagnostics, keep logs limited to safe structured facts such as item counts, item types, item ids, booleans, and presence flags like whether downstream `context_management` was present, whether Threadline stripped it upstream, or whether preserved completed output contained a compaction item. + +Do not log or echo `encrypted_content`, prompts, tool arguments, tokens, cookies, raw request bodies, or other opaque compaction payload fields. + +Never log secrets. + +Use stable event names as described in `docs/agent/conventions.md`. + +## Manual compaction round-trip probe + +Use this checklist when verifying VS Code and Codex round-trip behavior without dumping sensitive payloads: + +1. Confirm the incoming downstream `/v1/responses` request includes `context_management` and record only safe facts such as the configured compaction `type`, presence of `compact_threshold`, and whether `previous_response_id` is present. +2. Confirm the upstream `response.create` payload omits `context_management` after Threadline normalization, and confirm only safe structured stripping facts were recorded without logging prompts, tokens, or raw bodies. +3. Confirm the downstream stream or terminal `response.completed` includes a preserved `type: "compaction"` item by checking only safe structure such as item count, item `type`, item `id`, and whether `encrypted_content` is present. +4. Confirm the ordinary-request terminal result matches visibility rules: a preserved compaction-only completion is a valid `response.completed`, while a terminal path with no forwarded or retained observable output must become `threadline_no_observable_output`. +5. Confirm VS Code records the returned compaction item without exposing its opaque payload, using only safe indicators such as a compaction-related event name, presence flag, item id, or count. +6. Confirm the next downstream request round-trips the prior compaction item as input by matching only safe structure such as `type: "compaction"`, item `id`, and presence flags rather than comparing raw encrypted payload bytes in logs. + +If the downstream request included `context_management` but no preserved or forwarded compaction item ever returns downstream, do not treat auxiliary summary as covering a server-side compaction contract that the v1 bridge does not send upstream. + +## Protocol change checklist + +Before finalizing a protocol change, check that live upstream WebSockets remain pump-owned, idle pumps keep reading and answer Ping with Pong, response markers survive completion and recoverable idle closes, internal tool calls stay hidden downstream, tool outputs wait for intermediate completion, intermediate and final completions stay separate, long-running work uses jobs, job completion only updates local state, public errors are stable and safe, malformed inputs/upstream errors do not panic, and logs are structured and secret-free. diff --git a/docs/agent/workflow.md b/docs/agent/workflow.md new file mode 100644 index 0000000..236fe9e --- /dev/null +++ b/docs/agent/workflow.md @@ -0,0 +1,190 @@ +# Agent Workflow + +This document expands Threadline workflow rules for local state, validation, CI, security scanning, and final development summaries. + +Root `AGENTS.md` contains the always-on rules. If this file conflicts with root `AGENTS.md`, follow root `AGENTS.md`. + +## Scope + +Use this file before finishing a code change, deciding validation commands, editing GitHub Actions or CodeQL configuration, creating local-only notes, writing final development summaries, preparing commit/PR validation notes, or deciding what not to include in source files, tests, docs, commits, or PR text. + +## Workflow principles + +Keep development work reproducible. + +Keep committed files free of private local context. + +Keep validation results separate from source comments. + +Keep temporary orchestration notes out of public docs, tests, code comments, commit messages, and PR text. + +Report what changed, what was validated, and what risks remain. + +## Before changing files + +Before editing code, identify the affected area: + +* Protocol behavior: read `docs/agent/protocol.md`. +* Naming, comments, tests, logs, commits, or PR wording: read `docs/agent/conventions.md`. +* Module boundaries or large refactors: read `docs/agent/architecture.md`. +* Validation, CI, CodeQL, local notes, or summary wording: use this file. + +For small edits, still follow root `AGENTS.md`. For large edits, prefer focused changes over broad rewrites. + +## Local-only state + +Local orchestration notes must not be committed unless generalized into durable documentation. + +Use local-only files for scratch notes, debugging notes, and temporary coordination, such as `.threadline/notes.md`, `.threadline/debug-log.md`, and `.threadline/orchestration.md`. + +These files should be ignored by git. + +Do not copy local-only context into source comments, test names, public docs, commit messages, PR descriptions, fixtures, GitHub Actions names, CodeQL workflow names, or final public summaries. + +## Git ignore and secrets + +Use `.gitignore` for local state directories and private local configuration: `.threadline/`, `*.local.json`, `*.local.toml`, and `*.log`. + +Never commit or log access tokens, refresh tokens, cookies, full authorization headers, account identifiers, local credential files, production credentials, or private local paths that reveal credential locations. + +Do not store production credentials in test fixtures. + +Validation output and error reports must also avoid secrets. When summarizing a failure, describe the failing component and error class without copying sensitive data. + +## Development loop + +Use this general loop: + +1. Understand the affected behavior. +2. Check the relevant agent docs. +3. Make the smallest durable change that fits existing responsibilities. +4. Add or update tests when behavior changes. +5. Run formatting. +6. Run static checks. +7. Run tests. +8. Record validation results in the final development summary. + +Do not add source comments that say which step or phase produced the change. + +Do not add temporary branch names, local phase labels, or model-conversation artifacts to committed files. + +## Validation baseline + +Before considering a code change complete, run the relevant checks: `cargo fmt`, `cargo clippy --all-targets --all-features`, and `cargo test`. + +Prefer running all three for behavior changes. + +For documentation-only changes, Rust validation may be unnecessary, but the final summary should say that no code validation was needed. + +## Formatting and Clippy + +Run `cargo fmt` before final reporting. + +Run `cargo clippy --all-targets --all-features` for static checks. + +Treat new Clippy warnings as issues to fix unless there is a clear reason not to. + +If a warning is intentionally allowed, use the narrowest possible allow and explain the durable reason only when future maintainers need it. + +Do not silence warnings just to pass temporary work. + +## Tests + +Run `cargo test`. + +Add or update tests when changing retained sessions, WebSocket pump behavior, Ping/Pong, `previous_response_id` continuation, registry conflicts, internal tool lifecycle, job lifecycle, public errors, SSE translation, or security-sensitive behavior. + +Test names should describe stable behavior, not implementation phases. + +## Targeted validation + +When full validation is expensive or unnecessary, run the most relevant targeted checks first: + +```sh +cargo test registry +cargo test jobs +cargo test responses +cargo test ws_pump +``` + +After targeted checks pass, prefer the full baseline before final completion when the change affects shared behavior. + +## When validation cannot be run + +If a check cannot be run, record it in the final development summary. + +Include the skipped command, reason, partial validation, and remaining risk. + +Do not put validation excuses in source comments. + +Good final summary wording: `Validation: Not run: cargo test. Reason: Rust toolchain is unavailable in this environment.` + +## CI and CodeQL + +Keep GitHub Actions workflow names stable and descriptive. + +Workflow, job, and step names should describe durable purpose, not branch names, local phase labels, orchestration notes, or model-conversation context. + +Good examples include `name: Rust CI`, `jobs.test.name: Test`, and a step named `Run cargo test`. + +Use CodeQL for Rust security scanning. Prefer manual build mode so analysis sees the same crate graph that `cargo build` uses. + +Do not make CodeQL configuration depend on private local paths or local machine state. + +Do not commit temporary CodeQL experiments unless they are generalized into durable CI configuration. + +## Development summaries + +When reporting changes, use this shape: `Changed:`, `Validation:`, and `Risks:`. + +Keep summaries factual and brief. + +The `Changed` section should describe maintainer-visible behavior, not local orchestration steps. + +The `Validation` section should list commands run and results. Do not claim checks were run if they were not run. + +The `Risks` section should mention remaining uncertainty. Use `Risks: None known.` only when there is no specific remaining concern. + +## Final summary safety + +Final summaries must not include private local paths, access tokens, refresh tokens, cookies, authorization headers, account identifiers, raw credential file contents, transcript-only context, temporary phase labels, local machine details, or model-generation wording. + +Mention public repository paths only when they are part of the change. + +## Pull request readiness + +Before opening or finalizing a PR, check: + +* The change is focused. +* Root `AGENTS.md` rules were followed. +* Relevant split docs were consulted. +* Code comments explain durable design intent only. +* Test names describe behavior. +* Logs use structured fields and do not expose secrets. +* Public errors are stable and safe. +* Local-only files remain ignored. +* CI names are stable and descriptive. +* CodeQL configuration does not depend on private local state. +* Validation results are recorded honestly. +* Remaining risks are called out. + +## No background completion claims + +Do not say work will be completed later unless an explicit scheduled task or external workflow actually exists. + +Do not imply local jobs, CI, or background work have run unless they have actually run. + +When reporting status, distinguish clearly between changed, not changed, validated, not validated, recommended next action, and remaining risk. + +## Workflow checklist + +Before finalizing work, verify: + +* No local credentials or account identifiers were added. +* No `.threadline/` scratch notes were committed. +* No `*.local.json`, `*.local.toml`, or `*.log` files were committed. +* No source comment contains phase labels or transcript-only context. +* No test name contains local debugging history. +* No workflow/job/step name contains temporary branch or phase wording. +* Relevant Rust checks were run, or skipped checks are explained. +* The final summary uses `Changed`, `Validation`, and `Risks`. diff --git a/src/auth.rs b/src/auth.rs new file mode 100644 index 0000000..97c0876 --- /dev/null +++ b/src/auth.rs @@ -0,0 +1,790 @@ +use std::env; +use std::fmt; +use std::fs; +use std::path::{Path, PathBuf}; + +use serde::Deserialize; +use sha2::{Digest, Sha256}; +use thiserror::Error; + +const CODEX_KEYRING_SERVICE: &str = "Codex Auth"; + +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct AuthDiscoveryOptions { + pub chatgpt_local_home: Option, + pub codex_home: Option, + pub user_home: Option, +} + +impl AuthDiscoveryOptions { + pub fn from_env() -> Self { + Self { + chatgpt_local_home: env_path("CHATGPT_LOCAL_HOME"), + codex_home: env_path("CODEX_HOME"), + user_home: env_path("USERPROFILE").or_else(|| env_path("HOME")), + } + } +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum AuthSource { + CodexKeyring, + ChatgptLocalAuth, + CodexHomeAuth, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum RefreshBoundary { + NotAvailable, + RefreshTokenPresent, +} + +#[derive(Clone, PartialEq, Eq)] +pub struct LoadedUpstreamAuth { + pub bearer_token: String, + pub source: AuthSource, + pub refresh_boundary: RefreshBoundary, +} + +impl fmt::Debug for LoadedUpstreamAuth { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("LoadedUpstreamAuth") + .field("bearer_token", &"[redacted]") + .field("source", &self.source) + .field("refresh_boundary", &self.refresh_boundary) + .finish() + } +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum CredentialStoreErrorKind { + ServiceUnavailable, + MalformedPayload, +} + +#[derive(Debug, Clone, Error, PartialEq, Eq)] +#[error("{message}")] +struct CredentialStoreError { + kind: CredentialStoreErrorKind, + message: String, +} + +impl CredentialStoreError { + fn new(kind: CredentialStoreErrorKind, message: impl Into) -> Self { + Self { + kind, + message: message.into(), + } + } + + #[cfg(test)] + fn kind(&self) -> CredentialStoreErrorKind { + self.kind + } +} + +trait CredentialStore { + fn get_secret( + &self, + service: &str, + account: &str, + ) -> Result, CredentialStoreError>; +} + +#[derive(Debug, Default, Clone, Copy)] +struct OsKeyringCredentialStore; + +impl CredentialStore for OsKeyringCredentialStore { + fn get_secret( + &self, + service: &str, + account: &str, + ) -> Result, CredentialStoreError> { + let entry = keyring::Entry::new(service, account).map_err(|error| { + CredentialStoreError::new( + CredentialStoreErrorKind::ServiceUnavailable, + format!("failed to open OS credential entry: {error}"), + ) + })?; + match entry.get_password() { + Ok(secret) => Ok(Some(secret)), + Err(keyring::Error::NoEntry) => Ok(None), + Err(error) => Err(CredentialStoreError::new( + CredentialStoreErrorKind::ServiceUnavailable, + format!("failed to read OS credential entry: {error}"), + )), + } + } +} + +#[derive(Debug, Error, PartialEq, Eq)] +pub enum AuthLoadError { + #[error("Threadline could not find upstream credentials in any supported auth.json location.")] + MissingCredentials, + + #[error("Threadline could not read upstream credentials from {path}.")] + CredentialFileUnreadable { path: PathBuf }, + + #[error( + "Threadline found an auth.json file at {path}, but it did not contain a usable upstream token." + )] + CredentialFileMissingToken { path: PathBuf }, + + #[error("Threadline could not parse auth.json at {path}.")] + CredentialFileMalformed { path: PathBuf }, +} + +#[derive(Debug, Deserialize)] +struct StoredAuthFile { + #[serde(rename = "OPENAI_API_KEY")] + openai_api_key: Option, + tokens: Option, +} + +#[derive(Debug, Deserialize)] +struct StoredTokens { + access_token: Option, + refresh_token: Option, +} + +fn codex_keyring_service_and_account(codex_home: &Path) -> std::io::Result<(String, String)> { + Ok(( + CODEX_KEYRING_SERVICE.to_string(), + compute_codex_store_key(codex_home), + )) +} + +fn load_codex_keyring_auth( + store: &impl CredentialStore, + codex_home: &Path, +) -> Result, CredentialStoreError> { + let (service, account) = codex_keyring_service_and_account(codex_home).map_err(|_| { + CredentialStoreError::new( + CredentialStoreErrorKind::ServiceUnavailable, + "failed to compute Codex keyring account", + ) + })?; + let Some(secret) = store.get_secret(&service, &account)? else { + return Ok(None); + }; + + let file = serde_json::from_str::(&secret).map_err(|_| { + CredentialStoreError::new( + CredentialStoreErrorKind::MalformedPayload, + "Codex keyring payload could not be parsed.", + ) + })?; + + let token = file + .tokens + .as_ref() + .and_then(|tokens| non_empty(tokens.access_token.as_deref())) + .or_else(|| non_empty(file.openai_api_key.as_deref())); + + let Some(token) = token else { + return Err(CredentialStoreError::new( + CredentialStoreErrorKind::MalformedPayload, + "Codex keyring payload did not contain a usable upstream token.", + )); + }; + + let refresh_boundary = if file + .tokens + .as_ref() + .and_then(|tokens| non_empty(tokens.refresh_token.as_deref())) + .is_some() + { + RefreshBoundary::RefreshTokenPresent + } else { + RefreshBoundary::NotAvailable + }; + + Ok(Some(LoadedUpstreamAuth { + bearer_token: token.to_string(), + source: AuthSource::CodexKeyring, + refresh_boundary, + })) +} + +fn compute_codex_store_key(codex_home: &Path) -> String { + let canonical = codex_home + .canonicalize() + .unwrap_or_else(|_| codex_home.to_path_buf()); + let path_str = canonical.to_string_lossy(); + let mut hasher = Sha256::new(); + hasher.update(path_str.as_bytes()); + let digest = hasher.finalize(); + let hex = format!("{digest:x}"); + format!("cli|{}", &hex[..16]) +} + +pub fn load_upstream_auth( + options: &AuthDiscoveryOptions, +) -> Result { + load_upstream_auth_with_store(options, &OsKeyringCredentialStore) +} + +fn load_upstream_auth_with_store( + options: &AuthDiscoveryOptions, + store: &impl CredentialStore, +) -> Result { + if let Some(codex_home) = non_empty_path(options.codex_home.as_ref()) { + match load_codex_keyring_auth(store, codex_home) { + Ok(Some(auth)) => return Ok(auth), + Ok(None) => {} + Err(_) => {} + } + } + + for (source, root) in auth_search_roots(options) { + let path = root.join("auth.json"); + let metadata = match fs::metadata(&path) { + Ok(metadata) => metadata, + Err(error) if error.kind() == std::io::ErrorKind::NotFound => continue, + Err(_) => return Err(AuthLoadError::CredentialFileUnreadable { path }), + }; + + if metadata.is_dir() { + return Err(AuthLoadError::CredentialFileUnreadable { path }); + } + + let bytes = fs::read(&path) + .map_err(|_| AuthLoadError::CredentialFileUnreadable { path: path.clone() })?; + let file = serde_json::from_slice::(&bytes) + .map_err(|_| AuthLoadError::CredentialFileMalformed { path: path.clone() })?; + + let token = file + .tokens + .as_ref() + .and_then(|tokens| non_empty(tokens.access_token.as_deref())) + .or_else(|| non_empty(file.openai_api_key.as_deref())); + + let Some(token) = token else { + return Err(AuthLoadError::CredentialFileMissingToken { path }); + }; + + let refresh_boundary = if file + .tokens + .as_ref() + .and_then(|tokens| non_empty(tokens.refresh_token.as_deref())) + .is_some() + { + RefreshBoundary::RefreshTokenPresent + } else { + RefreshBoundary::NotAvailable + }; + + return Ok(LoadedUpstreamAuth { + bearer_token: token.to_string(), + source, + refresh_boundary, + }); + } + + Err(AuthLoadError::MissingCredentials) +} + +fn auth_search_roots(options: &AuthDiscoveryOptions) -> Vec<(AuthSource, PathBuf)> { + let mut roots = Vec::new(); + + if let Some(path) = non_empty_path(options.chatgpt_local_home.as_ref()) { + roots.push((AuthSource::ChatgptLocalAuth, path.to_path_buf())); + } + if let Some(path) = non_empty_path(options.codex_home.as_ref()) { + roots.push((AuthSource::CodexHomeAuth, path.to_path_buf())); + } + if let Some(user_home) = non_empty_path(options.user_home.as_ref()) { + roots.push(( + AuthSource::ChatgptLocalAuth, + user_home.join(".chatgpt-local"), + )); + roots.push((AuthSource::CodexHomeAuth, user_home.join(".codex"))); + } + + roots +} + +fn env_path(name: &str) -> Option { + env::var_os(name) + .filter(|value| !value.is_empty()) + .map(PathBuf::from) +} + +fn non_empty(value: Option<&str>) -> Option<&str> { + value.and_then(|value| { + let trimmed = value.trim(); + if trimmed.is_empty() { + None + } else { + Some(trimmed) + } + }) +} + +fn non_empty_path(path: Option<&PathBuf>) -> Option<&PathBuf> { + path.filter(|path| !path.as_os_str().is_empty()) +} + +#[cfg(test)] +#[derive(Default, Debug, Clone)] +struct FakeCredentialStore { + state: std::sync::Arc>, +} + +#[cfg(test)] +#[derive(Default, Debug)] +struct FakeCredentialStoreState { + secrets: std::collections::BTreeMap<(String, String), String>, + read_errors: std::collections::BTreeMap<(String, String), CredentialStoreError>, +} + +#[cfg(test)] +impl FakeCredentialStore { + fn seed_secret(&self, service: &str, account: &str, secret: &str) { + let mut state = self.state.lock().expect("fake credential store poisoned"); + state.secrets.insert( + (service.to_string(), account.to_string()), + secret.to_string(), + ); + } + + fn read_raw(&self, service: &str, account: &str) -> Option { + let state = self.state.lock().expect("fake credential store poisoned"); + state + .secrets + .get(&(service.to_string(), account.to_string())) + .cloned() + } + + fn with_service_error(service: &str, account: &str, error: CredentialStoreError) -> Self { + let store = Self::default(); + let mut state = store.state.lock().expect("fake credential store poisoned"); + state + .read_errors + .insert((service.to_string(), account.to_string()), error); + drop(state); + store + } +} + +#[cfg(test)] +impl CredentialStore for FakeCredentialStore { + fn get_secret( + &self, + service: &str, + account: &str, + ) -> Result, CredentialStoreError> { + let state = self.state.lock().expect("fake credential store poisoned"); + if let Some(error) = state + .read_errors + .get(&(service.to_string(), account.to_string())) + { + return Err(error.clone()); + } + + Ok(state + .secrets + .get(&(service.to_string(), account.to_string())) + .cloned()) + } +} + +#[cfg(test)] +mod tests { + use std::fs; + use std::path::Path; + + use serde_json::json; + use tempfile::TempDir; + + use super::{ + AuthDiscoveryOptions, AuthLoadError, AuthSource, CredentialStoreError, + CredentialStoreErrorKind, FakeCredentialStore, RefreshBoundary, + codex_keyring_service_and_account, load_codex_keyring_auth, load_upstream_auth, + load_upstream_auth_with_store, + }; + + fn seed_legacy_threadline_keyring_secret( + store: &FakeCredentialStore, + bearer_token: &str, + refresh_token: Option<&str>, + ) { + let payload = match refresh_token { + Some(refresh_token) => json!({ + "bearer_token": bearer_token, + "refresh_token": refresh_token, + }), + None => json!({ + "bearer_token": bearer_token, + }), + }; + store.seed_secret("Threadline Auth", "default", &payload.to_string()); + } + + fn seed_codex_keyring_payload( + store: &FakeCredentialStore, + codex_home: &Path, + access_token: &str, + refresh_token: Option<&str>, + ) { + let (service, account) = + codex_keyring_service_and_account(codex_home).expect("codex key should compute"); + let payload = match refresh_token { + Some(refresh_token) => json!({ + "tokens": { + "access_token": access_token, + "refresh_token": refresh_token + } + }), + None => json!({ + "tokens": { + "access_token": access_token + } + }), + }; + store.seed_secret(&service, &account, &payload.to_string()); + } + + #[test] + fn codex_store_key_matches_known_codex_home() { + let (service, account) = codex_keyring_service_and_account(Path::new("~/.codex")) + .expect("codex key should compute"); + + assert_eq!(service, "Codex Auth"); + assert_eq!(account, "cli|940db7b1d0e4eb40"); + } + + #[test] + fn codex_keyring_payload_is_read_without_rewriting_unknown_fields() { + let store = FakeCredentialStore::default(); + let (service, account) = codex_keyring_service_and_account(Path::new("~/.codex")) + .expect("codex key should compute"); + let original_payload = json!({ + "OPENAI_API_KEY": "", + "tokens": { + "access_token": "codex-access-token", + "refresh_token": "codex-refresh-token", + "unknown_nested": "keep-me" + }, + "last_refresh": "2026-06-07T00:00:00Z", + "unknown_top_level": { + "still": "here" + } + }) + .to_string(); + store.seed_secret(&service, &account, &original_payload); + + let auth = load_codex_keyring_auth(&store, Path::new("~/.codex")) + .expect("codex payload should load") + .expect("codex auth should exist"); + + assert_eq!(auth.bearer_token, "codex-access-token"); + assert_eq!(auth.source, AuthSource::CodexKeyring); + assert_eq!(auth.refresh_boundary, RefreshBoundary::RefreshTokenPresent); + assert_eq!( + store.read_raw(&service, &account).as_deref(), + Some(original_payload.as_str()) + ); + } + + #[test] + fn threadline_owned_keyring_entries_are_ignored_during_auth_loading() { + let temp = TempDir::new().expect("tempdir"); + let store = FakeCredentialStore::default(); + let codex_home = temp.path().join("codex-home"); + seed_legacy_threadline_keyring_secret( + &store, + "threadline-token", + Some("threadline-refresh"), + ); + seed_codex_keyring_payload(&store, &codex_home, "codex-token", Some("codex-refresh")); + + let options = AuthDiscoveryOptions { + chatgpt_local_home: None, + codex_home: Some(codex_home), + user_home: None, + }; + + let auth = load_upstream_auth_with_store(&options, &store) + .expect("codex keyring auth should load when legacy threadline secret exists"); + + assert_eq!(auth.bearer_token, "codex-token"); + assert_eq!(auth.source, AuthSource::CodexKeyring); + assert_eq!(auth.refresh_boundary, RefreshBoundary::RefreshTokenPresent); + } + + #[test] + fn codex_keyring_wins_when_present() { + let temp = TempDir::new().expect("tempdir"); + let store = FakeCredentialStore::default(); + let codex_home = temp.path().join("codex-home"); + seed_codex_keyring_payload(&store, &codex_home, "codex-token", Some("codex-refresh")); + + let options = AuthDiscoveryOptions { + chatgpt_local_home: None, + codex_home: Some(codex_home), + user_home: None, + }; + + let auth = load_upstream_auth_with_store(&options, &store) + .expect("codex keyring auth should load"); + + assert_eq!(auth.bearer_token, "codex-token"); + assert_eq!(auth.source, AuthSource::CodexKeyring); + assert_eq!(auth.refresh_boundary, RefreshBoundary::RefreshTokenPresent); + } + + #[test] + fn codex_auth_file_remains_fallback_when_keyring_is_missing() { + let temp = TempDir::new().expect("tempdir"); + let store = FakeCredentialStore::default(); + let codex_home = temp.path().join("codex-home"); + fs::create_dir_all(&codex_home).expect("codex home"); + fs::write( + codex_home.join("auth.json"), + serde_json::to_vec_pretty(&json!({"OPENAI_API_KEY": "codex-file-token"})) + .expect("json"), + ) + .expect("auth file"); + + let options = AuthDiscoveryOptions { + chatgpt_local_home: None, + codex_home: Some(codex_home), + user_home: None, + }; + + let auth = + load_upstream_auth_with_store(&options, &store).expect("codex auth file should load"); + + assert_eq!(auth.bearer_token, "codex-file-token"); + assert_eq!(auth.source, AuthSource::CodexHomeAuth); + assert_eq!(auth.refresh_boundary, RefreshBoundary::NotAvailable); + } + + #[test] + fn codex_keyring_service_failure_falls_through_to_codex_auth_file() { + let temp = TempDir::new().expect("tempdir"); + let codex_home = temp.path().join("codex-home"); + fs::create_dir_all(&codex_home).expect("codex home"); + fs::write( + codex_home.join("auth.json"), + serde_json::to_vec_pretty(&json!({"OPENAI_API_KEY": "codex-file-token"})) + .expect("json"), + ) + .expect("auth file"); + let (service, account) = + codex_keyring_service_and_account(&codex_home).expect("codex key should compute"); + let store = FakeCredentialStore::with_service_error( + &service, + &account, + CredentialStoreError::new( + CredentialStoreErrorKind::ServiceUnavailable, + "keyring backend unavailable", + ), + ); + + let options = AuthDiscoveryOptions { + chatgpt_local_home: None, + codex_home: Some(codex_home), + user_home: None, + }; + + let auth = load_upstream_auth_with_store(&options, &store) + .expect("codex auth file should load after keyring failure"); + + assert_eq!(auth.bearer_token, "codex-file-token"); + assert_eq!(auth.source, AuthSource::CodexHomeAuth); + assert_eq!(auth.refresh_boundary, RefreshBoundary::NotAvailable); + } + + #[test] + fn malformed_codex_keyring_payload_falls_through_to_supported_auth_file_roots() { + let temp = TempDir::new().expect("tempdir"); + let store = FakeCredentialStore::default(); + let codex_home = temp.path().join("codex-home"); + fs::create_dir_all(&codex_home).expect("codex home"); + fs::write( + codex_home.join("auth.json"), + serde_json::to_vec_pretty(&json!({"OPENAI_API_KEY": "codex-file-token"})) + .expect("json"), + ) + .expect("auth file"); + let (service, account) = + codex_keyring_service_and_account(&codex_home).expect("codex key should compute"); + store.seed_secret(&service, &account, r#"{"tokens":{"access_token":}}"#); + + let options = AuthDiscoveryOptions { + chatgpt_local_home: None, + codex_home: Some(codex_home), + user_home: None, + }; + + let auth = load_upstream_auth_with_store(&options, &store) + .expect("codex auth file should load after malformed keyring payload"); + + assert_eq!(auth.bearer_token, "codex-file-token"); + assert_eq!(auth.source, AuthSource::CodexHomeAuth); + assert_eq!(auth.refresh_boundary, RefreshBoundary::NotAvailable); + } + + #[test] + fn malformed_codex_keyring_payload_is_reported_without_exposing_secret_values() { + let store = FakeCredentialStore::default(); + let codex_home = Path::new("~/.codex"); + let (service, account) = + codex_keyring_service_and_account(codex_home).expect("codex key should compute"); + store.seed_secret( + &service, + &account, + r#"{"tokens":{"access_token":"leaked-secret"}"#, + ); + + let error = load_codex_keyring_auth(&store, codex_home) + .expect_err("malformed payload should surface as a keyring error"); + + assert_eq!(error.kind(), CredentialStoreErrorKind::MalformedPayload); + assert!(!error.to_string().contains("leaked-secret")); + } + + #[test] + fn codex_keyring_is_skipped_when_codex_home_is_unavailable() { + let temp = TempDir::new().expect("tempdir"); + let store = FakeCredentialStore::default(); + let chatgpt_home = temp.path().join("chatgpt-home"); + fs::create_dir_all(&chatgpt_home).expect("chatgpt home"); + fs::write( + chatgpt_home.join("auth.json"), + serde_json::to_vec_pretty(&json!({"OPENAI_API_KEY": "chatgpt-file-token"})) + .expect("json"), + ) + .expect("auth file"); + + let options = AuthDiscoveryOptions { + chatgpt_local_home: Some(chatgpt_home), + codex_home: None, + user_home: None, + }; + + let auth = load_upstream_auth_with_store(&options, &store) + .expect("chatgpt auth should load without codex home"); + + assert_eq!(auth.bearer_token, "chatgpt-file-token"); + assert_eq!(auth.source, AuthSource::ChatgptLocalAuth); + } + + #[test] + fn missing_credentials_return_secret_safe_error() { + let temp = TempDir::new().expect("tempdir"); + let store = FakeCredentialStore::default(); + let options = AuthDiscoveryOptions { + chatgpt_local_home: Some(temp.path().join("chatgpt-home")), + codex_home: Some(temp.path().join("codex-home")), + user_home: Some(temp.path().join("user-home")), + }; + + let error = + load_upstream_auth_with_store(&options, &store).expect_err("missing auth should fail"); + + assert_eq!(error, AuthLoadError::MissingCredentials); + assert!(!error.to_string().contains("codex-token")); + } + + #[test] + fn unreadable_auth_file_returns_secret_safe_error() { + let temp = TempDir::new().expect("tempdir"); + let chatgpt_home = temp.path().join("chatgpt-home"); + fs::create_dir_all(chatgpt_home.join("auth.json")).expect("make unreadable directory"); + + let options = AuthDiscoveryOptions { + chatgpt_local_home: Some(chatgpt_home), + codex_home: None, + user_home: None, + }; + + let error = load_upstream_auth(&options).expect_err("directory auth path should fail"); + + match &error { + AuthLoadError::CredentialFileUnreadable { path } => { + assert_eq!( + path.file_name().and_then(|part| part.to_str()), + Some("auth.json") + ); + } + other => panic!("unexpected error: {other:?}"), + } + assert!(!error.to_string().contains("secret-value")); + } + + #[test] + fn codex_auth_file_is_used_when_chatgpt_auth_is_missing() { + let temp = TempDir::new().expect("tempdir"); + let codex_home = temp.path().join("codex-home"); + fs::create_dir_all(&codex_home).expect("codex home"); + fs::write( + codex_home.join("auth.json"), + serde_json::to_vec_pretty(&json!({"OPENAI_API_KEY": "codex-file-token"})) + .expect("json"), + ) + .expect("auth file"); + + let options = AuthDiscoveryOptions { + chatgpt_local_home: Some(temp.path().join("chatgpt-home")), + codex_home: Some(codex_home), + user_home: None, + }; + + let auth = load_upstream_auth(&options).expect("codex auth should load"); + + assert_eq!(auth.bearer_token, "codex-file-token"); + assert_eq!(auth.source, AuthSource::CodexHomeAuth); + assert_eq!(auth.refresh_boundary, RefreshBoundary::NotAvailable); + } + + #[test] + fn chatgpt_auth_reports_refresh_capability_when_refresh_token_exists() { + let temp = TempDir::new().expect("tempdir"); + let chatgpt_home = temp.path().join("chatgpt-home"); + fs::create_dir_all(&chatgpt_home).expect("chatgpt home"); + fs::write( + chatgpt_home.join("auth.json"), + serde_json::to_vec_pretty(&json!({ + "tokens": { + "access_token": "chatgpt-access-token", + "refresh_token": "chatgpt-refresh-token" + } + })) + .expect("json"), + ) + .expect("auth file"); + + let options = AuthDiscoveryOptions { + chatgpt_local_home: Some(chatgpt_home), + codex_home: None, + user_home: None, + }; + + let auth = load_upstream_auth(&options).expect("chatgpt auth should load"); + + assert_eq!(auth.bearer_token, "chatgpt-access-token"); + assert_eq!(auth.source, AuthSource::ChatgptLocalAuth); + assert_eq!(auth.refresh_boundary, RefreshBoundary::RefreshTokenPresent); + } + + #[test] + fn loaded_upstream_auth_debug_redacts_bearer_token() { + let auth = super::LoadedUpstreamAuth { + bearer_token: "sensitive-token".to_string(), + source: AuthSource::CodexKeyring, + refresh_boundary: RefreshBoundary::NotAvailable, + }; + + let debug = format!("{auth:?}"); + + assert!(debug.contains("LoadedUpstreamAuth")); + assert!(debug.contains("bearer_token")); + assert!(debug.contains("[redacted]")); + assert!(debug.contains("CodexKeyring")); + assert!(debug.contains("NotAvailable")); + assert!(!debug.contains("sensitive-token")); + } +} diff --git a/src/cli.rs b/src/cli.rs new file mode 100644 index 0000000..eb45967 --- /dev/null +++ b/src/cli.rs @@ -0,0 +1,395 @@ +use clap::{Parser, Subcommand}; + +use crate::config::ThreadlineConfig; + +#[derive(Debug, Clone, Parser, PartialEq, Eq)] +#[command( + name = "threadline", + about = "Bridge VSCode BYOK /v1/responses requests to Codex upstream sessions.", + long_about = "Bridge VSCode BYOK /v1/responses requests to Codex upstream sessions. Run without a subcommand to start the local downstream server." +)] +pub struct ThreadlineCli { + #[command(flatten)] + pub server: ThreadlineConfig, + + #[command(subcommand)] + pub command: Option, +} + +#[derive(Debug, Clone, Subcommand, PartialEq, Eq)] +pub enum ThreadlineCommand { + #[command( + about = "Show sign in guidance for Codex credentials.", + long_about = "Show sign in guidance for Codex credentials. This command provides informational instructions only and does not store, delete, or inspect credentials." + )] + Login, +} + +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum ThreadlineCliAction { + StartServer(ThreadlineConfig), + LoginInstructions, +} + +impl ThreadlineCli { + pub fn into_action(self) -> ThreadlineCliAction { + match self.command { + None => ThreadlineCliAction::StartServer(self.server), + Some(ThreadlineCommand::Login) => ThreadlineCliAction::LoginInstructions, + } + } +} + +#[cfg(test)] +mod login_cli_tests { + use std::fs; + use std::path::PathBuf; + + use clap::{Command, CommandFactory, Parser}; + + use super::{ThreadlineCli, ThreadlineCliAction, ThreadlineCommand}; + + fn removed_model_flag() -> String { + ["--", "model-id"].concat() + } + + fn removed_model_env_var() -> String { + ["THREADLINE", "MODEL", "ID"].join("_") + } + + fn readme_text() -> String { + let readme_path = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("README.md"); + fs::read_to_string(readme_path).expect("readme should be readable") + } + + fn readme_section_containing(readme: &str, needle: &str) -> Option { + let start = readme.find(needle)?; + let section_start = readme[..start] + .rfind("\n\n") + .map(|idx| idx + 2) + .unwrap_or(0); + let section_end = readme[start..] + .find("\n\n") + .map(|idx| start + idx) + .unwrap_or(readme.len()); + + Some(readme[section_start..section_end].to_string()) + } + + fn login_subcommand(command: &Command) -> &Command { + command + .find_subcommand("login") + .expect("login subcommand should exist") + } + + fn command_about_text(command: &Command) -> String { + [command.get_about(), command.get_long_about()] + .into_iter() + .flatten() + .map(|value| value.to_string()) + .collect::>() + .join(" ") + .trim() + .to_string() + } + + #[test] + fn server_starts_by_default_without_subcommand() { + let cli = ThreadlineCli::try_parse_from(["threadline"]).expect("cli should parse"); + + assert!(cli.command.is_none()); + assert!(matches!( + cli.into_action(), + ThreadlineCliAction::StartServer(_) + )); + } + + #[test] + fn config_server_flags_survive_subcommand_refactor() { + let cli = ThreadlineCli::try_parse_from([ + "threadline", + "--host", + "0.0.0.0", + "--port", + "9100", + "--utility-port", + "9101", + "--profile", + "utility", + "--retained-session-capacity", + "9", + "--jobs-enabled", + ]) + .expect("top-level server flags should still parse"); + + assert_eq!(cli.server.host, "0.0.0.0"); + assert_eq!(cli.server.port, 9100); + assert_eq!(cli.server.utility_port, Some(9101)); + assert_eq!(cli.server.profile.to_string(), "utility"); + assert_eq!(cli.server.retained_session_capacity, 9); + assert!(cli.server.jobs_enabled); + } + + #[test] + fn removed_model_id_flag_is_rejected() { + let removed_flag = removed_model_flag(); + + ThreadlineCli::try_parse_from(["threadline", removed_flag.as_str(), "gpt-5.4"]) + .expect_err("removed model-id flag should not parse"); + } + + #[test] + fn clap_surface_excludes_removed_model_configuration() { + let command = ThreadlineCli::command(); + let long_flags: Vec<_> = command + .get_arguments() + .filter_map(|arg| arg.get_long()) + .collect(); + let env_vars: Vec<_> = command + .get_arguments() + .filter_map(|arg| arg.get_env()) + .filter_map(|name| name.to_str()) + .collect(); + let removed_env_var = removed_model_env_var(); + + assert!(!long_flags.contains(&"model-id")); + assert!(!env_vars.contains(&removed_env_var.as_str())); + } + + #[test] + fn readme_lists_only_supported_model_ids_without_model_configuration() { + let readme = readme_text(); + let removed_flag = removed_model_flag(); + let removed_env_var = removed_model_env_var(); + let supported_aliases_section = readme_section_containing(&readme, "Main profile aliases:") + .expect("README should document the supported model alias list"); + let unreleased_caveat = readme_section_containing( + &readme, + "The `gpt-5.6-sol`, `gpt-5.6-terra`, and `gpt-5.6-luna` entries are next models.", + ) + .expect("README should document the GPT-5.6 unreleased caveat"); + let raw_upstream_ids_section = + readme_section_containing(&readme, "The upstream model ids sent to Codex remain") + .expect("README should explain raw upstream model ids"); + let all_turns_caveat = + readme_section_containing(&readme, "Persistent CoT with `reasoning.context=all_turns`") + .expect("README should document the raw compatibility all-turns caveat"); + let custom_endpoint_json = + readme_section_containing(&readme, "\"id\": \"threadline-main-gpt-5.6-sol\"") + .expect("README should include the VS Code custom endpoint JSON example"); + + assert!(!readme.contains(&removed_flag)); + assert!(!readme.contains(&removed_env_var)); + + for visible_alias in [ + "threadline-main-gpt-5.6-sol", + "threadline-main-gpt-5.6-terra", + "threadline-main-gpt-5.6-luna", + ] { + assert!( + supported_aliases_section.contains(visible_alias), + "README should list supported alias {visible_alias} in the Supported Model Aliases section" + ); + assert!( + custom_endpoint_json.contains(visible_alias), + "README should include visible alias {visible_alias} in the VS Code custom endpoint JSON" + ); + } + + for raw_model_id in ["gpt-5.6-sol", "gpt-5.6-terra", "gpt-5.6-luna"] { + assert!( + raw_upstream_ids_section.contains(raw_model_id), + "README should explain raw upstream id {raw_model_id} in the upstream-id section" + ); + assert!( + all_turns_caveat.contains(raw_model_id), + "README should include raw compatibility id {raw_model_id} in the all-turns caveat" + ); + } + + assert!( + raw_upstream_ids_section + .contains("These visible ids are aliases for VS Code selection and routing."), + "README should distinguish visible aliases from raw upstream ids" + ); + assert!( + raw_upstream_ids_section.contains("The upstream model ids sent to Codex remain `gpt-*` ids such as `gpt-5.6-sol`, `gpt-5.6-terra`, `gpt-5.6-luna`"), + "README raw upstream-id explanation should explicitly list the raw gpt-5.6 ids" + ); + assert!( + unreleased_caveat.contains("Threadline currently covers local advertisement, validation, and `model`-field rewriting for those ids."), + "README should describe the GPT-5.6 entries as local-only coverage" + ); + assert!( + unreleased_caveat.contains("Live upstream behavior remains unverified until upstream release makes direct testing possible."), + "README should keep the GPT-5.6 caveat explicitly unverified upstream" + ); + assert!( + !unreleased_caveat.contains("live upstream verification"), + "README should not claim live upstream verification for unreleased GPT-5.6 ids" + ); + + for raw_model_id in ["gpt-5.5", "gpt-5.4", "gpt-5.4-mini", "gpt-5.3-codex-spark"] { + assert!( + raw_upstream_ids_section.contains(raw_model_id), + "README should list supported raw model id {raw_model_id} in the upstream-id section" + ); + } + + for all_turns_raw_model_id in ["gpt-5.5", "gpt-5.4"] { + assert!( + all_turns_caveat.contains(all_turns_raw_model_id), + "README should keep raw compatibility id {all_turns_raw_model_id} in the all-turns caveat" + ); + } + } + + #[test] + fn readme_documents_supported_configuration_flags() { + let readme = readme_text(); + + for (flag, env_var, stable_default) in [ + ("--host", "THREADLINE_HOST", Some("127.0.0.1")), + ("--port", "THREADLINE_PORT", Some("8100")), + ("--utility-port", "THREADLINE_UTILITY_PORT", Some("None")), + ( + "--codex-client-version", + "THREADLINE_CODEX_CLIENT_VERSION", + Some("0.136.0"), + ), + ( + "--retained-session-capacity", + "THREADLINE_RETAINED_SESSION_CAPACITY", + Some("64"), + ), + ("--jobs-enabled", "THREADLINE_JOBS_ENABLED", Some("false")), + ( + "--job-output-buffer-limit-bytes", + "THREADLINE_JOB_OUTPUT_BUFFER_LIMIT_BYTES", + Some("32768"), + ), + ( + "--job-retention-ttl-secs", + "THREADLINE_JOB_RETENTION_TTL_SECS", + Some("300"), + ), + ( + "--job-allowed-commands", + "THREADLINE_JOB_ALLOWED_COMMANDS", + None, + ), + ("--log-level", "THREADLINE_LOG_LEVEL", Some("info")), + ] { + assert!( + readme.contains(flag), + "README should document supported flag {flag}" + ); + assert!( + readme.contains(env_var), + "README should document environment variable {env_var}" + ); + + if let Some(stable_default) = stable_default { + assert!( + readme.contains(stable_default), + "README should document stable default {stable_default} for {flag}" + ); + } + } + + assert!( + readme.contains("comma-separated"), + "README should describe --job-allowed-commands as comma-separated" + ); + assert!( + readme.contains("exact program names"), + "README should describe --job-allowed-commands as exact program names" + ); + let job_allowed_section = readme_section_containing(&readme, "--job-allowed-commands") + .expect("README should describe --job-allowed-commands in one section"); + let normalized_job_allowed_section = job_allowed_section.to_ascii_lowercase(); + assert!( + ![ + "supports prefix matching", + "supports program prefixes", + "allows program prefixes", + "prefixes are allowed", + "matches command prefixes", + ] + .iter() + .any(|phrase| normalized_job_allowed_section.contains(phrase)), + "README should not describe --job-allowed-commands as prefix-based, got section {job_allowed_section:?}" + ); + + let removed_flag = removed_model_flag(); + let removed_env_var = removed_model_env_var(); + assert!(!readme.contains(&removed_flag)); + assert!(!readme.contains(&removed_env_var)); + } + + #[test] + fn readme_recommends_one_process_dual_listener_startup() { + let readme = readme_text(); + + assert!( + readme.contains("threadline --port 8100 --jobs-enabled --utility-port 8101"), + "README should recommend one-process dual-listener startup" + ); + assert!( + readme.contains("second stateless Utility listener in the same process"), + "README should explain that --utility-port starts a second stateless Utility listener in the same process" + ); + assert!( + readme.contains("fallback/debug"), + "README should keep two-process startup documented as fallback/debug guidance" + ); + } + + #[test] + fn login_command_accepts_bare_login_only() { + let cli = + ThreadlineCli::try_parse_from(["threadline", "login"]).expect("login should parse"); + + assert!(matches!(cli.command, Some(ThreadlineCommand::Login))); + assert_eq!(cli.into_action(), ThreadlineCliAction::LoginInstructions); + } + + #[test] + fn login_command_rejects_removed_nested_subcommands() { + for command in [ + ["threadline", "login", "store"], + ["threadline", "login", "status"], + ["threadline", "login", "logout"], + ] { + ThreadlineCli::try_parse_from(command) + .expect_err("removed login subcommand should not parse"); + } + } + + #[test] + fn login_help_describes_informational_credentials_guidance() { + let command = ThreadlineCli::command(); + let login_command = login_subcommand(&command); + let about_text = command_about_text(login_command); + let normalized_about = about_text.to_ascii_lowercase(); + + assert!( + !about_text.is_empty(), + "login subcommand should describe its informational help surface" + ); + assert!( + normalized_about.contains("sign in") || normalized_about.contains("login"), + "login help should mention sign-in guidance, got {about_text:?}" + ); + assert!( + normalized_about.contains("instruction") + || normalized_about.contains("guidance") + || normalized_about.contains("information"), + "login help should describe informational-only guidance, got {about_text:?}" + ); + assert!( + normalized_about.contains("does not") || normalized_about.contains("without storing"), + "login help should avoid implying credential storage behavior, got {about_text:?}" + ); + } +} diff --git a/src/codex_ws.rs b/src/codex_ws.rs new file mode 100644 index 0000000..6a68e85 --- /dev/null +++ b/src/codex_ws.rs @@ -0,0 +1,194 @@ +use thiserror::Error; +use tokio_tungstenite::tungstenite::{ + client::IntoClientRequest, + http::{HeaderValue, Request}, +}; +use uuid::Uuid; + +use crate::auth::LoadedUpstreamAuth; + +pub const RESPONSES_WEBSOCKETS_BETA_HEADER: &str = "responses_websockets=2026-02-06"; +#[cfg(test)] +const EXPECTED_CODEX_CLIENT_VERSION: &str = "0.136.0"; + +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct UpstreamSessionDescriptor { + pub session_id: String, + pub thread_id: String, + pub window_id: String, + pub turn_state: Option, +} + +impl UpstreamSessionDescriptor { + pub fn refresh_window(&mut self) { + self.window_id = new_request_id(); + } +} + +#[derive(Debug)] +pub struct CodexHandshake { + pub request: Request<()>, + pub session: UpstreamSessionDescriptor, + pub client_request_id: String, +} + +#[derive(Debug, Error)] +pub enum HandshakeBuildError { + #[error("Threadline could not build the upstream websocket request.")] + RequestBuildFailed, +} + +pub fn build_handshake_request( + url: &str, + auth: &LoadedUpstreamAuth, + codex_client_version: &str, + session: Option, +) -> Result { + let session = session.unwrap_or_else(|| UpstreamSessionDescriptor { + session_id: new_request_id(), + thread_id: new_request_id(), + window_id: new_request_id(), + turn_state: None, + }); + let client_request_id = new_request_id(); + + let mut request = url + .into_client_request() + .map_err(|_| HandshakeBuildError::RequestBuildFailed)?; + let headers = request.headers_mut(); + + headers.insert( + "authorization", + header_value(&format!("Bearer {}", auth.bearer_token))?, + ); + headers.insert( + "OpenAI-Beta", + header_value(RESPONSES_WEBSOCKETS_BETA_HEADER)?, + ); + headers.insert("originator", header_value("codex_vscode")?); + headers.insert( + "user-agent", + header_value(&format!( + "codex_vscode/{codex_client_version} Threadline/{}", + env!("CARGO_PKG_VERSION") + ))?, + ); + headers.insert("version", header_value(codex_client_version)?); + headers.insert("session-id", header_value(&session.session_id)?); + headers.insert("thread-id", header_value(&session.thread_id)?); + headers.insert("x-codex-window-id", header_value(&session.window_id)?); + headers.insert("x-client-request-id", header_value(&client_request_id)?); + + if let Some(turn_state) = &session.turn_state { + headers.insert("x-codex-turn-state", header_value(turn_state)?); + } + + Ok(CodexHandshake { + request, + session, + client_request_id, + }) +} + +fn header_value(value: &str) -> Result { + HeaderValue::from_str(value).map_err(|_| HandshakeBuildError::RequestBuildFailed) +} + +fn new_request_id() -> String { + Uuid::now_v7().to_string() +} + +#[cfg(test)] +mod tests { + use uuid::Uuid; + + use crate::auth::{AuthSource, LoadedUpstreamAuth, RefreshBoundary}; + + use super::{ + EXPECTED_CODEX_CLIENT_VERSION, HandshakeBuildError, RESPONSES_WEBSOCKETS_BETA_HEADER, + UpstreamSessionDescriptor, build_handshake_request, + }; + + fn test_auth() -> LoadedUpstreamAuth { + LoadedUpstreamAuth { + bearer_token: "top-secret-token".to_string(), + source: AuthSource::CodexKeyring, + refresh_boundary: RefreshBoundary::NotAvailable, + } + } + + #[test] + fn handshake_generates_required_headers_and_identifiers() { + let handshake = build_handshake_request( + "ws://localhost:9001/codex", + &test_auth(), + EXPECTED_CODEX_CLIENT_VERSION, + None, + ) + .expect("handshake should build"); + let headers = handshake.request.headers(); + + assert_eq!( + handshake.request.uri().to_string(), + "ws://localhost:9001/codex" + ); + assert_eq!(headers["connection"], "Upgrade"); + assert_eq!(headers["upgrade"], "websocket"); + assert!(headers.get("sec-websocket-key").is_some()); + assert_eq!(headers["sec-websocket-version"], "13"); + assert_eq!(headers["authorization"], "Bearer top-secret-token"); + assert_eq!(headers["openai-beta"], RESPONSES_WEBSOCKETS_BETA_HEADER); + assert_eq!(headers["originator"], "codex_vscode"); + assert_eq!( + headers["user-agent"], + format!( + "codex_vscode/{EXPECTED_CODEX_CLIENT_VERSION} Threadline/{}", + env!("CARGO_PKG_VERSION") + ) + ); + assert_eq!(headers["version"], EXPECTED_CODEX_CLIENT_VERSION); + Uuid::parse_str(headers["session-id"].to_str().unwrap()).expect("session id uuid"); + Uuid::parse_str(headers["thread-id"].to_str().unwrap()).expect("thread id uuid"); + Uuid::parse_str(headers["x-codex-window-id"].to_str().unwrap()).expect("window id uuid"); + Uuid::parse_str(headers["x-client-request-id"].to_str().unwrap()).expect("request id uuid"); + assert!(headers.get("x-codex-turn-state").is_none()); + } + + #[test] + fn handshake_reuses_supplied_session_context_and_turn_state() { + let session = UpstreamSessionDescriptor { + session_id: "session-123".to_string(), + thread_id: "thread-456".to_string(), + window_id: "window-789".to_string(), + turn_state: Some("turn-state-abc".to_string()), + }; + + let handshake = build_handshake_request( + "wss://example.invalid/upstream", + &test_auth(), + EXPECTED_CODEX_CLIENT_VERSION, + Some(session.clone()), + ) + .expect("handshake should build"); + let headers = handshake.request.headers(); + + assert_eq!(headers["session-id"], session.session_id); + assert_eq!(headers["thread-id"], session.thread_id); + assert_eq!(headers["x-codex-window-id"], session.window_id); + assert_eq!(headers["x-codex-turn-state"], "turn-state-abc"); + assert_ne!(headers["x-client-request-id"], "turn-state-abc"); + } + + #[test] + fn handshake_rejects_invalid_upstream_url() { + let error = build_handshake_request( + "not a websocket url", + &test_auth(), + EXPECTED_CODEX_CLIENT_VERSION, + None, + ) + .expect_err("invalid url should fail"); + + assert!(matches!(error, HandshakeBuildError::RequestBuildFailed)); + } +} diff --git a/src/config.rs b/src/config.rs new file mode 100644 index 0000000..d646b53 --- /dev/null +++ b/src/config.rs @@ -0,0 +1,480 @@ +use std::net::{IpAddr, SocketAddr}; +use std::sync::{LazyLock, Mutex}; +use std::time::Duration; + +use clap::{Args, Parser}; + +use crate::jobs::ThreadlineJobManagerConfig; +use crate::models::RouteProfile; + +const DEFAULT_HOST: &str = "127.0.0.1"; +const DEFAULT_PORT: u16 = 8100; +const DEFAULT_PROFILE: RouteProfile = RouteProfile::Main; +const DEFAULT_CODEX_CLIENT_VERSION: &str = "0.136.0"; +const DEFAULT_RETAINED_SESSION_CAPACITY: usize = 64; +const DEFAULT_JOBS_ENABLED: bool = false; +const DEFAULT_JOB_OUTPUT_BUFFER_LIMIT_BYTES: usize = 32 * 1024; +const DEFAULT_JOB_RETENTION_TTL_SECS: u64 = 300; +const DEFAULT_LOG_LEVEL: &str = "info"; + +static ACTIVE_JOB_MANAGER_CONFIG: LazyLock> = + LazyLock::new(|| Mutex::new(ThreadlineJobManagerConfig::default())); + +#[derive(Debug, Clone, Args, PartialEq, Eq)] +pub struct ThreadlineConfig { + #[arg( + long, + env = "THREADLINE_HOST", + default_value = DEFAULT_HOST, + value_name = "IP_ADDRESS", + help = "Listen address for the downstream HTTP server.", + long_help = "Listen address for the downstream HTTP server. Use an IP address that Threadline should bind for local /v1/responses requests." + )] + pub host: String, + + #[arg( + long, + env = "THREADLINE_PORT", + default_value_t = DEFAULT_PORT, + value_name = "PORT", + help = "Listen port for the downstream HTTP server.", + long_help = "Listen port for the downstream HTTP server. This controls which local TCP port accepts /v1/responses requests." + )] + pub port: u16, + + #[arg( + long, + env = "THREADLINE_UTILITY_PORT", + value_name = "PORT", + help = "Optional port for a second utility listener.", + long_help = "Optional port for a second utility listener. When set, Threadline can start a separate utility-profile listener on this port in addition to the main listener." + )] + pub utility_port: Option, + + #[arg( + long, + env = "THREADLINE_PROFILE", + default_value_t = DEFAULT_PROFILE, + value_name = "PROFILE", + help = "Route profile that controls advertised model aliases.", + long_help = "Route profile that controls advertised model aliases. Use main for retained-session routes and utility for utility-only model advertisement on this listener." + )] + pub profile: RouteProfile, + + #[arg( + long, + env = "THREADLINE_CODEX_CLIENT_VERSION", + default_value = DEFAULT_CODEX_CLIENT_VERSION, + value_name = "VERSION", + help = "Codex client version sent to the upstream backend.", + long_help = "Codex client version sent to the upstream backend. Set this when Threadline must match the Codex client version expected by the backend." + )] + pub codex_client_version: String, + + #[arg( + long, + env = "THREADLINE_RETAINED_SESSION_CAPACITY", + default_value_t = DEFAULT_RETAINED_SESSION_CAPACITY, + value_name = "COUNT", + help = "Maximum retained session capacity for response continuation.", + long_help = "Maximum retained session capacity for response continuation. Higher values allow more completed response markers to keep a retained session available for follow-up requests." + )] + pub retained_session_capacity: usize, + + #[arg( + long, + env = "THREADLINE_JOBS_ENABLED", + default_value_t = DEFAULT_JOBS_ENABLED, + value_name = "BOOL", + help = "Enable local job execution support.", + long_help = "Enable local job execution support. When enabled, Threadline may expose job tools for long-running local work instead of blocking a response." + )] + pub jobs_enabled: bool, + + #[arg( + long, + env = "THREADLINE_JOB_OUTPUT_BUFFER_LIMIT_BYTES", + default_value_t = DEFAULT_JOB_OUTPUT_BUFFER_LIMIT_BYTES, + value_name = "BYTES", + help = "Maximum buffered job output in bytes.", + long_help = "Maximum buffered job output in bytes. Older job output is dropped once the retained in-memory job output buffer reaches this byte limit." + )] + pub job_output_buffer_limit_bytes: usize, + + #[arg( + long, + env = "THREADLINE_JOB_RETENTION_TTL_SECS", + default_value_t = DEFAULT_JOB_RETENTION_TTL_SECS, + value_name = "SECONDS", + help = "Job retention time in seconds after completion.", + long_help = "Job retention time in seconds after completion. Finished job metadata and buffered output remain available until this retention window expires." + )] + pub job_retention_ttl_secs: u64, + + #[arg( + long, + env = "THREADLINE_JOB_ALLOWED_COMMANDS", + value_name = "PROGRAMS", + help = "Comma-separated exact program names allowed for jobs.", + long_help = "Comma-separated exact executable or program names allowed for jobs. Each configured entry is matched against the requested program name exactly." + )] + pub job_allowed_commands: Option, + + #[arg( + long, + env = "THREADLINE_LOG_LEVEL", + default_value = DEFAULT_LOG_LEVEL, + value_name = "LEVEL", + help = "Log verbosity for Threadline diagnostics.", + long_help = "Log verbosity for Threadline diagnostics. Use standard Rust tracing levels such as error, warn, info, debug, or trace." + )] + pub log_level: String, +} + +impl Default for ThreadlineConfig { + fn default() -> Self { + let config = Self { + host: DEFAULT_HOST.to_string(), + port: DEFAULT_PORT, + utility_port: None, + profile: DEFAULT_PROFILE, + codex_client_version: DEFAULT_CODEX_CLIENT_VERSION.to_string(), + retained_session_capacity: DEFAULT_RETAINED_SESSION_CAPACITY, + jobs_enabled: DEFAULT_JOBS_ENABLED, + job_output_buffer_limit_bytes: DEFAULT_JOB_OUTPUT_BUFFER_LIMIT_BYTES, + job_retention_ttl_secs: DEFAULT_JOB_RETENTION_TTL_SECS, + job_allowed_commands: None, + log_level: DEFAULT_LOG_LEVEL.to_string(), + }; + set_active_job_manager_config(config.job_manager_config()); + config + } +} + +impl ThreadlineConfig { + pub fn from_env() -> Self { + let config = crate::cli::ThreadlineCli::parse().server; + set_active_job_manager_config(config.job_manager_config()); + config + } + + pub fn bind_address(&self) -> Result { + let host: IpAddr = self.host.parse()?; + Ok(SocketAddr::from((host, self.port))) + } + + pub fn job_manager_config(&self) -> ThreadlineJobManagerConfig { + ThreadlineJobManagerConfig { + jobs_enabled: self.jobs_enabled, + output_buffer_limit_bytes: self.job_output_buffer_limit_bytes, + retention_ttl: Duration::from_secs(self.job_retention_ttl_secs), + allowed_commands: split_allowed_commands(self.job_allowed_commands.as_deref()), + } + } +} + +pub fn job_manager_config_from_environment() -> ThreadlineJobManagerConfig { + ThreadlineJobManagerConfig { + jobs_enabled: read_bool_env("THREADLINE_JOBS_ENABLED", DEFAULT_JOBS_ENABLED), + output_buffer_limit_bytes: read_usize_env( + "THREADLINE_JOB_OUTPUT_BUFFER_LIMIT_BYTES", + DEFAULT_JOB_OUTPUT_BUFFER_LIMIT_BYTES, + ), + retention_ttl: Duration::from_secs(read_u64_env( + "THREADLINE_JOB_RETENTION_TTL_SECS", + DEFAULT_JOB_RETENTION_TTL_SECS, + )), + allowed_commands: split_allowed_commands( + std::env::var("THREADLINE_JOB_ALLOWED_COMMANDS") + .ok() + .as_deref(), + ), + } +} + +pub fn active_job_manager_config() -> ThreadlineJobManagerConfig { + ACTIVE_JOB_MANAGER_CONFIG + .lock() + .expect("job manager config lock") + .clone() +} + +fn read_bool_env(name: &str, default: bool) -> bool { + std::env::var(name) + .ok() + .and_then(|value| value.parse::().ok()) + .unwrap_or(default) +} + +fn read_usize_env(name: &str, default: usize) -> usize { + std::env::var(name) + .ok() + .and_then(|value| value.parse::().ok()) + .unwrap_or(default) +} + +fn read_u64_env(name: &str, default: u64) -> u64 { + std::env::var(name) + .ok() + .and_then(|value| value.parse::().ok()) + .unwrap_or(default) +} + +fn split_allowed_commands(value: Option<&str>) -> Vec { + value + .into_iter() + .flat_map(|commands| commands.split(',')) + .map(str::trim) + .filter(|command| !command.is_empty()) + .map(ToString::to_string) + .collect() +} + +fn set_active_job_manager_config(config: ThreadlineJobManagerConfig) { + *ACTIVE_JOB_MANAGER_CONFIG + .lock() + .expect("job manager config lock") = config; +} + +#[cfg(test)] +mod tests { + use std::ffi::OsString; + use std::sync::Mutex; + + use clap::{Arg, Command, CommandFactory, Parser}; + + use crate::cli::ThreadlineCli; + use crate::models::RouteProfile; + + use super::{DEFAULT_CODEX_CLIENT_VERSION, ThreadlineConfig}; + + static THREADLINE_PROFILE_ENV_LOCK: Mutex<()> = Mutex::new(()); + static THREADLINE_UTILITY_PORT_ENV_LOCK: Mutex<()> = Mutex::new(()); + + struct ProfileEnvGuard { + original: Option, + } + + impl ProfileEnvGuard { + fn acquire() -> Self { + Self { + original: std::env::var_os("THREADLINE_PROFILE"), + } + } + } + + impl Drop for ProfileEnvGuard { + fn drop(&mut self) { + match self.original.take() { + Some(value) => unsafe { std::env::set_var("THREADLINE_PROFILE", value) }, + None => unsafe { std::env::remove_var("THREADLINE_PROFILE") }, + } + } + } + + struct UtilityPortEnvGuard { + original: Option, + } + + impl UtilityPortEnvGuard { + fn acquire() -> Self { + Self { + original: std::env::var_os("THREADLINE_UTILITY_PORT"), + } + } + } + + impl Drop for UtilityPortEnvGuard { + fn drop(&mut self) { + match self.original.take() { + Some(value) => unsafe { std::env::set_var("THREADLINE_UTILITY_PORT", value) }, + None => unsafe { std::env::remove_var("THREADLINE_UTILITY_PORT") }, + } + } + } + + fn arg_by_long_flag<'a>(command: &'a Command, long_flag: &str) -> &'a Arg { + command + .get_arguments() + .find(|arg| arg.get_long() == Some(long_flag)) + .unwrap_or_else(|| panic!("expected --{long_flag} to exist on ThreadlineCli")) + } + + fn argument_help_text(argument: &Arg) -> String { + [argument.get_help(), argument.get_long_help()] + .into_iter() + .flatten() + .map(|value| value.to_string()) + .collect::>() + .join(" ") + .trim() + .to_string() + } + + fn assert_help_mentions(argument: &Arg, long_flag: &str, expected_terms: &[&str]) { + let help_text = argument_help_text(argument); + let normalized_help = help_text.to_ascii_lowercase(); + let generated_flag_label = long_flag.to_ascii_lowercase(); + let generated_phrase_label = long_flag.replace('-', " ").to_ascii_lowercase(); + + assert!( + !help_text.is_empty(), + "expected --{long_flag} to have help or long_help text" + ); + assert!( + help_text.len() > long_flag.len() + 12, + "expected --{long_flag} help to be descriptive, got {help_text:?}" + ); + assert!( + normalized_help != generated_flag_label && normalized_help != generated_phrase_label, + "expected --{long_flag} help to add semantics beyond generated-only flag text, got {help_text:?}" + ); + + for term in expected_terms { + assert!( + normalized_help.contains(term), + "expected --{long_flag} help to mention {term:?}, got {help_text:?}" + ); + } + } + + #[test] + fn codex_client_version_defaults_to_installed_version() { + let config = ThreadlineCli::parse_from(["threadline"]).server; + let command = ThreadlineCli::command(); + let argument = command + .get_arguments() + .find(|arg| arg.get_long() == Some("codex-client-version")) + .expect("codex client version arg should exist"); + let default_values: Vec<_> = argument + .get_default_values() + .iter() + .map(|value| value.to_str().expect("utf-8 default value")) + .collect(); + + assert_eq!(config.codex_client_version, DEFAULT_CODEX_CLIENT_VERSION); + assert_eq!(default_values, vec![DEFAULT_CODEX_CLIENT_VERSION]); + } + + #[test] + fn codex_client_version_cli_override_wins() { + let config = + ThreadlineCli::try_parse_from(["threadline", "--codex-client-version", "9.9.9"]) + .expect("threadline config should accept a codex client version cli override") + .server; + + assert_eq!(config.codex_client_version, "9.9.9"); + } + + #[test] + fn cli_flag_help_describes_supported_configuration() { + let command = ThreadlineCli::command(); + + for (long_flag, expected_terms) in [ + ("host", &["listen", "address"][..]), + ("port", &["listen", "port"][..]), + ("utility-port", &["utility", "listener", "port"][..]), + ("profile", &["profile", "main", "utility"][..]), + ("codex-client-version", &["codex", "client version"][..]), + ( + "retained-session-capacity", + &["retained session", "capacity"][..], + ), + ("jobs-enabled", &["job", "enable"][..]), + ( + "job-output-buffer-limit-bytes", + &["job output", "bytes"][..], + ), + ( + "job-retention-ttl-secs", + &["job", "retention", "seconds"][..], + ), + ( + "job-allowed-commands", + &["comma-separated", "exact", "program"][..], + ), + ("log-level", &["log", "verbosity"][..]), + ] { + let argument = arg_by_long_flag(&command, long_flag); + assert_help_mentions(argument, long_flag, expected_terms); + } + } + + #[test] + fn profile_defaults_to_main() { + let _lock = THREADLINE_PROFILE_ENV_LOCK + .lock() + .expect("profile env lock"); + let _guard = ProfileEnvGuard::acquire(); + unsafe { std::env::remove_var("THREADLINE_PROFILE") }; + + let config = ThreadlineCli::parse_from(["threadline"]).server; + let command = ThreadlineCli::command(); + let argument = arg_by_long_flag(&command, "profile"); + let default_values: Vec<_> = argument + .get_default_values() + .iter() + .map(|value| value.to_str().expect("utf-8 default value")) + .collect(); + + assert_eq!(config.profile, RouteProfile::Main); + assert_eq!(default_values, vec!["main"]); + } + + #[test] + fn profile_accepts_explicit_utility_value() { + let config = ThreadlineCli::try_parse_from(["threadline", "--profile", "utility"]) + .expect("threadline config should accept utility profile") + .server; + + assert_eq!(config.profile, RouteProfile::Utility); + } + + #[test] + fn profile_rejects_invalid_value() { + ThreadlineCli::try_parse_from(["threadline", "--profile", "invalid"]) + .expect_err("threadline config should reject invalid profiles"); + } + + #[test] + fn profile_reads_threadline_profile_env_var() { + let _lock = THREADLINE_PROFILE_ENV_LOCK + .lock() + .expect("profile env lock"); + let _guard = ProfileEnvGuard::acquire(); + unsafe { std::env::set_var("THREADLINE_PROFILE", "utility") }; + + let config = ThreadlineCli::parse_from(["threadline"]).server; + + assert_eq!(config.profile, RouteProfile::Utility); + } + + #[test] + fn utility_port_defaults_to_none() { + let config = ThreadlineConfig::default(); + + assert_eq!(config.utility_port, None); + } + + #[test] + fn utility_port_accepts_cli_value() { + let config = ThreadlineCli::try_parse_from(["threadline", "--utility-port", "8101"]) + .expect("threadline config should accept a utility port cli override") + .server; + + assert_eq!(config.utility_port, Some(8101)); + } + + #[test] + fn utility_port_reads_threadline_utility_port_env_var() { + let _lock = THREADLINE_UTILITY_PORT_ENV_LOCK + .lock() + .expect("utility port env lock"); + let _guard = UtilityPortEnvGuard::acquire(); + unsafe { std::env::set_var("THREADLINE_UTILITY_PORT", "8101") }; + + let config = ThreadlineCli::parse_from(["threadline"]).server; + + assert_eq!(config.utility_port, Some(8101)); + } +} diff --git a/src/errors.rs b/src/errors.rs new file mode 100644 index 0000000..abafbc3 --- /dev/null +++ b/src/errors.rs @@ -0,0 +1,376 @@ +use std::borrow::Cow; + +use axum::Json; +use axum::http::StatusCode; +use axum::response::{IntoResponse, Response}; +use serde::Serialize; +use thiserror::Error; + +#[derive(Debug, Serialize)] +pub struct PublicErrorDocument { + pub error: PublicErrorPayload, +} + +#[derive(Debug, Serialize)] +pub struct PublicErrorPayload { + pub code: Cow<'static, str>, + pub message: Cow<'static, str>, + #[serde(rename = "type")] + pub error_type: Cow<'static, str>, +} + +#[derive(Debug, Error)] +pub enum ThreadlineError { + #[error("The /v1/responses bridge is not available yet.")] + ResponsesNotReady, + + #[error("The /v1/responses request body was not a valid JSON object.")] + InvalidResponsesRequest, + + #[error("The /v1/responses request must include a supported string model.")] + InvalidModel, + + #[error( + "reasoning.context=all_turns is not supported for this model. The model metadata has use_responses_lite=false." + )] + UnsupportedReasoningContext, + + #[error( + "Threadline could not find the retained session for the supplied previous_response_id." + )] + PreviousResponseNotFound, + + #[error("The retained session for this previous_response_id is already in use.")] + RetainedSessionConflict, + + #[error("Threadline has no free retained session capacity for another active response.")] + RetainedSessionCapacityExceeded, + + #[error("Threadline could not connect to the upstream Codex websocket.")] + UpstreamWebSocketConnectFailed, + + #[error("The upstream Codex websocket handshake was rejected with HTTP {status}.")] + UpstreamWebSocketHandshakeRejected { status: StatusCode }, + + #[error( + "The upstream Codex websocket closed before Threadline finished streaming the response." + )] + UpstreamWebSocketClosed, + + #[error( + "The upstream response.failed event cannot be streamed as a successful downstream response." + )] + UpstreamResponseFailed, + + #[error("The upstream websocket emitted an error event.")] + UpstreamErrorEvent, + + #[error("The upstream websocket emitted malformed JSON.")] + UpstreamInvalidJson, + + #[error("Threadline failed while executing an internal tool.")] + InternalToolFailed, + + #[error("Threadline could not find a job with that job_id.")] + JobNotFound, + + #[error("Threadline jobs are disabled.")] + JobsDisabled, + + #[error("The requested job command is not allowed by Threadline policy.")] + JobCommandNotAllowed, + + #[error("The Threadline job command failed.")] + JobCommandFailed, + + #[error("The Threadline job was cancelled.")] + JobCancelled, + + #[error("Threadline could not load upstream credentials.")] + UpstreamCredentialsUnavailable, + + #[error("Threadline is missing THREADLINE_UPSTREAM_URL for upstream websocket connections.")] + UpstreamUrlMissing, + + #[error("{0}")] + InvalidServerConfiguration(String), + + #[error("Invalid bind host: {0}")] + InvalidBindHost(String), +} + +impl ThreadlineError { + pub fn is_upstream_recoverable_close(&self) -> bool { + matches!( + self, + Self::UpstreamWebSocketClosed | Self::UpstreamWebSocketConnectFailed + ) + } + + pub fn status_code(&self) -> StatusCode { + match self { + Self::ResponsesNotReady => StatusCode::NOT_IMPLEMENTED, + Self::InvalidResponsesRequest => StatusCode::BAD_REQUEST, + Self::InvalidModel => StatusCode::BAD_REQUEST, + Self::UnsupportedReasoningContext => StatusCode::BAD_REQUEST, + Self::PreviousResponseNotFound => StatusCode::BAD_REQUEST, + Self::RetainedSessionConflict => StatusCode::CONFLICT, + Self::RetainedSessionCapacityExceeded => StatusCode::SERVICE_UNAVAILABLE, + Self::UpstreamWebSocketConnectFailed => StatusCode::BAD_GATEWAY, + Self::UpstreamWebSocketHandshakeRejected { status } => *status, + Self::UpstreamWebSocketClosed => StatusCode::BAD_GATEWAY, + Self::UpstreamResponseFailed => StatusCode::BAD_GATEWAY, + Self::UpstreamErrorEvent => StatusCode::BAD_GATEWAY, + Self::UpstreamInvalidJson => StatusCode::BAD_GATEWAY, + Self::InternalToolFailed => StatusCode::INTERNAL_SERVER_ERROR, + Self::JobNotFound => StatusCode::NOT_FOUND, + Self::JobsDisabled => StatusCode::FORBIDDEN, + Self::JobCommandNotAllowed => StatusCode::FORBIDDEN, + Self::JobCommandFailed => StatusCode::INTERNAL_SERVER_ERROR, + Self::JobCancelled => StatusCode::CONFLICT, + Self::UpstreamCredentialsUnavailable => StatusCode::INTERNAL_SERVER_ERROR, + Self::UpstreamUrlMissing => StatusCode::INTERNAL_SERVER_ERROR, + Self::InvalidServerConfiguration(_) => StatusCode::INTERNAL_SERVER_ERROR, + Self::InvalidBindHost(_) => StatusCode::INTERNAL_SERVER_ERROR, + } + } + + pub fn public_error(&self) -> PublicErrorPayload { + match self { + Self::ResponsesNotReady => borrowed_public_error( + "responses_not_ready", + "The /v1/responses bridge is not available yet.", + "not_implemented_error", + ), + Self::InvalidResponsesRequest => borrowed_public_error( + "invalid_request_error", + "The /v1/responses request body must be a JSON object.", + "invalid_request_error", + ), + Self::InvalidModel => borrowed_public_error( + "invalid_model", + "The /v1/responses request must include a supported string model.", + "invalid_request_error", + ), + Self::UnsupportedReasoningContext => borrowed_public_error( + "unsupported_reasoning_context", + "reasoning.context=all_turns is not supported for this model. The model metadata has use_responses_lite=false.", + "invalid_request_error", + ), + Self::PreviousResponseNotFound => borrowed_public_error( + "previous_response_not_found", + "Threadline could not find the retained session for that previous_response_id.", + "invalid_request_error", + ), + Self::RetainedSessionConflict => borrowed_public_error( + "retained_session_conflict", + "The retained session for that previous_response_id is already active.", + "conflict_error", + ), + Self::RetainedSessionCapacityExceeded => borrowed_public_error( + "retained_session_capacity_exceeded", + "Threadline has no free retained session capacity for another active response.", + "service_unavailable_error", + ), + Self::UpstreamWebSocketConnectFailed => borrowed_public_error( + "upstream_websocket_connect_failed", + "Threadline could not connect to the upstream Codex websocket.", + "bad_gateway_error", + ), + Self::UpstreamWebSocketHandshakeRejected { status } => PublicErrorPayload { + code: Cow::Borrowed("upstream_websocket_handshake_rejected"), + message: Cow::Owned(format_upstream_websocket_handshake_rejected_message( + *status, + )), + error_type: Cow::Borrowed("bad_gateway_error"), + }, + Self::UpstreamWebSocketClosed => borrowed_public_error( + "upstream_websocket_closed", + "The upstream Codex websocket closed before Threadline finished streaming the response.", + "bad_gateway_error", + ), + Self::UpstreamResponseFailed => borrowed_public_error( + "upstream_response_failed", + "The upstream response.failed event cannot be streamed as a successful downstream response.", + "bad_gateway_error", + ), + Self::UpstreamErrorEvent => borrowed_public_error( + "upstream_error_event", + "The upstream websocket emitted an error event.", + "bad_gateway_error", + ), + Self::UpstreamInvalidJson => borrowed_public_error( + "upstream_invalid_json", + "The upstream websocket emitted malformed JSON.", + "bad_gateway_error", + ), + Self::InternalToolFailed => borrowed_public_error( + "internal_tool_failed", + "Threadline failed while executing an internal tool.", + "internal_server_error", + ), + Self::JobNotFound => borrowed_public_error( + "job_not_found", + "Threadline could not find a job with that job_id.", + "invalid_request_error", + ), + Self::JobsDisabled => borrowed_public_error( + "jobs_disabled", + "Threadline jobs are disabled.", + "forbidden_error", + ), + Self::JobCommandNotAllowed => borrowed_public_error( + "job_command_not_allowed", + "The requested job command is not allowed by Threadline policy.", + "forbidden_error", + ), + Self::JobCommandFailed => borrowed_public_error( + "job_command_failed", + "The Threadline job command failed.", + "internal_server_error", + ), + Self::JobCancelled => borrowed_public_error( + "job_cancelled", + "The Threadline job was cancelled.", + "conflict_error", + ), + Self::UpstreamCredentialsUnavailable => borrowed_public_error( + "upstream_credentials_unavailable", + "Threadline could not load upstream credentials.", + "configuration_error", + ), + Self::UpstreamUrlMissing => borrowed_public_error( + "configuration_error", + "Threadline is missing THREADLINE_UPSTREAM_URL for upstream websocket connections.", + "configuration_error", + ), + Self::InvalidServerConfiguration(message) => PublicErrorPayload { + code: Cow::Borrowed("configuration_error"), + message: Cow::Owned(message.clone()), + error_type: Cow::Borrowed("configuration_error"), + }, + Self::InvalidBindHost(_) => borrowed_public_error( + "configuration_error", + "Threadline failed to resolve its configured bind address.", + "configuration_error", + ), + } + } + + pub fn public_error_document(&self) -> PublicErrorDocument { + PublicErrorDocument { + error: self.public_error(), + } + } +} + +fn borrowed_public_error( + code: &'static str, + message: &'static str, + error_type: &'static str, +) -> PublicErrorPayload { + PublicErrorPayload { + code: Cow::Borrowed(code), + message: Cow::Borrowed(message), + error_type: Cow::Borrowed(error_type), + } +} + +fn format_upstream_websocket_handshake_rejected_message(status: StatusCode) -> String { + match status.canonical_reason() { + Some(reason) => format!( + "The upstream Codex websocket handshake was rejected with HTTP {} {}.", + status.as_u16(), + reason + ), + None => format!( + "The upstream Codex websocket handshake was rejected with HTTP {}.", + status.as_u16() + ), + } +} + +impl IntoResponse for ThreadlineError { + fn into_response(self) -> Response { + let status = self.status_code(); + let payload = self.public_error_document(); + + (status, Json(payload)).into_response() + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn upstream_websocket_handshake_rejected_uses_upstream_status() { + let error = ThreadlineError::UpstreamWebSocketHandshakeRejected { + status: StatusCode::FORBIDDEN, + }; + + assert_eq!(error.status_code(), StatusCode::FORBIDDEN); + + let document = error.public_error_document(); + + assert_eq!( + document.error.code.as_ref(), + "upstream_websocket_handshake_rejected" + ); + assert_eq!( + document.error.message.as_ref(), + "The upstream Codex websocket handshake was rejected with HTTP 403 Forbidden." + ); + assert_eq!(document.error.error_type.as_ref(), "bad_gateway_error"); + } + + #[test] + fn upstream_websocket_handshake_rejected_propagates_exact_server_error_status() { + let error = ThreadlineError::UpstreamWebSocketHandshakeRejected { + status: StatusCode::SERVICE_UNAVAILABLE, + }; + + assert_eq!(error.status_code(), StatusCode::SERVICE_UNAVAILABLE); + assert_eq!( + error.public_error_document().error.message.as_ref(), + "The upstream Codex websocket handshake was rejected with HTTP 503 Service Unavailable." + ); + } + + #[test] + fn invalid_server_configuration_maps_to_configuration_error_with_original_message() { + let error = ThreadlineError::InvalidServerConfiguration( + "--utility-port must differ from --port".to_string(), + ); + + assert_eq!(error.status_code(), StatusCode::INTERNAL_SERVER_ERROR); + + let document = error.public_error_document(); + + assert_eq!(document.error.code.as_ref(), "configuration_error"); + assert_eq!( + document.error.message.as_ref(), + "--utility-port must differ from --port" + ); + assert_eq!(document.error.error_type.as_ref(), "configuration_error"); + } + + #[test] + fn unsupported_reasoning_context_maps_to_stable_invalid_request_error() { + let error = ThreadlineError::UnsupportedReasoningContext; + + assert_eq!(error.status_code(), StatusCode::BAD_REQUEST); + + let document = error.public_error_document(); + + assert_eq!( + document.error.code.as_ref(), + "unsupported_reasoning_context" + ); + assert_eq!( + document.error.message.as_ref(), + "reasoning.context=all_turns is not supported for this model. The model metadata has use_responses_lite=false." + ); + assert_eq!(document.error.error_type.as_ref(), "invalid_request_error"); + } +} diff --git a/src/http.rs b/src/http.rs new file mode 100644 index 0000000..c3032f3 --- /dev/null +++ b/src/http.rs @@ -0,0 +1,366 @@ +use std::sync::Arc; + +use axum::extract::State; +use axum::http::HeaderMap; +use axum::routing::{get, post}; +use axum::{Json, Router}; +use futures_util::future::BoxFuture; +use serde::Serialize; +use serde_json::Value; +use tokio_tungstenite::connect_async; +use tokio_tungstenite::tungstenite::Error as TungsteniteError; +use tracing::warn; + +use crate::auth::{AuthDiscoveryOptions, load_upstream_auth}; +use crate::codex_ws::build_handshake_request; +use crate::config::ThreadlineConfig; +use crate::errors::ThreadlineError; +use crate::models::{RouteProfile, advertised_model_ids_for_profile}; +use crate::registry::RetainedSessionRegistry; +use crate::responses::{ + ConnectedUpstream, DownstreamRequestMetadata, ResponsesRouteState, ThreadlineServices, + responses_handler, +}; +use crate::ws_pump::LiveUpstreamWebSocket; + +const MODEL_CREATED_UNSPECIFIED: u64 = 0; +const DEFAULT_UPSTREAM_URL: &str = "wss://chatgpt.com/backend-api/codex/responses"; +const INTERACTION_TYPE_HEADER: &str = "x-interaction-type"; + +#[derive(Clone)] +struct AppState { + profile: RouteProfile, + responses: ResponsesRouteState, +} + +#[derive(Serialize)] +struct HealthPayload { + status: &'static str, + service: &'static str, +} + +#[derive(Serialize)] +struct ModelListPayload { + object: &'static str, + data: Vec, +} + +#[derive(Serialize)] +struct ModelEntry { + id: String, + object: &'static str, + created: u64, + owned_by: &'static str, +} + +pub fn build_router(config: ThreadlineConfig) -> Router { + let connector = DefaultUpstreamConnector { + codex_client_version: config.codex_client_version.clone(), + }; + + build_router_with_services( + config, + ThreadlineServices::new(Arc::new(DefaultAuthProvider), Arc::new(connector)), + ) +} + +pub fn build_router_with_services( + config: ThreadlineConfig, + services: ThreadlineServices, +) -> Router { + let responses = ResponsesRouteState { + profile: config.profile, + registry: Arc::new(RetainedSessionRegistry::new( + config.retained_session_capacity, + )), + services, + }; + let state = AppState { + profile: config.profile, + responses, + }; + + Router::new() + .route("/health", get(health)) + .route("/v1/models", get(models)) + .route("/v1/responses", post(responses_route)) + .with_state(state) +} + +async fn health() -> Json { + Json(HealthPayload { + status: "ok", + service: "threadline", + }) +} + +async fn models(State(state): State) -> Json { + Json(ModelListPayload { + object: "list", + data: advertised_model_ids_for_profile(state.profile) + .iter() + .map(|model_id| ModelEntry { + id: (*model_id).to_string(), + object: "model", + created: MODEL_CREATED_UNSPECIFIED, + owned_by: "threadline", + }) + .collect(), + }) +} + +async fn responses_route( + State(state): State, + headers: HeaderMap, + Json(payload): Json, +) -> Result { + let request_metadata = extract_downstream_request_metadata(&headers); + responses_handler(State(state.responses), Json(payload), request_metadata).await +} + +fn extract_downstream_request_metadata(headers: &HeaderMap) -> DownstreamRequestMetadata { + let interaction_type = headers + .get_all(INTERACTION_TYPE_HEADER) + .iter() + .next() + .map(|value| value.as_bytes()); + + DownstreamRequestMetadata::from_interaction_type_header_bytes(interaction_type) +} + +#[derive(Clone)] +struct DefaultAuthProvider; + +impl crate::responses::UpstreamAuthProvider for DefaultAuthProvider { + fn load(&self) -> Result { + load_upstream_auth(&AuthDiscoveryOptions::from_env()) + .map_err(|_| ThreadlineError::UpstreamCredentialsUnavailable) + } +} + +#[derive(Clone)] +struct DefaultUpstreamConnector { + codex_client_version: String, +} + +impl DefaultUpstreamConnector { + fn upstream_url() -> String { + std::env::var("THREADLINE_UPSTREAM_URL") + .unwrap_or_else(|_| DEFAULT_UPSTREAM_URL.to_string()) + } +} + +fn upstream_connect_error_kind(error: &TungsteniteError) -> &'static str { + match error { + TungsteniteError::ConnectionClosed => "connection_closed", + TungsteniteError::AlreadyClosed => "already_closed", + TungsteniteError::Io(_) => "io", + TungsteniteError::Tls(_) => "tls", + TungsteniteError::Capacity(_) => "capacity", + TungsteniteError::Protocol(_) => "protocol", + TungsteniteError::WriteBufferFull(_) => "write_buffer_full", + TungsteniteError::Utf8 => "utf8", + TungsteniteError::AttackAttempt => "attack_attempt", + TungsteniteError::Url(_) => "url", + TungsteniteError::HttpFormat(_) => "http_format", + _ => unreachable!("http errors are handled before upstream_connect_error_kind"), + } +} + +fn map_upstream_connect_error(error: TungsteniteError) -> ThreadlineError { + match error { + TungsteniteError::Http(response) => { + let status = response.status(); + warn!( + upstream_status = status.as_u16(), + upstream_status_reason = status.canonical_reason().unwrap_or("unknown"), + "upstream_websocket_handshake_rejected" + ); + ThreadlineError::UpstreamWebSocketHandshakeRejected { status } + } + other => { + warn!( + error_kind = upstream_connect_error_kind(&other), + error = %other, + "upstream_websocket_connect_failed" + ); + ThreadlineError::UpstreamWebSocketConnectFailed + } + } +} + +impl crate::responses::UpstreamConnector for DefaultUpstreamConnector { + fn connect( + &self, + auth: crate::auth::LoadedUpstreamAuth, + session: Option, + ) -> BoxFuture<'static, Result> { + let codex_client_version = self.codex_client_version.clone(); + + Box::pin(async move { + let upstream_url = Self::upstream_url(); + let handshake = + build_handshake_request(&upstream_url, &auth, &codex_client_version, session) + .map_err(|_| ThreadlineError::UpstreamWebSocketConnectFailed)?; + let (stream, response) = connect_async(handshake.request) + .await + .map_err(map_upstream_connect_error)?; + let turn_state = response + .headers() + .get(crate::responses::TURN_STATE_HEADER) + .and_then(|value| value.to_str().ok()) + .map(ToString::to_string); + + Ok(ConnectedUpstream { + websocket: Arc::new(LiveUpstreamWebSocket::from_stream(stream)), + session: handshake.session, + turn_state, + }) + }) + } +} + +#[cfg(test)] +mod tests { + use axum::http::{HeaderValue, Response, StatusCode}; + use std::ffi::OsString; + use std::sync::Mutex; + use tokio_tungstenite::tungstenite::Error as TungsteniteError; + + use super::*; + use crate::responses::DownstreamInteractionType; + + static UPSTREAM_URL_ENV_LOCK: Mutex<()> = Mutex::new(()); + + struct UpstreamUrlEnvGuard { + original: Option, + } + + impl UpstreamUrlEnvGuard { + fn acquire() -> Self { + let original = std::env::var_os("THREADLINE_UPSTREAM_URL"); + Self { original } + } + } + + impl Drop for UpstreamUrlEnvGuard { + fn drop(&mut self) { + match self.original.take() { + Some(value) => unsafe { std::env::set_var("THREADLINE_UPSTREAM_URL", value) }, + None => unsafe { std::env::remove_var("THREADLINE_UPSTREAM_URL") }, + } + } + } + + #[test] + fn upstream_http_connect_error_maps_to_status_error() { + let error = TungsteniteError::Http( + Response::builder() + .status(StatusCode::UNAUTHORIZED) + .body(None) + .unwrap(), + ); + + let mapped = map_upstream_connect_error(error); + + assert!(matches!( + mapped, + ThreadlineError::UpstreamWebSocketHandshakeRejected { status } + if status == StatusCode::UNAUTHORIZED + )); + } + + #[test] + fn upstream_non_http_connect_error_remains_bad_gateway_failure() { + let error = TungsteniteError::Io(std::io::Error::other("dial failed")); + + let mapped = map_upstream_connect_error(error); + + assert!(matches!( + mapped, + ThreadlineError::UpstreamWebSocketConnectFailed + )); + assert_eq!(mapped.status_code(), StatusCode::BAD_GATEWAY); + } + + #[test] + fn upstream_connect_error_kind_uses_coarse_io_bucket() { + let error = TungsteniteError::Io(std::io::Error::other("dial failed")); + + assert_eq!(upstream_connect_error_kind(&error), "io"); + } + + #[test] + fn upstream_connect_error_kind_distinguishes_closed_connections() { + assert_eq!( + upstream_connect_error_kind(&TungsteniteError::ConnectionClosed), + "connection_closed" + ); + } + + #[test] + fn upstream_url_uses_default_when_env_is_unset() { + let _lock = UPSTREAM_URL_ENV_LOCK.lock().unwrap(); + let _guard = UpstreamUrlEnvGuard::acquire(); + unsafe { std::env::remove_var("THREADLINE_UPSTREAM_URL") }; + + assert_eq!( + DefaultUpstreamConnector::upstream_url(), + DEFAULT_UPSTREAM_URL + ); + } + + #[test] + fn upstream_url_prefers_env_override_when_present() { + let _lock = UPSTREAM_URL_ENV_LOCK.lock().unwrap(); + let _guard = UpstreamUrlEnvGuard::acquire(); + unsafe { + std::env::set_var( + "THREADLINE_UPSTREAM_URL", + "wss://example.invalid/backend-api/codex/responses", + ) + }; + + assert_eq!( + DefaultUpstreamConnector::upstream_url(), + "wss://example.invalid/backend-api/codex/responses" + ); + } + + #[test] + fn interaction_type_header_uses_first_duplicate_value() { + let mut headers = HeaderMap::new(); + headers.append( + INTERACTION_TYPE_HEADER, + HeaderValue::from_static("conversation-start"), + ); + headers.append( + INTERACTION_TYPE_HEADER, + HeaderValue::from_static("conversation-compaction"), + ); + + let metadata = extract_downstream_request_metadata(&headers); + + assert_eq!( + metadata.interaction_type(), + DownstreamInteractionType::Other + ); + + let mut headers = HeaderMap::new(); + headers.append( + INTERACTION_TYPE_HEADER, + HeaderValue::from_static(" conversation-compaction "), + ); + headers.append( + INTERACTION_TYPE_HEADER, + HeaderValue::from_static("conversation-start"), + ); + + let metadata = extract_downstream_request_metadata(&headers); + + assert_eq!( + metadata.interaction_type(), + DownstreamInteractionType::ConversationCompaction + ); + } +} diff --git a/src/jobs.rs b/src/jobs.rs new file mode 100644 index 0000000..81f5c8a --- /dev/null +++ b/src/jobs.rs @@ -0,0 +1,786 @@ +use std::collections::{HashMap, VecDeque}; +use std::future::Future; +use std::io::{BufReader, Read}; +use std::process::{Child, Command, Stdio}; +use std::sync::{Arc, Mutex}; +use std::thread; +use std::time::{Duration, Instant}; + +use serde_json::{Value, json}; +use tracing::debug; +use uuid::Uuid; + +const JOB_START_NEXT_ACTION_HINT: &str = "This job is running in the background. Continue other useful work if available, then poll status or read output later when needed."; + +#[derive(Debug, Clone)] +pub struct ThreadlineJobManager { + inner: Arc, +} + +#[derive(Debug)] +struct ThreadlineJobManagerInner { + config: ThreadlineJobManagerConfig, + entries: Mutex>>>, +} + +#[derive(Debug, Clone)] +pub struct ThreadlineJobManagerConfig { + pub jobs_enabled: bool, + pub output_buffer_limit_bytes: usize, + pub retention_ttl: Duration, + pub allowed_commands: Vec, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum JobState { + Starting, + Running, + Completed, + Failed, + Cancelled, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum JobTerminalState { + Completed, + Failed, + Cancelled, +} + +#[derive(Debug)] +struct JobEntry { + job_id: String, + name: String, + state: JobState, + output: JobOutputRingBuffer, + result: Option, + error: Option, + cancel_requested: bool, + child: Option>>, + finished_at: Option, +} + +#[derive(Debug, Clone)] +struct JobFailurePayload { + code: &'static str, + message: String, +} + +#[derive(Debug, Clone)] +pub struct ManagedJobContext { + entry: Arc>, +} + +#[derive(Debug)] +struct JobOutputRingBuffer { + limit: usize, + next_offset: u64, + truncated_before: u64, + buffered_bytes: usize, + segments: VecDeque, +} + +#[derive(Debug)] +struct JobOutputSegment { + offset: u64, + stream: &'static str, + text: String, +} + +impl Default for ThreadlineJobManagerConfig { + fn default() -> Self { + Self { + jobs_enabled: false, + output_buffer_limit_bytes: 32 * 1024, + retention_ttl: Duration::from_secs(300), + allowed_commands: Vec::new(), + } + } +} + +impl ThreadlineJobManager { + pub fn new(config: ThreadlineJobManagerConfig) -> Self { + Self { + inner: Arc::new(ThreadlineJobManagerInner { + config, + entries: Mutex::new(HashMap::new()), + }), + } + } + + pub fn spawn_job(&self, name: &str, task: F) -> Value + where + F: FnOnce(ManagedJobContext) -> Fut + Send + 'static, + Fut: Future + Send + 'static, + { + let context = self.insert_job(name); + let job_id = context.job_id(); + let spawned_context = context.clone(); + + tokio::spawn(async move { + task(spawned_context.clone()).await; + spawned_context.fail_if_unresolved(); + }); + + job_started_json(&job_id) + } + + pub fn start_command_json(&self, command: Vec) -> Value { + if !self.inner.config.jobs_enabled { + return stable_error("jobs_disabled", "Threadline jobs are disabled."); + } + if command.is_empty() { + return stable_error( + "invalid_job_request", + "threadline_start_job requires a non-empty command array.", + ); + } + + let program = command[0].clone(); + if !self.command_allowed(&program) { + return stable_error( + "job_command_not_allowed", + "The requested command is not allowed by the configured Threadline job policy.", + ); + } + + let context = self.insert_job("command"); + let job_id = context.job_id(); + thread::spawn(move || run_command_job(context, command)); + + job_started_json(&job_id) + } + + pub fn poll_json(&self, job_id: &str) -> Value { + match self.entry(job_id) { + Some(entry) => entry_snapshot(&entry), + None => job_not_found(job_id), + } + } + + pub fn read_output_json(&self, job_id: &str, offset: u64) -> Value { + let Some(entry) = self.entry(job_id) else { + return job_not_found(job_id); + }; + + let entry = entry.lock().expect("job entry lock"); + let effective_offset = offset.max(entry.output.truncated_before); + let output = entry.output.read_from(offset); + debug!( + job_id = %entry.job_id, + requested_offset = offset, + served_from_offset = effective_offset, + item_count = output.len(), + next_offset = entry.output.next_offset, + truncated_before = entry.output.truncated_before, + "job_output_offset_served" + ); + json!({ + "ok": true, + "job_id": entry.job_id, + "status": entry.state.as_str(), + "items": output, + "next_offset": entry.output.next_offset, + "truncated_before": entry.output.truncated_before, + }) + } + + pub fn get_result_json(&self, job_id: &str) -> Value { + let Some(entry) = self.entry(job_id) else { + return job_not_found(job_id); + }; + + let entry = entry.lock().expect("job entry lock"); + let error = entry.error.as_ref().map(|payload| { + json!({ + "code": payload.code, + "message": payload.message, + }) + }); + + json!({ + "ok": true, + "job_id": entry.job_id, + "status": entry.state.as_str(), + "result": entry.result, + "error": error, + }) + } + + pub fn cancel_json(&self, job_id: &str) -> Value { + let Some(entry) = self.entry(job_id) else { + return job_not_found(job_id); + }; + + let child = { + let mut entry = entry.lock().expect("job entry lock"); + entry.cancel_requested = true; + if !entry.state.is_terminal() { + entry.state = JobState::Cancelled; + entry.result = None; + entry.error = Some(JobFailurePayload { + code: "job_cancelled", + message: "The Threadline job was cancelled.".to_string(), + }); + entry.finished_at = Some(Instant::now()); + debug!( + job_id = %entry.job_id, + terminal_state = JobTerminalState::Cancelled.as_str(), + "job_terminal_state_changed" + ); + } + entry.child.clone() + }; + + if let Some(child) = child { + let _ = child.lock().expect("child lock").kill(); + } + + self.poll_json(job_id) + } + + pub fn prune_expired(&self) -> usize { + let now = Instant::now(); + let ttl = self.inner.config.retention_ttl; + let mut removed = 0usize; + + self.inner + .entries + .lock() + .expect("entries lock") + .retain(|_, entry| { + let keep = { + let entry = entry.lock().expect("job entry lock"); + match entry.finished_at { + Some(finished_at) => now.duration_since(finished_at) < ttl, + None => true, + } + }; + if !keep { + removed += 1; + } + keep + }); + + removed + } + + fn insert_job(&self, name: &str) -> ManagedJobContext { + let job_id = Uuid::now_v7().to_string(); + let entry = Arc::new(Mutex::new(JobEntry { + job_id, + name: name.to_string(), + state: JobState::Starting, + output: JobOutputRingBuffer::new(self.inner.config.output_buffer_limit_bytes), + result: None, + error: None, + cancel_requested: false, + child: None, + finished_at: None, + })); + + let job_id = entry.lock().expect("job entry lock").job_id.clone(); + self.inner + .entries + .lock() + .expect("entries lock") + .insert(job_id, Arc::clone(&entry)); + + ManagedJobContext { entry } + } + + fn entry(&self, job_id: &str) -> Option>> { + self.inner + .entries + .lock() + .expect("entries lock") + .get(job_id) + .cloned() + } + + fn command_allowed(&self, program: &str) -> bool { + self.inner + .config + .allowed_commands + .iter() + .any(|allowed| allowed == program) + } +} + +impl JobTerminalState { + pub fn as_str(self) -> &'static str { + match self { + Self::Completed => "completed", + Self::Failed => "failed", + Self::Cancelled => "cancelled", + } + } +} + +impl ManagedJobContext { + pub fn mark_running(&self) { + let mut entry = self.entry.lock().expect("job entry lock"); + if entry.state == JobState::Starting { + entry.state = JobState::Running; + } + } + + pub fn push_stdout(&self, text: &str) { + self.push_output("stdout", text); + } + + pub fn push_stderr(&self, text: &str) { + self.push_output("stderr", text); + } + + pub fn complete(&self, result: Value) { + let mut entry = self.entry.lock().expect("job entry lock"); + if entry.state.is_terminal() { + return; + } + + entry.state = JobState::Completed; + entry.result = Some(result); + entry.error = None; + entry.child = None; + entry.finished_at = Some(Instant::now()); + debug!( + job_id = %entry.job_id, + terminal_state = JobTerminalState::Completed.as_str(), + "job_terminal_state_changed" + ); + } + + pub fn fail(&self, code: &'static str, message: impl Into) { + let mut entry = self.entry.lock().expect("job entry lock"); + if entry.state.is_terminal() { + return; + } + + entry.state = JobState::Failed; + entry.result = None; + entry.error = Some(JobFailurePayload { + code, + message: message.into(), + }); + entry.child = None; + entry.finished_at = Some(Instant::now()); + debug!( + job_id = %entry.job_id, + terminal_state = JobTerminalState::Failed.as_str(), + error_code = code, + "job_terminal_state_changed" + ); + } + + pub fn is_cancelled(&self) -> bool { + self.entry.lock().expect("job entry lock").cancel_requested + } + + pub fn job_id(&self) -> String { + self.entry.lock().expect("job entry lock").job_id.clone() + } + + fn push_output(&self, stream: &'static str, text: &str) { + let mut entry = self.entry.lock().expect("job entry lock"); + if entry.state.is_terminal() || text.is_empty() { + return; + } + + let start_offset = entry.output.next_offset; + let truncated_before = entry.output.truncated_before; + entry.output.append(stream, text); + debug!( + job_id = %entry.job_id, + stream, + byte_count = text.len(), + start_offset, + next_offset = entry.output.next_offset, + truncated_before = entry.output.truncated_before, + "job_output_chunk_appended" + ); + if entry.output.truncated_before != truncated_before { + debug!( + job_id = %entry.job_id, + stream, + previous_truncated_before = truncated_before, + truncated_before = entry.output.truncated_before, + next_offset = entry.output.next_offset, + buffered_bytes = entry.output.buffered_bytes, + "job_output_truncation_advanced" + ); + } + } + + fn attach_child(&self, child: Arc>) { + let mut entry = self.entry.lock().expect("job entry lock"); + if entry.state.is_terminal() { + return; + } + entry.child = Some(child); + } + + fn clear_child(&self) { + self.entry.lock().expect("job entry lock").child = None; + } + + fn fail_if_unresolved(&self) { + let mut entry = self.entry.lock().expect("job entry lock"); + if entry.state.is_terminal() { + return; + } + + entry.state = JobState::Failed; + entry.result = None; + entry.error = Some(JobFailurePayload { + code: "job_did_not_finalize", + message: "The Threadline job ended without reporting a terminal state.".to_string(), + }); + entry.child = None; + entry.finished_at = Some(Instant::now()); + debug!( + job_id = %entry.job_id, + terminal_state = JobTerminalState::Failed.as_str(), + error_code = "job_did_not_finalize", + "job_terminal_state_changed" + ); + } +} + +impl JobOutputRingBuffer { + fn new(limit: usize) -> Self { + Self { + limit, + next_offset: 0, + truncated_before: 0, + buffered_bytes: 0, + segments: VecDeque::new(), + } + } + + fn append(&mut self, stream: &'static str, text: &str) { + let added = text.len(); + if added == 0 { + return; + } + + let offset = self.next_offset; + self.next_offset += added as u64; + + if self.limit == 0 { + self.truncated_before = self.next_offset; + return; + } + + self.buffered_bytes += added; + self.segments.push_back(JobOutputSegment { + offset, + stream, + text: text.to_string(), + }); + self.trim_to_limit(); + } + + fn read_from(&self, offset: u64) -> Vec { + let effective_offset = offset.max(self.truncated_before); + + self.segments + .iter() + .filter_map(|segment| { + let segment_end = segment.offset + segment.text.len() as u64; + if segment_end <= effective_offset { + return None; + } + + if effective_offset <= segment.offset { + return Some(json!({ + "offset": segment.offset, + "stream": segment.stream, + "text": segment.text, + })); + } + + let skip = (effective_offset - segment.offset) as usize; + let (skipped_bytes, trimmed_text) = trim_front_bytes(&segment.text, skip); + if trimmed_text.is_empty() { + return None; + } + + Some(json!({ + "offset": segment.offset + skipped_bytes as u64, + "stream": segment.stream, + "text": trimmed_text, + })) + }) + .collect() + } + + fn trim_to_limit(&mut self) { + while self.buffered_bytes > self.limit { + let overflow = self.buffered_bytes - self.limit; + let Some(front) = self.segments.front_mut() else { + break; + }; + + let available = front.text.len(); + let trim = overflow.min(available); + let (trimmed_bytes, trimmed_text) = trim_front_bytes(&front.text, trim); + front.text = trimmed_text; + front.offset += trimmed_bytes as u64; + self.buffered_bytes -= trimmed_bytes; + self.truncated_before += trimmed_bytes as u64; + + if front.text.is_empty() { + self.segments.pop_front(); + } + } + } +} + +impl JobState { + fn as_str(self) -> &'static str { + match self { + Self::Starting => "starting", + Self::Running => "running", + Self::Completed => "completed", + Self::Failed => "failed", + Self::Cancelled => "cancelled", + } + } + + fn is_terminal(self) -> bool { + matches!(self, Self::Completed | Self::Failed | Self::Cancelled) + } + + fn terminal_state(self) -> Option { + match self { + Self::Completed => Some(JobTerminalState::Completed), + Self::Failed => Some(JobTerminalState::Failed), + Self::Cancelled => Some(JobTerminalState::Cancelled), + Self::Starting | Self::Running => None, + } + } +} + +fn entry_snapshot(entry: &Arc>) -> Value { + let entry = entry.lock().expect("job entry lock"); + json!({ + "ok": true, + "job_id": entry.job_id, + "name": entry.name, + "status": entry.state.as_str(), + "finished": entry.state.is_terminal(), + "cancel_requested": entry.cancel_requested, + "terminal_state": entry.state.terminal_state().map(JobTerminalState::as_str), + }) +} + +fn job_not_found(job_id: &str) -> Value { + json!({ + "ok": false, + "code": "job_not_found", + "message": "Threadline could not find a job with that job_id.", + "job_id": job_id, + }) +} + +fn stable_error(code: &'static str, message: &'static str) -> Value { + json!({ + "ok": false, + "code": code, + "message": message, + }) +} + +fn job_started_json(job_id: &str) -> Value { + json!({ + "ok": true, + "job_id": job_id, + "status": JobState::Starting.as_str(), + "next_action_hint": JOB_START_NEXT_ACTION_HINT, + }) +} + +fn run_command_job(context: ManagedJobContext, command: Vec) { + context.mark_running(); + + let mut child = match spawn_command(&command) { + Ok(child) => child, + Err(error) => { + context.fail( + "job_command_spawn_failed", + format!("Threadline could not start the requested command: {error}"), + ); + return; + } + }; + + let stdout = child.stdout.take(); + let stderr = child.stderr.take(); + let child = Arc::new(Mutex::new(child)); + context.attach_child(Arc::clone(&child)); + + let stdout_reader = stdout.map(|stdout| spawn_output_reader(stdout, context.clone(), "stdout")); + let stderr_reader = stderr.map(|stderr| spawn_output_reader(stderr, context.clone(), "stderr")); + + let status = loop { + if context.is_cancelled() { + let _ = child.lock().expect("child lock").kill(); + } + + match child.lock().expect("child lock").try_wait() { + Ok(Some(status)) => break status, + Ok(None) => thread::sleep(Duration::from_millis(10)), + Err(error) => { + context.clear_child(); + join_reader(stdout_reader); + join_reader(stderr_reader); + context.fail( + "job_command_failed", + format!("Threadline could not observe the command status: {error}"), + ); + return; + } + } + }; + + join_reader(stdout_reader); + join_reader(stderr_reader); + context.clear_child(); + + if context.is_cancelled() { + return; + } + + if status.success() { + context.complete(json!({ + "kind": "command", + "command": command, + "exit_code": status.code(), + "success": true, + })); + } else { + context.fail( + "job_command_failed", + format!( + "The Threadline job command exited unsuccessfully with code {:?}.", + status.code() + ), + ); + } +} + +fn spawn_command(command: &[String]) -> Result { + let mut child = Command::new(&command[0]); + if command.len() > 1 { + child.args(&command[1..]); + } + + child.stdout(Stdio::piped()).stderr(Stdio::piped()).spawn() +} + +fn spawn_output_reader( + reader: R, + context: ManagedJobContext, + stream: &'static str, +) -> thread::JoinHandle<()> +where + R: std::io::Read + Send + 'static, +{ + thread::spawn(move || { + let mut reader = BufReader::new(reader); + let mut pending = Vec::new(); + let mut buffer = [0u8; 1024]; + + loop { + match reader.read(&mut buffer) { + Ok(0) => break, + Ok(read_bytes) => { + pending.extend_from_slice(&buffer[..read_bytes]); + flush_output_chunks(&context, stream, &mut pending); + } + Err(_) => break, + } + } + + if !pending.is_empty() { + let text = String::from_utf8_lossy(&pending).to_string(); + push_stream_output(&context, stream, &text); + } + }) +} + +fn flush_output_chunks(context: &ManagedJobContext, stream: &'static str, pending: &mut Vec) { + loop { + if let Some(newline_index) = pending.iter().position(|byte| *byte == b'\n') { + let chunk: Vec = pending.drain(..=newline_index).collect(); + let text = String::from_utf8_lossy(&chunk).to_string(); + push_stream_output(context, stream, &text); + continue; + } + + match std::str::from_utf8(pending) { + Ok(text) => { + if !text.is_empty() { + push_stream_output(context, stream, text); + pending.clear(); + } + break; + } + Err(error) => { + let valid_up_to = error.valid_up_to(); + if valid_up_to > 0 { + let text = + std::str::from_utf8(&pending[..valid_up_to]).expect("valid utf-8 prefix"); + push_stream_output(context, stream, text); + pending.drain(..valid_up_to); + continue; + } + + if error.error_len().is_none() { + break; + } + + let text = String::from_utf8_lossy(pending).to_string(); + push_stream_output(context, stream, &text); + pending.clear(); + break; + } + } + } +} + +fn push_stream_output(context: &ManagedJobContext, stream: &'static str, text: &str) { + if text.is_empty() { + return; + } + + match stream { + "stdout" => context.push_stdout(text), + "stderr" => context.push_stderr(text), + _ => {} + } +} + +fn join_reader(reader: Option>) { + if let Some(reader) = reader { + let _ = reader.join(); + } +} + +fn trim_front_bytes(text: &str, count: usize) -> (usize, String) { + if count == 0 { + return (0, text.to_string()); + } + + if count >= text.len() { + return (text.len(), String::new()); + } + + let mut start = count; + while start < text.len() && !text.is_char_boundary(start) { + start += 1; + } + + (start, text[start..].to_string()) +} diff --git a/src/lib.rs b/src/lib.rs new file mode 100644 index 0000000..6478830 --- /dev/null +++ b/src/lib.rs @@ -0,0 +1,12 @@ +pub mod auth; +pub mod cli; +pub mod codex_ws; +pub mod config; +pub mod errors; +pub mod http; +pub mod jobs; +pub mod models; +pub mod registry; +pub mod responses; +pub mod tools; +pub mod ws_pump; diff --git a/src/main.rs b/src/main.rs new file mode 100644 index 0000000..2f5afe2 --- /dev/null +++ b/src/main.rs @@ -0,0 +1,302 @@ +use std::process::ExitCode; + +use clap::Parser; +use threadline::cli::{ThreadlineCli, ThreadlineCliAction}; +use threadline::config::ThreadlineConfig; +use threadline::errors::ThreadlineError; +use threadline::http::build_router; +use threadline::models::RouteProfile; +use tracing::info; +use tracing_subscriber::EnvFilter; + +const LOGIN_INSTRUCTIONS_MESSAGE: &str = "Threadline does not store credentials. Sign in with Codex Desktop or Codex CLI, then run Threadline again."; +const UTILITY_PORT_REQUIRES_MAIN_PROFILE_MESSAGE: &str = + "--utility-port can only be used with the main profile"; +const UTILITY_PORT_MUST_DIFFER_MESSAGE: &str = "--utility-port must differ from --port"; + +#[tokio::main] +async fn main() -> ExitCode { + match run().await { + Ok(()) => ExitCode::SUCCESS, + Err(error) => { + eprintln!("threadline startup failed: {error}"); + ExitCode::FAILURE + } + } +} + +async fn run() -> Result<(), Box> { + let cli = ThreadlineCli::parse(); + + match cli.into_action() { + ThreadlineCliAction::StartServer(config) => run_server(config).await.map_err(Into::into), + ThreadlineCliAction::LoginInstructions => { + println!("{}", login_instructions_message()); + Ok(()) + } + } +} + +fn login_instructions_message() -> &'static str { + LOGIN_INSTRUCTIONS_MESSAGE +} + +async fn run_server(config: ThreadlineConfig) -> Result<(), ThreadlineError> { + init_tracing(&config); + + match config.utility_port { + Some(utility_port) => run_main_and_utility_servers(config, utility_port).await, + None => serve_config(config).await, + } +} + +async fn run_main_and_utility_servers( + main_config: ThreadlineConfig, + utility_port: u16, +) -> Result<(), ThreadlineError> { + let (main_config, utility_config) = split_main_and_utility_configs(main_config, utility_port)?; + + tokio::try_join!(serve_config(main_config), serve_config(utility_config))?; + + Ok(()) +} + +async fn serve_config(config: ThreadlineConfig) -> Result<(), ThreadlineError> { + let bind_address = config + .bind_address() + .map_err(|_| ThreadlineError::InvalidBindHost(config.host.clone()))?; + let profile = config.profile; + let listener = tokio::net::TcpListener::bind(bind_address) + .await + .map_err(|_| ThreadlineError::InvalidBindHost(bind_address.ip().to_string()))?; + let app = build_router(config); + + info!(address = %bind_address, profile = %profile, "threadline_http_server_started"); + + axum::serve(listener, app) + .await + .map_err(|_| ThreadlineError::InvalidBindHost(bind_address.ip().to_string())) +} + +fn split_main_and_utility_configs( + main_config: ThreadlineConfig, + utility_port: u16, +) -> Result<(ThreadlineConfig, ThreadlineConfig), ThreadlineError> { + if main_config.profile != RouteProfile::Main { + return Err(ThreadlineError::InvalidServerConfiguration( + UTILITY_PORT_REQUIRES_MAIN_PROFILE_MESSAGE.to_string(), + )); + } + + if main_config.port == utility_port { + return Err(ThreadlineError::InvalidServerConfiguration( + UTILITY_PORT_MUST_DIFFER_MESSAGE.to_string(), + )); + } + + let mut utility_config = main_config.clone(); + utility_config.port = utility_port; + utility_config.profile = RouteProfile::Utility; + utility_config.retained_session_capacity = 0; + utility_config.jobs_enabled = false; + + Ok((main_config, utility_config)) +} + +fn init_tracing(config: &ThreadlineConfig) { + let env_filter = EnvFilter::try_from_default_env() + .or_else(|_| EnvFilter::try_new(config.log_level.clone())) + .unwrap_or_else(|_| EnvFilter::new("info")); + + tracing_subscriber::fmt() + .with_env_filter(env_filter) + .with_target(false) + .compact() + .init(); +} + +#[cfg(test)] +mod tests { + use std::ffi::OsString; + use std::sync::Mutex; + + use clap::Parser; + use threadline::models::RouteProfile; + + use super::*; + + static THREADLINE_PROFILE_ENV_LOCK: Mutex<()> = Mutex::new(()); + static THREADLINE_UTILITY_PORT_ENV_LOCK: Mutex<()> = Mutex::new(()); + + struct ProfileEnvGuard { + original: Option, + } + + impl ProfileEnvGuard { + fn acquire() -> Self { + Self { + original: std::env::var_os("THREADLINE_PROFILE"), + } + } + } + + impl Drop for ProfileEnvGuard { + fn drop(&mut self) { + match self.original.take() { + Some(value) => unsafe { std::env::set_var("THREADLINE_PROFILE", value) }, + None => unsafe { std::env::remove_var("THREADLINE_PROFILE") }, + } + } + } + + struct UtilityPortEnvGuard { + original: Option, + } + + impl UtilityPortEnvGuard { + fn acquire() -> Self { + Self { + original: std::env::var_os("THREADLINE_UTILITY_PORT"), + } + } + } + + impl Drop for UtilityPortEnvGuard { + fn drop(&mut self) { + match self.original.take() { + Some(value) => unsafe { std::env::set_var("THREADLINE_UTILITY_PORT", value) }, + None => unsafe { std::env::remove_var("THREADLINE_UTILITY_PORT") }, + } + } + } + + fn utility_port_restricted_to_main_message() -> &'static str { + "--utility-port can only be used with the main profile" + } + + fn utility_port_must_differ_message() -> &'static str { + "--utility-port must differ from --port" + } + + #[test] + fn split_main_and_utility_configs_accepts_main_profile_with_different_port() { + let main_config = ThreadlineConfig { + port: 8100, + utility_port: Some(8101), + ..ThreadlineConfig::default() + }; + + let (resolved_main, utility_config) = + split_main_and_utility_configs(main_config.clone(), 8101).expect("config split"); + + assert_eq!(resolved_main, main_config); + assert_eq!(utility_config.port, 8101); + assert_eq!(utility_config.profile, RouteProfile::Utility); + } + + #[test] + fn split_main_and_utility_configs_rejects_utility_profile() { + let main_config = ThreadlineConfig { + profile: RouteProfile::Utility, + utility_port: Some(8101), + ..ThreadlineConfig::default() + }; + + let error = split_main_and_utility_configs(main_config, 8101) + .expect_err("utility profile should be rejected"); + + assert!(matches!( + error, + ThreadlineError::InvalidServerConfiguration(message) + if message == utility_port_restricted_to_main_message() + )); + } + + #[test] + fn split_main_and_utility_configs_rejects_same_port() { + let main_config = ThreadlineConfig { + port: 8100, + utility_port: Some(8100), + ..ThreadlineConfig::default() + }; + + let error = split_main_and_utility_configs(main_config, 8100) + .expect_err("same utility port should be rejected"); + + assert!(matches!( + error, + ThreadlineError::InvalidServerConfiguration(message) + if message == utility_port_must_differ_message() + )); + } + + #[test] + fn split_main_and_utility_configs_derives_stateless_utility_config() { + let main_config = ThreadlineConfig { + port: 8100, + utility_port: Some(8101), + retained_session_capacity: 9, + jobs_enabled: true, + ..ThreadlineConfig::default() + }; + + let (_, utility_config) = + split_main_and_utility_configs(main_config.clone(), 8101).expect("config split"); + + assert_eq!(utility_config.profile, RouteProfile::Utility); + assert_eq!(utility_config.port, 8101); + assert_eq!(utility_config.retained_session_capacity, 0); + assert!(!utility_config.jobs_enabled); + + let mut expected_utility = main_config; + expected_utility.port = 8101; + expected_utility.profile = RouteProfile::Utility; + expected_utility.retained_session_capacity = 0; + expected_utility.jobs_enabled = false; + + assert_eq!(utility_config, expected_utility); + } + + #[test] + fn split_main_and_utility_configs_preserves_main_job_and_retention_settings() { + let main_config = ThreadlineConfig { + retained_session_capacity: 11, + jobs_enabled: true, + utility_port: Some(8101), + ..ThreadlineConfig::default() + }; + + let (resolved_main, _) = + split_main_and_utility_configs(main_config.clone(), 8101).expect("config split"); + + assert_eq!(resolved_main.retained_session_capacity, 11); + assert!(resolved_main.jobs_enabled); + assert_eq!(resolved_main, main_config); + } + + #[test] + fn split_main_and_utility_configs_rejects_env_derived_utility_profile() { + let _profile_lock = THREADLINE_PROFILE_ENV_LOCK + .lock() + .expect("profile env lock"); + let _utility_port_lock = THREADLINE_UTILITY_PORT_ENV_LOCK + .lock() + .expect("utility port env lock"); + let _profile_guard = ProfileEnvGuard::acquire(); + let _utility_port_guard = UtilityPortEnvGuard::acquire(); + + unsafe { std::env::set_var("THREADLINE_PROFILE", "utility") }; + unsafe { std::env::set_var("THREADLINE_UTILITY_PORT", "8101") }; + + let config = ThreadlineCli::parse_from(["threadline"]).server; + let utility_port = config.utility_port.expect("utility port from env"); + let error = split_main_and_utility_configs(config, utility_port) + .expect_err("utility profile from env should be rejected"); + + assert!(matches!( + error, + ThreadlineError::InvalidServerConfiguration(message) + if message == utility_port_restricted_to_main_message() + )); + } +} diff --git a/src/models.rs b/src/models.rs new file mode 100644 index 0000000..9f51c51 --- /dev/null +++ b/src/models.rs @@ -0,0 +1,561 @@ +use std::sync::OnceLock; + +use clap::ValueEnum; +use serde_json::{Map, Value}; + +use crate::errors::ThreadlineError; + +#[derive(Clone, Copy, Debug, Eq, PartialEq, ValueEnum)] +pub enum RouteProfile { + Main, + Utility, +} + +impl std::fmt::Display for RouteProfile { + fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + let value = match self { + Self::Main => "main", + Self::Utility => "utility", + }; + + formatter.write_str(value) + } +} + +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub struct ModelAlias { + pub alias_id: &'static str, + pub upstream_model_id: &'static str, + pub profile: RouteProfile, + pub advertised: bool, + pub supports_reasoning_all_turns: bool, +} + +const MODEL_ALIAS_CATALOG: [ModelAlias; 14] = [ + ModelAlias { + alias_id: "threadline-main-gpt-5.6-sol", + upstream_model_id: "gpt-5.6-sol", + profile: RouteProfile::Main, + advertised: true, + supports_reasoning_all_turns: true, + }, + ModelAlias { + alias_id: "threadline-main-gpt-5.6-terra", + upstream_model_id: "gpt-5.6-terra", + profile: RouteProfile::Main, + advertised: true, + supports_reasoning_all_turns: true, + }, + ModelAlias { + alias_id: "threadline-main-gpt-5.6-luna", + upstream_model_id: "gpt-5.6-luna", + profile: RouteProfile::Main, + advertised: true, + supports_reasoning_all_turns: true, + }, + ModelAlias { + alias_id: "threadline-main-gpt-5.5", + upstream_model_id: "gpt-5.5", + profile: RouteProfile::Main, + advertised: true, + supports_reasoning_all_turns: true, + }, + ModelAlias { + alias_id: "threadline-main-gpt-5.4", + upstream_model_id: "gpt-5.4", + profile: RouteProfile::Main, + advertised: true, + supports_reasoning_all_turns: true, + }, + ModelAlias { + alias_id: "threadline-utility-gpt-5.4-mini", + upstream_model_id: "gpt-5.4-mini", + profile: RouteProfile::Utility, + advertised: true, + supports_reasoning_all_turns: true, + }, + ModelAlias { + alias_id: "threadline-utility-gpt-5.3-codex-spark", + upstream_model_id: "gpt-5.3-codex-spark", + profile: RouteProfile::Utility, + advertised: true, + supports_reasoning_all_turns: false, + }, + ModelAlias { + alias_id: "gpt-5.6-sol", + upstream_model_id: "gpt-5.6-sol", + profile: RouteProfile::Main, + advertised: false, + supports_reasoning_all_turns: false, + }, + ModelAlias { + alias_id: "gpt-5.6-terra", + upstream_model_id: "gpt-5.6-terra", + profile: RouteProfile::Main, + advertised: false, + supports_reasoning_all_turns: false, + }, + ModelAlias { + alias_id: "gpt-5.6-luna", + upstream_model_id: "gpt-5.6-luna", + profile: RouteProfile::Main, + advertised: false, + supports_reasoning_all_turns: false, + }, + ModelAlias { + alias_id: "gpt-5.5", + upstream_model_id: "gpt-5.5", + profile: RouteProfile::Main, + advertised: false, + supports_reasoning_all_turns: false, + }, + ModelAlias { + alias_id: "gpt-5.4", + upstream_model_id: "gpt-5.4", + profile: RouteProfile::Main, + advertised: false, + supports_reasoning_all_turns: false, + }, + ModelAlias { + alias_id: "gpt-5.4-mini", + upstream_model_id: "gpt-5.4-mini", + profile: RouteProfile::Main, + advertised: false, + supports_reasoning_all_turns: false, + }, + ModelAlias { + alias_id: "gpt-5.3-codex-spark", + upstream_model_id: "gpt-5.3-codex-spark", + profile: RouteProfile::Main, + advertised: false, + supports_reasoning_all_turns: false, + }, +]; + +static MAIN_ADVERTISED_MODEL_IDS: OnceLock> = OnceLock::new(); +static UTILITY_ADVERTISED_MODEL_IDS: OnceLock> = OnceLock::new(); + +fn advertised_model_ids_cache(profile: RouteProfile) -> &'static OnceLock> { + match profile { + RouteProfile::Main => &MAIN_ADVERTISED_MODEL_IDS, + RouteProfile::Utility => &UTILITY_ADVERTISED_MODEL_IDS, + } +} + +fn model_alias_by_id(model_id: &str) -> Option<&'static ModelAlias> { + MODEL_ALIAS_CATALOG + .iter() + .find(|alias| alias.alias_id == model_id) +} + +fn resolve_model_alias_for_profile( + model_id: &str, + profile: RouteProfile, +) -> Result<&'static ModelAlias, ThreadlineError> { + match model_alias_by_id(model_id) { + Some(alias) if alias.profile == profile => Ok(alias), + _ => Err(ThreadlineError::InvalidModel), + } +} + +pub fn supported_model_ids() -> &'static [&'static str] { + advertised_model_ids_for_profile(RouteProfile::Main) +} + +pub fn advertised_model_ids_for_profile(profile: RouteProfile) -> &'static [&'static str] { + advertised_model_ids_cache(profile) + .get_or_init(|| { + MODEL_ALIAS_CATALOG + .iter() + .filter(|alias| alias.profile == profile && alias.advertised) + .map(|alias| alias.alias_id) + .collect() + }) + .as_slice() +} + +pub fn is_supported_model(model_id: &str) -> bool { + model_alias_by_id(model_id).is_some() +} + +pub fn validate_request_model(payload: &Map) -> Result<&str, ThreadlineError> { + let alias = resolve_request_model_for_profile(payload, RouteProfile::Main)?; + Ok(alias.alias_id) +} + +pub fn resolve_request_model_for_profile( + payload: &Map, + profile: RouteProfile, +) -> Result<&'static ModelAlias, ThreadlineError> { + let model_id = payload + .get("model") + .and_then(Value::as_str) + .ok_or(ThreadlineError::InvalidModel)?; + + resolve_model_alias_for_profile(model_id, profile) +} + +#[cfg(test)] +mod tests { + use super::{ + RouteProfile, advertised_model_ids_for_profile, is_supported_model, + resolve_request_model_for_profile, supported_model_ids, validate_request_model, + }; + use serde_json::json; + + const NEW_MAIN_VISIBLE_MODEL_IDS: [&str; 3] = [ + "threadline-main-gpt-5.6-sol", + "threadline-main-gpt-5.6-terra", + "threadline-main-gpt-5.6-luna", + ]; + + const NEW_MAIN_RAW_COMPATIBILITY_IDS: [&str; 3] = + ["gpt-5.6-sol", "gpt-5.6-terra", "gpt-5.6-luna"]; + + #[test] + fn supported_model_ids_match_main_public_contract() { + assert_eq!( + supported_model_ids(), + &[ + "threadline-main-gpt-5.6-sol", + "threadline-main-gpt-5.6-terra", + "threadline-main-gpt-5.6-luna", + "threadline-main-gpt-5.5", + "threadline-main-gpt-5.4", + ] + ); + } + + #[test] + fn advertised_model_ids_are_filtered_by_profile() { + assert_eq!( + advertised_model_ids_for_profile(RouteProfile::Main), + &[ + "threadline-main-gpt-5.6-sol", + "threadline-main-gpt-5.6-terra", + "threadline-main-gpt-5.6-luna", + "threadline-main-gpt-5.5", + "threadline-main-gpt-5.4", + ] + ); + assert_eq!( + advertised_model_ids_for_profile(RouteProfile::Utility), + &[ + "threadline-utility-gpt-5.4-mini", + "threadline-utility-gpt-5.3-codex-spark", + ] + ); + } + + #[test] + fn supported_model_check_accepts_aliases_and_hidden_main_compatibility_ids() { + for model_id in NEW_MAIN_VISIBLE_MODEL_IDS { + assert!(is_supported_model(model_id)); + } + assert!(is_supported_model("threadline-main-gpt-5.5")); + assert!(is_supported_model("threadline-main-gpt-5.4")); + assert!(is_supported_model("threadline-utility-gpt-5.4-mini")); + assert!(is_supported_model("threadline-utility-gpt-5.3-codex-spark")); + for model_id in NEW_MAIN_RAW_COMPATIBILITY_IDS { + assert!(is_supported_model(model_id)); + } + assert!(is_supported_model("gpt-5.5")); + assert!(is_supported_model("gpt-5.4")); + assert!(is_supported_model("gpt-5.4-mini")); + assert!(is_supported_model("gpt-5.3-codex-spark")); + assert!(!is_supported_model("codex-mini-latest")); + } + + #[test] + fn resolve_request_model_for_profile_rewrites_visible_alias_to_upstream_model() { + for (alias_id, upstream_model_id) in [ + ("threadline-main-gpt-5.6-sol", "gpt-5.6-sol"), + ("threadline-main-gpt-5.6-terra", "gpt-5.6-terra"), + ("threadline-main-gpt-5.6-luna", "gpt-5.6-luna"), + ] { + let main = resolve_request_model_for_profile( + json!({ "model": alias_id }).as_object().unwrap(), + RouteProfile::Main, + ) + .unwrap(); + assert_eq!(main.alias_id, alias_id); + assert_eq!(main.upstream_model_id, upstream_model_id); + assert_eq!(main.profile, RouteProfile::Main); + assert!(main.advertised); + assert!(main.supports_reasoning_all_turns); + } + + let main = resolve_request_model_for_profile( + json!({ "model": "threadline-main-gpt-5.5" }) + .as_object() + .unwrap(), + RouteProfile::Main, + ) + .unwrap(); + assert_eq!(main.alias_id, "threadline-main-gpt-5.5"); + assert_eq!(main.upstream_model_id, "gpt-5.5"); + assert_eq!(main.profile, RouteProfile::Main); + assert!(main.advertised); + assert!(main.supports_reasoning_all_turns); + + let utility = resolve_request_model_for_profile( + json!({ "model": "threadline-utility-gpt-5.4-mini" }) + .as_object() + .unwrap(), + RouteProfile::Utility, + ) + .unwrap(); + assert_eq!(utility.alias_id, "threadline-utility-gpt-5.4-mini"); + assert_eq!(utility.upstream_model_id, "gpt-5.4-mini"); + assert_eq!(utility.profile, RouteProfile::Utility); + assert!(utility.advertised); + assert!(utility.supports_reasoning_all_turns); + } + + #[test] + fn resolve_request_model_for_profile_rejects_profile_mismatch() { + for model_id in NEW_MAIN_VISIBLE_MODEL_IDS { + assert_eq!( + resolve_request_model_for_profile( + json!({ "model": model_id }).as_object().unwrap(), + RouteProfile::Utility, + ) + .unwrap_err() + .to_string(), + "The /v1/responses request must include a supported string model." + ); + } + + assert_eq!( + resolve_request_model_for_profile( + json!({ "model": "threadline-utility-gpt-5.4-mini" }) + .as_object() + .unwrap(), + RouteProfile::Main, + ) + .unwrap_err() + .to_string(), + "The /v1/responses request must include a supported string model." + ); + assert_eq!( + resolve_request_model_for_profile( + json!({ "model": "threadline-main-gpt-5.4" }) + .as_object() + .unwrap(), + RouteProfile::Utility, + ) + .unwrap_err() + .to_string(), + "The /v1/responses request must include a supported string model." + ); + } + + #[test] + fn resolve_request_model_for_profile_rejects_unknown_model() { + assert_eq!( + resolve_request_model_for_profile( + json!({ "model": "codex-mini-latest" }).as_object().unwrap(), + RouteProfile::Main, + ) + .unwrap_err() + .to_string(), + "The /v1/responses request must include a supported string model." + ); + } + + #[test] + fn resolve_request_model_for_profile_accepts_hidden_main_compatibility_ids() { + for model_id in NEW_MAIN_RAW_COMPATIBILITY_IDS { + let compatibility = resolve_request_model_for_profile( + json!({ "model": model_id }).as_object().unwrap(), + RouteProfile::Main, + ) + .unwrap(); + assert_eq!(compatibility.alias_id, model_id); + assert_eq!(compatibility.upstream_model_id, model_id); + assert_eq!(compatibility.profile, RouteProfile::Main); + assert!(!compatibility.advertised); + assert!(!compatibility.supports_reasoning_all_turns); + + assert_eq!( + resolve_request_model_for_profile( + json!({ "model": model_id }).as_object().unwrap(), + RouteProfile::Utility, + ) + .unwrap_err() + .to_string(), + "The /v1/responses request must include a supported string model." + ); + } + + let compatibility = resolve_request_model_for_profile( + json!({ "model": "gpt-5.4-mini" }).as_object().unwrap(), + RouteProfile::Main, + ) + .unwrap(); + assert_eq!(compatibility.alias_id, "gpt-5.4-mini"); + assert_eq!(compatibility.upstream_model_id, "gpt-5.4-mini"); + assert_eq!(compatibility.profile, RouteProfile::Main); + assert!(!compatibility.advertised); + assert!(!compatibility.supports_reasoning_all_turns); + + assert_eq!( + resolve_request_model_for_profile( + json!({ "model": "gpt-5.4-mini" }).as_object().unwrap(), + RouteProfile::Utility, + ) + .unwrap_err() + .to_string(), + "The /v1/responses request must include a supported string model." + ); + } + + #[test] + fn resolve_request_model_for_profile_exposes_reasoning_all_turns_capability_by_alias() { + for model_id in NEW_MAIN_VISIBLE_MODEL_IDS { + let main_supported = resolve_request_model_for_profile( + json!({ "model": model_id }).as_object().unwrap(), + RouteProfile::Main, + ) + .unwrap(); + assert!(main_supported.supports_reasoning_all_turns); + } + + let main_supported = resolve_request_model_for_profile( + json!({ "model": "threadline-main-gpt-5.5" }) + .as_object() + .unwrap(), + RouteProfile::Main, + ) + .unwrap(); + assert!(main_supported.supports_reasoning_all_turns); + + let main_supported_secondary = resolve_request_model_for_profile( + json!({ "model": "threadline-main-gpt-5.4" }) + .as_object() + .unwrap(), + RouteProfile::Main, + ) + .unwrap(); + assert!(main_supported_secondary.supports_reasoning_all_turns); + + let utility_supported = resolve_request_model_for_profile( + json!({ "model": "threadline-utility-gpt-5.4-mini" }) + .as_object() + .unwrap(), + RouteProfile::Utility, + ) + .unwrap(); + assert!(utility_supported.supports_reasoning_all_turns); + + let utility_unsupported = resolve_request_model_for_profile( + json!({ "model": "threadline-utility-gpt-5.3-codex-spark" }) + .as_object() + .unwrap(), + RouteProfile::Utility, + ) + .unwrap(); + assert!(!utility_unsupported.supports_reasoning_all_turns); + + let hidden_compatibility = resolve_request_model_for_profile( + json!({ "model": "gpt-5.5" }).as_object().unwrap(), + RouteProfile::Main, + ) + .unwrap(); + assert!(!hidden_compatibility.supports_reasoning_all_turns); + + let hidden_compatibility_secondary = resolve_request_model_for_profile( + json!({ "model": "gpt-5.4" }).as_object().unwrap(), + RouteProfile::Main, + ) + .unwrap(); + assert!(!hidden_compatibility_secondary.supports_reasoning_all_turns); + + let hidden_compatibility_tertiary = resolve_request_model_for_profile( + json!({ "model": "gpt-5.4-mini" }).as_object().unwrap(), + RouteProfile::Main, + ) + .unwrap(); + assert!(!hidden_compatibility_tertiary.supports_reasoning_all_turns); + + let hidden_compatibility_quaternary = resolve_request_model_for_profile( + json!({ "model": "gpt-5.3-codex-spark" }) + .as_object() + .unwrap(), + RouteProfile::Main, + ) + .unwrap(); + assert!(!hidden_compatibility_quaternary.supports_reasoning_all_turns); + } + + #[test] + fn resolve_request_model_for_profile_keeps_invalid_model_for_wrong_profile_before_capability_use() + { + assert_eq!( + resolve_request_model_for_profile( + json!({ "model": "threadline-utility-gpt-5.4-mini" }) + .as_object() + .unwrap(), + RouteProfile::Main, + ) + .unwrap_err() + .to_string(), + "The /v1/responses request must include a supported string model." + ); + } + + #[test] + fn resolve_request_model_for_profile_keeps_invalid_model_for_unknown_alias_before_capability_use() + { + assert_eq!( + resolve_request_model_for_profile( + json!({ "model": "gpt-5.4-nano" }).as_object().unwrap(), + RouteProfile::Main, + ) + .unwrap_err() + .to_string(), + "The /v1/responses request must include a supported string model." + ); + } + + #[test] + fn validate_request_model_requires_main_supported_string_model() { + assert_eq!( + validate_request_model(json!({}).as_object().unwrap()) + .unwrap_err() + .to_string(), + "The /v1/responses request must include a supported string model." + ); + assert_eq!( + validate_request_model(json!({ "model": { "id": "gpt-5.4" } }).as_object().unwrap()) + .unwrap_err() + .to_string(), + "The /v1/responses request must include a supported string model." + ); + assert_eq!( + validate_request_model(json!({ "model": "codex-mini-latest" }).as_object().unwrap()) + .unwrap_err() + .to_string(), + "The /v1/responses request must include a supported string model." + ); + assert_eq!( + validate_request_model( + json!({ "model": "threadline-main-gpt-5.4" }) + .as_object() + .unwrap() + ) + .unwrap(), + "threadline-main-gpt-5.4" + ); + assert_eq!( + validate_request_model( + json!({ "model": "threadline-utility-gpt-5.4-mini" }) + .as_object() + .unwrap() + ) + .unwrap_err() + .to_string(), + "The /v1/responses request must include a supported string model." + ); + } +} diff --git a/src/registry.rs b/src/registry.rs new file mode 100644 index 0000000..05bbd15 --- /dev/null +++ b/src/registry.rs @@ -0,0 +1,335 @@ +use std::collections::HashMap; +use std::sync::{Arc, Mutex}; +use std::time::Instant; + +use crate::codex_ws::UpstreamSessionDescriptor; +use crate::ws_pump::LiveUpstreamWebSocket; +use tracing::debug; +use uuid::Uuid; + +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum RegistryAcquireError { + PreviousResponseNotFound, + RetainedSessionConflict, + RetainedSessionCapacityExceeded, +} + +pub struct RetainedSessionRegistry { + inner: Arc>, +} + +pub struct RetainedSessionLease { + entry_id: u64, + registry: Arc>, + session: UpstreamSessionDescriptor, + upstream: Option>, + removed: bool, + released: bool, +} + +impl std::fmt::Debug for RetainedSessionLease { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("RetainedSessionLease") + .field("entry_id", &self.entry_id) + .field("session", &self.session) + .field("has_live_upstream", &self.upstream.is_some()) + .field("removed", &self.removed) + .field("released", &self.released) + .finish() + } +} + +struct RegistryState { + capacity: usize, + next_entry_id: u64, + entries: HashMap, + markers: HashMap, +} + +struct RegistryEntry { + session: UpstreamSessionDescriptor, + window_generation: u64, + upstream: Option>, + in_use: bool, + recoverable: bool, + last_used: Instant, + markers: Vec, +} + +impl RetainedSessionRegistry { + pub fn new(capacity: usize) -> Self { + Self { + inner: Arc::new(Mutex::new(RegistryState { + capacity, + next_entry_id: 1, + entries: HashMap::new(), + markers: HashMap::new(), + })), + } + } + + pub async fn acquire_new(&self) -> Result { + let mut state = self.inner.lock().expect("registry mutex poisoned"); + if state.capacity == 0 { + return Err(RegistryAcquireError::RetainedSessionCapacityExceeded); + } + + if state.entries.len() >= state.capacity { + if let Some(entry_id) = state + .entries + .iter() + .filter(|(_, entry)| !entry.in_use) + .min_by_key(|(_, entry)| entry.last_used) + .map(|(entry_id, _)| *entry_id) + { + remove_entry(&mut state, entry_id); + } else { + return Err(RegistryAcquireError::RetainedSessionCapacityExceeded); + } + } + + let entry_id = state.next_entry_id; + state.next_entry_id += 1; + let session = UpstreamSessionDescriptor { + session_id: new_id(), + thread_id: new_id(), + window_id: new_id(), + turn_state: None, + }; + state.entries.insert( + entry_id, + RegistryEntry { + session: session.clone(), + window_generation: 0, + upstream: None, + in_use: true, + recoverable: true, + last_used: Instant::now(), + markers: Vec::new(), + }, + ); + + debug!( + session_id = %session.session_id, + thread_id = %session.thread_id, + window_id = %session.window_id, + "retained_session_acquired" + ); + + Ok(RetainedSessionLease { + entry_id, + registry: Arc::clone(&self.inner), + session, + upstream: None, + removed: false, + released: false, + }) + } + + pub async fn acquire_previous( + &self, + response_marker: &str, + ) -> Result { + let mut state = self.inner.lock().expect("registry mutex poisoned"); + let Some(entry_id) = state.markers.get(response_marker).copied() else { + return Err(RegistryAcquireError::PreviousResponseNotFound); + }; + let Some(entry) = state.entries.get_mut(&entry_id) else { + state.markers.remove(response_marker); + return Err(RegistryAcquireError::PreviousResponseNotFound); + }; + + if entry.in_use { + return Err(RegistryAcquireError::RetainedSessionConflict); + } + + if entry + .upstream + .as_ref() + .is_some_and(|upstream| upstream.is_closed()) + { + entry.upstream = None; + entry.recoverable = true; + refresh_entry_window(entry); + } + + entry.in_use = true; + entry.last_used = Instant::now(); + + debug!( + response_marker, + session_id = %entry.session.session_id, + thread_id = %entry.session.thread_id, + window_id = %entry.session.window_id, + "retained_session_acquired" + ); + + Ok(RetainedSessionLease { + entry_id, + registry: Arc::clone(&self.inner), + session: entry.session.clone(), + upstream: entry.upstream.clone(), + removed: false, + released: false, + }) + } +} + +impl RetainedSessionLease { + pub fn session(&self) -> &UpstreamSessionDescriptor { + &self.session + } + + pub fn has_live_upstream(&self) -> bool { + self.upstream.is_some() + } + + pub fn has_open_upstream(&self) -> bool { + self.upstream + .as_ref() + .is_some_and(|upstream| !upstream.is_closed()) + } + + pub fn upstream(&self) -> Option> { + self.upstream.clone() + } + + pub fn release(&mut self) { + if self.removed || self.released { + return; + } + + self.released = true; + + if let Ok(mut state) = self.registry.lock() + && let Some(entry) = state.entries.get_mut(&self.entry_id) + { + entry.in_use = false; + entry.last_used = Instant::now(); + debug!( + session_id = %entry.session.session_id, + thread_id = %entry.session.thread_id, + window_id = %entry.session.window_id, + "retained_session_released" + ); + } + } + + pub async fn record_completed_marker(&mut self, response_marker: impl Into) { + let response_marker = response_marker.into(); + let mut state = self.registry.lock().expect("registry mutex poisoned"); + let Some(entry) = state.entries.get_mut(&self.entry_id) else { + return; + }; + + if !entry + .markers + .iter() + .any(|marker| marker == &response_marker) + { + entry.markers.push(response_marker.clone()); + } + entry.last_used = Instant::now(); + state.markers.insert(response_marker, self.entry_id); + } + + pub async fn update_turn_state(&mut self, turn_state: Option) { + self.session.turn_state = turn_state.clone(); + let mut state = self.registry.lock().expect("registry mutex poisoned"); + if let Some(entry) = state.entries.get_mut(&self.entry_id) { + entry.session.turn_state = turn_state; + entry.last_used = Instant::now(); + } + } + + pub async fn replace_upstream(&mut self, upstream: Option>) { + self.upstream = upstream.clone(); + let mut state = self.registry.lock().expect("registry mutex poisoned"); + if let Some(entry) = state.entries.get_mut(&self.entry_id) { + entry.upstream = upstream; + entry.recoverable = true; + entry.last_used = Instant::now(); + } + } + + pub async fn mark_upstream_recoverable(&mut self) { + self.upstream = None; + let mut state = self.registry.lock().expect("registry mutex poisoned"); + if let Some(entry) = state.entries.get_mut(&self.entry_id) { + entry.upstream = None; + entry.recoverable = true; + refresh_entry_window(entry); + self.session = entry.session.clone(); + entry.last_used = Instant::now(); + } + } + + pub async fn mark_upstream_terminal(&mut self) { + let mut state = self.registry.lock().expect("registry mutex poisoned"); + remove_entry(&mut state, self.entry_id); + self.upstream = None; + self.removed = true; + self.released = true; + } +} + +impl Drop for RetainedSessionLease { + fn drop(&mut self) { + self.release(); + } +} + +fn refresh_entry_window(entry: &mut RegistryEntry) { + entry.window_generation += 1; + entry.session.refresh_window(); +} + +fn remove_entry(state: &mut RegistryState, entry_id: u64) { + let Some(entry) = state.entries.remove(&entry_id) else { + return; + }; + for marker in entry.markers { + if state.markers.get(&marker).copied() == Some(entry_id) { + state.markers.remove(&marker); + } + } +} + +fn new_id() -> String { + Uuid::now_v7().to_string() +} + +#[cfg(test)] +mod tests { + use super::*; + use std::time::Duration; + + #[tokio::test] + async fn recording_a_completed_marker_refreshes_last_used() { + let registry = RetainedSessionRegistry::new(1); + let mut lease = registry.acquire_new().await.expect("create session"); + + let initial_last_used = { + let state = registry.inner.lock().expect("registry mutex poisoned"); + state + .entries + .get(&lease.entry_id) + .expect("entry should exist") + .last_used + }; + + std::thread::sleep(Duration::from_millis(5)); + lease.record_completed_marker("response-1").await; + + let refreshed_last_used = { + let state = registry.inner.lock().expect("registry mutex poisoned"); + state + .entries + .get(&lease.entry_id) + .expect("entry should exist") + .last_used + }; + + assert!(refreshed_last_used > initial_last_used); + } +} diff --git a/src/responses/downstream.rs b/src/responses/downstream.rs new file mode 100644 index 0000000..2b4ba94 --- /dev/null +++ b/src/responses/downstream.rs @@ -0,0 +1,1921 @@ +use axum::body::Bytes; +use serde::Deserialize; +use serde_json::{Map, Value}; + +use crate::errors::ThreadlineError; + +const AUTO_CONTEXT_TOO_LARGE_PROMPT: &str = + "The conversation has grown too large for the context window and must be compacted now"; +const AUTO_SUMMARY_TAGS_INSTRUCTION: &str = + "Output your summary wrapped in and tags"; +const AUTO_ONLY_TASK_INSTRUCTION: &str = + "Your ONLY task right now is to produce a comprehensive summary"; +const NEW_AUTO_DETAILED_SUMMARY_INSTRUCTION: &str = "Your task is to create a comprehensive, detailed summary of the entire conversation that captures all essential information needed to seamlessly continue the work without any loss of context"; +const MANUAL_SUMMARY_PROMPT: &str = "Summarize the conversation history so far, paying special attention to the most recent agent commands and tool results"; +const MANUAL_STRUCTURE_INSTRUCTION: &str = + "Structure your summary using the enhanced format provided in the system message"; +const MANUAL_TOOL_RESULTS_INSTRUCTION: &str = "Include all important tool calls and their results"; +const SIMPLE_HISTORY_CONTEXT_OBSERVED: &str = + "The following is a compressed version of the preceeding history in the current conversation"; +const SIMPLE_HISTORY_CONTEXT_CORRECTED: &str = + "The following is a compressed version of the preceding history in the current conversation"; +const MAX_ALLOWLISTED_INTERACTION_TYPE_LEN: usize = 64; + +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] +pub(super) enum DownstreamRequestClassification { + #[default] + Normal, + AuxiliarySummary, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] +pub(crate) enum DownstreamInteractionType { + #[default] + None, + ConversationCompaction, + Other, +} + +impl DownstreamInteractionType { + pub(crate) fn label(self) -> &'static str { + match self { + Self::None => "none", + Self::ConversationCompaction => "conversation_compaction", + Self::Other => "other", + } + } +} + +#[derive(Debug, Clone, Copy, Default)] +pub(crate) struct DownstreamRequestMetadata { + interaction_type: DownstreamInteractionType, +} + +impl DownstreamRequestMetadata { + #[cfg(test)] + pub(crate) fn from_interaction_type_header_value(value: Option<&str>) -> Self { + Self { + interaction_type: normalize_interaction_type(value), + } + } + + pub(crate) fn from_interaction_type_header_bytes(value: Option<&[u8]>) -> Self { + let interaction_type = match value { + Some(raw) => match std::str::from_utf8(raw) { + Ok(text) => normalize_interaction_type(Some(text)), + Err(_) => DownstreamInteractionType::Other, + }, + None => DownstreamInteractionType::None, + }; + + Self { interaction_type } + } + + #[cfg(test)] + pub(crate) fn interaction_type(self) -> DownstreamInteractionType { + self.interaction_type + } +} + +#[derive(Debug, Deserialize)] +pub(super) struct DownstreamResponsesRequest { + #[serde(default)] + pub(super) previous_response_id: Option, + #[serde(skip)] + pub(super) classification: DownstreamRequestClassification, + #[serde(skip)] + routing_diagnostics: DownstreamRequestRoutingDiagnostics, + #[serde(flatten)] + pub(super) payload: serde_json::Map, +} + +impl DownstreamResponsesRequest { + pub(super) fn routing_diagnostics(&self) -> &DownstreamRequestRoutingDiagnostics { + &self.routing_diagnostics + } +} + +#[cfg_attr(not(test), allow(dead_code))] +pub(super) fn parse_downstream_request( + payload: Value, +) -> Result { + parse_downstream_request_with_metadata(payload, DownstreamRequestMetadata::default()) +} + +pub(super) fn parse_downstream_request_with_metadata( + payload: Value, + metadata: DownstreamRequestMetadata, +) -> Result { + let mut request = serde_json::from_value::(payload) + .map_err(|_| ThreadlineError::InvalidResponsesRequest)?; + let routing_diagnostics = collect_request_routing_diagnostics(&request.payload, metadata); + request.classification = classify_request(&routing_diagnostics); + request.routing_diagnostics = routing_diagnostics; + Ok(request) +} + +fn normalize_interaction_type(value: Option<&str>) -> DownstreamInteractionType { + let Some(value) = value.map(str::trim) else { + return DownstreamInteractionType::None; + }; + + if value.is_empty() { + return DownstreamInteractionType::None; + } + + if value.len() > MAX_ALLOWLISTED_INTERACTION_TYPE_LEN { + return DownstreamInteractionType::Other; + } + + if value.eq_ignore_ascii_case("conversation-compaction") { + DownstreamInteractionType::ConversationCompaction + } else { + DownstreamInteractionType::Other + } +} + +fn classify_request( + routing_diagnostics: &DownstreamRequestRoutingDiagnostics, +) -> DownstreamRequestClassification { + if routing_diagnostics.interaction_type_compaction_hit { + return DownstreamRequestClassification::AuxiliarySummary; + } + + if is_auxiliary_summary_request(&routing_diagnostics.summary_hits) { + DownstreamRequestClassification::AuxiliarySummary + } else { + DownstreamRequestClassification::Normal + } +} + +fn is_auxiliary_summary_request(summary_hits: &SummaryFingerprintHits) -> bool { + summary_hits.matches_auxiliary_summary() +} + +pub(super) fn looks_like_auxiliary_summary_conflict_fallback( + payload: &serde_json::Map, +) -> bool { + let Some(input) = payload.get("input") else { + return false; + }; + + collect_conflict_fallback_summary_fingerprints(input).matches_auxiliary_summary() +} + +pub(super) fn wants_reasoning_all_turns(payload: &serde_json::Map) -> bool { + payload + .get("reasoning") + .and_then(Value::as_object) + .and_then(|reasoning| reasoning.get("context")) + .and_then(Value::as_str) + == Some("all_turns") +} + +#[derive(Debug, Clone, Default)] +pub(super) struct DownstreamRequestRoutingDiagnostics { + pub(super) summary_hits: SummaryFingerprintHits, + pub(super) interaction_type: DownstreamInteractionType, + pub(super) interaction_type_compaction_hit: bool, + pub(super) tool_choice: Option, + pub(super) tools_count: usize, + pub(super) input_item_count: usize, + pub(super) last_input_role: Option, + pub(super) last_input_type: Option, +} + +#[derive(Debug, Clone, Default)] +pub(super) struct SummaryFingerprintHits { + pub(super) manual_summary_prompt_hit: bool, + pub(super) manual_structure_instruction_hit: bool, + pub(super) manual_tool_results_instruction_hit: bool, + pub(super) auto_context_too_large_hit: bool, + pub(super) auto_summary_tags_hit: bool, + pub(super) auto_only_task_hit: bool, + pub(super) simple_history_context_hit: bool, + pub(super) new_auto_detailed_summary_hit: bool, + pub(super) new_auto_user_history_hit: bool, + pub(super) new_auto_user_final_summary_prompt_hit: bool, + pub(super) summary_instruction_like_hit: bool, + manual_summary_prompt_instruction_like: bool, + manual_structure_instruction_instruction_like: bool, + manual_tool_results_instruction_instruction_like: bool, + auto_context_too_large_instruction_like: bool, + auto_summary_tags_instruction_like: bool, + auto_only_task_instruction_like: bool, + simple_history_context_instruction_like: bool, + new_auto_detailed_summary_instruction_like: bool, +} + +impl SummaryFingerprintHits { + pub(super) fn matches_auxiliary_summary(&self) -> bool { + let manual_primary = self.manual_summary_prompt_instruction_like; + let manual_secondary = self.manual_structure_instruction_instruction_like + || self.manual_tool_results_instruction_instruction_like + || self.simple_history_context_instruction_like; + let auto_primary = self.auto_context_too_large_instruction_like; + let auto_secondary = self.auto_summary_tags_instruction_like + || self.auto_only_task_instruction_like + || self.simple_history_context_instruction_like; + let new_auto_with_history = self.new_auto_detailed_summary_instruction_like + && self.new_auto_user_history_hit + && self.new_auto_user_final_summary_prompt_hit; + let new_foreground = self.new_auto_detailed_summary_instruction_like + && self.new_auto_user_final_summary_prompt_hit; + + (manual_primary && manual_secondary) + || (auto_primary && auto_secondary) + || new_auto_with_history + || new_foreground + } + + fn record_text(&mut self, text: &str, context: SummaryObservationContext<'_>) { + self.record_text_with_instruction_like(text, context.is_summary_instruction_like()); + + if text.contains(NEW_AUTO_DETAILED_SUMMARY_INSTRUCTION) { + self.new_auto_detailed_summary_hit = true; + self.new_auto_detailed_summary_instruction_like |= + context.is_summary_instruction_like(); + } + + if context.is_user_input_text() + && (text.contains(SIMPLE_HISTORY_CONTEXT_OBSERVED) + || text.contains(SIMPLE_HISTORY_CONTEXT_CORRECTED)) + { + self.new_auto_user_history_hit = true; + } + + if context.is_final_user_input_text() + && text.contains(MANUAL_SUMMARY_PROMPT) + && text.contains(MANUAL_STRUCTURE_INSTRUCTION) + && text.contains(MANUAL_TOOL_RESULTS_INSTRUCTION) + { + self.new_auto_user_final_summary_prompt_hit = true; + } + } + + fn record_text_with_instruction_like(&mut self, text: &str, instruction_like: bool) { + let had_instruction_like_hit = self.manual_summary_prompt_instruction_like + || self.manual_structure_instruction_instruction_like + || self.manual_tool_results_instruction_instruction_like + || self.auto_context_too_large_instruction_like + || self.auto_summary_tags_instruction_like + || self.auto_only_task_instruction_like + || self.simple_history_context_instruction_like; + + if text.contains(MANUAL_SUMMARY_PROMPT) { + self.manual_summary_prompt_hit = true; + self.manual_summary_prompt_instruction_like |= instruction_like; + } + if text.contains(MANUAL_STRUCTURE_INSTRUCTION) { + self.manual_structure_instruction_hit = true; + self.manual_structure_instruction_instruction_like |= instruction_like; + } + if text.contains(MANUAL_TOOL_RESULTS_INSTRUCTION) { + self.manual_tool_results_instruction_hit = true; + self.manual_tool_results_instruction_instruction_like |= instruction_like; + } + if text.contains(AUTO_CONTEXT_TOO_LARGE_PROMPT) { + self.auto_context_too_large_hit = true; + self.auto_context_too_large_instruction_like |= instruction_like; + } + if text.contains(AUTO_SUMMARY_TAGS_INSTRUCTION) { + self.auto_summary_tags_hit = true; + self.auto_summary_tags_instruction_like |= instruction_like; + } + if text.contains(AUTO_ONLY_TASK_INSTRUCTION) { + self.auto_only_task_hit = true; + self.auto_only_task_instruction_like |= instruction_like; + } + if text.contains(SIMPLE_HISTORY_CONTEXT_OBSERVED) + || text.contains(SIMPLE_HISTORY_CONTEXT_CORRECTED) + { + self.simple_history_context_hit = true; + self.simple_history_context_instruction_like |= instruction_like; + } + + let has_instruction_like_hit = self.manual_summary_prompt_instruction_like + || self.manual_structure_instruction_instruction_like + || self.manual_tool_results_instruction_instruction_like + || self.auto_context_too_large_instruction_like + || self.auto_summary_tags_instruction_like + || self.auto_only_task_instruction_like + || self.simple_history_context_instruction_like; + + self.summary_instruction_like_hit |= + instruction_like && (had_instruction_like_hit || has_instruction_like_hit); + } +} + +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)] +enum InputSourceCategory { + SummaryInstructionLike, + OrdinaryUserContent, + #[default] + UnknownInputContent, +} + +impl InputSourceCategory { + fn from_role(role: Option<&str>) -> Self { + match role { + Some("system" | "developer") => Self::SummaryInstructionLike, + Some("user") => Self::OrdinaryUserContent, + Some(_) | None => Self::UnknownInputContent, + } + } + + fn is_summary_instruction_like(self) -> bool { + matches!(self, Self::SummaryInstructionLike) + } +} + +#[derive(Clone, Copy, Debug, Default)] +struct SummaryObservationContext<'a> { + message_role: Option<&'a str>, + content_item_type: Option<&'a str>, + under_content_array: bool, + final_input_item: bool, + source_category: InputSourceCategory, +} + +impl SummaryObservationContext<'_> { + fn is_summary_instruction_like(self) -> bool { + self.under_content_array + && self.content_item_type == Some("input_text") + && self.source_category.is_summary_instruction_like() + } + + fn is_user_input_text(self) -> bool { + self.under_content_array + && self.content_item_type == Some("input_text") + && self.source_category == InputSourceCategory::OrdinaryUserContent + } + + fn is_final_user_input_text(self) -> bool { + self.final_input_item && self.is_user_input_text() + } +} + +fn collect_request_routing_diagnostics( + payload: &serde_json::Map, + metadata: DownstreamRequestMetadata, +) -> DownstreamRequestRoutingDiagnostics { + let input = payload.get("input"); + + DownstreamRequestRoutingDiagnostics { + summary_hits: collect_summary_fingerprints(input), + interaction_type: metadata.interaction_type, + interaction_type_compaction_hit: matches!( + metadata.interaction_type, + DownstreamInteractionType::ConversationCompaction + ), + tool_choice: safe_value_type_label(payload.get("tool_choice")), + tools_count: payload + .get("tools") + .and_then(Value::as_array) + .map_or(0, Vec::len), + input_item_count: input.map_or(0, input_item_count), + last_input_role: input.and_then(last_input_role), + last_input_type: input.and_then(last_input_type), + } +} + +fn collect_summary_fingerprints(input: Option<&Value>) -> SummaryFingerprintHits { + let Some(input) = input else { + return SummaryFingerprintHits::default(); + }; + + let mut fingerprints = SummaryFingerprintHits::default(); + collect_summary_fingerprints_into_input(input, &mut fingerprints); + fingerprints +} + +fn collect_conflict_fallback_summary_fingerprints(input: &Value) -> SummaryFingerprintHits { + let mut fingerprints = SummaryFingerprintHits::default(); + + match input { + Value::Array(items) => { + for item in items { + collect_conflict_fallback_summary_from_input_item(item, &mut fingerprints); + } + } + _ => collect_conflict_fallback_summary_from_input_item(input, &mut fingerprints), + } + + fingerprints +} + +fn collect_conflict_fallback_summary_from_input_item( + value: &Value, + fingerprints: &mut SummaryFingerprintHits, +) { + let Some(item) = value.as_object() else { + return; + }; + + match item.get("type").and_then(Value::as_str) { + Some("message") => { + let source_category = + InputSourceCategory::from_role(item.get("role").and_then(Value::as_str)); + if let Some(content) = item.get("content").and_then(Value::as_array) { + for content_item in content { + collect_conflict_fallback_summary_from_content_item( + content_item, + source_category, + fingerprints, + ); + } + } + } + Some("input_text") => { + let Some(text) = item.get("text").and_then(Value::as_str) else { + return; + }; + + // Direct top-level input_text summary prompts remain eligible as explicit fallback traffic. + fingerprints.record_text_with_instruction_like(text, true); + } + Some(_) | None => {} + } +} + +fn collect_conflict_fallback_summary_from_content_item( + value: &Value, + source_category: InputSourceCategory, + fingerprints: &mut SummaryFingerprintHits, +) { + let Some(item) = value.as_object() else { + return; + }; + + if item.get("type").and_then(Value::as_str) != Some("input_text") { + return; + } + + let Some(text) = item.get("text").and_then(Value::as_str) else { + return; + }; + + fingerprints.record_text( + text, + SummaryObservationContext { + content_item_type: Some("input_text"), + under_content_array: true, + source_category, + ..SummaryObservationContext::default() + }, + ); +} + +fn collect_summary_fingerprints_into_input( + value: &Value, + fingerprints: &mut SummaryFingerprintHits, +) { + match value { + Value::Array(items) => { + for (index, item) in items.iter().enumerate() { + let context = SummaryObservationContext { + final_input_item: index + 1 == items.len(), + ..SummaryObservationContext::default() + }; + collect_summary_fingerprints_from_input_item(item, context, fingerprints); + } + } + _ => collect_summary_fingerprints_into( + value, + SummaryObservationContext::default(), + fingerprints, + ), + } +} + +fn collect_summary_fingerprints_from_input_item<'a>( + value: &'a Value, + mut context: SummaryObservationContext<'a>, + fingerprints: &mut SummaryFingerprintHits, +) { + if let Some(item) = value.as_object() { + context.content_item_type = item.get("type").and_then(Value::as_str); + if context.content_item_type == Some("message") { + context.message_role = item.get("role").and_then(Value::as_str); + context.source_category = InputSourceCategory::from_role(context.message_role); + + if let Some(content) = item.get("content").and_then(Value::as_array) { + for content_item in content { + let mut content_context = context; + content_context.under_content_array = true; + content_context.content_item_type = content_item + .as_object() + .and_then(|object| object.get("type")) + .and_then(Value::as_str); + collect_summary_fingerprints_into(content_item, content_context, fingerprints); + } + return; + } + } + } + + collect_summary_fingerprints_into(value, context, fingerprints); +} + +fn collect_summary_fingerprints_into<'a>( + value: &'a Value, + context: SummaryObservationContext<'a>, + fingerprints: &mut SummaryFingerprintHits, +) { + match value { + Value::String(text) => fingerprints.record_text(text, context), + Value::Array(values) => { + for value in values { + collect_summary_fingerprints_into(value, context, fingerprints); + } + } + Value::Object(values) => { + for value in values.values() { + collect_summary_fingerprints_into(value, context, fingerprints); + } + } + Value::Null | Value::Bool(_) | Value::Number(_) => {} + } +} + +fn input_item_count(value: &Value) -> usize { + match value { + Value::Array(items) => items.len(), + Value::Null => 0, + _ => 1, + } +} + +fn last_input_role(value: &Value) -> Option { + let value = match value { + Value::Array(items) => items.last()?, + _ => value, + }; + + value + .as_object() + .and_then(|object| object.get("role")) + .and_then(Value::as_str) + .map(ToOwned::to_owned) +} + +fn last_input_type(value: &Value) -> Option { + let value = match value { + Value::Array(items) => items.last()?, + _ => value, + }; + + safe_value_type_label(Some(value)) +} + +fn safe_value_type_label(value: Option<&Value>) -> Option { + match value? { + Value::String(_) => Some("string".to_string()), + Value::Array(_) => Some("array".to_string()), + Value::Object(object) => object + .get("type") + .and_then(Value::as_str) + .map(ToOwned::to_owned) + .or_else(|| Some("object".to_string())), + Value::Bool(_) => Some("bool".to_string()), + Value::Number(_) => Some("number".to_string()), + Value::Null => Some("null".to_string()), + } +} + +pub(super) fn sse_payload_chunk(event: &str, payload: &str) -> Bytes { + Bytes::from(format!("event: {event}\ndata: {payload}\n\n")) +} + +pub(super) fn sse_json_chunk(event: &str, payload: &Value) -> Bytes { + let payload = serde_json::to_string(payload).expect("serialize downstream sse payload"); + sse_payload_chunk(event, &payload) +} + +pub(super) fn sse_done_chunk() -> Bytes { + Bytes::from_static(b"data: [DONE]\n\n") +} + +fn safe_object_clone(value: Option<&Value>) -> Option { + value.and_then(Value::as_object).cloned().map(Value::Object) +} + +fn sanitized_terminal_response(payload: &Value, status: &str) -> Map { + let source = payload.get("response").and_then(Value::as_object); + let mut response = Map::new(); + + if let Some(response_id) = source + .and_then(|value| value.get("id")) + .and_then(safe_scalar_field) + { + response.insert("id".to_string(), Value::String(response_id)); + } + + if let Some(model) = source + .and_then(|value| value.get("model")) + .and_then(safe_scalar_field) + { + response.insert("model".to_string(), Value::String(model)); + } + + if let Some(usage) = safe_object_clone(source.and_then(|value| value.get("usage"))) { + response.insert("usage".to_string(), usage); + } + + if let Some(output) = payload + .get("response") + .and_then(|value| value.get("output")) + .and_then(Value::as_array) + .cloned() + { + response.insert("output".to_string(), Value::Array(output)); + } + + response.insert("status".to_string(), Value::String(status.to_string())); + response +} + +pub(super) fn sse_terminal_response_failed_chunk(payload: &Value) -> Bytes { + let fallback = ThreadlineError::UpstreamResponseFailed.public_error(); + let error = payload.get("error"); + let mut response = sanitized_terminal_response(payload, "failed"); + response.insert( + "error".to_string(), + Value::Object(Map::from_iter([ + ( + "code".to_string(), + Value::String( + error + .and_then(|value| value.get("code")) + .and_then(safe_scalar_field) + .unwrap_or_else(|| fallback.code.into_owned()), + ), + ), + ( + "message".to_string(), + Value::String( + error + .and_then(|value| value.get("message")) + .and_then(safe_scalar_field) + .unwrap_or_else(|| fallback.message.into_owned()), + ), + ), + ])), + ); + + sse_json_chunk( + "response.failed", + &Value::Object(Map::from_iter([ + ( + "type".to_string(), + Value::String("response.failed".to_string()), + ), + ("response".to_string(), Value::Object(response)), + ])), + ) +} + +pub(super) fn sse_terminal_response_incomplete_chunk(payload: &Value) -> Bytes { + let mut response = sanitized_terminal_response(payload, "incomplete"); + + if let Some(reason) = payload + .get("response") + .and_then(|value| value.get("incomplete_details")) + .and_then(Value::as_object) + .and_then(|value| value.get("reason")) + .and_then(safe_scalar_field) + { + response.insert( + "incomplete_details".to_string(), + Value::Object(Map::from_iter([( + "reason".to_string(), + Value::String(reason), + )])), + ); + } + + sse_json_chunk( + "response.incomplete", + &Value::Object(Map::from_iter([ + ( + "type".to_string(), + Value::String("response.incomplete".to_string()), + ), + ("response".to_string(), Value::Object(response)), + ])), + ) +} + +pub(super) fn safe_scalar_field(value: &Value) -> Option { + match value { + Value::String(text) => Some(text.clone()), + Value::Number(number) => Some(number.to_string()), + Value::Bool(flag) => Some(flag.to_string()), + _ => None, + } +} + +pub(super) fn sse_error_chunk(error: &ThreadlineError) -> Bytes { + let payload = serde_json::to_value(error.public_error_document()) + .expect("convert threadline error payload to json value"); + sse_json_chunk("error", &payload) +} + +#[cfg(test)] +mod tests { + use super::{ + DownstreamInteractionType, DownstreamRequestClassification, DownstreamRequestMetadata, + looks_like_auxiliary_summary_conflict_fallback, parse_downstream_request, + parse_downstream_request_with_metadata, safe_scalar_field, sse_done_chunk, sse_error_chunk, + sse_json_chunk, sse_payload_chunk, sse_terminal_response_failed_chunk, + sse_terminal_response_incomplete_chunk, wants_reasoning_all_turns, + }; + use crate::errors::ThreadlineError; + use serde_json::{Value, json}; + + fn auxiliary_summary_text() -> &'static str { + concat!( + "The conversation has grown too large for the context window and must be compacted now", + "\n\n", + "Your ONLY task right now is to produce a comprehensive summary", + "\n", + "Output your summary wrapped in and tags" + ) + } + + fn auxiliary_summary_input_item() -> Value { + json!({ + "type": "message", + "role": "system", + "content": [ + { + "type": "input_text", + "text": auxiliary_summary_text() + } + ] + }) + } + + fn manual_summary_text() -> &'static str { + concat!( + "Summarize the conversation history so far, paying special attention to the most recent agent commands and tool results", + "\n\n", + "Structure your summary using the enhanced format provided in the system message", + "\n", + "Include all important tool calls and their results" + ) + } + + fn manual_summary_input_item() -> Value { + json!({ + "type": "message", + "role": "system", + "content": [ + { + "type": "input_text", + "text": manual_summary_text() + } + ] + }) + } + + fn manual_simple_summary_text() -> &'static str { + concat!( + "Summarize the conversation history so far, paying special attention to the most recent agent commands and tool results", + "\n\n", + "Include all important tool calls and their results" + ) + } + + fn manual_simple_summary_input_item() -> Value { + json!({ + "type": "message", + "role": "system", + "content": [ + { + "type": "input_text", + "text": manual_simple_summary_text() + } + ] + }) + } + + fn simple_history_context_text() -> &'static str { + "The following is a compressed version of the preceeding history in the current conversation" + } + + fn simple_history_context_input_item() -> Value { + json!({ + "type": "message", + "role": "system", + "content": [ + { + "type": "input_text", + "text": simple_history_context_text() + } + ] + }) + } + + fn new_auto_system_summary_text() -> &'static str { + "Your task is to create a comprehensive, detailed summary of the entire conversation that captures all essential information needed to seamlessly continue the work without any loss of context" + } + + fn new_auto_compressed_history_text() -> &'static str { + "The following is a compressed version of the preceeding history in the current conversation" + } + + fn new_auto_compressed_history_text_corrected() -> &'static str { + "The following is a compressed version of the preceding history in the current conversation" + } + + fn new_auto_final_summary_prompt_text() -> &'static str { + concat!( + "Summarize the conversation history so far, paying special attention to the most recent agent commands and tool results that triggered this summarization.", + " Structure your summary using the enhanced format provided in the system message.\n", + "Focus particularly on:\n", + "- The specific agent commands/tools that were just executed\n", + "- The results returned from these recent tool calls (truncate if very long but preserve key information)\n", + "- What the agent was actively working on when the token budget was exceeded\n", + "- How these recent operations connect to the overall user goals\n", + "Include all important tool calls and their results as part of the appropriate sections, with special emphasis on the most recent operations." + ) + } + + fn input_text_message(role: &str, text: &str) -> Value { + json!({ + "type": "message", + "role": role, + "content": [ + { + "type": "input_text", + "text": text + } + ] + }) + } + + #[test] + fn wants_reasoning_all_turns_matches_only_exact_string_value() { + assert!(wants_reasoning_all_turns( + json!({ + "reasoning": { + "context": "all_turns" + } + }) + .as_object() + .expect("object payload") + )); + + assert!(!wants_reasoning_all_turns( + json!({ + "reasoning": { + "context": "last_turn" + } + }) + .as_object() + .expect("object payload") + )); + } + + #[test] + fn wants_reasoning_all_turns_returns_false_for_missing_or_non_object_reasoning() { + assert!(!wants_reasoning_all_turns( + json!({ "input": "no reasoning" }) + .as_object() + .expect("object payload") + )); + + assert!(!wants_reasoning_all_turns( + json!({ "reasoning": "all_turns" }) + .as_object() + .expect("object payload") + )); + } + + #[test] + fn wants_reasoning_all_turns_returns_false_for_non_string_context() { + assert!(!wants_reasoning_all_turns( + json!({ + "reasoning": { + "context": true + } + }) + .as_object() + .expect("object payload") + )); + } + + fn new_auto_system_summary_input_item() -> Value { + input_text_message("system", new_auto_system_summary_text()) + } + + fn new_auto_compressed_history_input_item() -> Value { + input_text_message("user", new_auto_compressed_history_text()) + } + + fn new_auto_compressed_history_input_item_corrected() -> Value { + input_text_message("user", new_auto_compressed_history_text_corrected()) + } + + fn new_auto_final_summary_prompt_input_item() -> Value { + input_text_message("user", new_auto_final_summary_prompt_text()) + } + + fn classify_input(input: Vec) -> DownstreamRequestClassification { + parse_downstream_request(json!({ + "previous_response_id": "resp_123", + "input": input + })) + .expect("parse request") + .classification + } + + fn parse_input_with_metadata( + input: Vec, + metadata: DownstreamRequestMetadata, + ) -> super::DownstreamResponsesRequest { + parse_downstream_request_with_metadata( + json!({ + "previous_response_id": "resp_123", + "input": input + }), + metadata, + ) + .expect("parse request") + } + + fn sanitized_observed_auxiliary_summary_request() -> Value { + json!({ + "model": "gpt-5.4", + "previous_response_id": "resp_123", + "context_management": { + "type": "compaction", + "compact_threshold": 12345 + }, + "tools": [ + { + "type": "function", + "name": "user_tool", + "description": "User-defined tool", + "parameters": { + "type": "object", + "properties": {}, + "additionalProperties": false + } + }, + { + "type": "function", + "name": "threadline_echo", + "description": "Threadline internal tool", + "parameters": { + "type": "object", + "properties": { + "value": { + "type": "string" + } + }, + "required": ["value"], + "additionalProperties": false + } + } + ], + "input": [ + { + "type": "message", + "role": "user", + "content": [ + { + "type": "input_text", + "text": "Continue from the earlier answer." + } + ] + }, + auxiliary_summary_input_item() + ] + }) + } + + #[test] + fn parse_downstream_request_extracts_previous_response_id_and_payload() { + let request = parse_downstream_request(json!({ + "previous_response_id": "resp_123", + "model": "gpt-5.4", + "stream": true + })) + .expect("parse request"); + + assert_eq!(request.previous_response_id.as_deref(), Some("resp_123")); + assert_eq!(request.payload.get("model"), Some(&json!("gpt-5.4"))); + assert_eq!(request.payload.get("stream"), Some(&json!(true))); + assert!(!request.payload.contains_key("previous_response_id")); + } + + #[test] + fn parse_downstream_request_identifies_auxiliary_summary_request() { + let request = parse_downstream_request(sanitized_observed_auxiliary_summary_request()) + .expect("parse request"); + + assert_eq!( + request.classification, + DownstreamRequestClassification::AuxiliarySummary + ); + } + + #[test] + fn parse_downstream_request_does_not_classify_context_management_only() { + let request = parse_downstream_request(json!({ + "previous_response_id": "resp_123", + "context_management": { + "type": "auto" + }, + "input": [ + { + "type": "message", + "role": "user", + "content": [ + { + "type": "input_text", + "text": "Please continue the earlier task." + } + ] + } + ] + })) + .expect("parse request"); + + assert_eq!( + request.classification, + DownstreamRequestClassification::Normal + ); + } + + #[test] + fn parse_downstream_request_classifies_conversation_compaction_with_context_management() { + let request = parse_downstream_request_with_metadata( + json!({ + "previous_response_id": "resp_123", + "context_management": { + "type": "compaction", + "compact_threshold": 12345 + }, + "input": [ + { + "type": "message", + "role": "user", + "content": [ + { + "type": "input_text", + "text": "Please continue the earlier task." + } + ] + } + ] + }), + DownstreamRequestMetadata::from_interaction_type_header_value(Some( + "conversation-compaction", + )), + ) + .expect("parse request"); + + assert_eq!( + request.routing_diagnostics().interaction_type, + DownstreamInteractionType::ConversationCompaction + ); + assert!( + request + .routing_diagnostics() + .interaction_type_compaction_hit + ); + assert_eq!( + request.classification, + DownstreamRequestClassification::AuxiliarySummary + ); + } + + #[test] + fn parse_downstream_request_does_not_classify_fingerprints_outside_input() { + let request = parse_downstream_request(json!({ + "previous_response_id": "resp_123", + "metadata": { + "summary_prompt": auxiliary_summary_text() + }, + "input": [ + { + "type": "message", + "role": "user", + "content": [ + { + "type": "input_text", + "text": "Please continue the earlier task." + } + ] + } + ] + })) + .expect("parse request"); + + assert_eq!( + request.classification, + DownstreamRequestClassification::Normal + ); + } + + #[test] + fn parse_downstream_request_does_not_classify_partial_summary_quote() { + let request = parse_downstream_request(json!({ + "previous_response_id": "resp_123", + "input": [ + { + "type": "message", + "role": "system", + "content": [ + { + "type": "input_text", + "text": "The conversation has grown too large for the context window and must be compacted now" + } + ] + } + ] + })) + .expect("parse request"); + + assert_eq!( + request.classification, + DownstreamRequestClassification::Normal + ); + } + + #[test] + fn parse_downstream_request_does_not_classify_user_role_full_prompt_quote() { + let request = parse_downstream_request(json!({ + "previous_response_id": "resp_123", + "input": [ + { + "type": "message", + "role": "user", + "content": [ + { + "type": "input_text", + "text": auxiliary_summary_text() + } + ] + } + ] + })) + .expect("parse request"); + + assert_eq!( + request.classification, + DownstreamRequestClassification::Normal + ); + } + + #[test] + fn parse_downstream_request_classifies_manual_full_summary_prompt_fingerprints() { + assert_eq!( + classify_input(vec![ + json!({ + "type": "message", + "role": "user", + "content": [ + { + "type": "input_text", + "text": "Continue from the earlier answer." + } + ] + }), + manual_summary_input_item(), + ]), + DownstreamRequestClassification::AuxiliarySummary + ); + } + + #[test] + fn parse_downstream_request_classifies_manual_simple_summary_prompt_fingerprints() { + assert_eq!( + classify_input(vec![ + manual_simple_summary_input_item(), + json!({ + "type": "message", + "role": "user", + "content": [ + { + "type": "input_text", + "text": "Acknowledge the compaction request." + } + ] + }), + ]), + DownstreamRequestClassification::AuxiliarySummary + ); + } + + #[test] + fn parse_downstream_request_does_not_classify_user_role_manual_summary_quote_only() { + assert_eq!( + classify_input(vec![json!({ + "type": "message", + "role": "user", + "content": [ + { + "type": "input_text", + "text": format!("Quoted prompt: {}", manual_summary_text()) + } + ] + })]), + DownstreamRequestClassification::Normal + ); + } + + #[test] + fn parse_downstream_request_classifies_new_auto_compaction_prompt_fingerprints() { + assert_eq!( + classify_input(vec![ + new_auto_system_summary_input_item(), + new_auto_compressed_history_input_item(), + new_auto_final_summary_prompt_input_item(), + ]), + DownstreamRequestClassification::AuxiliarySummary + ); + } + + #[test] + fn parse_downstream_request_does_not_classify_new_auto_user_quote_only() { + assert_eq!( + classify_input(vec![ + new_auto_compressed_history_input_item(), + new_auto_final_summary_prompt_input_item(), + ]), + DownstreamRequestClassification::Normal + ); + } + + #[test] + fn parse_downstream_request_classifies_interaction_type_conversation_compaction() { + let request = parse_input_with_metadata( + vec![input_text_message( + "user", + "Please continue the earlier task.", + )], + DownstreamRequestMetadata::from_interaction_type_header_value(Some( + "conversation-compaction", + )), + ); + + assert_eq!( + request.routing_diagnostics().interaction_type, + DownstreamInteractionType::ConversationCompaction + ); + assert!( + request + .routing_diagnostics() + .interaction_type_compaction_hit + ); + assert_eq!( + request.classification, + DownstreamRequestClassification::AuxiliarySummary + ); + } + + #[test] + fn parse_downstream_request_trims_and_lowercases_interaction_type() { + let request = parse_input_with_metadata( + vec![input_text_message( + "user", + "Please continue the earlier task.", + )], + DownstreamRequestMetadata::from_interaction_type_header_value(Some( + " Conversation-Compaction ", + )), + ); + + assert_eq!( + request.routing_diagnostics().interaction_type, + DownstreamInteractionType::ConversationCompaction + ); + assert!( + request + .routing_diagnostics() + .interaction_type_compaction_hit + ); + assert_eq!( + request.classification, + DownstreamRequestClassification::AuxiliarySummary + ); + } + + #[test] + fn parse_downstream_request_ignores_unknown_empty_non_utf8_or_long_interaction_type() { + let long_value = "x".repeat(65); + + for (name, metadata, expected_interaction_type) in [ + ( + "missing", + DownstreamRequestMetadata::default(), + DownstreamInteractionType::None, + ), + ( + "empty", + DownstreamRequestMetadata::from_interaction_type_header_value(Some(" ")), + DownstreamInteractionType::None, + ), + ( + "unknown", + DownstreamRequestMetadata::from_interaction_type_header_value(Some( + "conversation-start", + )), + DownstreamInteractionType::Other, + ), + ( + "non_utf8", + DownstreamRequestMetadata::from_interaction_type_header_bytes(Some(b"\xFF")), + DownstreamInteractionType::Other, + ), + ( + "too_long", + DownstreamRequestMetadata::from_interaction_type_header_value(Some(&long_value)), + DownstreamInteractionType::Other, + ), + ] { + let request = parse_input_with_metadata( + vec![input_text_message( + "user", + "Please continue the earlier task.", + )], + metadata, + ); + + assert_eq!( + request.classification, + DownstreamRequestClassification::Normal, + "fixture should remain normal: {name}" + ); + assert_eq!( + request.routing_diagnostics().interaction_type, + expected_interaction_type, + "fixture should use allowlisted interaction type diagnostics: {name}" + ); + assert!( + !request + .routing_diagnostics() + .interaction_type_compaction_hit + ); + } + } + + #[test] + fn looks_like_auxiliary_summary_conflict_fallback_detects_nested_vscode_summary_shape_without_context_management() + { + let payload = json!({ + "input": [ + { + "type": "message", + "role": "system", + "content": [ + { + "type": "input_text", + "text": manual_summary_text() + }, + { + "type": "input_text", + "text": simple_history_context_text() + } + ] + } + ] + }); + + let payload = payload + .as_object() + .expect("payload object for conflict fallback test"); + + assert!(looks_like_auxiliary_summary_conflict_fallback(payload)); + } + + #[test] + fn looks_like_auxiliary_summary_conflict_fallback_rejects_nested_ordinary_quoted_text() { + let payload = json!({ + "input": [ + { + "type": "message", + "role": "user", + "content": [ + { + "type": "input_text", + "text": format!( + "Quoted prompt: {}", + manual_summary_text() + ) + } + ] + } + ] + }); + + let payload = payload + .as_object() + .expect("payload object for conflict fallback test"); + + assert!(!looks_like_auxiliary_summary_conflict_fallback(payload)); + } + + #[test] + fn parse_downstream_request_does_not_classify_new_auto_partial_fingerprints() { + for (name, input) in [ + ( + "system_plus_history_only", + vec![ + new_auto_system_summary_input_item(), + new_auto_compressed_history_input_item(), + ], + ), + ( + "history_plus_final_prompt_only", + vec![ + new_auto_compressed_history_input_item(), + new_auto_final_summary_prompt_input_item(), + ], + ), + ("system_only", vec![new_auto_system_summary_input_item()]), + ( + "history_only", + vec![new_auto_compressed_history_input_item()], + ), + ( + "final_prompt_only", + vec![new_auto_final_summary_prompt_input_item()], + ), + ] { + assert_eq!( + classify_input(input), + DownstreamRequestClassification::Normal, + "fixture should remain normal: {name}" + ); + } + } + + #[test] + fn parse_downstream_request_classifies_new_foreground_summary_prompt_without_compressed_history() + { + assert_eq!( + classify_input(vec![ + new_auto_system_summary_input_item(), + new_auto_final_summary_prompt_input_item(), + ]), + DownstreamRequestClassification::AuxiliarySummary + ); + } + + #[test] + fn parse_downstream_request_does_not_classify_user_role_foreground_prompt_quote_only() { + assert_eq!( + classify_input(vec![json!({ + "type": "message", + "role": "user", + "content": [ + { + "type": "input_text", + "text": format!("Quoted prompt: {}", new_auto_final_summary_prompt_text()) + } + ] + })]), + DownstreamRequestClassification::Normal + ); + } + + #[test] + fn parse_downstream_request_does_not_classify_new_foreground_partial_fingerprints() { + for (name, input) in [ + ("system_only", vec![new_auto_system_summary_input_item()]), + ( + "final_prompt_only", + vec![new_auto_final_summary_prompt_input_item()], + ), + ] { + assert_eq!( + classify_input(input), + DownstreamRequestClassification::Normal, + "fixture should remain normal: {name}" + ); + } + } + + #[test] + fn parse_downstream_request_does_not_classify_foreground_prompt_when_not_final_user_input() { + assert_eq!( + classify_input(vec![ + new_auto_system_summary_input_item(), + new_auto_final_summary_prompt_input_item(), + input_text_message("user", "Please continue the earlier task."), + ]), + DownstreamRequestClassification::Normal + ); + } + + #[test] + fn parse_downstream_request_does_not_classify_split_ordinary_conversation_quotes() { + assert_eq!( + classify_input(vec![ + input_text_message( + "user", + &format!( + "The user quoted this instruction earlier: {}", + new_auto_system_summary_text() + ), + ), + input_text_message( + "user", + &format!( + "The user later quoted this prompt too: {}", + new_auto_final_summary_prompt_text() + ), + ), + ]), + DownstreamRequestClassification::Normal + ); + } + + #[test] + fn parse_downstream_request_does_not_classify_new_foreground_fingerprints_outside_input_text() { + let request = parse_downstream_request(json!({ + "previous_response_id": "resp_123", + "metadata": { + "system_prompt": new_auto_system_summary_text(), + "final_prompt": new_auto_final_summary_prompt_text() + }, + "tools": [ + { + "type": "function", + "name": "echo", + "description": new_auto_final_summary_prompt_text(), + "parameters": { + "type": "object", + "properties": { + "summary": { + "type": "string" + } + } + } + } + ], + "input": [ + { + "type": "message", + "role": "system", + "content": [ + { + "type": "input_image", + "image_url": new_auto_system_summary_text() + } + ] + }, + { + "type": "message", + "role": "user", + "content": [ + { + "type": "input_text", + "text": "Please continue the earlier task." + } + ] + } + ] + })) + .expect("parse request"); + + assert_eq!( + request.classification, + DownstreamRequestClassification::Normal + ); + } + + #[test] + fn parse_downstream_request_does_not_classify_new_auto_fingerprints_outside_input_text() { + let request = parse_downstream_request(json!({ + "previous_response_id": "resp_123", + "metadata": { + "system_prompt": new_auto_system_summary_text(), + "history": new_auto_compressed_history_text(), + "final_prompt": new_auto_final_summary_prompt_text() + }, + "tools": [ + { + "type": "function", + "name": "echo", + "parameters": { + "type": "object", + "properties": { + "summary": { + "type": "string", + "description": new_auto_final_summary_prompt_text() + } + } + } + } + ], + "input": [ + { + "type": "message", + "role": "system", + "content": [ + { + "type": "input_image", + "image_url": new_auto_system_summary_text() + } + ] + }, + { + "type": "message", + "role": "user", + "content": [ + { + "type": "input_text", + "text": "Please continue the earlier task." + } + ] + } + ] + })) + .expect("parse request"); + + assert_eq!( + request.classification, + DownstreamRequestClassification::Normal + ); + } + + #[test] + fn parse_downstream_request_classifies_new_auto_compaction_prompt_with_corrected_history_spelling() + { + assert_eq!( + classify_input(vec![ + new_auto_system_summary_input_item(), + new_auto_compressed_history_input_item_corrected(), + new_auto_final_summary_prompt_input_item(), + ]), + DownstreamRequestClassification::AuxiliarySummary + ); + } + + #[test] + fn parse_downstream_request_classifies_auto_background_compaction_in_non_final_shapes() { + for (name, input) in [ + ( + "auto_summary_followed_by_user_message", + vec![ + auxiliary_summary_input_item(), + json!({ + "type": "message", + "role": "user", + "content": [ + { + "type": "input_text", + "text": "Please keep this request moving." + } + ] + }), + ], + ), + ( + "auto_summary_before_non_message_item", + vec![ + auxiliary_summary_input_item(), + json!({ + "type": "input_text", + "text": "Resume after compaction." + }), + ], + ), + ] { + assert_eq!( + classify_input(input), + DownstreamRequestClassification::AuxiliarySummary, + "fixture should classify as auxiliary summary: {name}" + ); + } + } + + #[test] + fn parse_downstream_request_classifies_simple_history_context_only_with_summary_prompt() { + for (name, input) in [ + ( + "simple_history_plus_manual_summary_prompt", + vec![ + simple_history_context_input_item(), + manual_summary_input_item(), + ], + ), + ( + "simple_history_plus_auto_summary_prompt", + vec![ + simple_history_context_input_item(), + auxiliary_summary_input_item(), + ], + ), + ] { + assert_eq!( + classify_input(input), + DownstreamRequestClassification::AuxiliarySummary, + "fixture should classify as auxiliary summary: {name}" + ); + } + } + + #[test] + fn parse_downstream_request_keeps_ordinary_and_quoted_summary_shapes_normal() { + for (name, input) in [ + ( + "ordinary_user_prompt", + vec![json!({ + "type": "message", + "role": "user", + "content": [ + { + "type": "input_text", + "text": "Please continue the earlier task." + } + ] + })], + ), + ( + "simple_history_only_context", + vec![simple_history_context_input_item()], + ), + ( + "user_role_full_prompt_quote_with_simple_history_context", + vec![ + simple_history_context_input_item(), + json!({ + "type": "message", + "role": "user", + "content": [ + { + "type": "input_text", + "text": concat!( + "Quoted prompt: ", + "Summarize the conversation history so far, paying special attention to the most recent agent commands and tool results", + "\n\n", + "Structure your summary using the enhanced format provided in the system message", + "\n", + "Include all important tool calls and their results" + ) + } + ] + }), + ], + ), + ] { + assert_eq!( + classify_input(input), + DownstreamRequestClassification::Normal, + "fixture should remain normal: {name}" + ); + } + } + + #[test] + fn sse_payload_chunk_keeps_single_line_frame() { + let chunk = sse_payload_chunk("response.output_text.delta", "{\"delta\":\"hi\"}"); + + assert_eq!( + std::str::from_utf8(&chunk).expect("utf8"), + "event: response.output_text.delta\ndata: {\"delta\":\"hi\"}\n\n" + ); + } + + #[test] + fn sse_json_chunk_serializes_compact_json() { + let chunk = sse_json_chunk("response.completed", &json!({"id":"resp_1","ok":true})); + + assert_eq!( + std::str::from_utf8(&chunk).expect("utf8"), + "event: response.completed\ndata: {\"id\":\"resp_1\",\"ok\":true}\n\n" + ); + } + + #[test] + fn sse_done_chunk_keeps_bare_done_payload() { + let chunk = sse_done_chunk(); + + assert_eq!( + std::str::from_utf8(&chunk).expect("utf8"), + "data: [DONE]\n\n" + ); + } + + #[test] + fn sse_terminal_response_failed_chunk_preserves_public_shape() { + let chunk = sse_terminal_response_failed_chunk(&json!({ + "type": "response.failed", + "response": { "id": 42 }, + "error": { + "code": "tool_timeout", + "message": "tool timed out" + } + })); + + assert_eq!( + std::str::from_utf8(&chunk).expect("utf8"), + concat!( + "event: response.failed\n", + "data: {\"response\":{\"error\":{\"code\":\"tool_timeout\",\"message\":\"tool timed out\"},\"id\":\"42\",\"status\":\"failed\"},\"type\":\"response.failed\"}\n\n" + ) + ); + } + + #[test] + fn sse_terminal_response_failed_chunk_uses_fallback_error_fields() { + let chunk = sse_terminal_response_failed_chunk(&json!({ + "type": "response.failed", + "response": {}, + "error": {} + })); + + assert_eq!( + std::str::from_utf8(&chunk).expect("utf8"), + concat!( + "event: response.failed\n", + "data: {\"response\":{\"error\":{\"code\":\"upstream_response_failed\",\"message\":\"The upstream response.failed event cannot be streamed as a successful downstream response.\"},\"status\":\"failed\"},\"type\":\"response.failed\"}\n\n" + ) + ); + } + + #[test] + fn sse_terminal_response_incomplete_chunk_preserves_safe_terminal_fields() { + let chunk = sse_terminal_response_incomplete_chunk(&json!({ + "response": { + "id": "response-1", + "model": "gpt-5.4", + "usage": { + "input_tokens": 4, + "output_tokens": 2, + "total_tokens": 6 + }, + "output": [ + { + "id": "assistant-1", + "type": "message", + "role": "assistant", + "content": [ + { + "type": "output_text", + "text": "partial" + } + ] + } + ], + "incomplete_details": { + "reason": "max_output_tokens", + "ignored": {"nested": true} + } + } + })); + + let payload = std::str::from_utf8(&chunk) + .expect("utf8 chunk") + .strip_prefix("event: response.incomplete\ndata: ") + .and_then(|text| text.strip_suffix("\n\n")) + .map(|text| serde_json::from_str::(text).expect("payload json")) + .expect("incomplete payload"); + + assert_eq!(payload["type"], "response.incomplete"); + assert_eq!(payload["response"]["id"], "response-1"); + assert_eq!(payload["response"]["model"], "gpt-5.4"); + assert_eq!(payload["response"]["usage"]["total_tokens"], 6); + assert_eq!( + payload["response"]["output"][0]["content"][0]["text"], + "partial" + ); + assert_eq!( + payload["response"]["incomplete_details"]["reason"], + "max_output_tokens" + ); + } + + #[test] + fn safe_scalar_field_accepts_only_scalar_values() { + assert_eq!( + safe_scalar_field(&json!("hello")), + Some("hello".to_string()) + ); + assert_eq!(safe_scalar_field(&json!(7)), Some("7".to_string())); + assert_eq!(safe_scalar_field(&json!(false)), Some("false".to_string())); + assert_eq!(safe_scalar_field(&Value::Null), None); + assert_eq!(safe_scalar_field(&json!([1, 2, 3])), None); + assert_eq!(safe_scalar_field(&json!({"a": 1})), None); + } + + #[test] + fn sse_error_chunk_preserves_public_error_shape() { + let chunk = sse_error_chunk(&ThreadlineError::UpstreamWebSocketClosed); + + assert_eq!( + std::str::from_utf8(&chunk).expect("utf8"), + concat!( + "event: error\n", + "data: {\"error\":{\"code\":\"upstream_websocket_closed\",\"message\":\"The upstream Codex websocket closed before Threadline finished streaming the response.\",\"type\":\"bad_gateway_error\"}}\n\n" + ) + ); + } +} diff --git a/src/responses/mod.rs b/src/responses/mod.rs new file mode 100644 index 0000000..144bd23 --- /dev/null +++ b/src/responses/mod.rs @@ -0,0 +1,589 @@ +use std::sync::Arc; + +use axum::body::Body; +use axum::extract::State; +use axum::http::{HeaderValue, Response, StatusCode, header}; +use axum::response::IntoResponse; +use serde_json::Value; +use tracing::debug; + +use crate::auth::LoadedUpstreamAuth; +use crate::errors::ThreadlineError; +use crate::models::{RouteProfile, resolve_request_model_for_profile}; +use crate::registry::{RegistryAcquireError, RetainedSessionLease, RetainedSessionRegistry}; +use crate::tools::{inject_internal_tools, is_internal_tool_name}; +use crate::ws_pump::LiveUpstreamWebSocket; + +mod downstream; +mod translation; +mod upstream; + +use self::downstream::{ + DownstreamRequestClassification, looks_like_auxiliary_summary_conflict_fallback, + parse_downstream_request_with_metadata, wants_reasoning_all_turns, +}; +use self::translation::{ResponseStreamLease, ResponseStreamState, response_stream}; +use self::upstream::send_response_create; + +#[cfg(test)] +pub(crate) use self::downstream::DownstreamInteractionType; +pub(crate) use self::downstream::DownstreamRequestMetadata; + +pub use self::upstream::{ + ConnectedUpstream, ThreadlineServices, UpstreamAuthProvider, UpstreamConnector, +}; + +pub const TURN_STATE_HEADER: &str = "x-codex-turn-state"; + +#[derive(Clone)] +pub struct ResponsesRouteState { + pub profile: RouteProfile, + pub registry: Arc, + pub services: ThreadlineServices, +} + +struct PreparedResponseRoute { + upstream: Arc, + lease: ResponseStreamLease, + previous_response_id: Option, + replay_stale_marker_on_pre_first_event_close: bool, + reconnect_attempted: bool, + upstream_request: serde_json::Map, + execute_internal_tools: bool, + apply_no_observable_output_failure: bool, +} + +#[derive(Clone, Copy)] +enum TransientRouteKind { + AuxiliarySummary, + Utility, +} + +impl TransientRouteKind { + fn label(self) -> &'static str { + match self { + Self::AuxiliarySummary => "auxiliary_summary", + Self::Utility => "utility", + } + } +} + +pub(crate) async fn responses_handler( + State(state): State, + axum::Json(payload): axum::Json, + request_metadata: DownstreamRequestMetadata, +) -> Result { + let mut request = parse_downstream_request_with_metadata(payload, request_metadata)?; + let model_alias = resolve_request_model_for_profile(&request.payload, state.profile)?; + if wants_reasoning_all_turns(&request.payload) && !model_alias.supports_reasoning_all_turns { + return Err(ThreadlineError::UnsupportedReasoningContext); + } + request.payload.insert( + "model".to_string(), + Value::String(model_alias.upstream_model_id.to_string()), + ); + let classification = request.classification; + let routing_diagnostics = request.routing_diagnostics().clone(); + let previous_response_id_present = request.previous_response_id.is_some(); + let context_management_present = request.payload.contains_key("context_management"); + debug!( + request_class = request_class_label(classification), + interaction_type = routing_diagnostics.interaction_type.label(), + interaction_type_compaction_hit = routing_diagnostics.interaction_type_compaction_hit, + previous_response_id_present, + context_management_present, + manual_summary_prompt_hit = routing_diagnostics.summary_hits.manual_summary_prompt_hit, + manual_structure_instruction_hit = routing_diagnostics + .summary_hits + .manual_structure_instruction_hit, + manual_tool_results_instruction_hit = routing_diagnostics + .summary_hits + .manual_tool_results_instruction_hit, + auto_context_too_large_hit = routing_diagnostics.summary_hits.auto_context_too_large_hit, + auto_summary_tags_hit = routing_diagnostics.summary_hits.auto_summary_tags_hit, + auto_only_task_hit = routing_diagnostics.summary_hits.auto_only_task_hit, + simple_history_context_hit = routing_diagnostics.summary_hits.simple_history_context_hit, + new_auto_detailed_summary_hit = routing_diagnostics + .summary_hits + .new_auto_detailed_summary_hit, + new_auto_user_history_hit = routing_diagnostics.summary_hits.new_auto_user_history_hit, + new_auto_user_final_summary_prompt_hit = routing_diagnostics + .summary_hits + .new_auto_user_final_summary_prompt_hit, + summary_instruction_like_hit = routing_diagnostics + .summary_hits + .summary_instruction_like_hit, + tool_choice = routing_diagnostics.tool_choice.as_deref().unwrap_or("none"), + tools_count = routing_diagnostics.tools_count, + input_item_count = routing_diagnostics.input_item_count, + last_input_role = routing_diagnostics + .last_input_role + .as_deref() + .unwrap_or("none"), + last_input_type = routing_diagnostics + .last_input_type + .as_deref() + .unwrap_or("none"), + "responses_request_routed" + ); + let base_request = request.payload; + let previous_response_id = request.previous_response_id; + let is_continuation_request = previous_response_id.is_some(); + let prepared = if state.profile == RouteProfile::Utility { + start_transient_route( + &state.services, + base_request, + classification, + TransientRouteKind::Utility, + ) + .await? + } else { + match classification { + DownstreamRequestClassification::Normal => { + match acquire_lease(&state.registry, previous_response_id.as_deref()).await { + Ok(mut lease) => { + let mut upstream_request = base_request.clone(); + inject_internal_tools(&mut upstream_request); + strip_context_management_for_upstream( + &mut upstream_request, + "normal", + classification, + ); + let mut reconnect_attempted = false; + let upstream = if let Some(previous_response_id) = &previous_response_id { + if !lease.has_open_upstream() { + debug!( + previous_response_id, + session_id = %lease.session().session_id, + thread_id = %lease.session().thread_id, + window_id = %lease.session().window_id, + stale_reason = "missing_or_closed_upstream", + "stale_previous_response_requires_client_replay" + ); + lease.release(); + return Err(ThreadlineError::PreviousResponseNotFound); + } + + upstream_request.insert( + "previous_response_id".to_string(), + Value::String(previous_response_id.clone()), + ); + + let upstream = lease.upstream().expect( + "open retained upstream must exist for continuation preflight", + ); + if let Err(error) = + send_response_create(&upstream, &upstream_request).await + { + let error = rewrite_stale_continuation_first_send_error(error); + if matches!(error, ThreadlineError::PreviousResponseNotFound) { + debug!( + previous_response_id, + session_id = %lease.session().session_id, + thread_id = %lease.session().thread_id, + window_id = %lease.session().window_id, + stale_reason = "first_send_closed", + "stale_previous_response_requires_client_replay" + ); + lease.release(); + return Err(ThreadlineError::PreviousResponseNotFound); + } + + return Err(error); + } + + tokio::task::yield_now().await; + if upstream.is_closed() { + debug!( + previous_response_id, + session_id = %lease.session().session_id, + thread_id = %lease.session().thread_id, + window_id = %lease.session().window_id, + stale_reason = "first_send_closed_after_enqueue", + "stale_previous_response_requires_client_replay" + ); + lease.release(); + return Err(ThreadlineError::PreviousResponseNotFound); + } + + upstream + } else { + let auth = state.services.auth_provider().load()?; + let mut upstream = + ensure_upstream(&state.services, &mut lease, auth).await?; + if let Err(error) = + send_response_create(&upstream, &upstream_request).await + { + if let Some(reconnected) = attempt_pre_first_event_reconnect( + &state.services, + &mut lease, + &upstream_request, + previous_response_id.as_deref(), + false, + &mut reconnect_attempted, + ) + .await? + { + upstream = reconnected; + } else { + return Err(error); + } + } + + upstream + }; + + PreparedResponseRoute { + upstream, + lease: ResponseStreamLease::Retained(lease), + previous_response_id, + replay_stale_marker_on_pre_first_event_close: is_continuation_request, + reconnect_attempted, + upstream_request, + execute_internal_tools: true, + apply_no_observable_output_failure: true, + } + } + Err(ThreadlineError::RetainedSessionConflict) => { + let reroute_reason = retained_session_conflict_reroute_reason( + &routing_diagnostics, + &base_request, + ); + if reroute_reason.is_none() { + return Err(ThreadlineError::RetainedSessionConflict); + } + let reroute_reason = reroute_reason.expect("reroute reason present"); + + debug!( + reroute_reason, + request_class = request_class_label(classification), + interaction_type = routing_diagnostics.interaction_type.label(), + interaction_type_compaction_hit = + routing_diagnostics.interaction_type_compaction_hit, + previous_response_id_present, + context_management_present, + manual_summary_prompt_hit = + routing_diagnostics.summary_hits.manual_summary_prompt_hit, + manual_structure_instruction_hit = routing_diagnostics + .summary_hits + .manual_structure_instruction_hit, + manual_tool_results_instruction_hit = routing_diagnostics + .summary_hits + .manual_tool_results_instruction_hit, + auto_context_too_large_hit = + routing_diagnostics.summary_hits.auto_context_too_large_hit, + auto_summary_tags_hit = + routing_diagnostics.summary_hits.auto_summary_tags_hit, + auto_only_task_hit = + routing_diagnostics.summary_hits.auto_only_task_hit, + simple_history_context_hit = + routing_diagnostics.summary_hits.simple_history_context_hit, + new_auto_detailed_summary_hit = routing_diagnostics + .summary_hits + .new_auto_detailed_summary_hit, + new_auto_user_history_hit = + routing_diagnostics.summary_hits.new_auto_user_history_hit, + new_auto_user_final_summary_prompt_hit = routing_diagnostics + .summary_hits + .new_auto_user_final_summary_prompt_hit, + summary_instruction_like_hit = routing_diagnostics + .summary_hits + .summary_instruction_like_hit, + fallback_summary_input_hit = reroute_reason == "fallback_summary_input", + tool_choice = + routing_diagnostics.tool_choice.as_deref().unwrap_or("none"), + tools_count = routing_diagnostics.tools_count, + input_item_count = routing_diagnostics.input_item_count, + last_input_role = routing_diagnostics + .last_input_role + .as_deref() + .unwrap_or("none"), + last_input_type = routing_diagnostics + .last_input_type + .as_deref() + .unwrap_or("none"), + "retained_session_conflict_rerouted" + ); + + start_transient_route( + &state.services, + base_request, + classification, + TransientRouteKind::AuxiliarySummary, + ) + .await? + } + Err(error) => return Err(error), + } + } + DownstreamRequestClassification::AuxiliarySummary => { + start_transient_route( + &state.services, + base_request, + classification, + TransientRouteKind::AuxiliarySummary, + ) + .await? + } + } + }; + + let stream = response_stream(ResponseStreamState { + services: state.services.clone(), + upstream: prepared.upstream, + lease: prepared.lease, + base_request: prepared.upstream_request, + pending_internal_outputs: Vec::new(), + previous_response_id: prepared.previous_response_id, + execute_internal_tools: prepared.execute_internal_tools, + suppressed_internal_output_indexes: std::collections::HashSet::new(), + upstream_event_seen: false, + replay_stale_marker_on_pre_first_event_close: prepared + .replay_stale_marker_on_pre_first_event_close, + reconnect_attempted: prepared.reconnect_attempted, + observable_output: Default::default(), + downstream_visible_text_sources: std::collections::HashSet::new(), + downstream_visible_text_delta_count: 0, + visible_assistant_text: Vec::new(), + last_unidentified_visible_text: None, + queued_synthetic_output_text_deltas: std::collections::VecDeque::new(), + queued_forwarded_event: None, + queued_final_completed: None, + final_done_pending: false, + apply_no_observable_output_failure: prepared.apply_no_observable_output_failure, + done: false, + }); + + let response = Response::builder() + .status(StatusCode::OK) + .header( + header::CONTENT_TYPE, + HeaderValue::from_static("text/event-stream"), + ) + .header(header::CACHE_CONTROL, HeaderValue::from_static("no-cache")) + .body(Body::from_stream(stream)) + .expect("build sse response"); + Ok(response) +} + +fn strip_threadline_tools(payload: &mut serde_json::Map) { + let Some(Value::Array(tools)) = payload.get_mut("tools") else { + return; + }; + + tools.retain(|tool| { + !tool + .get("name") + .and_then(Value::as_str) + .is_some_and(is_internal_tool_name) + }); +} + +fn strip_context_management_for_upstream( + payload: &mut serde_json::Map, + route_kind: &'static str, + classification: DownstreamRequestClassification, +) -> bool { + let stripped = payload.remove("context_management").is_some(); + if stripped { + debug!( + route_kind, + request_class = request_class_label(classification), + client_compaction_only = true, + "context_management_stripped" + ); + } + stripped +} + +fn request_class_label(classification: DownstreamRequestClassification) -> &'static str { + match classification { + DownstreamRequestClassification::Normal => "normal", + DownstreamRequestClassification::AuxiliarySummary => "auxiliary_summary", + } +} + +fn retained_session_conflict_reroute_reason( + routing_diagnostics: &self::downstream::DownstreamRequestRoutingDiagnostics, + payload: &serde_json::Map, +) -> Option<&'static str> { + if routing_diagnostics.interaction_type_compaction_hit { + return Some("interaction_type_compaction"); + } + + if routing_diagnostics.summary_hits.matches_auxiliary_summary() { + return Some("summary_fingerprint"); + } + + if looks_like_auxiliary_summary_conflict_fallback(payload) { + return Some("fallback_summary_input"); + } + + None +} + +fn rewrite_stale_continuation_first_send_error(error: ThreadlineError) -> ThreadlineError { + match error { + ThreadlineError::UpstreamWebSocketClosed => ThreadlineError::PreviousResponseNotFound, + other => other, + } +} + +async fn attempt_pre_first_event_reconnect( + services: &ThreadlineServices, + lease: &mut RetainedSessionLease, + request_payload: &serde_json::Map, + previous_response_id: Option<&str>, + upstream_event_seen: bool, + reconnect_attempted: &mut bool, +) -> Result>, ThreadlineError> { + let Some(previous_response_id) = previous_response_id else { + return Ok(None); + }; + + if upstream_event_seen || *reconnect_attempted { + return Ok(None); + } + + *reconnect_attempted = true; + lease.mark_upstream_recoverable().await; + debug!( + previous_response_id, + session_id = %lease.session().session_id, + thread_id = %lease.session().thread_id, + window_id = %lease.session().window_id, + "reconnect_continuation_attempt" + ); + + let auth = services.auth_provider().load()?; + let upstream = match ensure_upstream(services, lease, auth).await { + Ok(upstream) => upstream, + Err(error) => { + debug!( + previous_response_id, + session_id = %lease.session().session_id, + thread_id = %lease.session().thread_id, + "reconnect_continuation_failed" + ); + return Err(error); + } + }; + + if let Err(error) = send_response_create(&upstream, request_payload).await { + debug!( + previous_response_id, + session_id = %lease.session().session_id, + thread_id = %lease.session().thread_id, + "reconnect_continuation_failed" + ); + return Err(error); + } + + Ok(Some(upstream)) +} + +async fn acquire_lease( + registry: &RetainedSessionRegistry, + previous_response_id: Option<&str>, +) -> Result { + match previous_response_id { + Some(previous_response_id) => registry + .acquire_previous(previous_response_id) + .await + .map_err(map_registry_error), + None => registry.acquire_new().await.map_err(map_registry_error), + } +} + +async fn start_transient_route( + services: &ThreadlineServices, + mut upstream_request: serde_json::Map, + classification: DownstreamRequestClassification, + kind: TransientRouteKind, +) -> Result { + strip_threadline_tools(&mut upstream_request); + strip_context_management_for_upstream(&mut upstream_request, kind.label(), classification); + + upstream_request.remove("previous_response_id"); + + let auth = services.auth_provider().load()?; + let connected = services.connector().connect(auth, None).await?; + send_response_create(&connected.websocket, &upstream_request).await?; + + Ok(PreparedResponseRoute { + upstream: connected.websocket, + lease: ResponseStreamLease::TransientAuxiliary, + previous_response_id: None, + replay_stale_marker_on_pre_first_event_close: false, + reconnect_attempted: false, + upstream_request, + execute_internal_tools: false, + apply_no_observable_output_failure: false, + }) +} + +async fn ensure_upstream( + services: &ThreadlineServices, + lease: &mut RetainedSessionLease, + auth: LoadedUpstreamAuth, +) -> Result, ThreadlineError> { + if let Some(upstream) = lease.upstream() { + if !upstream.is_closed() { + return Ok(upstream); + } + + lease.mark_upstream_recoverable().await; + } + + let connected = services + .connector() + .connect(auth, Some(lease.session().clone())) + .await?; + let turn_state = connected + .turn_state + .clone() + .or_else(|| lease.session().turn_state.clone()); + lease.update_turn_state(turn_state).await; + lease + .replace_upstream(Some(Arc::clone(&connected.websocket))) + .await; + Ok(connected.websocket) +} + +fn map_registry_error(error: RegistryAcquireError) -> ThreadlineError { + match error { + RegistryAcquireError::PreviousResponseNotFound => ThreadlineError::PreviousResponseNotFound, + RegistryAcquireError::RetainedSessionConflict => ThreadlineError::RetainedSessionConflict, + RegistryAcquireError::RetainedSessionCapacityExceeded => { + ThreadlineError::RetainedSessionCapacityExceeded + } + } +} + +#[cfg(test)] +mod tests { + use super::rewrite_stale_continuation_first_send_error; + use crate::errors::ThreadlineError; + + #[test] + fn stale_continuation_first_send_rewrites_closed_upstream_to_previous_response_not_found() { + let rewritten = + rewrite_stale_continuation_first_send_error(ThreadlineError::UpstreamWebSocketClosed); + + assert!(matches!( + rewritten, + ThreadlineError::PreviousResponseNotFound + )); + } + + #[test] + fn stale_continuation_first_send_preserves_non_transport_errors() { + let preserved = + rewrite_stale_continuation_first_send_error(ThreadlineError::InvalidResponsesRequest); + + assert!(matches!( + preserved, + ThreadlineError::InvalidResponsesRequest + )); + } +} diff --git a/src/responses/translation.rs b/src/responses/translation.rs new file mode 100644 index 0000000..3fbe025 --- /dev/null +++ b/src/responses/translation.rs @@ -0,0 +1,2012 @@ +use std::collections::{HashSet, VecDeque}; +use std::convert::Infallible; +use std::mem; +use std::sync::Arc; + +use axum::body::Bytes; +use futures_util::stream; +use serde_json::Value; +use tracing::{debug, trace}; + +use crate::errors::ThreadlineError; +use crate::registry::RetainedSessionLease; +use crate::tools::{ + InternalToolCall, PendingInternalToolOutput, build_followup_input, + event_contains_internal_tool_name, is_internal_tool_name, +}; +use crate::ws_pump::LiveUpstreamWebSocket; + +use super::downstream::{ + safe_scalar_field, sse_done_chunk, sse_error_chunk, sse_json_chunk, + sse_terminal_response_failed_chunk, sse_terminal_response_incomplete_chunk, +}; +use super::upstream::{ThreadlineServices, send_followup_tool_outputs}; + +fn response_id_from_event(event: &Value) -> Option<&str> { + event + .get("response_id") + .and_then(Value::as_str) + .or_else(|| { + event + .get("response") + .and_then(|response| response.get("id")) + .and_then(Value::as_str) + }) +} + +fn output_index_from_event(event: &Value) -> Option { + event.get("output_index").and_then(Value::as_u64) +} + +fn is_upstream_previous_response_not_found_error( + error_code: Option<&str>, + error_message: Option<&str>, +) -> bool { + if error_code == Some("previous_response_not_found") { + return true; + } + + error_message.is_some_and(|message| { + message.contains("Previous response with id") && message.contains("not found") + }) +} + +pub(super) enum ResponseStreamLease { + Retained(RetainedSessionLease), + TransientAuxiliary, +} + +impl ResponseStreamLease { + fn release(&mut self) { + if let Self::Retained(lease) = self { + lease.release(); + } + } + + async fn record_completed_marker(&mut self, response_marker: &str) { + if let Self::Retained(lease) = self { + lease.record_completed_marker(response_marker).await; + } + } + + async fn mark_upstream_recoverable(&mut self) { + if let Self::Retained(lease) = self { + lease.mark_upstream_recoverable().await; + } + } + + async fn mark_upstream_terminal(&mut self) { + if let Self::Retained(lease) = self { + lease.mark_upstream_terminal().await; + } + } + + fn retained_mut(&mut self) -> Option<&mut RetainedSessionLease> { + match self { + Self::Retained(lease) => Some(lease), + Self::TransientAuxiliary => None, + } + } +} + +const RESPONSES_TRANSLATION_UPSTREAM_EVENT: &str = "responses_translation_upstream_event"; +const RESPONSES_TRANSLATION_DOWNSTREAM_SSE_EVENT: &str = + "responses_translation_downstream_sse_event"; +const RESPONSES_TRANSLATION_EVENT_SUPPRESSED: &str = "responses_translation_event_suppressed"; +const RESPONSES_TRANSLATION_NO_OBSERVABLE_OUTPUT_GUARD: &str = + "responses_translation_no_observable_output_guard"; + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +enum DownstreamTraceAction { + Forwarded, + Suppressed, + Terminal, + ErrorTranslated, +} + +impl DownstreamTraceAction { + fn as_str(self) -> &'static str { + match self { + Self::Forwarded => "forwarded", + Self::Suppressed => "suppressed", + Self::Terminal => "terminal", + Self::ErrorTranslated => "error-translated", + } + } +} + +#[derive(Debug, PartialEq, Eq)] +struct UpstreamEventTraceMetadata { + event_type: String, + response_id: Option, + item_type: Option, + item_name: Option, + call_id: Option, + arguments_length: Option, + delta_length: Option, + output_index: Option, + content_index: Option, + item_id: Option, + is_compaction: bool, + compaction_id: Option, + has_encrypted_content: Option, +} + +impl UpstreamEventTraceMetadata { + fn from_event(event: &Value) -> Self { + let item = event.get("item"); + let item_type = string_field(item.and_then(|value| value.get("type"))) + .or_else(|| string_field(event.get("item_type"))); + let item_id = string_field(event.get("item_id")) + .or_else(|| string_field(item.and_then(|value| value.get("id")))); + let is_compaction = item_type.as_deref() == Some("compaction"); + Self { + event_type: event + .get("type") + .and_then(Value::as_str) + .unwrap_or("message") + .to_string(), + response_id: response_id_from_event(event).map(ToString::to_string), + item_type, + item_name: string_field(item.and_then(|value| value.get("name"))) + .or_else(|| string_field(event.get("name"))) + .or_else(|| string_field(event.get("tool_name"))), + call_id: string_field(item.and_then(|value| value.get("call_id"))) + .or_else(|| string_field(event.get("call_id"))), + arguments_length: string_length_field( + item.and_then(|value| value.get("arguments")) + .or_else(|| event.get("arguments")), + ), + delta_length: string_length_field(event.get("delta")), + output_index: output_index_from_event(event), + content_index: event.get("content_index").and_then(Value::as_u64), + item_id: item_id.clone(), + is_compaction, + compaction_id: is_compaction.then_some(item_id).flatten(), + has_encrypted_content: is_compaction.then_some( + item.and_then(|value| value.get("encrypted_content")) + .is_some(), + ), + } + } +} + +#[derive(Debug, Default, PartialEq, Eq)] +struct DownstreamTraceDiagnostics { + response_id: Option, + synthetic_delta_source: Option<&'static str>, + visible_text_delta_count: Option, + visible_text_length: Option, + sanitized_internal_function_call_count: Option, + sanitized_compaction_count: Option, + completed_visible_message_count: Option, +} + +#[derive(Debug, PartialEq, Eq)] +struct DownstreamSseTraceMetadata { + translation_action: &'static str, + event_type: String, + response_id: Option, + item_type: Option, + item_name: Option, + call_id: Option, + arguments_length: Option, + delta_length: Option, + output_index: Option, + content_index: Option, + item_id: Option, + is_compaction: bool, + compaction_id: Option, + has_encrypted_content: Option, + synthetic_delta_source: Option<&'static str>, + visible_text_delta_count: Option, + visible_text_length: Option, + sanitized_internal_function_call_count: Option, + sanitized_compaction_count: Option, + completed_visible_message_count: Option, +} + +#[derive(Debug, PartialEq, Eq)] +struct NoObservableOutputGuardDiagnostics { + response_id: Option, + pending_internal_outputs_count: usize, + suppressed_internal_tool_call_count: usize, + forwarded_external_tool_call_count: usize, + forwarded_compaction_or_marker_count: usize, + visible_assistant_text_len: usize, + completed_output_item_types: Vec, + upstream_last_event_type: Option, + is_intermediate_completed: bool, +} + +fn downstream_sse_trace_metadata( + event: &Value, + action: DownstreamTraceAction, + diagnostics: Option<&DownstreamTraceDiagnostics>, +) -> DownstreamSseTraceMetadata { + let metadata = UpstreamEventTraceMetadata::from_event(event); + DownstreamSseTraceMetadata { + translation_action: action.as_str(), + event_type: metadata.event_type, + response_id: diagnostics + .and_then(|value| value.response_id.clone()) + .or(metadata.response_id), + item_type: metadata.item_type, + item_name: metadata.item_name, + call_id: metadata.call_id, + arguments_length: metadata.arguments_length, + delta_length: metadata.delta_length, + output_index: metadata.output_index, + content_index: metadata.content_index, + item_id: metadata.item_id, + is_compaction: metadata.is_compaction, + compaction_id: metadata.compaction_id, + has_encrypted_content: metadata.has_encrypted_content, + synthetic_delta_source: diagnostics.and_then(|value| value.synthetic_delta_source), + visible_text_delta_count: diagnostics.and_then(|value| value.visible_text_delta_count), + visible_text_length: diagnostics.and_then(|value| value.visible_text_length), + sanitized_internal_function_call_count: diagnostics + .and_then(|value| value.sanitized_internal_function_call_count), + sanitized_compaction_count: diagnostics.and_then(|value| value.sanitized_compaction_count), + completed_visible_message_count: diagnostics + .and_then(|value| value.completed_visible_message_count), + } +} + +fn trace_upstream_event(metadata: &UpstreamEventTraceMetadata) { + trace!( + event_type = %metadata.event_type, + response_id = ?metadata.response_id, + item_type = ?metadata.item_type, + item_name = ?metadata.item_name, + call_id = ?metadata.call_id, + arguments_length = ?metadata.arguments_length, + delta_length = ?metadata.delta_length, + output_index = ?metadata.output_index, + content_index = ?metadata.content_index, + item_id = ?metadata.item_id, + is_compaction = metadata.is_compaction, + compaction_id = ?metadata.compaction_id, + has_encrypted_content = ?metadata.has_encrypted_content, + "{RESPONSES_TRANSLATION_UPSTREAM_EVENT}" + ); +} + +fn trace_downstream_sse_event(metadata: &DownstreamSseTraceMetadata) { + trace!( + translation_action = metadata.translation_action, + event_type = %metadata.event_type, + response_id = ?metadata.response_id, + item_type = ?metadata.item_type, + item_name = ?metadata.item_name, + call_id = ?metadata.call_id, + arguments_length = ?metadata.arguments_length, + delta_length = ?metadata.delta_length, + output_index = ?metadata.output_index, + content_index = ?metadata.content_index, + item_id = ?metadata.item_id, + is_compaction = metadata.is_compaction, + compaction_id = ?metadata.compaction_id, + has_encrypted_content = ?metadata.has_encrypted_content, + synthetic_delta_source = ?metadata.synthetic_delta_source, + visible_text_delta_count = ?metadata.visible_text_delta_count, + visible_text_length = ?metadata.visible_text_length, + sanitized_internal_function_call_count = + ?metadata.sanitized_internal_function_call_count, + sanitized_compaction_count = ?metadata.sanitized_compaction_count, + completed_visible_message_count = ?metadata.completed_visible_message_count, + "{RESPONSES_TRANSLATION_DOWNSTREAM_SSE_EVENT}" + ); +} + +fn trace_suppressed_event(metadata: &UpstreamEventTraceMetadata) { + trace!( + translation_action = DownstreamTraceAction::Suppressed.as_str(), + event_type = %metadata.event_type, + response_id = ?metadata.response_id, + item_type = ?metadata.item_type, + item_name = ?metadata.item_name, + call_id = ?metadata.call_id, + arguments_length = ?metadata.arguments_length, + delta_length = ?metadata.delta_length, + output_index = ?metadata.output_index, + content_index = ?metadata.content_index, + item_id = ?metadata.item_id, + is_compaction = metadata.is_compaction, + compaction_id = ?metadata.compaction_id, + has_encrypted_content = ?metadata.has_encrypted_content, + "{RESPONSES_TRANSLATION_EVENT_SUPPRESSED}" + ); +} + +fn trace_no_observable_output_guard(metadata: &NoObservableOutputGuardDiagnostics) { + debug!( + response_id = ?metadata.response_id, + pending_internal_outputs_count = metadata.pending_internal_outputs_count, + suppressed_internal_tool_call_count = metadata.suppressed_internal_tool_call_count, + forwarded_external_tool_call_count = metadata.forwarded_external_tool_call_count, + forwarded_compaction_or_marker_count = metadata.forwarded_compaction_or_marker_count, + visible_assistant_text_len = metadata.visible_assistant_text_len, + completed_output_item_types = ?metadata.completed_output_item_types, + upstream_last_event_type = ?metadata.upstream_last_event_type, + is_intermediate_completed = metadata.is_intermediate_completed, + "{RESPONSES_TRANSLATION_NO_OBSERVABLE_OUTPUT_GUARD}" + ); +} + +fn string_field(value: Option<&Value>) -> Option { + value.and_then(Value::as_str).map(ToString::to_string) +} + +fn string_length_field(value: Option<&Value>) -> Option { + value.and_then(Value::as_str).map(str::len) +} + +#[derive(Clone, Debug, Eq, Hash, PartialEq)] +pub(super) struct VisibleTextSourceKey { + item_id: Option, + output_index: Option, + content_index: Option, +} + +impl VisibleTextSourceKey { + fn new(item_id: Option, output_index: Option, content_index: Option) -> Self { + Self { + item_id, + output_index, + content_index, + } + } + + fn dedupe_identity(&self) -> Option { + if let Some(item_id) = self.item_id.as_ref() { + return Some(VisibleTextDedupeIdentity::ItemId { + item_id: item_id.clone(), + content_index: self.content_index, + }); + } + + self.output_index + .map(|output_index| VisibleTextDedupeIdentity::OutputIndex { + output_index, + content_index: self.content_index, + }) + } +} + +#[derive(Clone, Debug, Eq, Hash, PartialEq)] +pub(super) enum VisibleTextDedupeIdentity { + ItemId { + item_id: String, + content_index: Option, + }, + OutputIndex { + output_index: u64, + content_index: Option, + }, +} + +#[derive(Clone, Debug, Eq, PartialEq)] +pub(super) struct VisibleAssistantText { + key: VisibleTextSourceKey, + text: String, +} + +fn response_output_text_delta_payload(key: &VisibleTextSourceKey, delta: &str) -> Option { + if delta.trim().is_empty() { + return None; + } + + let mut payload = serde_json::Map::new(); + payload.insert( + "type".to_string(), + Value::String("response.output_text.delta".to_string()), + ); + payload.insert("delta".to_string(), Value::String(delta.to_string())); + + if let Some(item_id) = key.item_id.as_ref() { + payload.insert("item_id".to_string(), Value::String(item_id.clone())); + } + if let Some(output_index) = key.output_index { + payload.insert("output_index".to_string(), Value::from(output_index)); + } + if let Some(content_index) = key.content_index { + payload.insert("content_index".to_string(), Value::from(content_index)); + } + + Some(Value::Object(payload)) +} + +fn visible_text_delta_source_key(event: &Value) -> VisibleTextSourceKey { + VisibleTextSourceKey::new( + string_field(event.get("item_id")), + event.get("output_index").and_then(Value::as_u64), + event.get("content_index").and_then(Value::as_u64), + ) +} + +fn synthesized_output_text_done_delta(event: &Value) -> Option<(VisibleTextSourceKey, Value)> { + let key = visible_text_delta_source_key(event); + let text = event.get("text").and_then(Value::as_str)?; + let payload = response_output_text_delta_payload(&key, text)?; + Some((key, payload)) +} + +fn message_output_text_delta_payloads( + item: &Value, + output_index: Option, +) -> Vec<(VisibleTextSourceKey, Value)> { + if item.get("type").and_then(Value::as_str) != Some("message") + || item.get("role").and_then(Value::as_str) != Some("assistant") + { + return Vec::new(); + } + + let Some(content) = item.get("content").and_then(Value::as_array) else { + return Vec::new(); + }; + + let item_id = string_field(item.get("id")); + let mut payloads = Vec::new(); + for (content_index, part) in content.iter().enumerate() { + if part.get("type").and_then(Value::as_str) != Some("output_text") { + continue; + } + + let Some(text) = part.get("text").and_then(Value::as_str) else { + continue; + }; + + let key = + VisibleTextSourceKey::new(item_id.clone(), output_index, Some(content_index as u64)); + let Some(payload) = response_output_text_delta_payload(&key, text) else { + continue; + }; + payloads.push((key, payload)); + } + + payloads +} + +fn synthesized_output_item_done_text_delta(event: &Value) -> Vec<(VisibleTextSourceKey, Value)> { + let Some(item) = event.get("item") else { + return Vec::new(); + }; + + message_output_text_delta_payloads(item, output_index_from_event(event)) +} + +fn synthesized_completed_output_text_delta(event: &Value) -> Vec<(VisibleTextSourceKey, Value)> { + let Some(output) = event + .get("response") + .and_then(|response| response.get("output")) + .and_then(Value::as_array) + else { + return Vec::new(); + }; + + let mut first_key: Option = None; + let mut combined_text = String::new(); + + for (output_index, item) in output.iter().enumerate() { + if item.get("type").and_then(Value::as_str) != Some("message") + || item.get("role").and_then(Value::as_str) != Some("assistant") + { + continue; + } + + let Some(content) = item.get("content").and_then(Value::as_array) else { + continue; + }; + + let item_id = string_field(item.get("id")); + for (content_index, part) in content.iter().enumerate() { + if part.get("type").and_then(Value::as_str) != Some("output_text") { + continue; + } + + let Some(text) = part.get("text").and_then(Value::as_str) else { + continue; + }; + + if text.trim().is_empty() { + continue; + } + + if first_key.is_none() { + first_key = Some(VisibleTextSourceKey::new( + item_id.clone(), + Some(output_index as u64), + Some(content_index as u64), + )); + } + + combined_text.push_str(text); + } + } + + let Some(key) = first_key else { + return Vec::new(); + }; + let Some(payload) = response_output_text_delta_payload(&key, &combined_text) else { + return Vec::new(); + }; + + vec![(key, payload)] +} + +fn record_visible_assistant_text( + visible_text: &mut Vec, + key: &VisibleTextSourceKey, + text: &str, +) { + if text.trim().is_empty() { + return; + } + + if let Some(existing) = visible_text.iter_mut().find(|entry| entry.key == *key) { + existing.text.push_str(text); + return; + } + + visible_text.push(VisibleAssistantText { + key: key.clone(), + text: text.to_string(), + }); +} + +fn track_visible_text_identity( + state: &mut ResponseStreamState, + key: &VisibleTextSourceKey, + text: &str, +) -> bool { + if let Some(identity) = key.dedupe_identity() { + state.downstream_visible_text_sources.insert(identity); + return true; + } + + if state.last_unidentified_visible_text.as_deref() == Some(text) { + return false; + } + + state.last_unidentified_visible_text = Some(text.to_string()); + true +} + +fn record_forwarded_visible_text_delta( + state: &mut ResponseStreamState, + key: VisibleTextSourceKey, + delta: &str, +) { + if !track_visible_text_identity(state, &key, delta) { + return; + } + + record_visible_assistant_text(&mut state.visible_assistant_text, &key, delta); +} + +fn queue_visible_text_delta( + state: &mut ResponseStreamState, + key: VisibleTextSourceKey, + payload: Value, + synthetic_delta_source: &'static str, + response_id: Option<&str>, +) -> bool { + let Some(delta) = payload.get("delta").and_then(Value::as_str) else { + return false; + }; + + if !key + .dedupe_identity() + .map(|identity| state.downstream_visible_text_sources.insert(identity)) + .unwrap_or_else(|| { + if state.last_unidentified_visible_text.as_deref() == Some(delta) { + false + } else { + state.last_unidentified_visible_text = Some(delta.to_string()); + true + } + }) + { + return false; + } + + record_visible_assistant_text(&mut state.visible_assistant_text, &key, delta); + state + .queued_synthetic_output_text_deltas + .push_back(QueuedSyntheticOutputTextDelta { + payload, + response_id: response_id.map(ToString::to_string), + synthetic_delta_source, + }); + true +} + +fn queue_visible_text_deltas( + state: &mut ResponseStreamState, + payloads: Vec<(VisibleTextSourceKey, Value)>, + synthetic_delta_source: &'static str, + response_id: Option<&str>, +) -> bool { + let mut queued = false; + for (key, payload) in payloads { + if queue_visible_text_delta(state, key, payload, synthetic_delta_source, response_id) { + queued = true; + } + } + + queued +} + +fn assistant_message_has_visible_output_text(item: &Value) -> bool { + if item.get("type").and_then(Value::as_str) != Some("message") + || item.get("role").and_then(Value::as_str) != Some("assistant") + { + return false; + } + + item.get("content") + .and_then(Value::as_array) + .is_some_and(|content| { + content.iter().any(|part| { + part.get("type").and_then(Value::as_str) == Some("output_text") + && part + .get("text") + .and_then(Value::as_str) + .is_some_and(|text| !text.trim().is_empty()) + }) + }) +} + +fn synthetic_assistant_message_id(response_id: Option<&str>) -> String { + match response_id { + Some(response_id) if !response_id.is_empty() => { + format!("synthetic_assistant_{response_id}") + } + _ => "synthetic_assistant".to_string(), + } +} + +fn synthetic_assistant_message( + response_id: Option<&str>, + visible_text: &[VisibleAssistantText], +) -> Option { + let content: Vec = visible_text + .iter() + .filter(|entry| !entry.text.trim().is_empty()) + .map(|entry| { + Value::Object(serde_json::Map::from_iter([ + ("type".to_string(), Value::String("output_text".to_string())), + ("text".to_string(), Value::String(entry.text.clone())), + ("annotations".to_string(), Value::Array(Vec::new())), + ])) + }) + .collect(); + + if content.is_empty() { + return None; + } + + let message_id = visible_text + .iter() + .find_map(|entry| entry.key.item_id.clone()) + .unwrap_or_else(|| synthetic_assistant_message_id(response_id)); + + Some(Value::Object(serde_json::Map::from_iter([ + ("id".to_string(), Value::String(message_id)), + ("type".to_string(), Value::String("message".to_string())), + ("role".to_string(), Value::String("assistant".to_string())), + ("content".to_string(), Value::Array(content)), + ]))) +} + +fn has_accumulated_visible_assistant_text(visible_text: &[VisibleAssistantText]) -> bool { + visible_text + .iter() + .any(|entry| !entry.text.trim().is_empty()) +} + +fn is_forwarded_external_tool_call_item(item: &Value) -> bool { + item.get("type").and_then(Value::as_str) == Some("function_call") + && item + .get("name") + .or_else(|| item.get("tool_name")) + .and_then(Value::as_str) + .is_some_and(|name| !is_internal_tool_name(name)) +} + +fn has_image_generation_result(item: &Value) -> bool { + item.get("type").and_then(Value::as_str) == Some("image_generation_call") + && item + .get("result") + .and_then(Value::as_str) + .is_some_and(|result| !result.trim().is_empty()) +} + +fn is_forwarded_marker_like_item(item: &Value) -> bool { + matches!( + item.get("type").and_then(Value::as_str), + Some("compaction") | Some("context") + ) +} + +#[derive(Clone, Debug, Default, PartialEq, Eq)] +pub(super) struct DownstreamObservableOutputState { + forwarded_visible_text_delta_count: usize, + forwarded_external_tool_call_count: usize, + forwarded_image_generation_count: usize, + forwarded_marker_like_output_count: usize, + final_visible_message_count: usize, + final_external_tool_call_count: usize, + final_image_generation_count: usize, + final_marker_like_output_count: usize, + last_upstream_event_type: Option, +} + +impl DownstreamObservableOutputState { + fn reset(&mut self) { + *self = Self::default(); + } + + fn has_observable_output(&self) -> bool { + self.forwarded_visible_text_delta_count > 0 + || self.forwarded_external_tool_call_count > 0 + || self.forwarded_image_generation_count > 0 + || self.forwarded_marker_like_output_count > 0 + || self.final_visible_message_count > 0 + || self.final_external_tool_call_count > 0 + || self.final_image_generation_count > 0 + || self.final_marker_like_output_count > 0 + } +} + +fn record_forwarded_observable_output( + observable_output: &mut DownstreamObservableOutputState, + event_type: &str, + event: &Value, +) { + match event_type { + "response.output_text.delta" => { + observable_output.forwarded_visible_text_delta_count += 1; + } + "response.function_call_arguments.delta" | "response.function_call_arguments.done" => { + observable_output.forwarded_external_tool_call_count += 1; + } + "response.output_item.added" | "response.output_item.done" => { + let Some(item) = event.get("item") else { + return; + }; + + if is_forwarded_external_tool_call_item(item) { + observable_output.forwarded_external_tool_call_count += 1; + } else if has_image_generation_result(item) { + observable_output.forwarded_image_generation_count += 1; + } else if is_forwarded_marker_like_item(item) { + observable_output.forwarded_marker_like_output_count += 1; + } + } + _ => {} + } +} + +fn record_completed_observable_output( + observable_output: &mut DownstreamObservableOutputState, + event: &Value, + diagnostics: &CompletedSanitizationDiagnostics, +) { + observable_output.final_visible_message_count = diagnostics.completed_visible_message_count; + observable_output.final_external_tool_call_count = 0; + observable_output.final_image_generation_count = 0; + observable_output.final_marker_like_output_count = 0; + + let Some(output) = event + .get("response") + .and_then(|response| response.get("output")) + .and_then(Value::as_array) + else { + return; + }; + + for item in output { + if is_forwarded_external_tool_call_item(item) { + observable_output.final_external_tool_call_count += 1; + } else if has_image_generation_result(item) { + observable_output.final_image_generation_count += 1; + } else if is_forwarded_marker_like_item(item) { + observable_output.final_marker_like_output_count += 1; + } + } +} + +fn has_downstream_observable_output(state: &ResponseStreamState) -> bool { + state.observable_output.has_observable_output() +} + +fn no_observable_output_guard_diagnostics( + state: &ResponseStreamState, + completed_event: &Value, + diagnostics: &CompletedSanitizationDiagnostics, +) -> NoObservableOutputGuardDiagnostics { + NoObservableOutputGuardDiagnostics { + response_id: response_id_from_event(completed_event).map(ToString::to_string), + pending_internal_outputs_count: state.pending_internal_outputs.len(), + suppressed_internal_tool_call_count: state.suppressed_internal_output_indexes.len() + + diagnostics.sanitized_internal_function_call_count, + forwarded_external_tool_call_count: state + .observable_output + .forwarded_external_tool_call_count + + state.observable_output.final_external_tool_call_count, + forwarded_compaction_or_marker_count: state + .observable_output + .forwarded_marker_like_output_count + + state.observable_output.final_marker_like_output_count, + visible_assistant_text_len: state + .visible_assistant_text + .iter() + .map(|entry| entry.text.len()) + .sum(), + completed_output_item_types: completed_event + .get("response") + .and_then(|response| response.get("output")) + .and_then(Value::as_array) + .map(|output| { + output + .iter() + .filter_map(|item| item.get("type").and_then(Value::as_str)) + .map(ToString::to_string) + .collect() + }) + .unwrap_or_default(), + upstream_last_event_type: state.observable_output.last_upstream_event_type.clone(), + is_intermediate_completed: !state.pending_internal_outputs.is_empty(), + } +} + +fn no_observable_output_failed_payload(response_id: Option<&str>) -> Value { + let mut response = serde_json::Map::new(); + if let Some(response_id) = response_id.filter(|value| !value.is_empty()) { + response.insert("id".to_string(), Value::String(response_id.to_string())); + } + + Value::Object(serde_json::Map::from_iter([ + ( + "type".to_string(), + Value::String("response.failed".to_string()), + ), + ("response".to_string(), Value::Object(response)), + ( + "error".to_string(), + Value::Object(serde_json::Map::from_iter([ + ( + "code".to_string(), + Value::String("threadline_no_observable_output".to_string()), + ), + ( + "message".to_string(), + Value::String("Response contained no observable output.".to_string()), + ), + ])), + ), + ])) +} + +fn terminal_failed_payload( + response: Option<&Value>, + response_id: Option<&str>, + code: impl Into, + message: impl Into, +) -> Value { + let mut response_object = response + .and_then(Value::as_object) + .cloned() + .unwrap_or_default(); + + if !response_object.contains_key("id") + && let Some(response_id) = response_id.filter(|value| !value.is_empty()) + { + response_object.insert("id".to_string(), Value::String(response_id.to_string())); + } + + Value::Object(serde_json::Map::from_iter([ + ( + "type".to_string(), + Value::String("response.failed".to_string()), + ), + ("response".to_string(), Value::Object(response_object)), + ( + "error".to_string(), + Value::Object(serde_json::Map::from_iter([ + ("code".to_string(), Value::String(code.into())), + ("message".to_string(), Value::String(message.into())), + ])), + ), + ])) +} + +fn terminal_failed_payload_from_error( + response: Option<&Value>, + response_id: Option<&str>, + error: &ThreadlineError, +) -> Value { + let public = error.public_error(); + terminal_failed_payload( + response, + response_id, + public.code.into_owned(), + public.message.into_owned(), + ) +} + +#[derive(Clone, Debug, Default, PartialEq, Eq)] +struct CompletedSanitizationDiagnostics { + sanitized_internal_function_call_count: usize, + sanitized_compaction_count: usize, + completed_visible_message_count: usize, +} + +fn sanitized_completed_event_with_diagnostics( + event: &Value, + visible_text: &[VisibleAssistantText], +) -> (Value, CompletedSanitizationDiagnostics) { + let mut sanitized = event.clone(); + let response_id = response_id_from_event(event); + let mut diagnostics = CompletedSanitizationDiagnostics::default(); + + let Some(response) = sanitized.get_mut("response").and_then(Value::as_object_mut) else { + return (sanitized, diagnostics); + }; + + let Some(original_output) = response.get("output").and_then(Value::as_array) else { + if let Some(message) = synthetic_assistant_message(response_id, visible_text) { + diagnostics.completed_visible_message_count = 1; + response.insert("output".to_string(), Value::Array(vec![message])); + } + return (sanitized, diagnostics); + }; + + let mut final_output = original_output + .iter() + .filter_map(|item| match item.get("type").and_then(Value::as_str) { + Some("function_call") + if item + .get("name") + .or_else(|| item.get("tool_name")) + .and_then(Value::as_str) + .is_some_and(is_internal_tool_name) => + { + diagnostics.sanitized_internal_function_call_count += 1; + None + } + _ => Some(item.clone()), + }) + .collect::>(); + + let has_visible_assistant_message = final_output + .iter() + .any(assistant_message_has_visible_output_text); + let mut output_changed = diagnostics.sanitized_internal_function_call_count > 0; + + if !has_visible_assistant_message + && let Some(message) = synthetic_assistant_message(response_id, visible_text) + { + final_output.push(message); + output_changed = true; + } + + diagnostics.completed_visible_message_count = final_output + .iter() + .filter(|item| assistant_message_has_visible_output_text(item)) + .count(); + + if output_changed { + response.insert("output".to_string(), Value::Array(final_output)); + } + + (sanitized, diagnostics) +} + +#[derive(Clone, Debug)] +pub(super) struct QueuedSyntheticOutputTextDelta { + payload: Value, + response_id: Option, + synthetic_delta_source: &'static str, +} + +#[derive(Clone, Debug)] +pub(super) struct QueuedCompletedEvent { + payload: Value, + diagnostics: CompletedSanitizationDiagnostics, +} + +#[derive(Clone, Debug)] +pub(super) struct QueuedForwardedEvent { + payload: Value, +} + +pub(super) struct ResponseStreamState { + pub(super) services: ThreadlineServices, + pub(super) upstream: Arc, + pub(super) lease: ResponseStreamLease, + pub(super) base_request: serde_json::Map, + pub(super) pending_internal_outputs: Vec, + pub(super) previous_response_id: Option, + pub(super) execute_internal_tools: bool, + pub(super) suppressed_internal_output_indexes: HashSet, + pub(super) upstream_event_seen: bool, + pub(super) replay_stale_marker_on_pre_first_event_close: bool, + pub(super) reconnect_attempted: bool, + pub(super) observable_output: DownstreamObservableOutputState, + pub(super) downstream_visible_text_sources: HashSet, + pub(super) downstream_visible_text_delta_count: usize, + pub(super) visible_assistant_text: Vec, + pub(super) last_unidentified_visible_text: Option, + pub(super) queued_synthetic_output_text_deltas: VecDeque, + pub(super) queued_forwarded_event: Option, + pub(super) queued_final_completed: Option, + pub(super) final_done_pending: bool, + pub(super) apply_no_observable_output_failure: bool, + pub(super) done: bool, +} + +pub(super) fn response_stream( + state: ResponseStreamState, +) -> impl futures_util::Stream> { + stream::unfold(state, |mut state| async move { + loop { + if let Some(synthetic_delta) = state.queued_synthetic_output_text_deltas.pop_front() { + let event_type = synthetic_delta + .payload + .get("type") + .and_then(Value::as_str) + .unwrap_or("message") + .to_string(); + state.downstream_visible_text_delta_count += 1; + let trace_diagnostics = DownstreamTraceDiagnostics { + response_id: synthetic_delta.response_id.clone(), + synthetic_delta_source: Some(synthetic_delta.synthetic_delta_source), + visible_text_delta_count: Some(1), + visible_text_length: synthetic_delta + .payload + .get("delta") + .and_then(Value::as_str) + .map(str::len), + ..Default::default() + }; + trace_downstream_sse_event(&downstream_sse_trace_metadata( + &synthetic_delta.payload, + DownstreamTraceAction::Forwarded, + Some(&trace_diagnostics), + )); + debug!(event_type, "translation_event_forwarded"); + return Some(( + Ok::(sse_json_chunk(&event_type, &synthetic_delta.payload)), + state, + )); + } + + if let Some(forwarded_event) = state.queued_forwarded_event.take() { + let event_type = forwarded_event + .payload + .get("type") + .and_then(Value::as_str) + .unwrap_or("message") + .to_string(); + trace_downstream_sse_event(&downstream_sse_trace_metadata( + &forwarded_event.payload, + DownstreamTraceAction::Forwarded, + Some(&DownstreamTraceDiagnostics::default()), + )); + debug!(event_type, "translation_event_forwarded"); + return Some(( + Ok::(sse_json_chunk(&event_type, &forwarded_event.payload)), + state, + )); + } + + if let Some(completed) = state.queued_final_completed.take() { + let response_id = response_id_from_event(&completed.payload); + if let Some(response_id) = response_id { + state.lease.record_completed_marker(response_id).await; + } + let trace_diagnostics = DownstreamTraceDiagnostics { + response_id: response_id.map(ToString::to_string), + visible_text_delta_count: Some(state.downstream_visible_text_delta_count), + sanitized_internal_function_call_count: Some( + completed.diagnostics.sanitized_internal_function_call_count, + ), + sanitized_compaction_count: Some( + completed.diagnostics.sanitized_compaction_count, + ), + completed_visible_message_count: Some( + completed.diagnostics.completed_visible_message_count, + ), + ..Default::default() + }; + trace_downstream_sse_event(&downstream_sse_trace_metadata( + &completed.payload, + DownstreamTraceAction::Terminal, + Some(&trace_diagnostics), + )); + debug!( + response_id, + event_type = "response.completed", + "translation_event_forwarded" + ); + debug!(response_id, "terminal_response_forwarded"); + state.lease.release(); + state.final_done_pending = true; + debug!(response_id, "final_done_queued"); + return Some(( + Ok::(sse_json_chunk( + "response.completed", + &completed.payload, + )), + state, + )); + } + + if state.final_done_pending { + state.final_done_pending = false; + state.done = true; + debug!("downstream_sse_done_sent"); + return Some((Ok::(sse_done_chunk()), state)); + } + + if state.done { + debug!("downstream_sse_stream_finished"); + return None; + } + + let next = match state.upstream.recv_text().await { + Ok(Some(text)) => text, + Ok(None) | Err(_) => match try_reconnect_or_terminal_error(&mut state).await { + Ok(Some(reconnected)) => { + state.upstream = reconnected; + continue; + } + Ok(None) => { + let failed_payload = terminal_failed_payload_from_error( + None, + None, + &ThreadlineError::UpstreamWebSocketClosed, + ); + trace_downstream_sse_event(&downstream_sse_trace_metadata( + &failed_payload, + DownstreamTraceAction::Terminal, + None, + )); + state.lease.mark_upstream_recoverable().await; + state.lease.release(); + state.final_done_pending = true; + return Some(( + Ok::(sse_terminal_response_failed_chunk( + &failed_payload, + )), + state, + )); + } + Err(error) => { + let failed_payload = terminal_failed_payload_from_error(None, None, &error); + trace_downstream_sse_event(&downstream_sse_trace_metadata( + &failed_payload, + DownstreamTraceAction::Terminal, + None, + )); + state.lease.mark_upstream_terminal().await; + state.lease.release(); + state.final_done_pending = true; + return Some(( + Ok::(sse_terminal_response_failed_chunk( + &failed_payload, + )), + state, + )); + } + }, + }; + + state.upstream_event_seen = true; + + if next.trim() == "[DONE]" { + let failed_payload = terminal_failed_payload( + None, + None, + "upstream_done_before_completed", + "The upstream websocket emitted [DONE] before Threadline received a terminal response event.", + ); + trace_downstream_sse_event(&downstream_sse_trace_metadata( + &failed_payload, + DownstreamTraceAction::Terminal, + None, + )); + state.lease.mark_upstream_terminal().await; + state.lease.release(); + state.final_done_pending = true; + return Some(( + Ok::(sse_terminal_response_failed_chunk(&failed_payload)), + state, + )); + } + + let parsed = match serde_json::from_str::(&next) { + Ok(parsed) => parsed, + Err(_) => { + state.lease.mark_upstream_terminal().await; + state.done = true; + return Some(( + Ok::(sse_error_chunk( + &ThreadlineError::UpstreamInvalidJson, + )), + state, + )); + } + }; + + let trace_metadata = UpstreamEventTraceMetadata::from_event(&parsed); + trace_upstream_event(&trace_metadata); + state.observable_output.last_upstream_event_type = + Some(trace_metadata.event_type.clone()); + + if state.execute_internal_tools { + let internal_tool_call = match InternalToolCall::from_event(&parsed) { + Ok(call) => call, + Err(error) => { + trace_downstream_sse_event(&downstream_sse_trace_metadata( + &parsed, + DownstreamTraceAction::ErrorTranslated, + None, + )); + let failed_payload = terminal_failed_payload_from_error( + parsed.get("response"), + response_id_from_event(&parsed), + &error, + ); + state.lease.mark_upstream_terminal().await; + state.lease.release(); + state.final_done_pending = true; + return Some(( + Ok::(sse_terminal_response_failed_chunk( + &failed_payload, + )), + state, + )); + } + }; + + if let Some(call) = internal_tool_call { + match call.execute() { + Ok(output) => { + if state + .pending_internal_outputs + .iter() + .any(|pending| pending.call_id() == output.call_id()) + { + let failed_payload = terminal_failed_payload_from_error( + parsed.get("response"), + response_id_from_event(&parsed), + &ThreadlineError::InternalToolFailed, + ); + trace_downstream_sse_event(&downstream_sse_trace_metadata( + &failed_payload, + DownstreamTraceAction::Terminal, + None, + )); + state.lease.mark_upstream_terminal().await; + state.lease.release(); + state.final_done_pending = true; + return Some(( + Ok::(sse_terminal_response_failed_chunk( + &failed_payload, + )), + state, + )); + } + state.pending_internal_outputs.push(output); + debug!( + pending_internal_output_count = + state.pending_internal_outputs.len(), + "internal_tool_executed" + ); + trace_suppressed_event(&trace_metadata); + continue; + } + Err(error) => { + trace_downstream_sse_event(&downstream_sse_trace_metadata( + &parsed, + DownstreamTraceAction::ErrorTranslated, + None, + )); + let failed_payload = terminal_failed_payload_from_error( + parsed.get("response"), + response_id_from_event(&parsed), + &error, + ); + state.lease.mark_upstream_terminal().await; + state.lease.release(); + state.final_done_pending = true; + return Some(( + Ok::(sse_terminal_response_failed_chunk( + &failed_payload, + )), + state, + )); + } + } + } + } + + let event_type = parsed + .get("type") + .and_then(Value::as_str) + .unwrap_or("message") + .to_string(); + + debug!(event_type, "upstream_event_received"); + + if !state.pending_internal_outputs.is_empty() + && (matches!( + event_type.as_str(), + "response.output_text.delta" | "response.output_text.done" + ) || (event_type == "response.output_item.done" + && !synthesized_output_item_done_text_delta(&parsed).is_empty())) + { + trace_suppressed_event(&trace_metadata); + debug!(event_type, "translation_event_suppressed_internal_tool"); + continue; + } + + if event_type.starts_with("response.output_item.") + && event_contains_internal_tool_name(&parsed) + { + if let Some(output_index) = output_index_from_event(&parsed) { + state + .suppressed_internal_output_indexes + .insert(output_index); + } + trace_suppressed_event(&trace_metadata); + debug!(event_type, "translation_event_suppressed_internal_tool"); + continue; + } + + if event_type == "response.function_call_arguments.delta" + && output_index_from_event(&parsed).is_some_and(|output_index| { + state + .suppressed_internal_output_indexes + .contains(&output_index) + }) + { + trace_suppressed_event(&trace_metadata); + debug!(event_type, "translation_event_suppressed_internal_tool"); + continue; + } + + match event_type.as_str() { + "response.completed" => { + let response_id = response_id_from_event(&parsed).map(ToString::to_string); + + if !state.pending_internal_outputs.is_empty() { + let Some(response_id) = response_id.as_deref() else { + let error = ThreadlineError::InternalToolFailed; + trace_downstream_sse_event(&downstream_sse_trace_metadata( + &parsed, + DownstreamTraceAction::ErrorTranslated, + None, + )); + let failed_payload = terminal_failed_payload_from_error( + parsed.get("response"), + response_id_from_event(&parsed), + &error, + ); + state.lease.mark_upstream_terminal().await; + state.lease.release(); + state.final_done_pending = true; + return Some(( + Ok::(sse_terminal_response_failed_chunk( + &failed_payload, + )), + state, + )); + }; + + let outputs = mem::take(&mut state.pending_internal_outputs); + let output_count = outputs.len(); + debug!( + response_id, + pending_internal_output_count = output_count, + "intermediate_completion_consumed" + ); + state.suppressed_internal_output_indexes.clear(); + let followup_input = build_followup_input(outputs); + if let Err(error) = send_followup_tool_outputs( + &state.upstream, + &state.base_request, + response_id, + followup_input, + ) + .await + { + let failed_payload = terminal_failed_payload_from_error( + parsed.get("response"), + Some(response_id), + &error, + ); + trace_downstream_sse_event(&downstream_sse_trace_metadata( + &failed_payload, + DownstreamTraceAction::Terminal, + None, + )); + state.lease.mark_upstream_terminal().await; + state.lease.release(); + state.final_done_pending = true; + return Some(( + Ok::(sse_terminal_response_failed_chunk( + &failed_payload, + )), + state, + )); + } + state.downstream_visible_text_sources.clear(); + state.downstream_visible_text_delta_count = 0; + state.visible_assistant_text.clear(); + state.last_unidentified_visible_text = None; + state.queued_synthetic_output_text_deltas.clear(); + state.observable_output.reset(); + debug!( + response_id, + output_count, + previous_response_id = state.previous_response_id.as_deref(), + "internal_tool_followup_sent" + ); + continue; + } + + if !has_accumulated_visible_assistant_text(&state.visible_assistant_text) { + queue_visible_text_deltas( + &mut state, + synthesized_completed_output_text_delta(&parsed), + "response.completed", + response_id.as_deref(), + ); + } + + let (sanitized_completed, diagnostics) = + sanitized_completed_event_with_diagnostics( + &parsed, + &state.visible_assistant_text, + ); + record_completed_observable_output( + &mut state.observable_output, + &sanitized_completed, + &diagnostics, + ); + + if state.apply_no_observable_output_failure + && !has_downstream_observable_output(&state) + { + let guard_diagnostics = + no_observable_output_guard_diagnostics(&state, &parsed, &diagnostics); + trace_no_observable_output_guard(&guard_diagnostics); + let failed_payload = + no_observable_output_failed_payload(response_id.as_deref()); + trace_downstream_sse_event(&downstream_sse_trace_metadata( + &failed_payload, + DownstreamTraceAction::Terminal, + None, + )); + state.lease.mark_upstream_terminal().await; + state.lease.release(); + state.final_done_pending = true; + debug!(response_id, event_type, "translation_event_forwarded"); + debug!(response_id, "terminal_response_forwarded"); + debug!(response_id, "final_done_queued"); + return Some(( + Ok::(sse_terminal_response_failed_chunk( + &failed_payload, + )), + state, + )); + } + + state.queued_final_completed = Some(QueuedCompletedEvent { + payload: sanitized_completed, + diagnostics, + }); + continue; + } + "response.failed" => { + trace_downstream_sse_event(&downstream_sse_trace_metadata( + &parsed, + DownstreamTraceAction::Terminal, + None, + )); + state.lease.mark_upstream_recoverable().await; + state.lease.release(); + state.final_done_pending = true; + debug!(event_type, "terminal_response_forwarded"); + debug!(event_type, "final_done_queued"); + return Some(( + Ok::(sse_terminal_response_failed_chunk(&parsed)), + state, + )); + } + "response.incomplete" => { + trace_downstream_sse_event(&downstream_sse_trace_metadata( + &parsed, + DownstreamTraceAction::Terminal, + None, + )); + state.lease.mark_upstream_terminal().await; + state.lease.release(); + state.final_done_pending = true; + debug!(event_type, "terminal_response_forwarded"); + debug!(event_type, "final_done_queued"); + return Some(( + Ok::(sse_terminal_response_incomplete_chunk(&parsed)), + state, + )); + } + "error" => { + let error = parsed.get("error"); + let error_code = error + .and_then(|value| value.get("code")) + .and_then(safe_scalar_field); + let error_message = error + .and_then(|value| value.get("message")) + .and_then(safe_scalar_field); + let status = parsed + .get("status") + .or_else(|| parsed.get("status_code")) + .and_then(safe_scalar_field); + let is_previous_response_not_found = + is_upstream_previous_response_not_found_error( + error_code.as_deref(), + error_message.as_deref(), + ); + + debug!( + event_type, + error_code, + error_message, + status, + is_previous_response_not_found, + "upstream_error_event" + ); + trace_downstream_sse_event(&downstream_sse_trace_metadata( + &parsed, + DownstreamTraceAction::ErrorTranslated, + None, + )); + let public_error = if is_previous_response_not_found { + ThreadlineError::PreviousResponseNotFound.public_error() + } else { + ThreadlineError::UpstreamErrorEvent.public_error() + }; + let public_message = public_error.message.clone().into_owned(); + let failed_payload = terminal_failed_payload( + parsed.get("response"), + response_id_from_event(&parsed), + public_error.code.into_owned(), + if is_previous_response_not_found { + public_message + } else { + error_message.unwrap_or(public_message) + }, + ); + if is_previous_response_not_found { + state.lease.mark_upstream_recoverable().await; + } else { + state.lease.mark_upstream_terminal().await; + } + state.lease.release(); + state.final_done_pending = true; + return Some(( + Ok::(sse_terminal_response_failed_chunk( + &failed_payload, + )), + state, + )); + } + _ => { + let mut trace_diagnostics = DownstreamTraceDiagnostics::default(); + if event_type == "response.output_text.delta" { + let key = visible_text_delta_source_key(&parsed); + if let Some(delta) = parsed.get("delta").and_then(Value::as_str) { + record_forwarded_visible_text_delta(&mut state, key, delta); + trace_diagnostics.visible_text_length = Some(delta.len()); + } + record_forwarded_observable_output( + &mut state.observable_output, + &event_type, + &parsed, + ); + state.downstream_visible_text_delta_count += 1; + trace_diagnostics.response_id = + response_id_from_event(&parsed).map(ToString::to_string); + trace_diagnostics.visible_text_delta_count = Some(1); + } else if event_type == "response.output_text.done" { + if let Some((key, payload)) = synthesized_output_text_done_delta(&parsed) + && queue_visible_text_delta( + &mut state, + key, + payload, + "response.output_text.done", + response_id_from_event(&parsed), + ) + { + state.queued_forwarded_event = + Some(QueuedForwardedEvent { payload: parsed }); + continue; + } + } else if event_type == "response.output_item.done" + && queue_visible_text_deltas( + &mut state, + synthesized_output_item_done_text_delta(&parsed), + "response.output_item.done", + response_id_from_event(&parsed), + ) + { + state.queued_forwarded_event = + Some(QueuedForwardedEvent { payload: parsed }); + continue; + } + record_forwarded_observable_output( + &mut state.observable_output, + &event_type, + &parsed, + ); + trace_downstream_sse_event(&downstream_sse_trace_metadata( + &parsed, + DownstreamTraceAction::Forwarded, + Some(&trace_diagnostics), + )); + debug!(event_type, "translation_event_forwarded"); + return Some(( + Ok::(sse_json_chunk(&event_type, &parsed)), + state, + )); + } + } + } + }) +} + +async fn try_reconnect_or_terminal_error( + state: &mut ResponseStreamState, +) -> Result>, ThreadlineError> { + if state.replay_stale_marker_on_pre_first_event_close && !state.upstream_event_seen { + state.lease.release(); + return Err(ThreadlineError::PreviousResponseNotFound); + } + + let Some(lease) = state.lease.retained_mut() else { + return Ok(None); + }; + + super::attempt_pre_first_event_reconnect( + &state.services, + lease, + &state.base_request, + state.previous_response_id.as_deref(), + state.upstream_event_seen, + &mut state.reconnect_attempted, + ) + .await +} + +#[cfg(test)] +mod tests { + use serde_json::json; + + use super::{ + CompletedSanitizationDiagnostics, DownstreamTraceAction, DownstreamTraceDiagnostics, + RESPONSES_TRANSLATION_DOWNSTREAM_SSE_EVENT, RESPONSES_TRANSLATION_EVENT_SUPPRESSED, + RESPONSES_TRANSLATION_NO_OBSERVABLE_OUTPUT_GUARD, RESPONSES_TRANSLATION_UPSTREAM_EVENT, + UpstreamEventTraceMetadata, VisibleAssistantText, VisibleTextSourceKey, + downstream_sse_trace_metadata, sanitized_completed_event_with_diagnostics, + }; + + #[test] + fn upstream_event_trace_metadata_redacts_argument_bodies_and_keeps_lengths() { + let arguments = "{\"input\":\"*** Begin Patch\\nsecret\\n*** End Patch\"}"; + let parsed = json!({ + "type": "response.output_item.added", + "output_index": 2, + "item_id": "item-visible", + "item": { + "type": "function_call", + "call_id": "call-visible", + "name": "apply_patch", + "arguments": arguments + } + }); + + let metadata = UpstreamEventTraceMetadata::from_event(&parsed); + + assert_eq!(metadata.event_type, "response.output_item.added"); + assert_eq!(metadata.item_type.as_deref(), Some("function_call")); + assert_eq!(metadata.item_name.as_deref(), Some("apply_patch")); + assert_eq!(metadata.call_id.as_deref(), Some("call-visible")); + assert_eq!(metadata.arguments_length, Some(arguments.len())); + assert_eq!(metadata.output_index, Some(2)); + assert_eq!(metadata.item_id.as_deref(), Some("item-visible")); + } + + #[test] + fn downstream_sse_trace_metadata_reports_action_without_delta_body() { + let delta = "{\"input\":\"*** Begin Patch"; + let parsed = json!({ + "type": "response.function_call_arguments.delta", + "output_index": 1, + "item_id": "fc_apply_patch_1", + "delta": delta + }); + + let metadata = + downstream_sse_trace_metadata(&parsed, DownstreamTraceAction::Forwarded, None); + + assert_eq!( + metadata.event_type, + "response.function_call_arguments.delta" + ); + assert_eq!( + metadata.translation_action, + DownstreamTraceAction::Forwarded.as_str() + ); + assert_eq!(metadata.delta_length, Some(delta.len())); + assert_eq!(metadata.output_index, Some(1)); + assert_eq!(metadata.item_id.as_deref(), Some("fc_apply_patch_1")); + assert_eq!(metadata.arguments_length, None); + assert_eq!(metadata.item_name, None); + } + + #[test] + fn synthetic_delta_emission_records_source_and_lengths_without_text() { + let synthetic_text = "synthesized visible assistant text"; + let parsed = json!({ + "type": "response.output_text.delta", + "item_id": "msg-visible", + "output_index": 3, + "content_index": 1, + "delta": synthetic_text + }); + + let metadata = downstream_sse_trace_metadata( + &parsed, + DownstreamTraceAction::Forwarded, + Some(&DownstreamTraceDiagnostics { + response_id: Some("response-visible".to_string()), + synthetic_delta_source: Some("response.output_text.done"), + visible_text_delta_count: Some(1), + visible_text_length: Some(synthetic_text.len()), + ..Default::default() + }), + ); + let metadata_debug = format!("{metadata:?}"); + + assert_eq!(metadata.translation_action, "forwarded"); + assert_eq!(metadata.event_type, "response.output_text.delta"); + assert_eq!(metadata.response_id.as_deref(), Some("response-visible")); + assert_eq!(metadata.item_id.as_deref(), Some("msg-visible")); + assert_eq!(metadata.output_index, Some(3)); + assert_eq!(metadata.content_index, Some(1)); + assert_eq!( + metadata.synthetic_delta_source, + Some("response.output_text.done") + ); + assert_eq!(metadata.visible_text_delta_count, Some(1)); + assert_eq!(metadata.visible_text_length, Some(synthetic_text.len())); + assert!(!metadata_debug.contains(synthetic_text)); + } + + #[test] + fn sanitized_completed_event_preserves_compaction_output() { + let parsed = json!({ + "type": "response.completed", + "response": { + "id": "response-compaction-only", + "output": [ + { + "type": "compaction", + "id": "compaction-1", + "encrypted_content": "opaque-compaction-payload" + } + ] + } + }); + + let (sanitized, diagnostics) = sanitized_completed_event_with_diagnostics(&parsed, &[]); + let sanitized_output = sanitized["response"]["output"] + .as_array() + .expect("sanitized output array"); + + assert_eq!(diagnostics, CompletedSanitizationDiagnostics::default()); + assert_eq!(sanitized_output.len(), 1); + assert_eq!(sanitized_output[0]["type"], "compaction"); + assert_eq!(sanitized_output[0]["id"], "compaction-1"); + assert_eq!( + sanitized_output[0]["encrypted_content"], + "opaque-compaction-payload" + ); + } + + #[test] + fn sanitized_completed_event_removes_internal_function_calls_but_preserves_compaction() { + let encrypted_content = "opaque-compaction-payload"; + let internal_arguments = "{\"token\":\"secret\"}"; + let visible_text = vec![VisibleAssistantText { + key: VisibleTextSourceKey::new(Some("message-visible".to_string()), Some(2), Some(0)), + text: "visible assistant answer".to_string(), + }]; + let parsed = json!({ + "type": "response.completed", + "response": { + "id": "response-sanitized", + "output": [ + { + "type": "function_call", + "id": "fc-internal", + "name": "threadline_echo", + "arguments": internal_arguments + }, + { + "type": "compaction", + "id": "compaction-1", + "encrypted_content": encrypted_content + } + ] + } + }); + + let (sanitized, diagnostics) = + sanitized_completed_event_with_diagnostics(&parsed, &visible_text); + let diagnostics_debug = format!("{diagnostics:?}"); + let sanitized_output = sanitized["response"]["output"] + .as_array() + .expect("sanitized output array"); + + assert_eq!( + diagnostics, + CompletedSanitizationDiagnostics { + sanitized_internal_function_call_count: 1, + sanitized_compaction_count: 0, + completed_visible_message_count: 1, + } + ); + assert_eq!(sanitized_output.len(), 2); + assert_eq!(sanitized_output[0]["type"], "compaction"); + assert_eq!(sanitized_output[0]["id"], "compaction-1"); + assert_eq!(sanitized_output[0]["encrypted_content"], encrypted_content); + assert_eq!(sanitized_output[1]["type"], "message"); + assert!(!diagnostics_debug.contains(encrypted_content)); + assert!(!diagnostics_debug.contains(internal_arguments)); + } + + #[test] + fn completed_sanitization_preserves_concrete_non_text_observable_output() { + let parsed = json!({ + "type": "response.completed", + "response": { + "id": "response-non-text-observable", + "output": [ + { + "type": "function_call", + "id": "fc-internal", + "name": "threadline_echo", + "arguments": "{\"value\":\"hidden\"}" + }, + { + "type": "function_call", + "id": "fc-external", + "call_id": "call-visible", + "name": "apply_patch", + "arguments": "{\"input\":\"*** Begin Patch\\n*** End Patch\"}" + }, + { + "type": "image_generation_call", + "id": "img-1", + "result": "image-asset-1" + }, + { + "type": "context", + "id": "ctx-1", + "summary": "context snapshot" + }, + { + "type": "compaction", + "id": "cmp-1", + "encrypted_content": "opaque" + } + ] + } + }); + + let (sanitized, diagnostics) = sanitized_completed_event_with_diagnostics(&parsed, &[]); + let sanitized_output = sanitized["response"]["output"] + .as_array() + .expect("sanitized output array"); + + assert_eq!( + diagnostics, + CompletedSanitizationDiagnostics { + sanitized_internal_function_call_count: 1, + sanitized_compaction_count: 0, + completed_visible_message_count: 0, + } + ); + assert_eq!(sanitized_output.len(), 4); + assert_eq!(sanitized_output[0]["id"], "fc-external"); + assert_eq!(sanitized_output[0]["name"], "apply_patch"); + assert_eq!(sanitized_output[1]["id"], "img-1"); + assert_eq!(sanitized_output[1]["type"], "image_generation_call"); + assert_eq!(sanitized_output[2]["id"], "ctx-1"); + assert_eq!(sanitized_output[2]["type"], "context"); + assert_eq!(sanitized_output[3]["id"], "cmp-1"); + assert_eq!(sanitized_output[3]["type"], "compaction"); + } + + #[test] + fn upstream_event_trace_metadata_reports_compaction_without_encrypted_content() { + let encrypted_content = "opaque-compaction-payload"; + let parsed = json!({ + "type": "response.output_item.done", + "output_index": 4, + "item": { + "type": "compaction", + "id": "compaction-visible", + "encrypted_content": encrypted_content + } + }); + + let metadata = UpstreamEventTraceMetadata::from_event(&parsed); + let metadata_debug = format!("{metadata:?}"); + + assert_eq!(metadata.event_type, "response.output_item.done"); + assert_eq!(metadata.item_type.as_deref(), Some("compaction")); + assert!(metadata.is_compaction); + assert_eq!( + metadata.compaction_id.as_deref(), + Some("compaction-visible") + ); + assert_eq!(metadata.output_index, Some(4)); + assert_eq!(metadata.has_encrypted_content, Some(true)); + assert!(!metadata_debug.contains(encrypted_content)); + } + + #[test] + fn upstream_event_trace_metadata_reports_compaction_without_blob_when_missing() { + let parsed = json!({ + "type": "response.output_item.added", + "output_index": 0, + "item": { + "type": "compaction", + "id": "compaction-empty" + } + }); + + let metadata = UpstreamEventTraceMetadata::from_event(&parsed); + + assert!(metadata.is_compaction); + assert_eq!(metadata.compaction_id.as_deref(), Some("compaction-empty")); + assert_eq!(metadata.has_encrypted_content, Some(false)); + } + + #[test] + fn translation_trace_event_names_remain_stable() { + assert_eq!( + RESPONSES_TRANSLATION_UPSTREAM_EVENT, + "responses_translation_upstream_event" + ); + assert_eq!( + RESPONSES_TRANSLATION_DOWNSTREAM_SSE_EVENT, + "responses_translation_downstream_sse_event" + ); + assert_eq!( + RESPONSES_TRANSLATION_EVENT_SUPPRESSED, + "responses_translation_event_suppressed" + ); + assert_eq!( + RESPONSES_TRANSLATION_NO_OBSERVABLE_OUTPUT_GUARD, + "responses_translation_no_observable_output_guard" + ); + } +} diff --git a/src/responses/upstream.rs b/src/responses/upstream.rs new file mode 100644 index 0000000..9d0ab98 --- /dev/null +++ b/src/responses/upstream.rs @@ -0,0 +1,293 @@ +use std::sync::Arc; + +use futures_util::future::BoxFuture; +use serde_json::{Map, Value}; + +use crate::auth::LoadedUpstreamAuth; +use crate::codex_ws::UpstreamSessionDescriptor; +use crate::errors::ThreadlineError; +use crate::ws_pump::LiveUpstreamWebSocket; + +const UNSUPPORTED_RESPONSE_FIELDS: &[&str] = &[ + "max_output_tokens", + "max_tokens", + "max_completion_tokens", + "truncation", +]; + +pub trait UpstreamAuthProvider: Send + Sync { + fn load(&self) -> Result; +} + +pub trait UpstreamConnector: Send + Sync { + fn connect( + &self, + auth: LoadedUpstreamAuth, + session: Option, + ) -> BoxFuture<'static, Result>; +} + +#[derive(Clone)] +pub struct ThreadlineServices { + auth_provider: Arc, + connector: Arc, +} + +pub struct ConnectedUpstream { + pub websocket: Arc, + pub session: UpstreamSessionDescriptor, + pub turn_state: Option, +} + +impl ThreadlineServices { + pub fn new( + auth_provider: Arc, + connector: Arc, + ) -> Self { + Self { + auth_provider, + connector, + } + } + + pub fn auth_provider(&self) -> &Arc { + &self.auth_provider + } + + pub fn connector(&self) -> &Arc { + &self.connector + } +} + +pub(super) fn build_response_create_payload(request: Value) -> Result { + let mut payload = require_payload_object(request)?; + payload.insert( + "type".to_string(), + Value::String("response.create".to_string()), + ); + payload.insert("store".to_string(), Value::Bool(false)); + + if matches!(payload.get("instructions"), None | Some(Value::Null)) { + payload.insert("instructions".to_string(), Value::String(String::new())); + } + + remove_codex_unsupported_response_fields(&mut payload); + normalize_codex_reasoning_fields(&mut payload); + Ok(Value::Object(payload)) +} + +pub(super) fn remove_codex_unsupported_response_fields(payload: &mut Map) { + for field in UNSUPPORTED_RESPONSE_FIELDS { + payload.remove(*field); + } +} + +pub(super) fn normalize_codex_reasoning_fields(payload: &mut Map) { + let remove_reasoning = match payload.get_mut("reasoning").and_then(Value::as_object_mut) { + Some(reasoning) => { + // VS Code briefly sent `reasoning.summary = "off"` to disable reasoning summaries. + // Codex/Responses API does not accept "off"; disabling summaries means omitting + // the `summary` field entirely. Preserve valid values and only strip this known + // compatibility value. + if reasoning.get("summary").and_then(Value::as_str) == Some("off") { + reasoning.remove("summary"); + } + + reasoning.is_empty() + } + None => false, + }; + + if remove_reasoning { + payload.remove("reasoning"); + } +} + +pub(super) async fn send_response_create( + upstream: &LiveUpstreamWebSocket, + request_payload: &Map, +) -> Result<(), ThreadlineError> { + let payload = build_response_create_payload(Value::Object(request_payload.clone()))?; + let text = serde_json::to_string(&payload).expect("serialize response.create payload"); + upstream + .send_text(text) + .await + .map_err(|_| ThreadlineError::UpstreamWebSocketClosed) +} + +pub(super) fn build_followup_tool_outputs_payload( + request: Value, + previous_response_id: &str, + input: Value, +) -> Result { + let mut payload = require_payload_object(request)?; + payload.insert( + "previous_response_id".to_string(), + Value::String(previous_response_id.to_string()), + ); + payload.insert("input".to_string(), input); + build_response_create_payload(Value::Object(payload)) +} + +pub(super) async fn send_followup_tool_outputs( + upstream: &LiveUpstreamWebSocket, + request_payload: &Map, + previous_response_id: &str, + input: Value, +) -> Result<(), ThreadlineError> { + let payload = build_followup_tool_outputs_payload( + Value::Object(request_payload.clone()), + previous_response_id, + input, + )?; + let text = serde_json::to_string(&payload).expect("serialize followup response.create payload"); + upstream + .send_text(text) + .await + .map_err(|_| ThreadlineError::UpstreamWebSocketClosed) +} + +fn require_payload_object(payload: Value) -> Result, ThreadlineError> { + match payload { + Value::Object(object) => Ok(object), + _ => Err(ThreadlineError::InvalidResponsesRequest), + } +} + +#[cfg(test)] +mod tests { + use serde_json::json; + + use super::{build_followup_tool_outputs_payload, build_response_create_payload}; + + #[test] + fn build_response_create_payload_sets_required_defaults_and_filters_unsupported_fields() { + let payload = build_response_create_payload(json!({ + "model": "gpt-test", + "instructions": null, + "max_output_tokens": 32, + "max_tokens": 64, + "max_completion_tokens": 96, + "truncation": "auto" + })) + .expect("response.create payload"); + + assert_eq!(payload["type"], "response.create"); + assert_eq!(payload["store"], false); + assert_eq!(payload["instructions"], ""); + assert!(payload.get("max_output_tokens").is_none()); + assert!(payload.get("max_tokens").is_none()); + assert!(payload.get("max_completion_tokens").is_none()); + assert!(payload.get("truncation").is_none()); + } + + #[test] + fn build_response_create_payload_omits_off_reasoning_summary() { + let payload = build_response_create_payload(json!({ + "model": "gpt-test", + "instructions": "keep", + "reasoning": { + "effort": "medium", + "summary": "off" + } + })) + .expect("response.create payload"); + + assert_eq!( + payload["reasoning"], + json!({ + "effort": "medium" + }) + ); + } + + #[test] + fn build_response_create_payload_removes_empty_reasoning_after_off_summary() { + let payload = build_response_create_payload(json!({ + "model": "gpt-test", + "instructions": "keep", + "reasoning": { + "summary": "off" + } + })) + .expect("response.create payload"); + + assert!(payload.get("reasoning").is_none()); + } + + #[test] + fn build_response_create_payload_preserves_context_management_and_previous_response_id() { + let payload = build_response_create_payload(json!({ + "model": "gpt-test", + "instructions": "keep", + "previous_response_id": "resp_123", + "context_management": { + "type": "compaction", + "compact_threshold": 12345 + }, + "reasoning": { + "effort": "high", + "summary": "auto" + }, + "include": ["reasoning.encrypted_content"], + "max_output_tokens": 32, + "max_tokens": 64, + "max_completion_tokens": 96, + "truncation": "auto" + })) + .expect("response.create payload"); + + assert_eq!(payload["type"], "response.create"); + assert_eq!(payload["store"], false); + assert_eq!(payload["instructions"], "keep"); + assert_eq!(payload["previous_response_id"], "resp_123"); + assert_eq!( + payload["context_management"], + json!({ + "type": "compaction", + "compact_threshold": 12345 + }) + ); + assert_eq!( + payload["reasoning"], + json!({ + "effort": "high", + "summary": "auto" + }) + ); + assert_eq!(payload["include"], json!(["reasoning.encrypted_content"])); + assert!(payload.get("max_output_tokens").is_none()); + assert!(payload.get("max_tokens").is_none()); + assert!(payload.get("max_completion_tokens").is_none()); + assert!(payload.get("truncation").is_none()); + } + + #[test] + fn build_followup_tool_outputs_payload_preserves_previous_response_id_and_output_shape() { + let payload = build_followup_tool_outputs_payload( + json!({ + "model": "gpt-test", + "instructions": "keep", + "truncation": "auto" + }), + "resp_intermediate", + json!([ + { + "type": "function_call_output", + "call_id": "call_123", + "output": "done" + } + ]), + ) + .expect("followup payload"); + + assert_eq!(payload["type"], "response.create"); + assert_eq!(payload["store"], false); + assert_eq!(payload["previous_response_id"], "resp_intermediate"); + assert_eq!(payload["instructions"], "keep"); + assert_eq!(payload["input"][0]["type"], "function_call_output"); + assert_eq!(payload["input"][0]["call_id"], "call_123"); + assert_eq!(payload["input"][0]["output"], "done"); + assert!(payload.get("truncation").is_none()); + } +} diff --git a/src/tools.rs b/src/tools.rs new file mode 100644 index 0000000..cc09df2 --- /dev/null +++ b/src/tools.rs @@ -0,0 +1,367 @@ +use std::sync::OnceLock; + +use serde_json::{Map, Value, json}; +use tracing::debug; + +use crate::config::active_job_manager_config; +use crate::errors::ThreadlineError; +use crate::jobs::ThreadlineJobManager; + +pub const INTERNAL_TOOL_PREFIX: &str = "threadline_"; +const ECHO_TOOL_NAME: &str = "threadline_echo"; +const START_JOB_TOOL_NAME: &str = "threadline_start_job"; +const POLL_JOB_TOOL_NAME: &str = "threadline_poll_job"; +const READ_JOB_OUTPUT_TOOL_NAME: &str = "threadline_read_job_output"; +const GET_JOB_RESULT_TOOL_NAME: &str = "threadline_get_job_result"; +const CANCEL_JOB_TOOL_NAME: &str = "threadline_cancel_job"; + +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct PendingInternalToolOutput { + call_id: String, + output: String, +} + +#[derive(Debug, Clone, PartialEq)] +pub struct InternalToolCall { + call_id: String, + name: String, + arguments: Value, +} + +impl PendingInternalToolOutput { + pub fn new(call_id: impl Into, output: impl Into) -> Self { + Self { + call_id: call_id.into(), + output: output.into(), + } + } + + pub fn call_id(&self) -> &str { + &self.call_id + } + + pub fn into_followup_input(self) -> Value { + json!({ + "type": "function_call_output", + "call_id": self.call_id, + "output": self.output, + }) + } +} + +impl InternalToolCall { + pub fn from_event(event: &Value) -> Result, ThreadlineError> { + let Some(event_type) = event.get("type").and_then(Value::as_str) else { + return Ok(None); + }; + if event_type != "response.output_item.done" { + return Ok(None); + } + + let Some(item) = event.get("item") else { + return Ok(None); + }; + if item.get("type").and_then(Value::as_str) != Some("function_call") { + return Ok(None); + } + + let Some(name) = item.get("name").and_then(Value::as_str) else { + return Ok(None); + }; + if !is_internal_tool_name(name) { + return Ok(None); + } + + let call_id = item + .get("call_id") + .and_then(Value::as_str) + .ok_or(ThreadlineError::InternalToolFailed)?; + if call_id.trim().is_empty() { + return Err(ThreadlineError::InternalToolFailed); + } + let arguments = parse_arguments(item.get("arguments"))?; + + debug!(call_id = %call_id, tool_name = name, "internal_tool_detected"); + + Ok(Some(Self { + call_id: call_id.to_string(), + name: name.to_string(), + arguments, + })) + } + + pub fn execute(self) -> Result { + self.execute_with_job_manager(&global_job_manager()) + } + + pub fn execute_with_job_manager( + self, + job_manager: &ThreadlineJobManager, + ) -> Result { + match self.name.as_str() { + ECHO_TOOL_NAME => Ok(PendingInternalToolOutput::new( + self.call_id, + extract_echo_output(&self.arguments), + )), + START_JOB_TOOL_NAME => Ok(PendingInternalToolOutput::new( + self.call_id, + start_job_output(job_manager, &self.arguments).to_string(), + )), + POLL_JOB_TOOL_NAME => Ok(PendingInternalToolOutput::new( + self.call_id, + poll_job_output(job_manager, &self.arguments).to_string(), + )), + READ_JOB_OUTPUT_TOOL_NAME => Ok(PendingInternalToolOutput::new( + self.call_id, + read_job_output(job_manager, &self.arguments).to_string(), + )), + GET_JOB_RESULT_TOOL_NAME => Ok(PendingInternalToolOutput::new( + self.call_id, + get_job_result_output(job_manager, &self.arguments).to_string(), + )), + CANCEL_JOB_TOOL_NAME => Ok(PendingInternalToolOutput::new( + self.call_id, + cancel_job_output(job_manager, &self.arguments).to_string(), + )), + _ => Err(ThreadlineError::InternalToolFailed), + } + } +} + +pub fn inject_internal_tools(payload: &mut Map) { + let internal_tools = internal_tool_definitions(); + + match payload.get_mut("tools") { + Some(Value::Array(existing_tools)) => { + for tool in internal_tools { + let tool_name = tool.get("name").and_then(Value::as_str); + let already_present = tool_name.is_some_and(|name| { + existing_tools + .iter() + .any(|existing| existing.get("name").and_then(Value::as_str) == Some(name)) + }); + if !already_present { + existing_tools.push(tool); + } + } + } + Some(Value::Null) | None => { + payload.insert("tools".to_string(), Value::Array(internal_tools)); + } + Some(_) => {} + } +} + +pub fn build_followup_input(outputs: Vec) -> Value { + Value::Array( + outputs + .into_iter() + .map(PendingInternalToolOutput::into_followup_input) + .collect(), + ) +} + +pub fn is_internal_tool_name(name: &str) -> bool { + name.starts_with(INTERNAL_TOOL_PREFIX) +} + +pub fn event_contains_internal_tool_name(event: &Value) -> bool { + let Some(item) = event.get("item") else { + return false; + }; + if item.get("type").and_then(Value::as_str) != Some("function_call") { + return false; + } + + value_contains_internal_tool_name(item) +} + +fn parse_arguments(arguments: Option<&Value>) -> Result { + match arguments { + Some(Value::String(text)) => { + serde_json::from_str(text).map_err(|_| ThreadlineError::InternalToolFailed) + } + Some(Value::Null) | None => Ok(Value::Object(Map::new())), + Some(value) => Ok(value.clone()), + } +} + +fn extract_echo_output(arguments: &Value) -> String { + arguments + .get("value") + .and_then(Value::as_str) + .map(ToString::to_string) + .unwrap_or_else(|| arguments.to_string()) +} + +fn value_contains_internal_tool_name(value: &Value) -> bool { + match value { + Value::Object(map) => map.iter().any(|(key, nested)| { + ((key == "name" || key == "tool_name") + && nested.as_str().is_some_and(is_internal_tool_name)) + || value_contains_internal_tool_name(nested) + }), + Value::Array(items) => items.iter().any(value_contains_internal_tool_name), + _ => false, + } +} + +fn internal_tool_definitions() -> Vec { + vec![ + json!({ + "type": "function", + "name": ECHO_TOOL_NAME, + "description": "Return the provided value so Threadline can satisfy local tool loops without involving downstream clients.", + "parameters": { + "type": "object", + "properties": { + "value": { + "type": "string" + } + }, + "required": ["value"], + "additionalProperties": false + } + }), + json!({ + "type": "function", + "name": START_JOB_TOOL_NAME, + "description": "Start a background Threadline job for an allowed local command, return immediately with a job id, and avoid busy-polling when independent work is still available.", + "parameters": { + "type": "object", + "properties": { + "command": { + "type": "array", + "items": {"type": "string"}, + "minItems": 1 + } + }, + "required": ["command"], + "additionalProperties": false + } + }), + json!({ + "type": "function", + "name": POLL_JOB_TOOL_NAME, + "description": "Check a previously started Threadline job at a natural checkpoint for status updates, not in a tight loop.", + "parameters": { + "type": "object", + "properties": { + "job_id": {"type": "string"} + }, + "required": ["job_id"], + "additionalProperties": false + } + }), + json!({ + "type": "function", + "name": READ_JOB_OUTPUT_TOOL_NAME, + "description": "Read incremental output from a Threadline job using a previous output offset; preserve next_offset for the next read and notice truncated_before if older buffered output was dropped.", + "parameters": { + "type": "object", + "properties": { + "job_id": {"type": "string"}, + "offset": {"type": "integer", "minimum": 0} + }, + "required": ["job_id"], + "additionalProperties": false + } + }), + json!({ + "type": "function", + "name": GET_JOB_RESULT_TOOL_NAME, + "description": "Get the current or terminal result payload for a Threadline job after a terminal poll state or before final claims that depend on success or failure.", + "parameters": { + "type": "object", + "properties": { + "job_id": {"type": "string"} + }, + "required": ["job_id"], + "additionalProperties": false + } + }), + json!({ + "type": "function", + "name": CANCEL_JOB_TOOL_NAME, + "description": "Cancel a stuck or no-longer-useful Threadline job, then poll or get the result to confirm the terminal state.", + "parameters": { + "type": "object", + "properties": { + "job_id": {"type": "string"} + }, + "required": ["job_id"], + "additionalProperties": false + } + }), + ] +} + +fn global_job_manager() -> ThreadlineJobManager { + static JOB_MANAGER: OnceLock = OnceLock::new(); + + JOB_MANAGER + .get_or_init(|| ThreadlineJobManager::new(active_job_manager_config())) + .clone() +} + +fn start_job_output(job_manager: &ThreadlineJobManager, arguments: &Value) -> Value { + match extract_string_list(arguments.get("command")) { + Some(command) => job_manager.start_command_json(command), + None => invalid_job_request("threadline_start_job requires a command array of strings."), + } +} + +fn poll_job_output(job_manager: &ThreadlineJobManager, arguments: &Value) -> Value { + match extract_job_id(arguments) { + Some(job_id) => job_manager.poll_json(&job_id), + None => invalid_job_request("threadline_poll_job requires a job_id string."), + } +} + +fn read_job_output(job_manager: &ThreadlineJobManager, arguments: &Value) -> Value { + match extract_job_id(arguments) { + Some(job_id) => job_manager.read_output_json(&job_id, extract_offset(arguments)), + None => invalid_job_request("threadline_read_job_output requires a job_id string."), + } +} + +fn get_job_result_output(job_manager: &ThreadlineJobManager, arguments: &Value) -> Value { + match extract_job_id(arguments) { + Some(job_id) => job_manager.get_result_json(&job_id), + None => invalid_job_request("threadline_get_job_result requires a job_id string."), + } +} + +fn cancel_job_output(job_manager: &ThreadlineJobManager, arguments: &Value) -> Value { + match extract_job_id(arguments) { + Some(job_id) => job_manager.cancel_json(&job_id), + None => invalid_job_request("threadline_cancel_job requires a job_id string."), + } +} + +fn extract_string_list(value: Option<&Value>) -> Option> { + let items = value?.as_array()?; + items + .iter() + .map(|item| item.as_str().map(ToString::to_string)) + .collect() +} + +fn extract_job_id(arguments: &Value) -> Option { + arguments + .get("job_id") + .and_then(Value::as_str) + .map(ToString::to_string) +} + +fn extract_offset(arguments: &Value) -> u64 { + arguments.get("offset").and_then(Value::as_u64).unwrap_or(0) +} + +fn invalid_job_request(message: &'static str) -> Value { + json!({ + "ok": false, + "code": "invalid_job_request", + "message": message, + }) +} diff --git a/src/ws_pump.rs b/src/ws_pump.rs new file mode 100644 index 0000000..ed96024 --- /dev/null +++ b/src/ws_pump.rs @@ -0,0 +1,234 @@ +use std::sync::Arc; +use std::sync::atomic::{AtomicBool, Ordering}; + +use futures_util::{SinkExt, StreamExt}; +use thiserror::Error; +use tokio::io::{AsyncRead, AsyncWrite}; +use tokio::sync::{Mutex, mpsc}; +use tokio::task::JoinHandle; +use tokio_tungstenite::WebSocketStream; +use tokio_tungstenite::tungstenite::Message; +use tracing::debug; + +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct UpstreamCloseMetadata { + pub code: Option, + pub reason: Option, + pub error: Option, +} + +#[derive(Debug, Error, PartialEq, Eq)] +pub enum UpstreamWebSocketError { + #[error("Threadline could not queue an outbound upstream websocket message.")] + OutboundQueueClosed, +} + +pub struct LiveUpstreamWebSocket { + outbound_tx: mpsc::Sender, + inbound_rx: Mutex>, + close_metadata: Arc>>, + is_closed: Arc, + task: JoinHandle<()>, +} + +enum OutboundCommand { + Text(String), +} + +const OUTBOUND_CHANNEL_CAPACITY: usize = 32; + +impl LiveUpstreamWebSocket { + pub fn from_stream(_stream: WebSocketStream) -> Self + where + S: AsyncRead + AsyncWrite + Unpin + Send + 'static, + { + let (mut writer, mut reader) = _stream.split(); + let (outbound_tx, mut outbound_rx) = mpsc::channel(OUTBOUND_CHANNEL_CAPACITY); + let (inbound_tx, inbound_rx) = mpsc::unbounded_channel(); + let close_metadata = Arc::new(Mutex::new(None)); + let task_close_metadata = Arc::clone(&close_metadata); + let is_closed = Arc::new(AtomicBool::new(false)); + let task_is_closed = Arc::clone(&is_closed); + + let task = tokio::spawn(async move { + debug!( + outbound_capacity = OUTBOUND_CHANNEL_CAPACITY, + "ws_pump_started" + ); + loop { + tokio::select! { + outbound = outbound_rx.recv() => match outbound { + Some(OutboundCommand::Text(text)) => { + if let Err(error) = writer.send(Message::Text(text)).await { + record_error(&task_close_metadata, error.to_string()).await; + break; + } + } + None => { + record_close( + &task_close_metadata, + outbound_channel_closed_metadata(), + ) + .await; + break; + } + }, + inbound = reader.next() => match inbound { + Some(Ok(Message::Text(text))) => { + let _ = inbound_tx.send(text.to_string()); + } + Some(Ok(Message::Binary(bytes))) => { + let _ = inbound_tx.send(String::from_utf8_lossy(bytes.as_ref()).into_owned()); + } + Some(Ok(Message::Ping(payload))) => { + let payload_len = payload.len(); + debug!(payload_len, "ws_pump_ping_received"); + if let Err(error) = writer.send(Message::Pong(payload)).await { + record_error(&task_close_metadata, error.to_string()).await; + break; + } + debug!(payload_len, "ws_pump_pong_sent"); + } + Some(Ok(Message::Pong(_))) => {} + Some(Ok(Message::Close(frame))) => { + let metadata = UpstreamCloseMetadata { + code: frame.as_ref().map(|frame| u16::from(frame.code)), + reason: frame.as_ref().map(|frame| frame.reason.to_string()), + error: None, + }; + record_close(&task_close_metadata, metadata).await; + break; + } + Some(Ok(Message::Frame(_))) => {} + Some(Err(error)) => { + record_error(&task_close_metadata, error.to_string()).await; + break; + } + None => break, + } + } + } + + let mut guard = task_close_metadata.lock().await; + if guard.is_none() { + *guard = Some(UpstreamCloseMetadata { + code: None, + reason: None, + error: None, + }); + } + task_is_closed.store(true, Ordering::SeqCst); + }); + + Self { + outbound_tx, + inbound_rx: Mutex::new(inbound_rx), + close_metadata, + is_closed, + task, + } + } + + pub async fn send_text(&self, text: impl Into) -> Result<(), UpstreamWebSocketError> { + self.outbound_tx + .send(OutboundCommand::Text(text.into())) + .await + .map_err(|_| UpstreamWebSocketError::OutboundQueueClosed) + } + + pub async fn recv_text(&self) -> Result, UpstreamWebSocketError> { + Ok(self.inbound_rx.lock().await.recv().await) + } + + pub fn is_closed(&self) -> bool { + self.is_closed.load(Ordering::SeqCst) + } + + pub async fn close_metadata(&self) -> Option { + self.close_metadata.lock().await.clone() + } +} + +impl Drop for LiveUpstreamWebSocket { + fn drop(&mut self) { + self.task.abort(); + } +} + +async fn record_close( + target: &Arc>>, + metadata: UpstreamCloseMetadata, +) { + *target.lock().await = Some(metadata); +} + +async fn record_error(target: &Arc>>, error: String) { + *target.lock().await = Some(UpstreamCloseMetadata { + code: None, + reason: None, + error: Some(error), + }); +} + +fn outbound_channel_closed_metadata() -> UpstreamCloseMetadata { + UpstreamCloseMetadata { + code: None, + reason: Some("outbound channel closed".to_string()), + error: None, + } +} + +#[cfg(test)] +mod tests { + use super::*; + use std::time::Duration; + use tokio::net::TcpListener; + use tokio::time::timeout; + use tokio_tungstenite::accept_async; + use tokio_tungstenite::connect_async; + + async fn connect_test_pump() -> LiveUpstreamWebSocket { + let listener = TcpListener::bind("127.0.0.1:0") + .await + .expect("bind listener"); + let address = listener.local_addr().expect("local addr"); + let accept_task = tokio::spawn(async move { + let (stream, _) = listener.accept().await.expect("accept client"); + let websocket = accept_async(stream).await.expect("accept websocket"); + let (_sink, _stream) = websocket.split(); + tokio::time::sleep(Duration::from_secs(5)).await; + }); + + let (stream, _) = connect_async(format!("ws://{address}")) + .await + .expect("connect websocket"); + let pump = LiveUpstreamWebSocket::from_stream(stream); + + drop(accept_task); + pump + } + + #[tokio::test] + async fn websocket_pump_records_metadata_when_outbound_channel_closes() { + let mut pump = connect_test_pump().await; + let (replacement_tx, replacement_rx) = mpsc::channel(1); + drop(replacement_rx); + let original_tx = std::mem::replace(&mut pump.outbound_tx, replacement_tx); + drop(original_tx); + + let metadata = timeout(Duration::from_secs(2), async { + loop { + if pump.is_closed() + && let Some(metadata) = pump.close_metadata().await + { + break metadata; + } + tokio::time::sleep(Duration::from_millis(10)).await; + } + }) + .await + .expect("pump should close after outbound sender drop"); + + assert_eq!(metadata, outbound_channel_closed_metadata()); + } +} diff --git a/tests/http_surface.rs b/tests/http_surface.rs new file mode 100644 index 0000000..81e7afb --- /dev/null +++ b/tests/http_surface.rs @@ -0,0 +1,574 @@ +use axum::body::{Body, to_bytes}; +use axum::http::{Request, StatusCode}; +use futures_util::future::BoxFuture; +use serde_json::{Value, json}; +use std::sync::Arc; +use tower::ServiceExt; + +use threadline::auth::LoadedUpstreamAuth; +use threadline::codex_ws::UpstreamSessionDescriptor; +use threadline::config::ThreadlineConfig; +use threadline::errors::ThreadlineError; +use threadline::http::build_router; +use threadline::http::build_router_with_services; +use threadline::models::RouteProfile; +use threadline::responses::{ + ConnectedUpstream, ThreadlineServices, UpstreamAuthProvider, UpstreamConnector, +}; + +#[derive(Clone)] +struct MissingAuthProvider; + +impl UpstreamAuthProvider for MissingAuthProvider { + fn load(&self) -> Result { + Err(ThreadlineError::UpstreamCredentialsUnavailable) + } +} + +#[derive(Clone)] +struct UnusedConnector; + +impl UpstreamConnector for UnusedConnector { + fn connect( + &self, + _auth: LoadedUpstreamAuth, + _session: Option, + ) -> BoxFuture<'static, Result> { + Box::pin(async { panic!("connector should not be called when auth loading fails") }) + } +} + +const NEW_MAIN_VISIBLE_MODEL_IDS: [&str; 3] = [ + "threadline-main-gpt-5.6-sol", + "threadline-main-gpt-5.6-terra", + "threadline-main-gpt-5.6-luna", +]; + +const NEW_MAIN_RAW_COMPATIBILITY_MODEL_IDS: [&str; 3] = + ["gpt-5.6-sol", "gpt-5.6-terra", "gpt-5.6-luna"]; + +const ADVERTISED_MAIN_MODEL_IDS: [&str; 5] = [ + "threadline-main-gpt-5.6-sol", + "threadline-main-gpt-5.6-terra", + "threadline-main-gpt-5.6-luna", + "threadline-main-gpt-5.5", + "threadline-main-gpt-5.4", +]; + +const ADVERTISED_UTILITY_MODEL_IDS: [&str; 2] = [ + "threadline-utility-gpt-5.4-mini", + "threadline-utility-gpt-5.3-codex-spark", +]; + +const ACCEPTED_MAIN_MODEL_IDS: [&str; 12] = [ + "threadline-main-gpt-5.6-sol", + "threadline-main-gpt-5.6-terra", + "threadline-main-gpt-5.6-luna", + "threadline-main-gpt-5.5", + "threadline-main-gpt-5.4", + "gpt-5.6-sol", + "gpt-5.6-terra", + "gpt-5.6-luna", + "gpt-5.5", + "gpt-5.4", + "gpt-5.4-mini", + "gpt-5.3-codex-spark", +]; + +const UNSUPPORTED_MODEL_IDS: [&str; 4] = [ + "threadline-utility-gpt-5.4-mini", + "threadline-utility-gpt-5.3-codex-spark", + "codex-mini-latest", + "threadline-test-unsupported", +]; + +const HIDDEN_MAIN_COMPATIBILITY_MODEL_IDS: [&str; 7] = [ + "gpt-5.6-sol", + "gpt-5.6-terra", + "gpt-5.6-luna", + "gpt-5.5", + "gpt-5.4", + "gpt-5.4-mini", + "gpt-5.3-codex-spark", +]; + +async fn read_json_body(response: axum::response::Response) -> Value { + let body = to_bytes(response.into_body(), usize::MAX).await.unwrap(); + serde_json::from_slice(&body).unwrap() +} + +fn invalid_model_payload(model: Value) -> Value { + json!({ "model": model }) +} + +async fn post_responses_json(app: axum::Router, payload: Value) -> axum::response::Response { + app.oneshot( + Request::builder() + .method("POST") + .uri("/v1/responses") + .header("content-type", "application/json") + .body(Body::from(payload.to_string())) + .unwrap(), + ) + .await + .unwrap() +} + +fn assert_invalid_model_error(payload: &Value) { + assert_eq!(payload["error"]["type"], "invalid_request_error"); + assert_eq!(payload["error"]["code"], "invalid_model"); +} + +fn assert_unsupported_reasoning_context_error(payload: &Value) { + assert_eq!(payload["error"]["type"], "invalid_request_error"); + assert_eq!(payload["error"]["code"], "unsupported_reasoning_context"); + assert_eq!( + payload["error"]["message"], + "reasoning.context=all_turns is not supported for this model. The model metadata has use_responses_lite=false." + ); +} + +fn utility_config() -> ThreadlineConfig { + ThreadlineConfig { + profile: RouteProfile::Utility, + ..ThreadlineConfig::default() + } +} + +#[tokio::test] +async fn health_endpoint_reports_ok() { + let app = build_router(ThreadlineConfig::default()); + + let response = app + .oneshot( + Request::builder() + .uri("/health") + .body(Body::empty()) + .unwrap(), + ) + .await + .unwrap(); + + assert_eq!(response.status(), StatusCode::OK); + + let body = to_bytes(response.into_body(), usize::MAX).await.unwrap(); + let payload: Value = serde_json::from_slice(&body).unwrap(); + + assert_eq!(payload["status"], "ok"); + assert_eq!(payload["service"], "threadline"); +} + +#[tokio::test] +async fn models_endpoint_returns_supported_models() { + let app = build_router(ThreadlineConfig::default()); + + let response = app + .oneshot( + Request::builder() + .uri("/v1/models") + .body(Body::empty()) + .unwrap(), + ) + .await + .unwrap(); + + assert_eq!(response.status(), StatusCode::OK); + + let payload = read_json_body(response).await; + + assert_eq!(payload["object"], "list"); + let models = payload["data"].as_array().expect("models list"); + assert_eq!(models.len(), ADVERTISED_MAIN_MODEL_IDS.len()); + + for (model, expected_id) in models.iter().zip(ADVERTISED_MAIN_MODEL_IDS) { + assert_eq!(model["id"], expected_id); + assert_eq!(model["object"], "model"); + assert_eq!(model["created"], 0); + assert_eq!(model["owned_by"], "threadline"); + } +} + +#[tokio::test] +async fn models_endpoint_returns_only_utility_profile_models() { + let app = build_router(ThreadlineConfig { + profile: RouteProfile::Utility, + ..ThreadlineConfig::default() + }); + + let response = app + .oneshot( + Request::builder() + .uri("/v1/models") + .body(Body::empty()) + .unwrap(), + ) + .await + .unwrap(); + + assert_eq!(response.status(), StatusCode::OK); + + let payload = read_json_body(response).await; + + assert_eq!(payload["object"], "list"); + let models = payload["data"].as_array().expect("models list"); + assert_eq!(models.len(), ADVERTISED_UTILITY_MODEL_IDS.len()); + + for (model, expected_id) in models.iter().zip(ADVERTISED_UTILITY_MODEL_IDS) { + assert_eq!(model["id"], expected_id); + assert_eq!(model["object"], "model"); + assert_eq!(model["created"], 0); + assert_eq!(model["owned_by"], "threadline"); + } +} + +#[tokio::test] +async fn responses_endpoint_rejects_missing_model() { + let app = build_router_with_services( + ThreadlineConfig::default(), + ThreadlineServices::new(Arc::new(MissingAuthProvider), Arc::new(UnusedConnector)), + ); + + let response = post_responses_json(app, json!({})).await; + + assert_eq!(response.status(), StatusCode::BAD_REQUEST); + + let payload = read_json_body(response).await; + assert_invalid_model_error(&payload); +} + +#[tokio::test] +async fn responses_endpoint_rejects_non_string_model() { + let app = build_router_with_services( + ThreadlineConfig::default(), + ThreadlineServices::new(Arc::new(MissingAuthProvider), Arc::new(UnusedConnector)), + ); + + let response = + post_responses_json(app, invalid_model_payload(json!({ "id": "gpt-5.4" }))).await; + + assert_eq!(response.status(), StatusCode::BAD_REQUEST); + + let payload = read_json_body(response).await; + assert_invalid_model_error(&payload); +} + +#[tokio::test] +async fn responses_model_rejects_missing_non_string_unknown_and_profile_mismatch_cases() { + let cases = [ + (ThreadlineConfig::default(), json!({})), + ( + ThreadlineConfig::default(), + invalid_model_payload(json!({ "id": "gpt-5.4" })), + ), + ( + ThreadlineConfig::default(), + invalid_model_payload(json!("codex-mini-latest")), + ), + ( + ThreadlineConfig::default(), + invalid_model_payload(json!("threadline-utility-gpt-5.4-mini")), + ), + ( + utility_config(), + invalid_model_payload(json!("threadline-main-gpt-5.4")), + ), + ]; + + for (config, payload) in cases { + let app = build_router_with_services( + config, + ThreadlineServices::new(Arc::new(MissingAuthProvider), Arc::new(UnusedConnector)), + ); + + let response = post_responses_json(app, payload).await; + + assert_eq!(response.status(), StatusCode::BAD_REQUEST); + + let body = read_json_body(response).await; + assert_invalid_model_error(&body); + } +} + +#[tokio::test] +async fn responses_model_accepts_main_compatibility_ids_on_main() { + for model_id in HIDDEN_MAIN_COMPATIBILITY_MODEL_IDS { + let app = build_router_with_services( + ThreadlineConfig::default(), + ThreadlineServices::new(Arc::new(MissingAuthProvider), Arc::new(UnusedConnector)), + ); + + let response = post_responses_json(app, json!({ "model": model_id })).await; + + assert_eq!(response.status(), StatusCode::INTERNAL_SERVER_ERROR); + + let payload = read_json_body(response).await; + assert_eq!(payload["error"]["code"], "upstream_credentials_unavailable"); + assert_eq!(payload["error"]["type"], "configuration_error"); + } +} + +#[tokio::test] +async fn responses_utility_profile_rejects_new_main_visible_aliases_before_auth_loading() { + for model_id in NEW_MAIN_VISIBLE_MODEL_IDS { + let app = build_router_with_services( + utility_config(), + ThreadlineServices::new(Arc::new(MissingAuthProvider), Arc::new(UnusedConnector)), + ); + + let response = post_responses_json(app, json!({ "model": model_id })).await; + + assert_eq!( + response.status(), + StatusCode::BAD_REQUEST, + "model_id={model_id}" + ); + + let payload = read_json_body(response).await; + assert_invalid_model_error(&payload); + } +} + +#[tokio::test] +async fn responses_utility_profile_rejects_new_main_compatibility_ids_before_auth_loading() { + for model_id in NEW_MAIN_RAW_COMPATIBILITY_MODEL_IDS { + let app = build_router_with_services( + utility_config(), + ThreadlineServices::new(Arc::new(MissingAuthProvider), Arc::new(UnusedConnector)), + ); + + let response = post_responses_json(app, json!({ "model": model_id })).await; + + assert_eq!( + response.status(), + StatusCode::BAD_REQUEST, + "model_id={model_id}" + ); + + let payload = read_json_body(response).await; + assert_invalid_model_error(&payload); + } +} + +#[tokio::test] +async fn responses_endpoint_rejects_each_unsupported_model() { + for model_id in UNSUPPORTED_MODEL_IDS { + let app = build_router_with_services( + ThreadlineConfig::default(), + ThreadlineServices::new(Arc::new(MissingAuthProvider), Arc::new(UnusedConnector)), + ); + + let response = post_responses_json(app, invalid_model_payload(json!(model_id))).await; + + assert_eq!( + response.status(), + StatusCode::BAD_REQUEST, + "model_id={model_id}" + ); + + let payload = read_json_body(response).await; + assert_invalid_model_error(&payload); + } +} + +#[tokio::test] +async fn responses_endpoint_rejects_unsupported_model_before_lease_acquisition() { + for model_id in UNSUPPORTED_MODEL_IDS { + let app = build_router(ThreadlineConfig { + retained_session_capacity: 0, + ..ThreadlineConfig::default() + }); + + let response = post_responses_json( + app, + json!({ + "model": model_id, + "previous_response_id": "response-missing" + }), + ) + .await; + + assert_eq!( + response.status(), + StatusCode::BAD_REQUEST, + "model_id={model_id}" + ); + + let payload = read_json_body(response).await; + assert_invalid_model_error(&payload); + } +} + +#[tokio::test] +async fn responses_endpoint_rejects_unsupported_model_before_auth_loading_and_upstream_connection() +{ + for model_id in UNSUPPORTED_MODEL_IDS { + let app = build_router_with_services( + ThreadlineConfig::default(), + ThreadlineServices::new(Arc::new(MissingAuthProvider), Arc::new(UnusedConnector)), + ); + + let response = post_responses_json(app, invalid_model_payload(json!(model_id))).await; + + assert_eq!( + response.status(), + StatusCode::BAD_REQUEST, + "model_id={model_id}" + ); + + let payload = read_json_body(response).await; + assert_invalid_model_error(&payload); + } +} + +#[tokio::test] +async fn responses_endpoint_rejects_reasoning_all_turns_for_unsupported_model_before_auth_or_upstream() + { + let app = build_router_with_services( + utility_config(), + ThreadlineServices::new(Arc::new(MissingAuthProvider), Arc::new(UnusedConnector)), + ); + + let response = post_responses_json( + app, + json!({ + "model": "threadline-utility-gpt-5.3-codex-spark", + "input": "utility-all-turns", + "reasoning": { + "context": "all_turns" + } + }), + ) + .await; + + assert_eq!(response.status(), StatusCode::BAD_REQUEST); + + let payload = read_json_body(response).await; + assert_unsupported_reasoning_context_error(&payload); +} + +#[tokio::test] +async fn responses_endpoint_allows_non_persistent_request_for_reasoning_all_turns_unsupported_model_to_reach_existing_auth_path() + { + let app = build_router_with_services( + utility_config(), + ThreadlineServices::new(Arc::new(MissingAuthProvider), Arc::new(UnusedConnector)), + ); + + let response = post_responses_json( + app, + json!({ + "model": "threadline-utility-gpt-5.3-codex-spark", + "input": "utility-non-persistent", + "reasoning": { + "effort": "high" + } + }), + ) + .await; + + assert_eq!(response.status(), StatusCode::INTERNAL_SERVER_ERROR); + + let payload = read_json_body(response).await; + assert_eq!(payload["error"]["code"], "upstream_credentials_unavailable"); + assert_eq!(payload["error"]["type"], "configuration_error"); + assert_ne!(payload["error"]["code"], "unsupported_reasoning_context"); +} + +#[tokio::test] +async fn responses_endpoint_rejects_unsupported_reasoning_all_turns_before_retained_session_lease() +{ + let app = build_router(ThreadlineConfig { + retained_session_capacity: 0, + ..ThreadlineConfig::default() + }); + + let response = post_responses_json( + app, + json!({ + "model": "gpt-5.4", + "input": "main-all-turns", + "previous_response_id": "response-lease", + "reasoning": { + "context": "all_turns" + } + }), + ) + .await; + + assert_eq!(response.status(), StatusCode::BAD_REQUEST); + + let payload = read_json_body(response).await; + assert_unsupported_reasoning_context_error(&payload); +} + +#[tokio::test] +async fn responses_endpoint_rejects_new_raw_main_compatibility_ids_for_reasoning_all_turns_before_auth_or_upstream() + { + for model_id in NEW_MAIN_RAW_COMPATIBILITY_MODEL_IDS { + let app = build_router_with_services( + ThreadlineConfig::default(), + ThreadlineServices::new(Arc::new(MissingAuthProvider), Arc::new(UnusedConnector)), + ); + + let response = post_responses_json( + app, + json!({ + "model": model_id, + "input": "main-all-turns-next-model", + "reasoning": { + "context": "all_turns" + } + }), + ) + .await; + + assert_eq!( + response.status(), + StatusCode::BAD_REQUEST, + "model_id={model_id}" + ); + + let payload = read_json_body(response).await; + assert_unsupported_reasoning_context_error(&payload); + } +} + +#[tokio::test] +async fn responses_endpoint_accepts_each_supported_model_before_missing_auth_error() { + for model_id in ACCEPTED_MAIN_MODEL_IDS { + let app = build_router_with_services( + ThreadlineConfig::default(), + ThreadlineServices::new(Arc::new(MissingAuthProvider), Arc::new(UnusedConnector)), + ); + + let response = post_responses_json(app, json!({ "model": model_id })).await; + + assert_eq!(response.status(), StatusCode::INTERNAL_SERVER_ERROR); + + let payload = read_json_body(response).await; + assert_eq!(payload["error"]["code"], "upstream_credentials_unavailable"); + assert_eq!(payload["error"]["type"], "configuration_error"); + } +} + +#[tokio::test] +async fn responses_endpoint_reports_configuration_error_for_allowed_model_when_upstream_credentials_are_unavailable() + { + let app = build_router_with_services( + ThreadlineConfig::default(), + ThreadlineServices::new(Arc::new(MissingAuthProvider), Arc::new(UnusedConnector)), + ); + + let response = post_responses_json(app, json!({ "model": "gpt-5.4" })).await; + + assert_eq!(response.status(), StatusCode::INTERNAL_SERVER_ERROR); + + let payload = read_json_body(response).await; + + assert_eq!(payload["error"]["code"], "upstream_credentials_unavailable"); + assert_eq!(payload["error"]["type"], "configuration_error"); + assert_eq!( + payload["error"]["message"], + "Threadline could not load upstream credentials." + ); +} diff --git a/tests/internal_tools.rs b/tests/internal_tools.rs new file mode 100644 index 0000000..32e15a6 --- /dev/null +++ b/tests/internal_tools.rs @@ -0,0 +1,3029 @@ +use axum::body::{Body, Bytes, to_bytes}; +use axum::http::{Request, Response, StatusCode}; +use futures_util::{StreamExt, future::BoxFuture}; +use serde_json::{Value, json}; +use std::collections::VecDeque; +use std::io; +use std::io::Write; +use std::sync::Arc; +use std::sync::Mutex as StdMutex; +use std::sync::OnceLock; +use std::time::Instant; +use tokio::sync::Mutex; +use tokio::time::{Duration, sleep, timeout}; +use tokio_tungstenite::connect_async; +use tokio_tungstenite::tungstenite::Message; +use tower::ServiceExt; +use tracing_subscriber::fmt::MakeWriter; +use uuid::Uuid; + +#[path = "support/scripted_ws.rs"] +mod scripted_ws; + +use scripted_ws::ScriptedWebSocketServer; +use threadline::auth::{AuthSource, LoadedUpstreamAuth, RefreshBoundary}; +use threadline::codex_ws::UpstreamSessionDescriptor; +use threadline::config::ThreadlineConfig; +use threadline::errors::ThreadlineError; +use threadline::http::build_router_with_services; +use threadline::jobs::{ThreadlineJobManager, ThreadlineJobManagerConfig}; +use threadline::responses::{ + ConnectedUpstream, ThreadlineServices, UpstreamAuthProvider, UpstreamConnector, +}; +use threadline::tools::{ + InternalToolCall, event_contains_internal_tool_name, inject_internal_tools, +}; +use threadline::ws_pump::LiveUpstreamWebSocket; + +const JOB_START_NEXT_ACTION_HINT: &str = "This job is running in the background. Continue other useful work if available, then poll status or read output later when needed."; + +#[derive(Clone)] +struct StaticAuthProvider; + +impl UpstreamAuthProvider for StaticAuthProvider { + fn load(&self) -> Result { + Ok(LoadedUpstreamAuth { + bearer_token: "test-token".to_string(), + source: AuthSource::CodexKeyring, + refresh_boundary: RefreshBoundary::NotAvailable, + }) + } +} + +struct PlannedConnection { + server: Arc, + turn_state: Option, +} + +#[derive(Clone)] +struct RecordingConnector { + plans: Arc>>, +} + +impl RecordingConnector { + fn new(plans: Vec) -> Self { + Self { + plans: Arc::new(Mutex::new(plans.into())), + } + } +} + +impl UpstreamConnector for RecordingConnector { + fn connect( + &self, + _auth: LoadedUpstreamAuth, + session: Option, + ) -> BoxFuture<'static, Result> { + let plans = Arc::clone(&self.plans); + Box::pin(async move { + let session = session.unwrap_or_else(new_session_descriptor); + let plan = plans + .lock() + .await + .pop_front() + .expect("planned websocket connection"); + + let (stream, _) = connect_async(plan.server.url()) + .await + .map_err(|_| ThreadlineError::UpstreamWebSocketConnectFailed)?; + + Ok(ConnectedUpstream { + websocket: Arc::new(LiveUpstreamWebSocket::from_stream(stream)), + session, + turn_state: plan.turn_state, + }) + }) + } +} + +fn build_test_router(connector: Arc) -> axum::Router { + build_router_with_services( + ThreadlineConfig::default(), + ThreadlineServices::new(Arc::new(StaticAuthProvider), connector), + ) +} + +async fn post_responses(app: axum::Router, payload: Value) -> Response { + app.oneshot( + Request::builder() + .method("POST") + .uri("/v1/responses") + .header("content-type", "application/json") + .body(Body::from(payload.to_string())) + .expect("request"), + ) + .await + .expect("response") +} + +fn message_text(message: Message) -> String { + match message { + Message::Text(text) => text.to_string(), + other => panic!("expected text message, got {other:?}"), + } +} + +fn new_session_descriptor() -> UpstreamSessionDescriptor { + UpstreamSessionDescriptor { + session_id: Uuid::now_v7().to_string(), + thread_id: Uuid::now_v7().to_string(), + window_id: Uuid::now_v7().to_string(), + turn_state: None, + } +} + +fn auxiliary_summary_text() -> &'static str { + concat!( + "The conversation has grown too large for the context window and must be compacted now", + "\n\n", + "Your ONLY task right now is to produce a comprehensive summary", + "\n", + "Output your summary wrapped in and tags" + ) +} + +fn auxiliary_summary_input_item() -> Value { + json!({ + "type": "message", + "role": "system", + "content": [ + { + "type": "input_text", + "text": auxiliary_summary_text() + } + ] + }) +} + +fn downstream_function_tool(name: &str) -> Value { + json!({ + "type": "function", + "name": name, + "description": format!("{name} description"), + "parameters": { + "type": "object", + "properties": {}, + "additionalProperties": false + } + }) +} + +fn auxiliary_summary_request_with_tools(tools: Vec) -> Value { + json!({ + "model": "gpt-5.4", + "input": [ + { + "type": "message", + "role": "user", + "content": [ + { + "type": "input_text", + "text": "Continue from the earlier answer." + } + ] + }, + auxiliary_summary_input_item() + ], + "tools": tools + }) +} + +fn new_auto_system_summary_text() -> &'static str { + "Your task is to create a comprehensive, detailed summary of the entire conversation that captures all essential information needed to seamlessly continue the work without any loss of context" +} + +fn new_auto_compressed_history_text() -> &'static str { + concat!( + "The following is a compressed version of the preceeding history in the current conversation. ", + "The first message is kept, some history may be truncated after that:" + ) +} + +fn new_auto_final_summary_prompt_text() -> &'static str { + concat!( + "Summarize the conversation history so far, paying special attention to the most recent agent commands and tool results that triggered this summarization. ", + "Structure your summary using the enhanced format provided in the system message.\n", + "Focus particularly on:\n", + "- The specific agent commands/tools that were just executed\n", + "- The results returned from these recent tool calls (truncate if very long but preserve key information)\n", + "- What the agent was actively working on when the token budget was exceeded\n", + "- How these recent operations connect to the overall user goals\n", + "Include all important tool calls and their results as part of the appropriate sections, with special emphasis on the most recent operations." + ) +} + +fn new_auto_summary_request_with_tools(tools: Vec) -> Value { + json!({ + "model": "gpt-5.4", + "input": [ + { + "type": "message", + "role": "user", + "content": [ + { + "type": "input_text", + "text": "Continue from the earlier answer." + } + ] + }, + { + "type": "message", + "role": "system", + "content": [ + { + "type": "input_text", + "text": new_auto_system_summary_text() + } + ] + }, + { + "type": "message", + "role": "user", + "content": [ + { + "type": "input_text", + "text": new_auto_compressed_history_text() + } + ] + }, + { + "type": "message", + "role": "user", + "content": [ + { + "type": "input_text", + "text": new_auto_final_summary_prompt_text() + } + ] + } + ], + "tools": tools + }) +} + +fn shell_program() -> String { + if cfg!(windows) { + "pwsh".to_string() + } else { + "sh".to_string() + } +} + +fn shell_command(script: &str) -> Vec { + if cfg!(windows) { + vec![ + "pwsh".to_string(), + "-NoProfile".to_string(), + "-Command".to_string(), + script.to_string(), + ] + } else { + vec!["sh".to_string(), "-lc".to_string(), script.to_string()] + } +} + +fn shell_job_manager() -> ThreadlineJobManager { + ThreadlineJobManager::new(ThreadlineJobManagerConfig { + jobs_enabled: true, + output_buffer_limit_bytes: 1024, + retention_ttl: Duration::from_secs(60), + allowed_commands: vec![shell_program()], + }) +} + +async fn wait_for_terminal_result( + manager: &ThreadlineJobManager, + job_id: &str, + timeout: Duration, +) -> Value { + let deadline = Instant::now() + timeout; + loop { + let result = manager.get_result_json(job_id); + if result["status"] == "completed" + || result["status"] == "failed" + || result["status"] == "cancelled" + { + return result; + } + assert!( + Instant::now() < deadline, + "timed out waiting for terminal job result" + ); + sleep(Duration::from_millis(10)).await; + } +} + +fn split_sse_frames(body: &str) -> Vec<&str> { + body.split("\n\n") + .filter(|frame| !frame.trim().is_empty()) + .collect() +} + +fn sse_event_and_data(frame: &str) -> (&str, &str) { + let mut event = None; + let mut data = None; + let mut unexpected_lines = Vec::new(); + + for (index, line) in frame.lines().enumerate() { + if let Some(value) = line.strip_prefix("event: ") { + assert!( + event.replace(value).is_none(), + "expected exactly one event line in SSE frame, found duplicate at line {}: {frame}", + index + 1 + ); + continue; + } + + if let Some(value) = line.strip_prefix("data: ") { + assert!( + data.replace(value).is_none(), + "expected compact single-line SSE data payload, found duplicate data line at line {}: {frame}", + index + 1 + ); + continue; + } + + unexpected_lines.push(format!("line {}: {line}", index + 1)); + } + + assert!( + unexpected_lines.is_empty(), + "expected exactly one event line and one compact data line in SSE frame; unexpected lines: {}. Frame: {frame}", + unexpected_lines.join(" | ") + ); + + ( + event.unwrap_or_else(|| panic!("missing event line in SSE frame: {frame}")), + data.unwrap_or_else(|| panic!("missing data line in SSE frame: {frame}")), + ) +} + +fn assert_done_frame(frame: &str) { + assert_eq!( + frame, "data: [DONE]", + "expected a bare downstream DONE frame without an event line" + ); +} + +type SharedBytes = Arc>>; +type ActiveTraceBytes = StdMutex>; + +#[derive(Clone)] +struct SharedLogBuffer { + bytes: SharedBytes, +} + +impl SharedLogBuffer { + fn new() -> Self { + Self { + bytes: Arc::new(StdMutex::new(Vec::new())), + } + } + + fn logs(&self) -> String { + String::from_utf8(self.bytes.lock().expect("log buffer lock").clone()) + .expect("utf8 trace logs") + } +} + +struct SharedLogWriter { + bytes: SharedBytes, +} + +static TRACE_CAPTURE_LOCK: OnceLock> = OnceLock::new(); +static ACTIVE_TRACE_BUFFER: OnceLock = OnceLock::new(); +static TRACE_SUBSCRIBER_INIT: OnceLock<()> = OnceLock::new(); + +fn trace_capture_lock() -> &'static Mutex<()> { + TRACE_CAPTURE_LOCK.get_or_init(|| Mutex::new(())) +} + +fn active_trace_buffer() -> &'static ActiveTraceBytes { + ACTIVE_TRACE_BUFFER.get_or_init(|| StdMutex::new(None)) +} + +fn ensure_test_trace_subscriber() { + TRACE_SUBSCRIBER_INIT.get_or_init(|| { + let subscriber = tracing_subscriber::fmt() + .with_max_level(tracing::Level::TRACE) + .without_time() + .with_ansi(false) + .with_writer(GlobalTraceCapture) + .finish(); + tracing::subscriber::set_global_default(subscriber) + .expect("global trace subscriber should only initialize once"); + }); +} + +#[derive(Clone, Copy)] +struct GlobalTraceCapture; + +struct GlobalTraceWriter; + +impl Write for GlobalTraceWriter { + fn write(&mut self, buf: &[u8]) -> io::Result { + if let Some(bytes) = active_trace_buffer() + .lock() + .expect("active trace buffer lock") + .as_ref() + { + bytes + .lock() + .expect("log buffer lock") + .extend_from_slice(buf); + } + Ok(buf.len()) + } + + fn flush(&mut self) -> io::Result<()> { + Ok(()) + } +} + +impl<'a> MakeWriter<'a> for GlobalTraceCapture { + type Writer = GlobalTraceWriter; + + fn make_writer(&'a self) -> Self::Writer { + GlobalTraceWriter + } +} + +struct TraceCaptureGuard { + _lock: tokio::sync::MutexGuard<'static, ()>, + log_buffer: SharedLogBuffer, +} + +impl TraceCaptureGuard { + async fn begin() -> Self { + let lock = trace_capture_lock().lock().await; + ensure_test_trace_subscriber(); + let log_buffer = SharedLogBuffer::new(); + *active_trace_buffer() + .lock() + .expect("active trace buffer lock") = Some(Arc::clone(&log_buffer.bytes)); + Self { + _lock: lock, + log_buffer, + } + } + + fn logs(&self) -> String { + self.log_buffer.logs() + } +} + +impl Drop for TraceCaptureGuard { + fn drop(&mut self) { + *active_trace_buffer() + .lock() + .expect("active trace buffer lock") = None; + } +} + +impl Write for SharedLogWriter { + fn write(&mut self, buf: &[u8]) -> io::Result { + self.bytes + .lock() + .expect("log buffer lock") + .extend_from_slice(buf); + Ok(buf.len()) + } + + fn flush(&mut self) -> io::Result<()> { + Ok(()) + } +} + +impl<'a> MakeWriter<'a> for SharedLogBuffer { + type Writer = SharedLogWriter; + + fn make_writer(&'a self) -> Self::Writer { + SharedLogWriter { + bytes: Arc::clone(&self.bytes), + } + } +} + +async fn next_sse_frame( + body_stream: &mut (impl futures_util::Stream> + Unpin), + pending: &mut String, +) -> String { + loop { + if let Some(frame_end) = pending.find("\n\n") { + let frame = pending[..frame_end].to_string(); + pending.drain(..frame_end + 2); + if !frame.trim().is_empty() { + return frame; + } + continue; + } + + let chunk = match body_stream.next().await { + Some(Ok(chunk)) => chunk, + Some(Err(error)) => panic!("expected SSE chunk, got body error: {error}"), + None => panic!("expected another SSE frame before EOF"), + }; + pending.push_str(std::str::from_utf8(&chunk).expect("utf8 sse chunk")); + } +} + +#[tokio::test] +async fn internal_tool_outputs_are_sent_after_intermediate_response_completes() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(Arc::new(connector)); + + let response = post_responses( + app, + json!({ + "model": "gpt-5.4", + "input": "run internal tool loop", + "max_output_tokens": 512, + "max_tokens": 256, + "max_completion_tokens": 128, + "truncation": "auto", + "tools": [ + { + "type": "function", + "name": "downstream_tool", + "description": "preserve me", + "parameters": {"type": "object"}, + "strict": true + } + ] + }), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let body_task = tokio::spawn(async move { + to_bytes(response.into_body(), usize::MAX) + .await + .expect("body bytes") + }); + + let first_request: Value = serde_json::from_str(&message_text( + server.recv_client_message().await.expect("initial request"), + )) + .expect("initial request json"); + assert_eq!(first_request["type"], "response.create"); + assert_eq!(first_request["instructions"], ""); + assert!(first_request.get("response").is_none()); + assert!(first_request.get("max_output_tokens").is_none()); + assert!(first_request.get("max_tokens").is_none()); + assert!(first_request.get("max_completion_tokens").is_none()); + assert!(first_request.get("truncation").is_none()); + + let tools = first_request["tools"].as_array().expect("tools array"); + assert_eq!(tools[0]["name"], "downstream_tool"); + assert_eq!(tools[0]["strict"], true); + assert!( + tools + .iter() + .any(|tool| { tool["name"] == "threadline_echo" && tool["type"] == "function" }) + ); + + server + .send_text( + r#"{"type":"response.output_item.done","item":{"type":"function_call","call_id":"call-1","name":"threadline_echo","arguments":"{\"value\":\"alpha\"}"}}"#, + ) + .await; + server + .send_text( + r#"{"type":"response.output_item.done","item":{"type":"function_call","call_id":"call-2","name":"threadline_echo","arguments":"{\"value\":\"beta\"}"}}"#, + ) + .await; + + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-intermediate"}}"#) + .await; + + let followup_request: Value = serde_json::from_str(&message_text( + server + .recv_client_message() + .await + .expect("followup request"), + )) + .expect("followup request json"); + assert_eq!(followup_request["type"], "response.create"); + assert_eq!(followup_request["store"], false); + assert_eq!(followup_request["instructions"], ""); + assert!(followup_request.get("response").is_none()); + assert!(followup_request.get("max_output_tokens").is_none()); + assert!(followup_request.get("max_tokens").is_none()); + assert!(followup_request.get("max_completion_tokens").is_none()); + assert!(followup_request.get("truncation").is_none()); + assert_eq!( + followup_request["previous_response_id"], + "response-intermediate" + ); + + let followup_input = followup_request["input"] + .as_array() + .expect("followup input array"); + assert_eq!(followup_input.len(), 2); + assert_eq!(followup_input[0]["type"], "function_call_output"); + assert_eq!(followup_input[0]["call_id"], "call-1"); + assert_eq!(followup_input[0]["output"], "alpha"); + assert_eq!(followup_input[1]["call_id"], "call-2"); + assert_eq!(followup_input[1]["output"], "beta"); + + server + .send_text(r#"{"type":"response.output_text.delta","delta":"final answer"}"#) + .await; + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-final"}}"#) + .await; + + let body = body_task.await.expect("body task"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + let (delta_event, delta_data) = sse_event_and_data(frames[0]); + let delta_payload: Value = serde_json::from_str(delta_data).expect("delta json"); + let (completed_event, completed_data) = sse_event_and_data(frames[1]); + let completed_payload: Value = serde_json::from_str(completed_data).expect("completed json"); + + assert_eq!(frames.len(), 3); + assert_eq!(delta_event, "response.output_text.delta"); + assert_eq!( + delta_payload, + json!({"type":"response.output_text.delta","delta":"final answer"}) + ); + assert_eq!(completed_event, "response.completed"); + assert_eq!(completed_payload["response"]["id"], "response-final"); + assert_eq!( + completed_payload["response"]["output"][0]["content"][0]["text"], + "final answer" + ); + assert_done_frame(frames[2]); + assert!(!body_text.contains("threadline_echo")); + assert!(!body_text.contains("response-intermediate")); + assert!(!body_text.contains("event: response.output_item.done")); + assert!(server.take_pending_client_messages().await.is_empty()); +} + +#[tokio::test] +async fn internal_tool_intermediate_text_does_not_leak_and_followup_fallback_still_runs() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(Arc::new(connector)); + + let response = post_responses( + app, + json!({ + "model": "gpt-5.4", + "input": "hide intermediate assistant text during internal tool follow-up" + }), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let body_task = tokio::spawn(async move { + to_bytes(response.into_body(), usize::MAX) + .await + .expect("body bytes") + }); + + let _ = server.recv_client_message().await.expect("initial request"); + + server + .send_text( + r#"{"type":"response.output_item.done","item":{"type":"function_call","call_id":"call-1","name":"threadline_echo","arguments":"{\"value\":\"alpha\"}"}}"#, + ) + .await; + server + .send_text( + r#"{"type":"response.output_text.done","item_id":"msg-intermediate","output_index":0,"content_index":0,"text":"hidden intermediate assistant text"}"#, + ) + .await; + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-intermediate"}}"#) + .await; + + let followup_request: Value = serde_json::from_str(&message_text( + server + .recv_client_message() + .await + .expect("followup request"), + )) + .expect("followup request json"); + assert_eq!(followup_request["type"], "response.create"); + assert_eq!( + followup_request["input"] + .as_array() + .expect("followup input array")[0]["output"], + "alpha" + ); + + let final_completed = json!({ + "type": "response.completed", + "response": { + "id": "response-final", + "output": [ + { + "type": "function_call", + "name": "threadline_echo", + "call_id": "call-hidden-final" + }, + { + "id": "msg-final", + "type": "message", + "role": "assistant", + "content": [ + { + "type": "output_text", + "text": "final follow-up answer" + } + ] + } + ] + } + }); + server.send_text(&final_completed.to_string()).await; + + let body = body_task.await.expect("body task"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + + assert_eq!(frames.len(), 3); + + let synthetic_delta_frame = sse_event_and_data(frames[0]); + assert_eq!(synthetic_delta_frame.0, "response.output_text.delta"); + assert_eq!( + serde_json::from_str::(synthetic_delta_frame.1).expect("synthetic delta json"), + json!({ + "type": "response.output_text.delta", + "delta": "final follow-up answer", + "item_id": "msg-final", + "output_index": 1, + "content_index": 0 + }) + ); + + let completed_frame = sse_event_and_data(frames[1]); + assert_eq!(completed_frame.0, "response.completed"); + assert_eq!( + serde_json::from_str::(completed_frame.1).expect("completed json"), + json!({ + "type": "response.completed", + "response": { + "id": "response-final", + "output": [ + { + "id": "msg-final", + "type": "message", + "role": "assistant", + "content": [ + { + "type": "output_text", + "text": "final follow-up answer" + } + ] + } + ] + } + }) + ); + + assert_done_frame(frames[2]); + assert!(!body_text.contains("hidden intermediate assistant text")); + assert!(!body_text.contains("response.output_text.done")); + assert!(!body_text.contains("threadline_echo")); + assert!(!body_text.contains("response-intermediate")); +} + +#[tokio::test] +async fn internal_tool_intermediate_output_item_done_text_does_not_leak() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(Arc::new(connector)); + + let response = post_responses( + app, + json!({ + "model": "gpt-5.4", + "input": "hide intermediate output_item.done assistant text during internal tool follow-up" + }), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let body_task = tokio::spawn(async move { + to_bytes(response.into_body(), usize::MAX) + .await + .expect("body bytes") + }); + + let _ = server.recv_client_message().await.expect("initial request"); + + server + .send_text( + r#"{"type":"response.output_item.done","item":{"type":"function_call","call_id":"call-1","name":"threadline_echo","arguments":"{\"value\":\"alpha\"}"}}"#, + ) + .await; + server + .send_text( + r#"{"type":"response.output_item.done","output_index":0,"item":{"id":"msg-intermediate","type":"message","role":"assistant","content":[{"type":"output_text","text":"hidden intermediate assistant text"}]}}"#, + ) + .await; + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-intermediate"}}"#) + .await; + + let followup_request: Value = serde_json::from_str(&message_text( + server + .recv_client_message() + .await + .expect("followup request"), + )) + .expect("followup request json"); + assert_eq!(followup_request["type"], "response.create"); + assert_eq!( + followup_request["input"] + .as_array() + .expect("followup input array")[0]["output"], + "alpha" + ); + + let final_done_event = json!({ + "type": "response.output_item.done", + "output_index": 0, + "item": { + "id": "msg-final", + "type": "message", + "role": "assistant", + "content": [ + { + "type": "output_text", + "text": "final follow-up answer from output_item.done" + } + ] + } + }); + server.send_text(&final_done_event.to_string()).await; + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-final"}}"#) + .await; + + let body = body_task.await.expect("body task"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + + assert_eq!(frames.len(), 4); + + let delta_frame = sse_event_and_data(frames[0]); + assert_eq!(delta_frame.0, "response.output_text.delta"); + assert_eq!( + serde_json::from_str::(delta_frame.1).expect("delta json"), + json!({ + "type": "response.output_text.delta", + "delta": "final follow-up answer from output_item.done", + "item_id": "msg-final", + "output_index": 0, + "content_index": 0 + }) + ); + + let done_frame = sse_event_and_data(frames[1]); + assert_eq!(done_frame.0, "response.output_item.done"); + assert_eq!(done_frame.1, final_done_event.to_string()); + + let completed_frame = sse_event_and_data(frames[2]); + let completed_payload: Value = serde_json::from_str(completed_frame.1).expect("completed json"); + assert_eq!(completed_frame.0, "response.completed"); + assert_eq!(completed_payload["response"]["id"], "response-final"); + assert_eq!( + completed_payload["response"]["output"][0]["content"][0]["text"], + "final follow-up answer from output_item.done" + ); + + assert_done_frame(frames[3]); + assert!(!body_text.contains("hidden intermediate assistant text")); + assert!(!body_text.contains("msg-intermediate")); + assert!(!body_text.contains("threadline_echo")); + assert!(!body_text.contains("response-intermediate")); +} + +#[tokio::test] +async fn internal_tool_pre_done_events_are_hidden_from_downstream() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(Arc::new(connector)); + + let response = post_responses( + app, + json!({ + "model": "gpt-5.4", + "input": "run internal tool loop", + }), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let body_task = tokio::spawn(async move { + to_bytes(response.into_body(), usize::MAX) + .await + .expect("body bytes") + }); + + let _ = server.recv_client_message().await.expect("initial request"); + + server + .send_text( + r#"{"type":"response.output_item.added","item":{"type":"function_call","call_id":"call-1","name":"threadline_echo","arguments":"{\"value\":\"alpha\"}"}}"#, + ) + .await; + server + .send_text( + r#"{"type":"response.output_item.done","item":{"type":"function_call","call_id":"call-1","name":"threadline_echo","arguments":"{\"value\":\"alpha\"}"}}"#, + ) + .await; + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-intermediate"}}"#) + .await; + + let followup_request: Value = serde_json::from_str(&message_text( + server + .recv_client_message() + .await + .expect("followup request"), + )) + .expect("followup request json"); + assert_eq!(followup_request["type"], "response.create"); + assert!(followup_request.get("response").is_none()); + + server + .send_text(r#"{"type":"response.output_text.delta","delta":"final answer"}"#) + .await; + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-final"}}"#) + .await; + + let body = body_task.await.expect("body task"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + let (delta_event, delta_data) = sse_event_and_data(frames[0]); + let (completed_event, completed_data) = sse_event_and_data(frames[1]); + + assert_eq!(frames.len(), 3); + assert_eq!(delta_event, "response.output_text.delta"); + assert_eq!( + serde_json::from_str::(delta_data).expect("delta json"), + json!({"type":"response.output_text.delta","delta":"final answer"}) + ); + assert_eq!(completed_event, "response.completed"); + let completed_payload: Value = serde_json::from_str(completed_data).expect("completed json"); + assert_eq!(completed_payload["response"]["id"], "response-final"); + assert_eq!( + completed_payload["response"]["output"][0]["content"][0]["text"], + "final answer" + ); + assert_done_frame(frames[2]); + assert!(!body_text.contains("event: response.output_item.added")); + assert!(!body_text.contains("threadline_echo")); + assert!(!body_text.contains("response-intermediate")); +} + +#[tokio::test] +async fn summary_request_does_not_inject_threadline_internal_tools() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(Arc::new(connector)); + + let response = post_responses( + app, + auxiliary_summary_request_with_tools(vec![downstream_function_tool("downstream_tool")]), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let body_task = tokio::spawn(async move { + to_bytes(response.into_body(), usize::MAX) + .await + .expect("body bytes") + }); + + let first_request: Value = serde_json::from_str(&message_text( + server.recv_client_message().await.expect("initial request"), + )) + .expect("initial request json"); + let tools = first_request["tools"].as_array().expect("tools array"); + + assert!(tools.iter().any(|tool| tool["name"] == "downstream_tool")); + assert!( + !tools.iter().any(|tool| { + tool["name"] + .as_str() + .is_some_and(|name| name.starts_with("threadline_")) + }), + "expected classified summary request to skip internal tool injection: {tools:?}" + ); + + server + .send_text(r#"{"type":"response.output_text.delta","delta":"summary answer"}"#) + .await; + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-summary"}}"#) + .await; + + let body = body_task.await.expect("body task"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + assert!(!body_text.contains("threadline_")); +} + +#[tokio::test] +async fn summary_request_strips_downstream_threadline_prefixed_tools() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(Arc::new(connector)); + + let response = post_responses( + app, + auxiliary_summary_request_with_tools(vec![ + downstream_function_tool("downstream_tool"), + downstream_function_tool("threadline_echo"), + downstream_function_tool("external_web_search"), + ]), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let body_task = tokio::spawn(async move { + to_bytes(response.into_body(), usize::MAX) + .await + .expect("body bytes") + }); + + let first_request: Value = serde_json::from_str(&message_text( + server.recv_client_message().await.expect("initial request"), + )) + .expect("initial request json"); + let tools = first_request["tools"].as_array().expect("tools array"); + let tool_names: Vec<&str> = tools + .iter() + .map(|tool| tool["name"].as_str().expect("tool name")) + .collect(); + + assert_eq!( + tool_names.len(), + 2, + "expected only non-Threadline tools upstream" + ); + assert!(tool_names.contains(&"downstream_tool")); + assert!(tool_names.contains(&"external_web_search")); + assert!( + !tool_names + .iter() + .any(|name| name.starts_with("threadline_")), + "expected classified summary request to strip downstream threadline_* tools: {tool_names:?}" + ); + + server + .send_text(r#"{"type":"response.output_text.delta","delta":"summary answer"}"#) + .await; + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-summary"}}"#) + .await; + + let body = body_task.await.expect("body task"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + assert!(!body_text.contains("threadline_")); +} + +#[tokio::test] +async fn summary_request_does_not_execute_threadline_tool_call_events() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(Arc::new(connector)); + + let response = post_responses( + app, + auxiliary_summary_request_with_tools(vec![downstream_function_tool("downstream_tool")]), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let body_task = tokio::spawn(async move { + to_bytes(response.into_body(), usize::MAX) + .await + .expect("body bytes") + }); + + let first_request: Value = serde_json::from_str(&message_text( + server.recv_client_message().await.expect("initial request"), + )) + .expect("initial request json"); + let tools = first_request["tools"].as_array().expect("tools array"); + + assert!( + !tools.iter().any(|tool| { + tool["name"] + .as_str() + .is_some_and(|name| name.starts_with("threadline_")) + }), + "expected classified summary request to exclude internal tools before streaming" + ); + + server + .send_text( + r#"{"type":"response.output_item.done","item":{"type":"function_call","call_id":"call-1","name":"threadline_echo","arguments":"{\"value\":\"alpha\"}"}}"#, + ) + .await; + server + .send_text( + r#"{"type":"response.completed","response":{"id":"response-summary-intermediate"}}"#, + ) + .await; + + let maybe_followup = + tokio::time::timeout(Duration::from_millis(100), server.recv_client_message()).await; + let saw_followup_request = matches!(maybe_followup, Ok(Some(_))); + + if saw_followup_request { + server + .send_text(r#"{"type":"response.output_text.delta","delta":"summary answer"}"#) + .await; + server + .send_text( + r#"{"type":"response.completed","response":{"id":"response-summary-final"}}"#, + ) + .await; + } + + let body = body_task.await.expect("body task"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + + assert!( + !saw_followup_request, + "expected no local internal-tool followup request during summary stream" + ); + assert!(!body_text.contains("threadline_echo")); + assert!(!body_text.contains("call-1")); +} + +#[tokio::test] +async fn summary_request_new_auto_shape_does_not_inject_or_execute_threadline_tools() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(Arc::new(connector)); + + let response = post_responses( + app, + new_auto_summary_request_with_tools(vec![ + downstream_function_tool("downstream_tool"), + downstream_function_tool("threadline_echo"), + ]), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let body_task = tokio::spawn(async move { + to_bytes(response.into_body(), usize::MAX) + .await + .expect("body bytes") + }); + + let first_request: Value = serde_json::from_str(&message_text( + server.recv_client_message().await.expect("initial request"), + )) + .expect("initial request json"); + let tools = first_request["tools"].as_array().expect("tools array"); + + assert!(tools.iter().any(|tool| tool["name"] == "downstream_tool")); + assert!( + !tools.iter().any(|tool| { + tool["name"] + .as_str() + .is_some_and(|name| name.starts_with("threadline_")) + }), + "expected new auto summary request to strip threadline_* tools before streaming" + ); + + server + .send_text( + r#"{"type":"response.output_item.done","item":{"type":"function_call","call_id":"call-1","name":"threadline_echo","arguments":"{\"value\":\"alpha\"}"}}"#, + ) + .await; + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-summary-new-auto"}}"#) + .await; + + let maybe_followup = + tokio::time::timeout(Duration::from_millis(100), server.recv_client_message()).await; + assert!( + !matches!(maybe_followup, Ok(Some(_))), + "expected new auto summary request to avoid internal tool follow-up traffic" + ); + + let body = body_task.await.expect("body task"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + assert!(!body_text.contains("threadline_echo")); + assert!(!body_text.contains("response.output_item.done")); +} + +#[tokio::test] +async fn non_internal_tool_events_continue_streaming_without_local_followup() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(Arc::new(connector)); + + let response = post_responses( + app, + json!({ + "model": "gpt-5.4", + "input": "run downstream tool", + "tools": [ + { + "type": "function", + "name": "downstream_tool", + "description": "visible tool", + "parameters": {"type": "object"} + } + ] + }), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let body_task = tokio::spawn(async move { + to_bytes(response.into_body(), usize::MAX) + .await + .expect("body bytes") + }); + + let _ = server.recv_client_message().await.expect("initial request"); + + server + .send_text(concat!( + "{\n", + " \"type\": \"response.output_item.done\",\n", + " \"item\": {\n", + " \"type\": \"function_call\",\n", + " \"call_id\": \"call-visible\",\n", + " \"name\": \"downstream_tool\",\n", + " \"arguments\": \"{}\"\n", + " }\n", + "}" + )) + .await; + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-visible"}}"#) + .await; + + let body = body_task.await.expect("body task"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + let tool_frame = sse_event_and_data(frames[0]); + let completed_frame = sse_event_and_data(frames[1]); + let tool_payload = json!({ + "type": "response.output_item.done", + "item": { + "type": "function_call", + "call_id": "call-visible", + "name": "downstream_tool", + "arguments": "{}" + } + }); + let completed_payload = json!({ + "type": "response.completed", + "response": { + "id": "response-visible" + } + }); + + assert_eq!(frames.len(), 3); + assert_eq!(tool_frame.0, "response.output_item.done"); + assert_eq!(tool_frame.1, tool_payload.to_string()); + assert_eq!( + serde_json::from_str::(tool_frame.1).expect("tool payload json"), + tool_payload + ); + assert_eq!(completed_frame.0, "response.completed"); + assert_eq!(completed_frame.1, completed_payload.to_string()); + assert_eq!( + serde_json::from_str::(completed_frame.1).expect("completed payload json"), + completed_payload + ); + assert_done_frame(frames[2]); + assert!(!body_text.contains(" \"type\": \"response.output_item.done\"")); + assert!(server.take_pending_client_messages().await.is_empty()); +} + +#[tokio::test] +async fn non_internal_tool_added_and_done_events_stream_before_response_completed() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(Arc::new(connector)); + + let response = post_responses( + app, + json!({ + "model": "gpt-5.4", + "input": "stream observed downstream tool events", + "tools": [ + { + "type": "function", + "name": "downstream_tool", + "description": "visible tool", + "parameters": {"type": "object"} + } + ] + }), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let mut body_stream = response.into_body().into_data_stream(); + let mut pending = String::new(); + + let _ = server.recv_client_message().await.expect("initial request"); + + server + .send_text( + r#"{"type":"response.output_item.added","item":{"type":"function_call","call_id":"call-visible","name":"downstream_tool","arguments":"{}"}}"#, + ) + .await; + server + .send_text( + r#"{"type":"response.output_item.done","item":{"type":"function_call","call_id":"call-visible","name":"downstream_tool","arguments":"{}"}}"#, + ) + .await; + let added_payload = json!({ + "type": "response.output_item.added", + "item": { + "type": "function_call", + "call_id": "call-visible", + "name": "downstream_tool", + "arguments": "{}" + } + }); + let done_payload = json!({ + "type": "response.output_item.done", + "item": { + "type": "function_call", + "call_id": "call-visible", + "name": "downstream_tool", + "arguments": "{}" + } + }); + let completed_payload = json!({ + "type": "response.completed", + "response": { + "id": "response-visible" + } + }); + + let added_frame = next_sse_frame(&mut body_stream, &mut pending).await; + let (added_event, added_data) = sse_event_and_data(&added_frame); + assert_eq!(added_event, "response.output_item.added"); + assert_eq!(added_data, added_payload.to_string()); + + let done_frame = next_sse_frame(&mut body_stream, &mut pending).await; + let (done_event, done_data) = sse_event_and_data(&done_frame); + assert_eq!(done_event, "response.output_item.done"); + assert_eq!(done_data, done_payload.to_string()); + + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-visible"}}"#) + .await; + + let completed_frame = next_sse_frame(&mut body_stream, &mut pending).await; + let (completed_event, completed_data) = sse_event_and_data(&completed_frame); + assert_eq!(completed_event, "response.completed"); + assert_eq!(completed_data, completed_payload.to_string()); + + let done_sentinel = next_sse_frame(&mut body_stream, &mut pending).await; + assert_done_frame(&done_sentinel); + assert!( + body_stream.next().await.is_none(), + "expected EOF after DONE" + ); + assert!(server.take_pending_client_messages().await.is_empty()); +} + +#[tokio::test] +async fn explicit_upstream_failure_remains_terminal_failure_after_forwarded_external_tool_event() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(Arc::new(connector)); + + let response = post_responses( + app, + json!({ + "model": "gpt-5.4", + "input": "forward tool before terminal upstream failure", + "tools": [ + { + "type": "function", + "name": "downstream_tool", + "description": "visible tool", + "parameters": {"type": "object"} + } + ] + }), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let body_task = tokio::spawn(async move { + to_bytes(response.into_body(), usize::MAX) + .await + .expect("body bytes") + }); + + let _ = server.recv_client_message().await.expect("initial request"); + + server + .send_text( + r#"{"type":"response.output_item.done","item":{"type":"function_call","call_id":"call-visible","name":"downstream_tool","arguments":"{}"}}"#, + ) + .await; + server + .send_text( + r#"{"type":"response.failed","response":{"id":"response-visible-failed"},"error":{"code":"upstream_response_failed","message":"failed after visible tool"}}"#, + ) + .await; + + let body = body_task.await.expect("body task"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + + assert_eq!(frames.len(), 3); + let tool_frame = sse_event_and_data(frames[0]); + assert_eq!(tool_frame.0, "response.output_item.done"); + assert_eq!( + serde_json::from_str::(tool_frame.1).expect("tool payload json"), + json!({ + "type": "response.output_item.done", + "item": { + "type": "function_call", + "call_id": "call-visible", + "name": "downstream_tool", + "arguments": "{}" + } + }) + ); + + let failed_frame = sse_event_and_data(frames[1]); + let failed_payload: Value = serde_json::from_str(failed_frame.1).expect("failed json"); + assert_eq!(failed_frame.0, "response.failed"); + assert_eq!(failed_payload["response"]["id"], "response-visible-failed"); + assert_eq!( + failed_payload["response"]["error"]["code"], + "upstream_response_failed" + ); + assert_eq!( + failed_payload["response"]["error"]["message"], + "failed after visible tool" + ); + assert_done_frame(frames[2]); +} + +#[tokio::test] +async fn internal_tool_added_and_done_events_stay_hidden_until_intermediate_completion() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(Arc::new(connector)); + + let response = post_responses( + app, + json!({ + "model": "gpt-5.4", + "input": "run hidden internal tool loop", + }), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let body_task = tokio::spawn(async move { + to_bytes(response.into_body(), usize::MAX) + .await + .expect("body bytes") + }); + + let _ = server.recv_client_message().await.expect("initial request"); + + server + .send_text( + r#"{"type":"response.output_item.added","item":{"type":"function_call","call_id":"call-1","name":"threadline_echo","arguments":"{\"value\":\"alpha\"}"}}"#, + ) + .await; + server + .send_text( + r#"{"type":"response.output_item.done","item":{"type":"function_call","call_id":"call-1","name":"threadline_echo","arguments":"{\"value\":\"alpha\"}"}}"#, + ) + .await; + + assert!( + server.take_pending_client_messages().await.is_empty(), + "expected no follow-up request before the intermediate completion arrives" + ); + + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-intermediate"}}"#) + .await; + + let followup_request: Value = serde_json::from_str(&message_text( + server + .recv_client_message() + .await + .expect("followup request"), + )) + .expect("followup request json"); + assert_eq!(followup_request["type"], "response.create"); + + let followup_input = followup_request["input"] + .as_array() + .expect("followup input array"); + assert_eq!(followup_input.len(), 1); + assert_eq!(followup_input[0]["type"], "function_call_output"); + assert_eq!(followup_input[0]["call_id"], "call-1"); + assert_eq!(followup_input[0]["output"], "alpha"); + + server + .send_text(r#"{"type":"response.output_text.delta","delta":"final answer"}"#) + .await; + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-final"}}"#) + .await; + + let body = body_task.await.expect("body task"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + let delta_frame = sse_event_and_data(frames[0]); + let completed_frame = sse_event_and_data(frames[1]); + + assert_eq!(frames.len(), 3); + assert_eq!(delta_frame.0, "response.output_text.delta"); + assert_eq!( + serde_json::from_str::(delta_frame.1).expect("delta json"), + json!({"type":"response.output_text.delta","delta":"final answer"}) + ); + assert_eq!(completed_frame.0, "response.completed"); + let completed_payload: Value = serde_json::from_str(completed_frame.1).expect("completed json"); + assert_eq!(completed_payload["response"]["id"], "response-final"); + assert_eq!( + completed_payload["response"]["output"][0]["content"][0]["text"], + "final answer" + ); + assert_done_frame(frames[2]); + assert!(!body_text.contains("response.output_item.added")); + assert!(!body_text.contains("response.output_item.done")); + assert!(!body_text.contains("threadline_echo")); + assert!(!body_text.contains("response-intermediate")); +} + +#[tokio::test] +async fn intermediate_internal_tool_completion_keeps_marker_active_until_followup_finishes() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(Arc::new(connector)); + + let seed = post_responses(app.clone(), json!({"model":"gpt-5.4","input":"seed"})).await; + assert_eq!(seed.status(), StatusCode::OK); + let _ = server.recv_client_message().await.expect("seed request"); + server + .send_text( + r#"{"type":"response.completed","response":{"id":"response-1","output":[{"id":"assistant-seed","type":"message","role":"assistant","content":[{"type":"output_text","text":"seed answer"}]}]}}"#, + ) + .await; + let _ = to_bytes(seed.into_body(), usize::MAX) + .await + .expect("seed body"); + + let active = post_responses( + app.clone(), + json!({ + "model": "gpt-5.4", + "input": "run hidden internal tool loop", + "previous_response_id": "response-1" + }), + ) + .await; + assert_eq!(active.status(), StatusCode::OK); + + let body_task = tokio::spawn(async move { + to_bytes(active.into_body(), usize::MAX) + .await + .expect("body bytes") + }); + + let active_request: Value = serde_json::from_str(&message_text( + server.recv_client_message().await.expect("active request"), + )) + .expect("active request json"); + assert_eq!(active_request["previous_response_id"], "response-1"); + + server + .send_text( + r#"{"type":"response.output_item.added","item":{"type":"function_call","call_id":"call-1","name":"threadline_echo","arguments":"{\"value\":\"alpha\"}"}}"#, + ) + .await; + server + .send_text( + r#"{"type":"response.output_item.done","item":{"type":"function_call","call_id":"call-1","name":"threadline_echo","arguments":"{\"value\":\"alpha\"}"}}"#, + ) + .await; + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-intermediate"}}"#) + .await; + + let followup_request: Value = serde_json::from_str(&message_text( + server + .recv_client_message() + .await + .expect("followup request"), + )) + .expect("followup request json"); + assert_eq!( + followup_request["previous_response_id"], + "response-intermediate" + ); + + let conflict = post_responses( + app.clone(), + json!({ + "model": "gpt-5.4", + "input": "conflict-before-followup-finish", + "previous_response_id": "response-1" + }), + ) + .await; + assert_eq!(conflict.status(), StatusCode::CONFLICT); + + assert!( + server.take_pending_client_messages().await.is_empty(), + "expected no extra upstream request while the follow-up response is still active" + ); + + server + .send_text(r#"{"type":"response.output_text.delta","delta":"final answer"}"#) + .await; + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-final"}}"#) + .await; + + let body = body_task.await.expect("body task"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + + assert_eq!(frames.len(), 3); + let delta_frame = sse_event_and_data(frames[0]); + assert_eq!(delta_frame.0, "response.output_text.delta"); + assert_eq!( + serde_json::from_str::(delta_frame.1).expect("delta json"), + json!({"type":"response.output_text.delta","delta":"final answer"}) + ); + + let completed_frame = sse_event_and_data(frames[1]); + assert_eq!(completed_frame.0, "response.completed"); + let completed_payload: Value = serde_json::from_str(completed_frame.1).expect("completed json"); + assert_eq!(completed_payload["response"]["id"], "response-final"); + assert_eq!( + completed_payload["response"]["output"][0]["content"][0]["text"], + "final answer" + ); + + assert_done_frame(frames[2]); +} + +#[tokio::test] +async fn internal_tool_intermediate_completion_sends_followup_without_downstream_failure() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(Arc::new(connector)); + + let response = post_responses( + app, + json!({ + "model": "gpt-5.4", + "input": "hold downstream terminal state until internal follow-up request exists" + }), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let mut body_stream = response.into_body().into_data_stream(); + let mut pending = String::new(); + + let _ = server.recv_client_message().await.expect("initial request"); + + server + .send_text( + r#"{"type":"response.output_item.added","item":{"type":"function_call","call_id":"call-1","name":"threadline_echo","arguments":"{\"value\":\"alpha\"}"}}"#, + ) + .await; + server + .send_text( + r#"{"type":"response.output_item.done","item":{"type":"function_call","call_id":"call-1","name":"threadline_echo","arguments":"{\"value\":\"alpha\"}"}}"#, + ) + .await; + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-intermediate"}}"#) + .await; + + let premature_frame = timeout( + Duration::from_millis(100), + next_sse_frame(&mut body_stream, &mut pending), + ) + .await; + assert!( + premature_frame.is_err(), + "expected no downstream terminal frame before the internal follow-up request is observed" + ); + + let followup_request: Value = serde_json::from_str(&message_text( + server + .recv_client_message() + .await + .expect("followup request"), + )) + .expect("followup request json"); + assert_eq!(followup_request["type"], "response.create"); + assert_eq!( + followup_request["previous_response_id"], + "response-intermediate" + ); + + server + .send_text(r#"{"type":"response.output_text.delta","delta":"final answer"}"#) + .await; + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-final"}}"#) + .await; + + let delta_frame = next_sse_frame(&mut body_stream, &mut pending).await; + let completed_frame = next_sse_frame(&mut body_stream, &mut pending).await; + let done_sentinel = next_sse_frame(&mut body_stream, &mut pending).await; + + assert_eq!( + sse_event_and_data(&delta_frame).0, + "response.output_text.delta" + ); + assert_eq!(sse_event_and_data(&completed_frame).0, "response.completed"); + assert_done_frame(&done_sentinel); + assert!( + body_stream.next().await.is_none(), + "expected EOF after DONE" + ); +} + +#[tokio::test] +async fn internal_tool_argument_deltas_are_not_forwarded_downstream() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(Arc::new(connector)); + + let response = post_responses( + app, + json!({ + "model": "gpt-5.4", + "input": "hide internal tool argument deltas", + "tools": [ + { + "type": "function", + "name": "apply_patch", + "description": "visible tool", + "parameters": {"type": "object"} + } + ] + }), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let body_task = tokio::spawn(async move { + to_bytes(response.into_body(), usize::MAX) + .await + .expect("body bytes") + }); + + let _ = server.recv_client_message().await.expect("initial request"); + + server + .send_text( + r#"{"type":"response.output_item.added","output_index":0,"item":{"type":"function_call","call_id":"call-internal","name":"threadline_echo","arguments":"{\"value\":\"alpha\"}"}}"#, + ) + .await; + server + .send_text( + r#"{"type":"response.output_item.added","output_index":1,"item":{"type":"function_call","call_id":"call-visible","name":"apply_patch","arguments":""}}"#, + ) + .await; + server + .send_text( + r#"{"type":"response.function_call_arguments.delta","output_index":0,"item_id":"item-internal","delta":"{\"value\":\"secret-internal\"}"}"#, + ) + .await; + server + .send_text( + r#"{"type":"response.function_call_arguments.delta","output_index":1,"item_id":"item-visible","delta":"{\"input\":\"*** Begin "}"#, + ) + .await; + server + .send_text( + r#"{"type":"response.function_call_arguments.delta","item_id":"item-uncorrelated","delta":"{\"input\":\"still-visible-without-index\"}"}"#, + ) + .await; + server + .send_text( + r#"{"type":"response.output_item.done","output_index":0,"item":{"type":"function_call","call_id":"call-internal","name":"threadline_echo","arguments":"{\"value\":\"alpha\"}"}}"#, + ) + .await; + server + .send_text( + r#"{"type":"response.output_item.done","output_index":1,"item":{"type":"function_call","call_id":"call-visible","name":"apply_patch","arguments":"{\"input\":\"*** Begin Patch\\n*** End Patch\"}"}}"#, + ) + .await; + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-intermediate"}}"#) + .await; + + let followup_request: Value = serde_json::from_str(&message_text( + server + .recv_client_message() + .await + .expect("followup request"), + )) + .expect("followup request json"); + assert_eq!(followup_request["type"], "response.create"); + assert_eq!( + followup_request["input"] + .as_array() + .expect("followup input")[0]["output"], + "alpha" + ); + + server + .send_text(r#"{"type":"response.output_text.delta","delta":"final answer"}"#) + .await; + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-final"}}"#) + .await; + + let body = body_task.await.expect("body task"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + + let visible_added = json!({ + "type": "response.output_item.added", + "output_index": 1, + "item": { + "type": "function_call", + "call_id": "call-visible", + "name": "apply_patch", + "arguments": "" + } + }); + let visible_delta = json!({ + "type": "response.function_call_arguments.delta", + "output_index": 1, + "item_id": "item-visible", + "delta": "{\"input\":\"*** Begin " + }); + let uncorrelated_delta = json!({ + "type": "response.function_call_arguments.delta", + "item_id": "item-uncorrelated", + "delta": "{\"input\":\"still-visible-without-index\"}" + }); + let visible_done = json!({ + "type": "response.output_item.done", + "output_index": 1, + "item": { + "type": "function_call", + "call_id": "call-visible", + "name": "apply_patch", + "arguments": "{\"input\":\"*** Begin Patch\\n*** End Patch\"}" + } + }); + let final_delta = json!({ + "type": "response.output_text.delta", + "delta": "final answer" + }); + assert_eq!(frames.len(), 7); + + let added_frame = sse_event_and_data(frames[0]); + assert_eq!(added_frame.0, "response.output_item.added"); + assert_eq!(added_frame.1, visible_added.to_string()); + + let visible_delta_frame = sse_event_and_data(frames[1]); + assert_eq!( + visible_delta_frame.0, + "response.function_call_arguments.delta" + ); + assert_eq!(visible_delta_frame.1, visible_delta.to_string()); + + let uncorrelated_delta_frame = sse_event_and_data(frames[2]); + assert_eq!( + uncorrelated_delta_frame.0, + "response.function_call_arguments.delta" + ); + assert_eq!(uncorrelated_delta_frame.1, uncorrelated_delta.to_string()); + + let done_frame = sse_event_and_data(frames[3]); + assert_eq!(done_frame.0, "response.output_item.done"); + assert_eq!(done_frame.1, visible_done.to_string()); + + let final_delta_frame = sse_event_and_data(frames[4]); + assert_eq!(final_delta_frame.0, "response.output_text.delta"); + assert_eq!(final_delta_frame.1, final_delta.to_string()); + + let completed_frame = sse_event_and_data(frames[5]); + assert_eq!(completed_frame.0, "response.completed"); + let completed_payload: Value = serde_json::from_str(completed_frame.1).expect("completed json"); + assert_eq!(completed_payload["response"]["id"], "response-final"); + assert_eq!( + completed_payload["response"]["output"][0]["content"][0]["text"], + "final answer" + ); + + assert_done_frame(frames[6]); + assert!(!body_text.contains("call-internal")); + assert!(!body_text.contains("threadline_echo")); + assert!(!body_text.contains("secret-internal")); + assert!(!body_text.contains("response-intermediate")); +} + +#[tokio::test] +async fn internal_tool_done_suppression_emits_stable_trace_event() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(Arc::new(connector)); + + let response = post_responses( + app, + json!({ + "model": "gpt-5.4", + "input": "trace suppressed internal tool completion", + "stream": true + }), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let trace_capture = TraceCaptureGuard::begin().await; + + let body_task = tokio::spawn(async move { + to_bytes(response.into_body(), usize::MAX) + .await + .expect("body bytes") + }); + + let _ = server.recv_client_message().await.expect("initial request"); + + server + .send_text( + r#"{"type":"response.output_item.done","output_index":0,"item":{"type":"function_call","call_id":"call-internal","name":"threadline_echo","arguments":"{\"value\":\"secret-internal\"}"}}"#, + ) + .await; + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-intermediate"}}"#) + .await; + + let followup_request: Value = serde_json::from_str(&message_text( + server + .recv_client_message() + .await + .expect("followup request"), + )) + .expect("followup request json"); + assert_eq!(followup_request["type"], "response.create"); + assert_eq!( + followup_request["input"] + .as_array() + .expect("followup input")[0]["output"], + "secret-internal" + ); + + server + .send_text(r#"{"type":"response.output_text.delta","delta":"final answer"}"#) + .await; + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-final"}}"#) + .await; + + let body = body_task.await.expect("body task"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + + assert_eq!(frames.len(), 3); + + let final_delta = sse_event_and_data(frames[0]); + assert_eq!(final_delta.0, "response.output_text.delta"); + + let final_completed = sse_event_and_data(frames[1]); + assert_eq!(final_completed.0, "response.completed"); + + assert_done_frame(frames[2]); + assert!(!body_text.contains("threadline_echo")); + assert!(!body_text.contains("secret-internal")); + + let logs = trace_capture.logs(); + assert!( + logs.contains("responses_translation_event_suppressed") + && logs.contains("event_type=response.output_item.done"), + "expected stable suppression trace for successful internal tool completion, logs were: {logs}" + ); + assert!(!logs.contains("secret-internal")); +} + +#[tokio::test] +async fn internal_tool_followup_completed_only_text_is_synthesized_as_final_delta() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(Arc::new(connector)); + + let response = post_responses( + app, + json!({ + "model": "gpt-5.4", + "input": "synthesize final follow-up completed-only assistant text", + }), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let body_task = tokio::spawn(async move { + to_bytes(response.into_body(), usize::MAX) + .await + .expect("body bytes") + }); + + let _ = server.recv_client_message().await.expect("initial request"); + + server + .send_text( + r#"{"type":"response.output_item.added","item":{"type":"function_call","call_id":"call-1","name":"threadline_echo","arguments":"{\"value\":\"alpha\"}"}}"#, + ) + .await; + server + .send_text( + r#"{"type":"response.output_item.done","item":{"type":"function_call","call_id":"call-1","name":"threadline_echo","arguments":"{\"value\":\"alpha\"}"}}"#, + ) + .await; + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-intermediate"}}"#) + .await; + + let followup_request: Value = serde_json::from_str(&message_text( + server + .recv_client_message() + .await + .expect("followup request"), + )) + .expect("followup request json"); + assert_eq!(followup_request["type"], "response.create"); + + let followup_input = followup_request["input"] + .as_array() + .expect("followup input array"); + assert_eq!(followup_input.len(), 1); + assert_eq!(followup_input[0]["type"], "function_call_output"); + assert_eq!(followup_input[0]["call_id"], "call-1"); + assert_eq!(followup_input[0]["output"], "alpha"); + + let final_completed = json!({ + "type": "response.completed", + "response": { + "id": "response-final", + "output": [ + { + "id": "msg-final", + "type": "message", + "role": "assistant", + "content": [ + { + "type": "output_text", + "text": "final follow-up answer" + } + ] + } + ] + } + }); + server.send_text(&final_completed.to_string()).await; + + let body = body_task.await.expect("body task"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + + assert_eq!(frames.len(), 3); + + let synthetic_delta_frame = sse_event_and_data(frames[0]); + assert_eq!(synthetic_delta_frame.0, "response.output_text.delta"); + assert_eq!( + serde_json::from_str::(synthetic_delta_frame.1).expect("synthetic delta json"), + json!({ + "type": "response.output_text.delta", + "delta": "final follow-up answer", + "item_id": "msg-final", + "output_index": 0, + "content_index": 0 + }) + ); + + let completed_frame = sse_event_and_data(frames[1]); + assert_eq!(completed_frame.0, "response.completed"); + assert_eq!( + serde_json::from_str::(completed_frame.1).expect("completed json"), + final_completed + ); + + assert_done_frame(frames[2]); + assert!(!body_text.contains("response.output_item.added")); + assert!(!body_text.contains("response.output_item.done")); + assert!(!body_text.contains("threadline_echo")); + assert!(!body_text.contains("response-intermediate")); +} + +#[tokio::test] +async fn internal_tool_followup_output_item_done_message_becomes_final_completed_output() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(Arc::new(connector)); + + let response = post_responses( + app, + json!({ + "model": "gpt-5.4", + "input": "surface follow-up output_item.done assistant text as final completed output" + }), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let body_task = tokio::spawn(async move { + timeout( + Duration::from_secs(2), + to_bytes(response.into_body(), usize::MAX), + ) + .await + .expect("body timeout") + .expect("body bytes") + }); + + let _ = server.recv_client_message().await.expect("initial request"); + + server + .send_text( + r#"{"type":"response.output_item.added","item":{"type":"function_call","call_id":"call-1","name":"threadline_echo","arguments":"{\"value\":\"alpha\"}"}}"#, + ) + .await; + server + .send_text( + r#"{"type":"response.output_item.done","item":{"type":"function_call","call_id":"call-1","name":"threadline_echo","arguments":"{\"value\":\"alpha\"}"}}"#, + ) + .await; + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-intermediate"}}"#) + .await; + + let followup_request: Value = serde_json::from_str(&message_text( + server + .recv_client_message() + .await + .expect("followup request"), + )) + .expect("followup request json"); + assert_eq!(followup_request["type"], "response.create"); + + let final_done_event = json!({ + "type": "response.output_item.done", + "output_index": 0, + "item": { + "id": "msg-final", + "type": "message", + "role": "assistant", + "content": [ + { + "type": "output_text", + "text": "final follow-up answer from output_item.done" + } + ] + } + }); + server.send_text(&final_done_event.to_string()).await; + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-final"}}"#) + .await; + + let body = body_task.await.expect("body task"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + + assert_eq!(frames.len(), 4); + + let delta_frame = sse_event_and_data(frames[0]); + assert_eq!(delta_frame.0, "response.output_text.delta"); + assert_eq!( + serde_json::from_str::(delta_frame.1).expect("delta json"), + json!({ + "type": "response.output_text.delta", + "delta": "final follow-up answer from output_item.done", + "item_id": "msg-final", + "output_index": 0, + "content_index": 0 + }) + ); + + let done_frame = sse_event_and_data(frames[1]); + assert_eq!(done_frame.0, "response.output_item.done"); + assert_eq!(done_frame.1, final_done_event.to_string()); + + let completed_frame = sse_event_and_data(frames[2]); + let completed_payload: Value = serde_json::from_str(completed_frame.1).expect("completed json"); + assert_eq!(completed_frame.0, "response.completed"); + assert_eq!(completed_payload["response"]["id"], "response-final"); + assert_eq!( + completed_payload["response"]["output"][0]["content"][0]["text"], + "final follow-up answer from output_item.done" + ); + + assert_done_frame(frames[3]); + assert!(!body_text.contains("threadline_echo")); + assert!(!body_text.contains("response-intermediate")); +} + +#[tokio::test] +async fn intermediate_internal_tool_completion_does_not_record_marker() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(Arc::new(connector)); + + let response = post_responses( + app.clone(), + json!({ + "model": "gpt-5.4", + "input": "do not record intermediate internal completion markers" + }), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let body_task = tokio::spawn(async move { + timeout( + Duration::from_secs(2), + to_bytes(response.into_body(), usize::MAX), + ) + .await + .expect("body timeout") + .expect("body bytes") + }); + + let _ = server.recv_client_message().await.expect("initial request"); + + server + .send_text( + r#"{"type":"response.output_item.added","item":{"type":"function_call","call_id":"call-1","name":"threadline_echo","arguments":"{\"value\":\"alpha\"}"}}"#, + ) + .await; + server + .send_text( + r#"{"type":"response.output_item.done","item":{"type":"function_call","call_id":"call-1","name":"threadline_echo","arguments":"{\"value\":\"alpha\"}"}}"#, + ) + .await; + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-intermediate"}}"#) + .await; + + let _ = server + .recv_client_message() + .await + .expect("followup request"); + + server + .send_text(r#"{"type":"response.output_text.delta","delta":"final answer"}"#) + .await; + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-final"}}"#) + .await; + + let _ = body_task.await.expect("body task"); + + let rejected = post_responses( + app, + json!({ + "model": "gpt-5.4", + "input": "invalid-intermediate-resume", + "previous_response_id": "response-intermediate" + }), + ) + .await; + assert_eq!(rejected.status(), StatusCode::BAD_REQUEST); + let rejected_body = to_bytes(rejected.into_body(), usize::MAX) + .await + .expect("rejected body"); + let rejected_payload: Value = serde_json::from_slice(&rejected_body).expect("rejected json"); + assert_eq!( + rejected_payload["error"]["code"], + "previous_response_not_found" + ); +} + +#[tokio::test] +async fn internal_tool_followup_failure_emits_response_failed_without_internal_leak() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(Arc::new(connector)); + + let response = post_responses( + app.clone(), + json!({ + "model": "gpt-5.4", + "input": "normalize internal-tool follow-up failure" + }), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let body_task = tokio::spawn(async move { + timeout( + Duration::from_secs(2), + to_bytes(response.into_body(), usize::MAX), + ) + .await + .expect("body timeout") + .expect("body bytes") + }); + + let _ = server.recv_client_message().await.expect("initial request"); + + server + .send_text( + r#"{"type":"response.output_item.added","item":{"type":"function_call","call_id":"call-1","name":"threadline_echo","arguments":"{\"value\":\"alpha\"}"}}"#, + ) + .await; + server + .send_text( + r#"{"type":"response.output_item.done","item":{"type":"function_call","call_id":"call-1","name":"threadline_echo","arguments":"{\"value\":\"alpha\"}"}}"#, + ) + .await; + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-intermediate"}}"#) + .await; + + let _ = server + .recv_client_message() + .await + .expect("followup request"); + + server + .send_text( + r#"{"type":"response.failed","response":{"id":"response-followup-failed","model":"gpt-5.4","usage":{"input_tokens":4,"output_tokens":0,"total_tokens":4}},"error":{"code":"upstream_response_failed","message":"followup failed"}}"#, + ) + .await; + + let body = body_task.await.expect("body task"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + + assert_eq!(frames.len(), 2); + + let failed_frame = sse_event_and_data(frames[0]); + let failed_payload: Value = serde_json::from_str(failed_frame.1).expect("failed json"); + assert_eq!(failed_frame.0, "response.failed"); + assert_eq!(failed_payload["response"]["id"], "response-followup-failed"); + assert_eq!( + failed_payload["response"]["error"]["code"], + "upstream_response_failed" + ); + assert_done_frame(frames[1]); + assert!(!body_text.contains("threadline_echo")); + assert!(!body_text.contains("response-intermediate")); + + let rejected = post_responses( + app, + json!({ + "model": "gpt-5.4", + "input": "invalid-followup-failed-resume", + "previous_response_id": "response-followup-failed" + }), + ) + .await; + assert_eq!(rejected.status(), StatusCode::BAD_REQUEST); + let rejected_body = to_bytes(rejected.into_body(), usize::MAX) + .await + .expect("rejected body"); + let rejected_payload: Value = serde_json::from_slice(&rejected_body).expect("rejected json"); + assert_eq!( + rejected_payload["error"]["code"], + "previous_response_not_found" + ); +} + +#[tokio::test] +async fn visible_followup_function_call_argument_delta_is_forwarded_when_output_index_is_reused() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(Arc::new(connector)); + + let response = post_responses( + app, + json!({ + "model": "gpt-5.4", + "input": "reuse output index after internal tool follow-up", + "tools": [ + { + "type": "function", + "name": "apply_patch", + "description": "visible tool", + "parameters": {"type": "object"} + } + ] + }), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let body_task = tokio::spawn(async move { + to_bytes(response.into_body(), usize::MAX) + .await + .expect("body bytes") + }); + + let _ = server.recv_client_message().await.expect("initial request"); + + server + .send_text( + r#"{"type":"response.output_item.added","output_index":0,"item":{"type":"function_call","call_id":"call-internal","name":"threadline_echo","arguments":"{\"value\":\"alpha\"}"}}"#, + ) + .await; + server + .send_text( + r#"{"type":"response.output_item.done","output_index":0,"item":{"type":"function_call","call_id":"call-internal","name":"threadline_echo","arguments":"{\"value\":\"alpha\"}"}}"#, + ) + .await; + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-intermediate"}}"#) + .await; + + let followup_request: Value = serde_json::from_str(&message_text( + server + .recv_client_message() + .await + .expect("followup request"), + )) + .expect("followup request json"); + assert_eq!(followup_request["type"], "response.create"); + assert_eq!( + followup_request["previous_response_id"], + "response-intermediate" + ); + assert_eq!( + followup_request["input"] + .as_array() + .expect("followup input")[0]["output"], + "alpha" + ); + + server + .send_text( + r#"{"type":"response.output_item.added","output_index":0,"item":{"type":"function_call","call_id":"call-visible","name":"apply_patch","arguments":""}}"#, + ) + .await; + server + .send_text( + r#"{"type":"response.function_call_arguments.delta","output_index":0,"item_id":"item-visible","delta":"{\"input\":\"*** Begin Patch"}"#, + ) + .await; + server + .send_text( + r#"{"type":"response.output_item.done","output_index":0,"item":{"type":"function_call","call_id":"call-visible","name":"apply_patch","arguments":"{\"input\":\"*** Begin Patch\\n*** End Patch\"}"}}"#, + ) + .await; + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-final"}}"#) + .await; + + let body = body_task.await.expect("body task"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + + let visible_added = json!({ + "type": "response.output_item.added", + "output_index": 0, + "item": { + "type": "function_call", + "call_id": "call-visible", + "name": "apply_patch", + "arguments": "" + } + }); + let visible_delta = json!({ + "type": "response.function_call_arguments.delta", + "output_index": 0, + "item_id": "item-visible", + "delta": "{\"input\":\"*** Begin Patch" + }); + let visible_done = json!({ + "type": "response.output_item.done", + "output_index": 0, + "item": { + "type": "function_call", + "call_id": "call-visible", + "name": "apply_patch", + "arguments": "{\"input\":\"*** Begin Patch\\n*** End Patch\"}" + } + }); + let final_completed = json!({ + "type": "response.completed", + "response": { + "id": "response-final" + } + }); + + assert_eq!(frames.len(), 5); + + let added_frame = sse_event_and_data(frames[0]); + assert_eq!(added_frame.0, "response.output_item.added"); + assert_eq!(added_frame.1, visible_added.to_string()); + + let delta_frame = sse_event_and_data(frames[1]); + assert_eq!(delta_frame.0, "response.function_call_arguments.delta"); + assert_eq!(delta_frame.1, visible_delta.to_string()); + + let done_frame = sse_event_and_data(frames[2]); + assert_eq!(done_frame.0, "response.output_item.done"); + assert_eq!(done_frame.1, visible_done.to_string()); + + let completed_frame = sse_event_and_data(frames[3]); + assert_eq!(completed_frame.0, "response.completed"); + assert_eq!(completed_frame.1, final_completed.to_string()); + + assert_done_frame(frames[4]); + assert!(!body_text.contains("call-internal")); + assert!(!body_text.contains("threadline_echo")); + assert!(!body_text.contains("response-intermediate")); +} + +#[tokio::test] +async fn internal_tool_followup_empty_final_does_not_reuse_intermediate_observability() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(Arc::new(connector)); + + let response = post_responses( + app, + json!({ + "model": "gpt-5.4", + "input": "final empty completion after internal follow-up" + }), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let body_task = tokio::spawn(async move { + timeout( + Duration::from_secs(2), + to_bytes(response.into_body(), usize::MAX), + ) + .await + .expect("body timeout") + .expect("body bytes") + }); + + let _ = server.recv_client_message().await.expect("initial request"); + + server + .send_text( + r#"{"type":"response.output_item.added","item":{"type":"function_call","call_id":"call-1","name":"threadline_echo","arguments":"{\"value\":\"alpha\"}"}}"#, + ) + .await; + server + .send_text( + r#"{"type":"response.output_item.done","item":{"type":"function_call","call_id":"call-1","name":"threadline_echo","arguments":"{\"value\":\"alpha\"}"}}"#, + ) + .await; + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-intermediate"}}"#) + .await; + + let followup_request: Value = serde_json::from_str(&message_text( + server + .recv_client_message() + .await + .expect("followup request"), + )) + .expect("followup request json"); + assert_eq!(followup_request["type"], "response.create"); + assert_eq!( + followup_request["previous_response_id"], + "response-intermediate" + ); + + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-final-empty"}}"#) + .await; + + let body = body_task.await.expect("body task"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + + assert_eq!(frames.len(), 2); + let failed_frame = sse_event_and_data(frames[0]); + let failed_payload: Value = serde_json::from_str(failed_frame.1).expect("failed json"); + assert_eq!(failed_frame.0, "response.failed"); + assert_eq!(failed_payload["response"]["id"], "response-final-empty"); + assert_eq!( + failed_payload["response"]["error"]["code"], + "threadline_no_observable_output" + ); + assert_done_frame(frames[1]); + assert!(!body_text.contains("threadline_echo")); + assert!(!body_text.contains("response-intermediate")); +} + +#[test] +fn start_job_tool_returns_stable_disabled_json_by_default() { + let event = json!({ + "type": "response.output_item.done", + "item": { + "type": "function_call", + "call_id": "call-start", + "name": "threadline_start_job", + "arguments": { + "command": ["echo", "hello"] + } + } + }); + + let call = InternalToolCall::from_event(&event) + .expect("tool parse") + .expect("internal tool call"); + let output = call.execute().expect("tool output").into_followup_input(); + + assert_eq!(output["type"], "function_call_output"); + assert_eq!(output["call_id"], "call-start"); + + let payload: Value = serde_json::from_str(output["output"].as_str().expect("output string")) + .expect("json payload"); + assert_eq!(payload["ok"], false); + assert_eq!(payload["code"], "jobs_disabled"); + assert_eq!(payload.get("next_action_hint"), None); + + let invalid_event = json!({ + "type": "response.output_item.done", + "item": { + "type": "function_call", + "call_id": "call-start-invalid", + "name": "threadline_start_job", + "arguments": { + "command": "echo hello" + } + } + }); + + let invalid_call = InternalToolCall::from_event(&invalid_event) + .expect("invalid tool parse") + .expect("invalid internal tool call"); + let invalid_output = invalid_call + .execute() + .expect("invalid tool output") + .into_followup_input(); + let invalid_payload: Value = serde_json::from_str( + invalid_output["output"] + .as_str() + .expect("invalid output string"), + ) + .expect("invalid json payload"); + assert_eq!(invalid_payload["ok"], false); + assert_eq!(invalid_payload["code"], "invalid_job_request"); + assert_eq!(invalid_payload.get("next_action_hint"), None); +} + +#[test] +fn injected_job_tool_definitions_include_contract_phrases_and_preserve_schema() { + let mut payload = serde_json::Map::new(); + inject_internal_tools(&mut payload); + + let tools = payload["tools"].as_array().expect("tools array"); + let find_tool = |name: &str| { + tools + .iter() + .find(|tool| tool["name"] == name) + .unwrap_or_else(|| panic!("missing tool definition: {name}")) + }; + + let start = find_tool("threadline_start_job"); + let start_description = start["description"].as_str().expect("start description"); + assert!(start_description.contains("background")); + assert!(start_description.contains("return immediately")); + assert!(start_description.contains("busy-poll")); + assert_eq!(start["parameters"]["required"], json!(["command"])); + assert_eq!(start["parameters"]["additionalProperties"], false); + assert_eq!(start["parameters"]["properties"]["command"]["minItems"], 1); + + let poll = find_tool("threadline_poll_job"); + let poll_description = poll["description"].as_str().expect("poll description"); + assert!(poll_description.contains("natural checkpoint")); + assert!(poll_description.contains("tight loop")); + assert_eq!(poll["parameters"]["required"], json!(["job_id"])); + assert_eq!(poll["parameters"]["additionalProperties"], false); + + let read_output = find_tool("threadline_read_job_output"); + let read_description = read_output["description"] + .as_str() + .expect("read description"); + assert!(read_description.contains("next_offset")); + assert!(read_description.contains("truncated_before")); + assert_eq!(read_output["parameters"]["required"], json!(["job_id"])); + assert_eq!(read_output["parameters"]["additionalProperties"], false); + assert_eq!( + read_output["parameters"]["properties"]["offset"]["minimum"], + 0 + ); + + let result = find_tool("threadline_get_job_result"); + let result_description = result["description"].as_str().expect("result description"); + assert!(result_description.contains("before final claims")); + assert!(result_description.contains("success or failure")); + assert_eq!(result["parameters"]["required"], json!(["job_id"])); + assert_eq!(result["parameters"]["additionalProperties"], false); + + let cancel = find_tool("threadline_cancel_job"); + let cancel_description = cancel["description"].as_str().expect("cancel description"); + assert!(cancel_description.contains("stuck")); + assert!(cancel_description.contains("poll or get the result")); + assert_eq!(cancel["parameters"]["required"], json!(["job_id"])); + assert_eq!(cancel["parameters"]["additionalProperties"], false); +} + +#[tokio::test] +async fn start_job_tool_serializes_success_hint_in_function_call_output() { + let manager = shell_job_manager(); + let command = if cfg!(windows) { + shell_command("Write-Output 'tool success'; Start-Sleep -Milliseconds 50") + } else { + shell_command("printf 'tool success\n'; sleep 0.05") + }; + + let call = InternalToolCall::from_event(&json!({ + "type": "response.output_item.done", + "item": { + "type": "function_call", + "call_id": "call-start-success", + "name": "threadline_start_job", + "arguments": {"command": command} + } + })) + .expect("start parse") + .expect("start call"); + + let output = call + .execute_with_job_manager(&manager) + .expect("start output") + .into_followup_input(); + let payload: Value = + serde_json::from_str(output["output"].as_str().expect("start output string")) + .expect("start json payload"); + + assert_eq!(output["type"], "function_call_output"); + assert_eq!(output["call_id"], "call-start-success"); + assert_eq!(payload["ok"], true); + assert_eq!(payload["status"], "starting"); + assert_eq!(payload["next_action_hint"], JOB_START_NEXT_ACTION_HINT); + + let job_id = payload["job_id"].as_str().expect("job id").to_string(); + let result = wait_for_terminal_result(&manager, &job_id, Duration::from_millis(1500)).await; + assert_eq!(result["status"], "completed"); + assert_eq!(result["result"]["success"], true); +} + +#[tokio::test] +async fn job_tool_outputs_are_serialized_as_function_call_output_json() { + let manager = ThreadlineJobManager::new(ThreadlineJobManagerConfig { + jobs_enabled: true, + output_buffer_limit_bytes: 1024, + retention_ttl: Duration::from_secs(60), + allowed_commands: vec![], + }); + let started = manager.spawn_job("tool-job", move |context| async move { + context.mark_running(); + context.push_stdout("hello\n"); + context.complete(json!({"summary": "done"})); + }); + let job_id = started["job_id"].as_str().expect("job id").to_string(); + sleep(Duration::from_millis(20)).await; + + let poll_call = InternalToolCall::from_event(&json!({ + "type": "response.output_item.done", + "item": { + "type": "function_call", + "call_id": "call-poll", + "name": "threadline_poll_job", + "arguments": {"job_id": job_id} + } + })) + .expect("poll parse") + .expect("poll call"); + let poll_output = poll_call + .execute_with_job_manager(&manager) + .expect("poll output") + .into_followup_input(); + let poll_payload: Value = + serde_json::from_str(poll_output["output"].as_str().expect("poll output string")) + .expect("poll json"); + assert_eq!(poll_payload["status"], "completed"); + + let read_call = InternalToolCall::from_event(&json!({ + "type": "response.output_item.done", + "item": { + "type": "function_call", + "call_id": "call-read", + "name": "threadline_read_job_output", + "arguments": {"job_id": job_id, "offset": 0} + } + })) + .expect("read parse") + .expect("read call"); + let read_output = read_call + .execute_with_job_manager(&manager) + .expect("read output") + .into_followup_input(); + let read_payload: Value = + serde_json::from_str(read_output["output"].as_str().expect("read output string")) + .expect("read json"); + assert_eq!(read_payload["items"][0]["text"], "hello\n"); + + let result_call = InternalToolCall::from_event(&json!({ + "type": "response.output_item.done", + "item": { + "type": "function_call", + "call_id": "call-result", + "name": "threadline_get_job_result", + "arguments": {"job_id": job_id} + } + })) + .expect("result parse") + .expect("result call"); + let result_output = result_call + .execute_with_job_manager(&manager) + .expect("result output") + .into_followup_input(); + let result_payload: Value = serde_json::from_str( + result_output["output"] + .as_str() + .expect("result output string"), + ) + .expect("result json"); + assert_eq!(result_payload["result"]["summary"], "done"); +} + +#[test] +fn compaction_item_with_threadline_like_name_is_not_an_internal_tool_call() { + let event = json!({ + "type": "response.output_item.done", + "item": { + "type": "compaction", + "id": "cmp_1", + "name": "threadline_echo", + "tool_name": "threadline_echo", + "encrypted_content": "opaque" + } + }); + + assert!( + InternalToolCall::from_event(&event) + .expect("compaction parse") + .is_none(), + "expected compaction items to bypass internal function-call handling" + ); +} + +#[test] +fn internal_tool_name_detection_does_not_match_non_function_compaction_items() { + let compaction_added = json!({ + "type": "response.output_item.added", + "output_index": 0, + "item": { + "type": "compaction", + "id": "cmp_1", + "name": "threadline_echo", + "encrypted_content": "opaque" + } + }); + let compaction_done = json!({ + "type": "response.output_item.done", + "output_index": 0, + "item": { + "type": "compaction", + "id": "cmp_1", + "tool_name": "threadline_echo", + "encrypted_content": "opaque" + } + }); + let internal_done = json!({ + "type": "response.output_item.done", + "output_index": 0, + "item": { + "type": "function_call", + "call_id": "call-internal", + "name": "threadline_echo", + "arguments": "{\"value\":\"opaque\"}" + } + }); + + assert!( + !event_contains_internal_tool_name(&compaction_added), + "expected non-function compaction added event to remain visible across translation" + ); + assert!( + !event_contains_internal_tool_name(&compaction_done), + "expected non-function compaction done event to remain visible across translation" + ); + assert!( + event_contains_internal_tool_name(&internal_done), + "expected actual internal function_call item to stay suppressed" + ); +} + +#[test] +fn actual_internal_function_call_with_threadline_name_remains_suppressed() { + let event = json!({ + "type": "response.output_item.done", + "output_index": 0, + "item": { + "type": "function_call", + "call_id": "call-internal", + "name": "threadline_echo", + "arguments": "{\"value\":\"secret-internal\"}" + } + }); + + assert!( + InternalToolCall::from_event(&event) + .expect("internal parse") + .is_some(), + "expected actual threadline function_call item to remain an internal tool" + ); + assert!( + event_contains_internal_tool_name(&event), + "expected actual threadline function_call item to remain suppressible" + ); +} diff --git a/tests/jobs.rs b/tests/jobs.rs new file mode 100644 index 0000000..a32cd1c --- /dev/null +++ b/tests/jobs.rs @@ -0,0 +1,447 @@ +use std::time::Instant; + +use serde_json::json; +use threadline::jobs::{JobTerminalState, ThreadlineJobManager, ThreadlineJobManagerConfig}; +use tokio::sync::oneshot; +use tokio::time::{Duration, sleep}; + +const JOB_START_NEXT_ACTION_HINT: &str = "This job is running in the background. Continue other useful work if available, then poll status or read output later when needed."; + +fn shell_program() -> String { + if cfg!(windows) { + "pwsh".to_string() + } else { + "sh".to_string() + } +} + +fn shell_command(script: &str) -> Vec { + if cfg!(windows) { + vec![ + "pwsh".to_string(), + "-NoProfile".to_string(), + "-Command".to_string(), + script.to_string(), + ] + } else { + vec!["sh".to_string(), "-lc".to_string(), script.to_string()] + } +} + +fn shell_job_manager() -> ThreadlineJobManager { + ThreadlineJobManager::new(ThreadlineJobManagerConfig { + jobs_enabled: true, + output_buffer_limit_bytes: 1024, + retention_ttl: Duration::from_secs(60), + allowed_commands: vec![shell_program()], + }) +} + +async fn wait_for_output_items( + manager: &ThreadlineJobManager, + job_id: &str, + offset: u64, + expected_item_count: usize, + timeout: Duration, +) -> serde_json::Value { + let deadline = Instant::now() + timeout; + loop { + let output = manager.read_output_json(job_id, offset); + if output["items"] + .as_array() + .map(|items| items.len() >= expected_item_count) + .unwrap_or(false) + { + return output; + } + assert!( + Instant::now() < deadline, + "timed out waiting for job output" + ); + sleep(Duration::from_millis(10)).await; + } +} + +async fn wait_for_terminal_result( + manager: &ThreadlineJobManager, + job_id: &str, + timeout: Duration, +) -> serde_json::Value { + let deadline = Instant::now() + timeout; + loop { + let result = manager.get_result_json(job_id); + if result["status"] == "completed" + || result["status"] == "failed" + || result["status"] == "cancelled" + { + return result; + } + assert!( + Instant::now() < deadline, + "timed out waiting for terminal job result" + ); + sleep(Duration::from_millis(10)).await; + } +} + +async fn assert_no_output_for( + manager: &ThreadlineJobManager, + job_id: &str, + offset: u64, + duration: Duration, +) { + let deadline = Instant::now() + duration; + loop { + let output = manager.read_output_json(job_id, offset); + assert_eq!(output["items"], json!([])); + if Instant::now() >= deadline { + return; + } + sleep(Duration::from_millis(10)).await; + } +} + +#[tokio::test] +async fn job_manager_transitions_through_starting_running_and_completed() { + let manager = ThreadlineJobManager::new(ThreadlineJobManagerConfig { + jobs_enabled: true, + output_buffer_limit_bytes: 1024, + retention_ttl: Duration::from_secs(60), + allowed_commands: vec![], + }); + let (start_tx, start_rx) = oneshot::channel(); + let (finish_tx, finish_rx) = oneshot::channel(); + let (running_tx, running_rx) = oneshot::channel(); + + let start = manager.spawn_job("contract-job", move |context| async move { + let _ = start_rx.await; + context.mark_running(); + let _ = running_tx.send(()); + let _ = finish_rx.await; + context.push_stdout("alpha\n"); + context.complete(json!({"summary": "done"})); + }); + + assert!(start["ok"].as_bool().unwrap_or(false)); + let job_id = start["job_id"].as_str().expect("job id"); + assert_eq!(start["status"], "starting"); + assert_eq!(start["next_action_hint"], JOB_START_NEXT_ACTION_HINT); + + let initial_poll = manager.poll_json(job_id); + assert_eq!(initial_poll["status"], "starting"); + + let _ = start_tx.send(()); + let _ = running_rx.await; + + let running_poll = manager.poll_json(job_id); + assert_eq!(running_poll["status"], "running"); + assert_eq!(running_poll["finished"], false); + + let _ = finish_tx.send(()); + sleep(Duration::from_millis(20)).await; + + let completed_poll = manager.poll_json(job_id); + assert_eq!(completed_poll["status"], "completed"); + assert_eq!(completed_poll["finished"], true); + + let result = manager.get_result_json(job_id); + assert_eq!(result["status"], "completed"); + assert_eq!(result["result"]["summary"], "done"); +} + +#[tokio::test] +async fn job_output_reads_are_incremental_and_bounded() { + let manager = ThreadlineJobManager::new(ThreadlineJobManagerConfig { + jobs_enabled: true, + output_buffer_limit_bytes: 10, + retention_ttl: Duration::from_secs(60), + allowed_commands: vec![], + }); + + let start = manager.spawn_job("buffer-job", move |context| async move { + context.mark_running(); + context.push_stdout("12345"); + context.push_stdout("67890"); + context.push_stderr("abc"); + context.complete(json!({"summary": "buffered"})); + }); + + let job_id = start["job_id"].as_str().expect("job id"); + sleep(Duration::from_millis(20)).await; + + let output = manager.read_output_json(job_id, 0); + assert_eq!(output["status"], "completed"); + assert_eq!(output["truncated_before"], 3); + assert_eq!(output["next_offset"], 13); + + let items = output["items"].as_array().expect("items array"); + assert_eq!(items.len(), 3); + assert_eq!(items[0]["offset"], 3); + assert_eq!(items[0]["text"], "45"); + assert_eq!(items[1]["text"], "67890"); + assert_eq!(items[2]["stream"], "stderr"); + assert_eq!(items[2]["text"], "abc"); + + let incremental = manager.read_output_json(job_id, 13); + assert_eq!(incremental["items"], json!([])); + assert_eq!(incremental["next_offset"], 13); +} + +#[tokio::test] +async fn job_output_limit_and_offsets_use_utf8_bytes() { + let manager = ThreadlineJobManager::new(ThreadlineJobManagerConfig { + jobs_enabled: true, + output_buffer_limit_bytes: 8, + retention_ttl: Duration::from_secs(60), + allowed_commands: vec![], + }); + + let start = manager.spawn_job("utf8-buffer-job", move |context| async move { + context.mark_running(); + context.push_stdout("éé"); + context.push_stderr("🙂"); + context.push_stdout("a"); + context.complete(json!({"summary": "utf8 buffered"})); + }); + + let job_id = start["job_id"].as_str().expect("job id"); + sleep(Duration::from_millis(20)).await; + + let output = manager.read_output_json(job_id, 0); + assert_eq!(output["status"], "completed"); + assert_eq!(output["truncated_before"], 2); + assert_eq!(output["next_offset"], 9); + + let items = output["items"].as_array().expect("items array"); + assert_eq!(items.len(), 3); + assert_eq!(items[0]["offset"], 2); + assert_eq!(items[0]["stream"], "stdout"); + assert_eq!(items[0]["text"], "é"); + assert_eq!(items[1]["offset"], 4); + assert_eq!(items[1]["stream"], "stderr"); + assert_eq!(items[1]["text"], "🙂"); + assert_eq!(items[2]["offset"], 8); + assert_eq!(items[2]["stream"], "stdout"); + assert_eq!(items[2]["text"], "a"); + + let incremental = manager.read_output_json(job_id, 4); + let incremental_items = incremental["items"].as_array().expect("items array"); + assert_eq!(incremental_items.len(), 2); + assert_eq!(incremental_items[0]["offset"], 4); + assert_eq!(incremental_items[0]["text"], "🙂"); + assert_eq!(incremental_items[1]["offset"], 8); + assert_eq!(incremental_items[1]["text"], "a"); + assert_eq!(incremental["next_offset"], 9); +} + +#[tokio::test] +async fn completed_and_cancelled_jobs_persist_until_ttl_cleanup() { + let manager = ThreadlineJobManager::new(ThreadlineJobManagerConfig { + jobs_enabled: true, + output_buffer_limit_bytes: 1024, + retention_ttl: Duration::from_millis(30), + allowed_commands: vec![], + }); + + let completed = manager.spawn_job("ttl-complete", move |context| async move { + context.mark_running(); + context.complete(json!({"summary": "done"})); + }); + let completed_id = completed["job_id"] + .as_str() + .expect("completed job id") + .to_string(); + + let cancelled = manager.spawn_job("ttl-cancel", move |context| async move { + context.mark_running(); + while !context.is_cancelled() { + sleep(Duration::from_millis(5)).await; + } + }); + let cancelled_id = cancelled["job_id"] + .as_str() + .expect("cancelled job id") + .to_string(); + + sleep(Duration::from_millis(20)).await; + + let cancel = manager.cancel_json(&cancelled_id); + assert_eq!(cancel["status"], "cancelled"); + assert_eq!( + cancel["terminal_state"], + JobTerminalState::Cancelled.as_str() + ); + + assert_eq!(manager.poll_json(&completed_id)["status"], "completed"); + assert_eq!(manager.poll_json(&cancelled_id)["status"], "cancelled"); + + sleep(Duration::from_millis(40)).await; + assert_eq!(manager.prune_expired(), 2); + + assert_eq!(manager.poll_json(&completed_id)["code"], "job_not_found"); + assert_eq!(manager.poll_json(&cancelled_id)["code"], "job_not_found"); +} + +#[tokio::test] +async fn disabled_jobs_and_disallowed_commands_are_rejected_with_stable_json() { + let disabled_manager = ThreadlineJobManager::new(ThreadlineJobManagerConfig { + jobs_enabled: false, + output_buffer_limit_bytes: 1024, + retention_ttl: Duration::from_secs(60), + allowed_commands: vec![], + }); + + let disabled = disabled_manager.start_command_json(vec!["echo".to_string()]); + assert_eq!(disabled["ok"], false); + assert_eq!(disabled["code"], "jobs_disabled"); + assert_eq!(disabled.get("next_action_hint"), None); + + let invalid = disabled_manager.start_command_json(Vec::new()); + assert_eq!(invalid["ok"], false); + assert_eq!(invalid["code"], "jobs_disabled"); + assert_eq!(invalid.get("next_action_hint"), None); + + let restricted_manager = ThreadlineJobManager::new(ThreadlineJobManagerConfig { + jobs_enabled: true, + output_buffer_limit_bytes: 1024, + retention_ttl: Duration::from_secs(60), + allowed_commands: vec!["allowed".to_string()], + }); + + let rejected = restricted_manager.start_command_json(vec!["echo".to_string()]); + assert_eq!(rejected["ok"], false); + assert_eq!(rejected["code"], "job_command_not_allowed"); + assert_eq!(rejected.get("next_action_hint"), None); + + let empty = restricted_manager.start_command_json(Vec::new()); + assert_eq!(empty["ok"], false); + assert_eq!(empty["code"], "invalid_job_request"); + assert_eq!(empty.get("next_action_hint"), None); + + assert_eq!( + restricted_manager.poll_json("missing")["code"], + "job_not_found" + ); + assert_eq!( + restricted_manager.read_output_json("missing", 0)["code"], + "job_not_found" + ); + assert_eq!( + restricted_manager.get_result_json("missing")["code"], + "job_not_found" + ); + assert_eq!( + restricted_manager.cancel_json("missing")["code"], + "job_not_found" + ); +} + +#[tokio::test] +async fn command_job_stdout_without_newline_becomes_visible_before_exit() { + let manager = shell_job_manager(); + let command = if cfg!(windows) { + shell_command("[Console]::Out.Write('partial stdout'); Start-Sleep -Milliseconds 1000") + } else { + shell_command("printf 'partial stdout'; sleep 1.0") + }; + + let start = manager.start_command_json(command); + assert_eq!(start["status"], "starting"); + assert_eq!(start["next_action_hint"], JOB_START_NEXT_ACTION_HINT); + + let job_id = start["job_id"].as_str().expect("job id").to_string(); + let output = wait_for_output_items(&manager, &job_id, 0, 1, Duration::from_millis(1200)).await; + + assert_eq!(manager.poll_json(&job_id)["status"], "running"); + let items = output["items"].as_array().expect("items array"); + assert_eq!(items.len(), 1); + assert_eq!(items[0]["stream"], "stdout"); + assert_eq!(items[0]["offset"], 0); + assert_eq!(items[0]["text"], "partial stdout"); + assert_eq!(output["next_offset"], 14); + + let result = wait_for_terminal_result(&manager, &job_id, Duration::from_millis(2000)).await; + assert_eq!(result["status"], "completed"); + assert_eq!(result["result"]["success"], true); +} + +#[tokio::test] +async fn command_job_stderr_without_newline_becomes_visible_before_exit() { + let manager = shell_job_manager(); + let command = if cfg!(windows) { + shell_command("[Console]::Error.Write('partial stderr'); Start-Sleep -Milliseconds 1000") + } else { + shell_command("printf 'partial stderr' >&2; sleep 1.0") + }; + + let start = manager.start_command_json(command); + assert_eq!(start["status"], "starting"); + + let job_id = start["job_id"].as_str().expect("job id").to_string(); + let output = wait_for_output_items(&manager, &job_id, 0, 1, Duration::from_millis(1200)).await; + + assert_eq!(manager.poll_json(&job_id)["status"], "running"); + let items = output["items"].as_array().expect("items array"); + assert_eq!(items.len(), 1); + assert_eq!(items[0]["stream"], "stderr"); + assert_eq!(items[0]["offset"], 0); + assert_eq!(items[0]["text"], "partial stderr"); + assert_eq!(output["next_offset"], 14); + + let result = wait_for_terminal_result(&manager, &job_id, Duration::from_millis(2000)).await; + assert_eq!(result["status"], "completed"); + assert_eq!(result["result"]["success"], true); +} + +#[tokio::test] +async fn command_job_split_utf8_bytes_wait_for_valid_prefix_and_keep_byte_offsets() { + let manager = shell_job_manager(); + let command = if cfg!(windows) { + shell_command( + "$stdout = [Console]::OpenStandardOutput(); \ + $emojiBytes = [byte[]](0xF0, 0x9F, 0x99, 0x82); \ + $asciiBytes = [byte[]](0x61); \ + $stdout.Write($emojiBytes, 0, 2); \ + $stdout.Flush(); \ + Start-Sleep -Milliseconds 1000; \ + $stdout.Write($emojiBytes, 2, 2); \ + $stdout.Flush(); \ + Start-Sleep -Milliseconds 800; \ + $stdout.Write($asciiBytes, 0, 1); \ + $stdout.Flush(); \ + Start-Sleep -Milliseconds 500", + ) + } else { + shell_command( + r"printf '\360\237'; sleep 1.0; printf '\231\202'; sleep 0.8; printf 'a'; sleep 0.5", + ) + }; + + let start = manager.start_command_json(command); + let job_id = start["job_id"].as_str().expect("job id").to_string(); + + assert_no_output_for(&manager, &job_id, 0, Duration::from_millis(300)).await; + assert_eq!(manager.poll_json(&job_id)["status"], "running"); + + let emoji_output = + wait_for_output_items(&manager, &job_id, 0, 1, Duration::from_millis(1800)).await; + let emoji_items = emoji_output["items"].as_array().expect("items array"); + assert_eq!(emoji_items.len(), 1); + assert_eq!(emoji_items[0]["stream"], "stdout"); + assert_eq!(emoji_items[0]["offset"], 0); + assert_eq!(emoji_items[0]["text"], "🙂"); + assert_eq!(emoji_output["next_offset"], 4); + + let ascii_output = + wait_for_output_items(&manager, &job_id, 4, 1, Duration::from_millis(1400)).await; + let ascii_items = ascii_output["items"].as_array().expect("items array"); + assert_eq!(ascii_items.len(), 1); + assert_eq!(ascii_items[0]["stream"], "stdout"); + assert_eq!(ascii_items[0]["offset"], 4); + assert_eq!(ascii_items[0]["text"], "a"); + assert_eq!(ascii_output["next_offset"], 5); + + let result = wait_for_terminal_result(&manager, &job_id, Duration::from_millis(1200)).await; + assert_eq!(result["status"], "completed"); + assert_eq!(result["result"]["success"], true); +} diff --git a/tests/reconnect.rs b/tests/reconnect.rs new file mode 100644 index 0000000..620a1d2 --- /dev/null +++ b/tests/reconnect.rs @@ -0,0 +1,699 @@ +use std::collections::VecDeque; +use std::sync::Arc; + +use axum::body::{Body, to_bytes}; +use axum::http::{Request, Response, StatusCode}; +use futures_util::future::BoxFuture; +use serde_json::{Value, json}; +use tokio::sync::Mutex; +use tokio::time::{Duration, timeout}; +use tokio_tungstenite::connect_async; +use tower::ServiceExt; +use uuid::Uuid; + +#[path = "support/scripted_ws.rs"] +mod scripted_ws; + +use scripted_ws::ScriptedWebSocketServer; +use threadline::auth::{AuthSource, LoadedUpstreamAuth, RefreshBoundary}; +use threadline::codex_ws::UpstreamSessionDescriptor; +use threadline::config::ThreadlineConfig; +use threadline::errors::ThreadlineError; +use threadline::http::build_router_with_services; +use threadline::responses::{ + ConnectedUpstream, ThreadlineServices, UpstreamAuthProvider, UpstreamConnector, +}; +use threadline::ws_pump::LiveUpstreamWebSocket; + +#[derive(Clone)] +struct StaticAuthProvider; + +impl UpstreamAuthProvider for StaticAuthProvider { + fn load(&self) -> Result { + Ok(LoadedUpstreamAuth { + bearer_token: "test-token".to_string(), + source: AuthSource::CodexKeyring, + refresh_boundary: RefreshBoundary::NotAvailable, + }) + } +} + +struct PlannedConnection { + server: Arc, + turn_state: Option, + wait_until_closed_before_return: bool, +} + +#[derive(Clone)] +struct RecordingConnector { + plans: Arc>>, + sessions: Arc>>, +} + +impl RecordingConnector { + fn new(plans: Vec) -> Self { + Self { + plans: Arc::new(Mutex::new(plans.into())), + sessions: Arc::new(Mutex::new(Vec::new())), + } + } + + async fn recorded_sessions(&self) -> Vec { + self.sessions.lock().await.clone() + } +} + +impl UpstreamConnector for RecordingConnector { + fn connect( + &self, + _auth: LoadedUpstreamAuth, + session: Option, + ) -> BoxFuture<'static, Result> { + let plans = Arc::clone(&self.plans); + let sessions = Arc::clone(&self.sessions); + Box::pin(async move { + let session = session.unwrap_or_else(new_session_descriptor); + let plan = plans + .lock() + .await + .pop_front() + .expect("planned websocket connection"); + sessions.lock().await.push(session.clone()); + + let (stream, _) = connect_async(plan.server.url()) + .await + .map_err(|_| ThreadlineError::UpstreamWebSocketConnectFailed)?; + let websocket = Arc::new(LiveUpstreamWebSocket::from_stream(stream)); + + if plan.wait_until_closed_before_return { + timeout(Duration::from_secs(1), async { + while !websocket.is_closed() { + tokio::task::yield_now().await; + } + }) + .await + .expect("disconnected websocket should close promptly"); + } + + Ok(ConnectedUpstream { + websocket, + session, + turn_state: plan.turn_state, + }) + }) + } +} + +fn build_test_router(connector: Arc) -> axum::Router { + build_router_with_services( + ThreadlineConfig::default(), + ThreadlineServices::new(Arc::new(StaticAuthProvider), connector), + ) +} + +async fn post_responses(app: axum::Router, payload: Value) -> Response { + app.oneshot( + Request::builder() + .method("POST") + .uri("/v1/responses") + .header("content-type", "application/json") + .body(Body::from(payload.to_string())) + .expect("request"), + ) + .await + .expect("response") +} + +fn new_session_descriptor() -> UpstreamSessionDescriptor { + UpstreamSessionDescriptor { + session_id: Uuid::now_v7().to_string(), + thread_id: Uuid::now_v7().to_string(), + window_id: Uuid::now_v7().to_string(), + turn_state: None, + } +} + +fn auxiliary_summary_text() -> &'static str { + concat!( + "The conversation has grown too large for the context window and must be compacted now", + "\n\n", + "Your ONLY task right now is to produce a comprehensive summary", + "\n", + "Output your summary wrapped in and tags" + ) +} + +fn auxiliary_summary_input_item() -> Value { + json!({ + "type": "message", + "role": "system", + "content": [ + { + "type": "input_text", + "text": auxiliary_summary_text() + } + ] + }) +} + +fn auxiliary_summary_request(previous_response_id: Option<&str>) -> Value { + let mut payload = json!({ + "model": "gpt-5.4", + "input": [ + { + "type": "message", + "role": "user", + "content": [ + { + "type": "input_text", + "text": "Continue from the earlier answer." + } + ] + }, + auxiliary_summary_input_item() + ] + }); + + if let Some(previous_response_id) = previous_response_id { + payload["previous_response_id"] = json!(previous_response_id); + } + + payload +} + +fn assistant_text_completed_event(response_id: &str, text: &str) -> Value { + json!({ + "type": "response.completed", + "response": { + "id": response_id, + "output": [ + { + "type": "message", + "role": "assistant", + "content": [ + { + "type": "output_text", + "text": text + } + ] + } + ] + } + }) +} + +fn split_sse_frames(body: &str) -> Vec<&str> { + body.split("\n\n") + .filter(|frame| !frame.trim().is_empty()) + .collect() +} + +fn sse_event_and_data(frame: &str) -> (&str, &str) { + let mut event = None; + let mut data = None; + let mut unexpected_lines = Vec::new(); + + for (index, line) in frame.lines().enumerate() { + if let Some(value) = line.strip_prefix("event: ") { + assert!( + event.replace(value).is_none(), + "expected exactly one event line in SSE frame, found duplicate at line {}: {frame}", + index + 1 + ); + continue; + } + + if let Some(value) = line.strip_prefix("data: ") { + assert!( + data.replace(value).is_none(), + "expected compact single-line SSE data payload, found duplicate data line at line {}: {frame}", + index + 1 + ); + continue; + } + + unexpected_lines.push(format!("line {}: {line}", index + 1)); + } + + assert!( + unexpected_lines.is_empty(), + "expected exactly one event line and one compact data line in SSE frame; unexpected lines: {}. Frame: {frame}", + unexpected_lines.join(" | ") + ); + + ( + event.unwrap_or_else(|| panic!("missing event line in SSE frame: {frame}")), + data.unwrap_or_else(|| panic!("missing data line in SSE frame: {frame}")), + ) +} + +fn assert_done_frame(frame: &str) { + assert_eq!( + frame, "data: [DONE]", + "expected a bare downstream DONE frame without an event line" + ); +} + +fn assert_response_failed_payload(payload: &Value, expected_code: &str) { + assert_eq!(payload["type"], "response.failed"); + assert_eq!(payload["response"]["status"], "failed"); + assert_eq!(payload["response"]["error"]["code"], expected_code); + assert!( + payload["response"]["error"]["message"] + .as_str() + .is_some_and(|message| !message.is_empty()), + "expected a stable non-empty terminal failure message: {payload:?}" + ); +} + +async fn seed_marker(app: axum::Router, server: &ScriptedWebSocketServer, marker: &str) { + let response = post_responses(app, json!({"model":"gpt-5.4","input":"seed"})).await; + assert_eq!(response.status(), StatusCode::OK); + let _ = server.recv_client_message().await.expect("seed request"); + server + .send_text(&assistant_text_completed_event(marker, "seed completion").to_string()) + .await; + let _ = to_bytes(response.into_body(), usize::MAX) + .await + .expect("seed body"); +} + +#[tokio::test] +async fn reconnect_fallback_is_not_attempted_for_non_continuation_requests() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + wait_until_closed_before_return: false, + }]); + let app = build_test_router(Arc::new(connector.clone())); + + let response = post_responses(app, json!({"model":"gpt-5.4","input":"first"})).await; + assert_eq!(response.status(), StatusCode::OK); + let _ = timeout(Duration::from_secs(1), server.recv_client_message()) + .await + .expect("initial request timeout") + .expect("initial request"); + server.send_close(1000, "closed-before-event").await; + + let body = to_bytes(response.into_body(), usize::MAX) + .await + .expect("body"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + let (event, data) = sse_event_and_data(frames.first().expect("failed frame")); + let payload: Value = serde_json::from_str(data).expect("failed json"); + + assert_eq!(frames.len(), 2); + assert_eq!(event, "response.failed"); + assert_response_failed_payload(&payload, "upstream_websocket_closed"); + assert!( + !body_text.contains("event: error\n"), + "expected terminal websocket close to use the downstream response.failed contract: {body_text}" + ); + assert_done_frame(frames[1]); + + let sessions = connector.recorded_sessions().await; + assert_eq!(sessions.len(), 1); +} + +#[tokio::test] +async fn live_retained_continuation_close_before_first_send_returns_previous_response_not_found_without_reconnect_or_resend() + { + let retained_server = Arc::new(ScriptedWebSocketServer::start().await); + let unexpected_reconnect_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![ + PlannedConnection { + server: Arc::clone(&retained_server), + turn_state: Some("turn-state-1".to_string()), + wait_until_closed_before_return: false, + }, + PlannedConnection { + server: Arc::clone(&unexpected_reconnect_server), + turn_state: None, + wait_until_closed_before_return: false, + }, + ]); + let app = build_test_router(Arc::new(connector.clone())); + + seed_marker(app.clone(), &retained_server, "response-1").await; + + let response_task = tokio::spawn({ + let app = app.clone(); + async move { + post_responses( + app, + json!({ + "model":"gpt-5.4", + "input":"followup", + "previous_response_id":"response-1" + }), + ) + .await + } + }); + + retained_server.abort_connection().await; + + let response = timeout(Duration::from_secs(1), response_task) + .await + .expect("continuation response timeout") + .expect("continuation response task"); + assert_eq!( + response.status(), + StatusCode::BAD_REQUEST, + "retained close before first send should fail before SSE starts" + ); + + let retained_message = timeout( + Duration::from_millis(250), + retained_server.recv_client_message(), + ) + .await + .expect("retained close should resolve the pending client receive"); + assert!( + retained_message.is_none(), + "expected the retained upstream to close before resending the same previous_response_id" + ); + + let no_reconnect = timeout( + Duration::from_millis(250), + unexpected_reconnect_server.recv_client_message(), + ) + .await; + assert!(no_reconnect.is_err()); + + let body = to_bytes(response.into_body(), usize::MAX) + .await + .expect("body"); + let payload: Value = serde_json::from_slice(&body).expect("json body"); + assert_eq!(payload["error"]["code"], "previous_response_not_found"); + + let sessions = connector.recorded_sessions().await; + assert_eq!(sessions.len(), 1); +} + +#[tokio::test] +async fn reconnect_fallback_is_not_attempted_after_any_upstream_event() { + let seed_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&seed_server), + turn_state: Some("turn-state-1".to_string()), + wait_until_closed_before_return: false, + }]); + let app = build_test_router(Arc::new(connector.clone())); + + seed_marker(app.clone(), &seed_server, "response-1").await; + + let response = post_responses( + app, + json!({ + "model":"gpt-5.4", + "input":"followup", + "previous_response_id":"response-1" + }), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let _ = timeout(Duration::from_secs(1), seed_server.recv_client_message()) + .await + .expect("continuation request timeout") + .expect("continuation request"); + seed_server + .send_text(r#"{"type":"response.created","response":{"id":"response-created"}}"#) + .await; + seed_server.send_close(1000, "closed-after-event").await; + + let body = timeout( + Duration::from_secs(1), + to_bytes(response.into_body(), usize::MAX), + ) + .await + .expect("body timeout") + .expect("body"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + let (created_event, created_data) = sse_event_and_data(frames.first().expect("created frame")); + let (failed_event, failed_data) = sse_event_and_data(frames.get(1).expect("failed frame")); + let created_payload: Value = serde_json::from_str(created_data).expect("created json"); + let failed_payload: Value = serde_json::from_str(failed_data).expect("failed json"); + + assert_eq!(frames.len(), 3); + assert_eq!(created_event, "response.created"); + assert_eq!( + created_payload, + json!({"type":"response.created","response":{"id":"response-created"}}) + ); + assert_eq!(failed_event, "response.failed"); + assert_response_failed_payload(&failed_payload, "upstream_websocket_closed"); + assert!( + !body_text.contains("event: error\n"), + "expected terminal websocket close to use the downstream response.failed contract: {body_text}" + ); + assert_done_frame(frames[2]); + + let sessions = connector.recorded_sessions().await; + assert_eq!(sessions.len(), 1); +} + +#[tokio::test] +async fn retained_continuation_close_after_send_before_first_upstream_event_replays_stale_marker_without_reconnect() + { + let retained_server = Arc::new(ScriptedWebSocketServer::start().await); + let unexpected_reconnect_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![ + PlannedConnection { + server: Arc::clone(&retained_server), + turn_state: Some("turn-state-1".to_string()), + wait_until_closed_before_return: false, + }, + PlannedConnection { + server: Arc::clone(&unexpected_reconnect_server), + turn_state: None, + wait_until_closed_before_return: false, + }, + ]); + let app = build_test_router(Arc::new(connector.clone())); + + seed_marker(app.clone(), &retained_server, "response-1").await; + + let response = post_responses( + app, + json!({ + "model":"gpt-5.4", + "input":"followup", + "previous_response_id":"response-1" + }), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let body_task = tokio::spawn(async move { + timeout( + Duration::from_secs(1), + to_bytes(response.into_body(), usize::MAX), + ) + .await + .expect("body timeout") + .expect("body bytes") + }); + + let retained_message = timeout( + Duration::from_secs(1), + retained_server.recv_client_message(), + ) + .await + .expect("continuation request timeout") + .expect("continuation request"); + let retained_message = retained_message.into_text().expect("text request"); + let retained_payload: Value = serde_json::from_str(&retained_message).expect("request json"); + assert_eq!(retained_payload["previous_response_id"], "response-1"); + + retained_server + .send_close(1000, "closed-before-first-event") + .await; + + let body = body_task.await.expect("body task"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + let (failed_event, failed_data) = sse_event_and_data(frames.first().expect("failed frame")); + let failed_payload: Value = serde_json::from_str(failed_data).expect("failed json"); + + assert_eq!(frames.len(), 2); + assert_eq!(failed_event, "response.failed"); + assert_response_failed_payload(&failed_payload, "previous_response_not_found"); + assert_done_frame(frames[1]); + + let no_reconnect = timeout( + Duration::from_millis(250), + unexpected_reconnect_server.recv_client_message(), + ) + .await; + assert!(no_reconnect.is_err()); + + let sessions = connector.recorded_sessions().await; + assert_eq!(sessions.len(), 1); +} + +#[tokio::test] +async fn stale_continuation_with_spare_reconnect_plans_returns_previous_response_not_found_without_reconnect() + { + let seed_server = Arc::new(ScriptedWebSocketServer::start().await); + let first_attempt_server = Arc::new(ScriptedWebSocketServer::start().await); + let reconnect_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![ + PlannedConnection { + server: Arc::clone(&seed_server), + turn_state: Some("turn-state-1".to_string()), + wait_until_closed_before_return: false, + }, + PlannedConnection { + server: Arc::clone(&first_attempt_server), + turn_state: None, + wait_until_closed_before_return: false, + }, + PlannedConnection { + server: Arc::clone(&reconnect_server), + turn_state: None, + wait_until_closed_before_return: false, + }, + ]); + let app = build_test_router(Arc::new(connector.clone())); + + seed_marker(app.clone(), &seed_server, "response-1").await; + seed_server.send_close(1000, "seed complete").await; + tokio::time::sleep(Duration::from_millis(50)).await; + + let response = post_responses( + app, + json!({ + "model":"gpt-5.4", + "input":"followup", + "previous_response_id":"response-1" + }), + ) + .await; + assert_eq!(response.status(), StatusCode::BAD_REQUEST); + + let no_first_attempt = timeout( + Duration::from_millis(250), + first_attempt_server.recv_client_message(), + ) + .await; + assert!(no_first_attempt.is_err()); + + let no_reconnect = timeout( + Duration::from_millis(250), + reconnect_server.recv_client_message(), + ) + .await; + assert!(no_reconnect.is_err()); + + let body = to_bytes(response.into_body(), usize::MAX) + .await + .expect("body"); + let payload: Value = serde_json::from_slice(&body).expect("json body"); + assert_eq!(payload["error"]["code"], "previous_response_not_found"); + + let sessions = connector.recorded_sessions().await; + assert_eq!(sessions.len(), 1); +} + +#[tokio::test] +async fn stale_continuation_returns_previous_response_not_found_before_sse_without_reconnect_or_resend() + { + let seed_server = Arc::new(ScriptedWebSocketServer::start().await); + let unexpected_reconnect_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![ + PlannedConnection { + server: Arc::clone(&seed_server), + turn_state: Some("turn-state-1".to_string()), + wait_until_closed_before_return: false, + }, + PlannedConnection { + server: Arc::clone(&unexpected_reconnect_server), + turn_state: None, + wait_until_closed_before_return: false, + }, + ]); + let app = build_test_router(Arc::new(connector.clone())); + + seed_marker(app.clone(), &seed_server, "response-1").await; + seed_server.send_close(1000, "seed complete").await; + tokio::time::sleep(Duration::from_millis(50)).await; + + let response = post_responses( + app, + json!({ + "model":"gpt-5.4", + "input":"followup", + "previous_response_id":"response-1" + }), + ) + .await; + assert_eq!( + response.status(), + StatusCode::BAD_REQUEST, + "stale retained continuation should fail before SSE starts" + ); + + let no_reconnect = timeout( + Duration::from_millis(250), + unexpected_reconnect_server.recv_client_message(), + ) + .await; + assert!(no_reconnect.is_err()); + + let body = to_bytes(response.into_body(), usize::MAX) + .await + .expect("body"); + let payload: Value = serde_json::from_slice(&body).expect("json body"); + assert_eq!(payload["error"]["code"], "previous_response_not_found"); + + let sessions = connector.recorded_sessions().await; + assert_eq!(sessions.len(), 1); +} + +#[tokio::test] +async fn summary_request_first_send_failure_does_not_reconnect_as_continuation() { + let seed_server = Arc::new(ScriptedWebSocketServer::start().await); + let first_attempt_server = + Arc::new(ScriptedWebSocketServer::start_disconnect_after_handshake().await); + let unexpected_reconnect_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![ + PlannedConnection { + server: Arc::clone(&seed_server), + turn_state: Some("turn-state-1".to_string()), + wait_until_closed_before_return: false, + }, + PlannedConnection { + server: Arc::clone(&first_attempt_server), + turn_state: None, + wait_until_closed_before_return: true, + }, + PlannedConnection { + server: Arc::clone(&unexpected_reconnect_server), + turn_state: None, + wait_until_closed_before_return: false, + }, + ]); + let app = build_test_router(Arc::new(connector.clone())); + + seed_marker(app.clone(), &seed_server, "response-1").await; + seed_server.send_close(1000, "seed complete").await; + tokio::time::sleep(Duration::from_millis(50)).await; + + let response = post_responses(app, auxiliary_summary_request(Some("response-1"))).await; + assert_eq!(response.status(), StatusCode::BAD_GATEWAY); + + let no_reconnect = timeout( + Duration::from_millis(250), + unexpected_reconnect_server.recv_client_message(), + ) + .await; + assert!(no_reconnect.is_err()); + + let sessions = connector.recorded_sessions().await; + assert_eq!(sessions.len(), 2); +} diff --git a/tests/registry.rs b/tests/registry.rs new file mode 100644 index 0000000..1f0eb17 --- /dev/null +++ b/tests/registry.rs @@ -0,0 +1,107 @@ +use threadline::registry::{RegistryAcquireError, RetainedSessionRegistry}; + +#[tokio::test] +async fn previous_response_marker_reuses_the_same_retained_session_after_release() { + let registry = RetainedSessionRegistry::new(2); + let mut first = registry.acquire_new().await.expect("create session"); + let original_session = first.session().clone(); + + first + .update_turn_state(Some("turn-state-1".to_string())) + .await; + first.record_completed_marker("response-1").await; + + drop(first); + + let followup = registry + .acquire_previous("response-1") + .await + .expect("reuse previous response session"); + + assert_eq!(followup.session().session_id, original_session.session_id); + assert_eq!(followup.session().thread_id, original_session.thread_id); + assert_eq!(followup.session().window_id, original_session.window_id); + assert_eq!( + followup.session().turn_state.as_deref(), + Some("turn-state-1") + ); +} + +#[tokio::test] +async fn released_lease_can_be_reacquired_before_drop() { + let registry = RetainedSessionRegistry::new(1); + let mut lease = registry.acquire_new().await.expect("create session"); + let original_session = lease.session().clone(); + + lease.record_completed_marker("response-1").await; + lease.release(); + + let reacquired = registry + .acquire_previous("response-1") + .await + .expect("released marker should be reacquired before drop"); + + assert_eq!(reacquired.session().session_id, original_session.session_id); + assert_eq!(reacquired.session().thread_id, original_session.thread_id); + assert_eq!(reacquired.session().window_id, original_session.window_id); +} + +#[tokio::test] +async fn active_lease_still_conflicts() { + let registry = RetainedSessionRegistry::new(2); + let mut first = registry.acquire_new().await.expect("create session"); + + first.record_completed_marker("response-1").await; + + let error = registry + .acquire_previous("response-1") + .await + .expect_err("leased marker should conflict"); + + assert_eq!(error, RegistryAcquireError::RetainedSessionConflict); +} + +#[tokio::test] +async fn missing_previous_response_marker_returns_not_found() { + let registry = RetainedSessionRegistry::new(2); + + let error = registry + .acquire_previous("missing-response") + .await + .expect_err("unknown marker should fail"); + + assert_eq!(error, RegistryAcquireError::PreviousResponseNotFound); +} + +#[tokio::test] +async fn capacity_exhaustion_returns_a_stable_error_while_all_sessions_are_leased() { + let registry = RetainedSessionRegistry::new(1); + let _lease = registry.acquire_new().await.expect("first session"); + + let error = registry + .acquire_new() + .await + .expect_err("leased capacity should fail"); + + assert_eq!(error, RegistryAcquireError::RetainedSessionCapacityExceeded); +} + +#[tokio::test] +async fn recoverable_close_preserves_marker_continuity_without_a_live_socket() { + let registry = RetainedSessionRegistry::new(1); + let mut lease = registry.acquire_new().await.expect("create session"); + let original_session = lease.session().clone(); + + lease.record_completed_marker("response-2").await; + lease.mark_upstream_recoverable().await; + drop(lease); + + let reacquired = registry + .acquire_previous("response-2") + .await + .expect("recoverable marker should survive"); + + assert_eq!(reacquired.session().session_id, original_session.session_id); + assert!(!reacquired.has_live_upstream()); + assert!(reacquired.upstream().is_none()); +} diff --git a/tests/responses_bridge.rs b/tests/responses_bridge.rs new file mode 100644 index 0000000..9c28bde --- /dev/null +++ b/tests/responses_bridge.rs @@ -0,0 +1,7061 @@ +use std::collections::VecDeque; +use std::io::{self, Write}; +use std::sync::Mutex as StdMutex; +use std::sync::OnceLock; +use std::sync::{Arc, Weak}; +use std::time::Duration; + +use axum::body::{Body, Bytes, to_bytes}; +use axum::http::{Request, Response, StatusCode}; +use futures_util::{StreamExt, future::BoxFuture, stream}; +use serde_json::{Value, json}; +use tokio::sync::Mutex; +use tokio::time::{sleep, timeout}; +use tokio_tungstenite::connect_async; +use tokio_tungstenite::tungstenite::Message; +use tower::ServiceExt; +use tracing_subscriber::fmt::MakeWriter; +use uuid::Uuid; + +#[path = "support/scripted_ws.rs"] +mod scripted_ws; + +use scripted_ws::ScriptedWebSocketServer; +use threadline::auth::{AuthSource, LoadedUpstreamAuth, RefreshBoundary}; +use threadline::codex_ws::UpstreamSessionDescriptor; +use threadline::config::ThreadlineConfig; +use threadline::errors::ThreadlineError; +use threadline::http::build_router_with_services; +use threadline::models::RouteProfile; +use threadline::responses::{ + ConnectedUpstream, ThreadlineServices, UpstreamAuthProvider, UpstreamConnector, +}; +use threadline::ws_pump::LiveUpstreamWebSocket; + +#[derive(Clone)] +struct StaticAuthProvider; + +impl UpstreamAuthProvider for StaticAuthProvider { + fn load(&self) -> Result { + Ok(LoadedUpstreamAuth { + bearer_token: "test-token".to_string(), + source: AuthSource::CodexKeyring, + refresh_boundary: RefreshBoundary::NotAvailable, + }) + } +} + +struct PlannedConnection { + server: Arc, + turn_state: Option, +} + +#[derive(Clone)] +struct RecordingConnector { + plans: Arc>>, + sessions: Arc>>, + requested_sessions: Arc>>>, + websockets: Arc>>>, +} + +impl RecordingConnector { + fn new(plans: Vec) -> Self { + Self { + plans: Arc::new(Mutex::new(plans.into())), + sessions: Arc::new(Mutex::new(Vec::new())), + requested_sessions: Arc::new(Mutex::new(Vec::new())), + websockets: Arc::new(Mutex::new(Vec::new())), + } + } + + async fn recorded_sessions(&self) -> Vec { + self.sessions.lock().await.clone() + } + + async fn recorded_requested_sessions(&self) -> Vec> { + self.requested_sessions.lock().await.clone() + } + + async fn recorded_websockets(&self) -> Vec> { + self.websockets.lock().await.clone() + } +} + +impl UpstreamConnector for RecordingConnector { + fn connect( + &self, + _auth: LoadedUpstreamAuth, + session: Option, + ) -> BoxFuture<'static, Result> { + let plans = Arc::clone(&self.plans); + let sessions = Arc::clone(&self.sessions); + let requested_sessions = Arc::clone(&self.requested_sessions); + let websockets = Arc::clone(&self.websockets); + Box::pin(async move { + requested_sessions.lock().await.push(session.clone()); + let session = session.unwrap_or_else(new_session_descriptor); + let plan = plans + .lock() + .await + .pop_front() + .expect("planned websocket connection"); + sessions.lock().await.push(session.clone()); + + let (stream, _) = connect_async(plan.server.url()) + .await + .map_err(|_| ThreadlineError::UpstreamWebSocketConnectFailed)?; + + let websocket = Arc::new(LiveUpstreamWebSocket::from_stream(stream)); + websockets.lock().await.push(Arc::downgrade(&websocket)); + + Ok(ConnectedUpstream { + websocket, + session, + turn_state: plan.turn_state, + }) + }) + } +} + +#[derive(Clone)] +struct FailingConnector; + +impl UpstreamConnector for FailingConnector { + fn connect( + &self, + _auth: LoadedUpstreamAuth, + _session: Option, + ) -> BoxFuture<'static, Result> { + Box::pin(async { Err(ThreadlineError::UpstreamWebSocketConnectFailed) }) + } +} + +fn build_test_router( + config: ThreadlineConfig, + connector: Arc, +) -> axum::Router { + build_router_with_services( + config, + ThreadlineServices::new(Arc::new(StaticAuthProvider), connector), + ) +} + +async fn post_responses(app: axum::Router, payload: Value) -> Response { + app.oneshot( + Request::builder() + .method("POST") + .uri("/v1/responses") + .header("content-type", "application/json") + .body(Body::from(payload.to_string())) + .expect("request"), + ) + .await + .expect("response") +} + +async fn post_responses_with_headers( + app: axum::Router, + payload: Value, + headers: &[(&str, &str)], +) -> Response { + let mut builder = Request::builder() + .method("POST") + .uri("/v1/responses") + .header("content-type", "application/json"); + + for (name, value) in headers { + builder = builder.header(*name, *value); + } + + app.oneshot( + builder + .body(Body::from(payload.to_string())) + .expect("request"), + ) + .await + .expect("response") +} + +fn message_text(message: Message) -> String { + match message { + Message::Text(text) => text.to_string(), + other => panic!("expected text message, got {other:?}"), + } +} + +fn new_session_descriptor() -> UpstreamSessionDescriptor { + UpstreamSessionDescriptor { + session_id: Uuid::now_v7().to_string(), + thread_id: Uuid::now_v7().to_string(), + window_id: Uuid::now_v7().to_string(), + turn_state: None, + } +} + +fn assistant_text_completed_event(response_id: &str, text: &str) -> Value { + json!({ + "type": "response.completed", + "response": { + "id": response_id, + "output": [ + { + "type": "message", + "role": "assistant", + "content": [ + { + "type": "output_text", + "text": text + } + ] + } + ] + } + }) +} + +fn no_observable_output_failed_event(response_id: &str) -> Value { + json!({ + "type": "response.failed", + "response": { + "id": response_id, + "status": "failed", + "error": { + "code": "threadline_no_observable_output", + "message": "Response contained no observable output." + } + } + }) +} + +fn assert_codex_unsupported_response_fields_are_absent(payload: &Value) { + for field_name in [ + "max_output_tokens", + "max_tokens", + "max_completion_tokens", + "truncation", + ] { + assert!( + payload.get(field_name).is_none(), + "expected upstream response.create payload to omit {field_name}, got {payload:?}" + ); + } +} + +fn split_sse_frames(body: &str) -> Vec<&str> { + body.split("\n\n") + .filter(|frame| !frame.trim().is_empty()) + .collect() +} + +fn sse_event_and_data(frame: &str) -> (&str, &str) { + let mut event = None; + let mut data = None; + let mut unexpected_lines = Vec::new(); + + for (index, line) in frame.lines().enumerate() { + if let Some(value) = line.strip_prefix("event: ") { + assert!( + event.replace(value).is_none(), + "expected exactly one event line in SSE frame, found duplicate at line {}: {frame}", + index + 1 + ); + continue; + } + + if let Some(value) = line.strip_prefix("data: ") { + assert!( + data.replace(value).is_none(), + "expected compact single-line SSE data payload, found duplicate data line at line {}: {frame}", + index + 1 + ); + continue; + } + + unexpected_lines.push(format!("line {}: {line}", index + 1)); + } + + assert!( + unexpected_lines.is_empty(), + "expected exactly one event line and one compact data line in SSE frame; unexpected lines: {}. Frame: {frame}", + unexpected_lines.join(" | ") + ); + + ( + event.unwrap_or_else(|| panic!("missing event line in SSE frame: {frame}")), + data.unwrap_or_else(|| panic!("missing data line in SSE frame: {frame}")), + ) +} + +fn assert_done_frame(frame: &str) { + assert_eq!( + frame, "data: [DONE]", + "expected a bare downstream DONE frame without an event line" + ); +} + +fn auxiliary_summary_text() -> &'static str { + concat!( + "The conversation has grown too large for the context window and must be compacted now", + "\n\n", + "Your ONLY task right now is to produce a comprehensive summary", + "\n", + "Output your summary wrapped in and tags" + ) +} + +fn manual_summary_text() -> &'static str { + concat!( + "Summarize the conversation history so far, paying special attention to the most recent agent commands and tool results", + "\n\n", + "Structure your summary using the enhanced format provided in the system message", + "\n", + "Include all important tool calls and their results" + ) +} + +fn manual_simple_summary_text() -> &'static str { + concat!( + "Summarize the conversation history so far, paying special attention to the most recent agent commands and tool results", + "\n\n", + "Include all important tool calls and their results" + ) +} + +fn simple_history_context_text() -> &'static str { + "The following is a compressed version of the preceeding history in the current conversation" +} + +fn new_auto_system_summary_text() -> &'static str { + "Your task is to create a comprehensive, detailed summary of the entire conversation that captures all essential information needed to seamlessly continue the work without any loss of context" +} + +fn new_auto_compressed_history_text() -> &'static str { + concat!( + "The following is a compressed version of the preceeding history in the current conversation. ", + "The first message is kept, some history may be truncated after that:" + ) +} + +fn new_auto_final_summary_prompt_text() -> &'static str { + concat!( + "Summarize the conversation history so far, paying special attention to the most recent agent commands and tool results that triggered this summarization. ", + "Structure your summary using the enhanced format provided in the system message.\n", + "Focus particularly on:\n", + "- The specific agent commands/tools that were just executed\n", + "- The results returned from these recent tool calls (truncate if very long but preserve key information)\n", + "- What the agent was actively working on when the token budget was exceeded\n", + "- How these recent operations connect to the overall user goals\n", + "Include all important tool calls and their results as part of the appropriate sections, with special emphasis on the most recent operations." + ) +} + +#[derive(Clone, Copy)] +enum SummaryRequestShape { + Auto, + ManualFull, + ManualSimple, + NewAuto, + NewForeground, +} + +impl SummaryRequestShape { + fn response_id(self) -> &'static str { + match self { + Self::Auto => "response-summary-auto", + Self::ManualFull => "response-summary-manual-full", + Self::ManualSimple => "response-summary-manual-simple", + Self::NewAuto => "response-summary-new-auto", + Self::NewForeground => "response-summary-new-foreground", + } + } + + fn label(self) -> &'static str { + match self { + Self::Auto => "auto", + Self::ManualFull => "manual_full", + Self::ManualSimple => "manual_simple", + Self::NewAuto => "new_auto", + Self::NewForeground => "new_foreground", + } + } + + fn summary_input_items(self) -> Vec { + match self { + Self::Auto => vec![auxiliary_summary_input_item()], + Self::ManualFull => vec![json!({ + "type": "message", + "role": "system", + "content": [ + { + "type": "input_text", + "text": manual_summary_text() + } + ] + })], + Self::ManualSimple => vec![json!({ + "type": "message", + "role": "system", + "content": [ + { + "type": "input_text", + "text": manual_simple_summary_text() + } + ] + })], + Self::NewAuto => vec![ + json!({ + "type": "message", + "role": "system", + "content": [ + { + "type": "input_text", + "text": new_auto_system_summary_text() + } + ] + }), + json!({ + "type": "message", + "role": "user", + "content": [ + { + "type": "input_text", + "text": new_auto_compressed_history_text() + } + ] + }), + json!({ + "type": "message", + "role": "user", + "content": [ + { + "type": "input_text", + "text": new_auto_final_summary_prompt_text() + } + ] + }), + ], + Self::NewForeground => vec![ + json!({ + "type": "message", + "role": "system", + "content": [ + { + "type": "input_text", + "text": new_auto_system_summary_text() + } + ] + }), + json!({ + "type": "message", + "role": "user", + "content": [ + { + "type": "input_text", + "text": new_auto_final_summary_prompt_text() + } + ] + }), + ], + } + } +} + +fn auxiliary_summary_input_item() -> Value { + json!({ + "type": "message", + "role": "system", + "content": [ + { + "type": "input_text", + "text": auxiliary_summary_text() + } + ] + }) +} + +fn quoted_manual_summary_prompt_input_item() -> Value { + json!({ + "type": "message", + "role": "user", + "content": [ + { + "type": "input_text", + "text": concat!( + "Quoted prompt: ", + "Summarize the conversation history so far, paying special attention to the most recent agent commands and tool results", + "\n\n", + "Structure your summary using the enhanced format provided in the system message", + "\n", + "Include all important tool calls and their results" + ) + } + ] + }) +} + +fn simple_history_context_input_item() -> Value { + json!({ + "type": "message", + "role": "system", + "content": [ + { + "type": "input_text", + "text": simple_history_context_text() + } + ] + }) +} + +fn retained_conflict_fallback_summary_input_item() -> Value { + json!({ + "type": "input_text", + "text": manual_summary_text() + }) +} + +fn summary_request_with_input(previous_response_id: Option<&str>, input: Vec) -> Value { + let mut payload = json!({ + "model": "gpt-5.4", + "context_management": { + "type": "compaction", + "compact_threshold": 12345 + }, + "tools": [ + { + "type": "function", + "name": "user_tool", + "description": "User-defined tool", + "parameters": { + "type": "object", + "properties": {}, + "additionalProperties": false + } + }, + { + "type": "function", + "name": "threadline_echo", + "description": "Threadline internal tool", + "parameters": { + "type": "object", + "properties": { + "value": { + "type": "string" + } + }, + "required": ["value"], + "additionalProperties": false + } + } + ], + "input": input + }); + + if let Some(previous_response_id) = previous_response_id { + payload["previous_response_id"] = json!(previous_response_id); + } + + payload +} + +fn auxiliary_summary_request(previous_response_id: Option<&str>) -> Value { + summary_request_with_shape(previous_response_id, SummaryRequestShape::Auto) +} + +fn retained_conflict_fallback_summary_request(previous_response_id: Option<&str>) -> Value { + summary_request_with_input( + previous_response_id, + vec![retained_conflict_fallback_summary_input_item()], + ) +} + +fn summary_request_with_shape( + previous_response_id: Option<&str>, + shape: SummaryRequestShape, +) -> Value { + let mut input = vec![json!({ + "type": "message", + "role": "user", + "content": [ + { + "type": "input_text", + "text": "Continue from the earlier answer." + } + ] + })]; + input.extend(shape.summary_input_items()); + summary_request_with_input(previous_response_id, input) +} + +async fn next_body_chunk( + body_stream: &mut (impl futures_util::Stream> + Unpin), +) -> Bytes { + body_stream + .next() + .await + .expect("expected body chunk before EOF") + .expect("body chunk") +} + +#[tokio::test] +async fn stale_previous_response_id_returns_not_found_without_reconnect_or_conflict() { + let first_server = Arc::new(ScriptedWebSocketServer::start().await); + let second_server = Arc::new(ScriptedWebSocketServer::start().await); + let third_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![ + PlannedConnection { + server: Arc::clone(&first_server), + turn_state: Some("turn-state-1".to_string()), + }, + PlannedConnection { + server: Arc::clone(&second_server), + turn_state: None, + }, + PlannedConnection { + server: Arc::clone(&third_server), + turn_state: None, + }, + ]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector.clone())); + + let first_response = + post_responses(app.clone(), json!({"model":"gpt-5.4","input":"first"})).await; + assert_eq!(first_response.status(), StatusCode::OK); + + let first_payload: Value = serde_json::from_str(&message_text( + first_server + .recv_client_message() + .await + .expect("first request message"), + )) + .expect("first request json"); + assert_eq!(first_payload["type"], "response.create"); + + first_server + .send_text(r#"{"type":"response.created","response":{"id":"response-1"}}"#) + .await; + first_server + .send_text(&assistant_text_completed_event("response-1", "first completion").to_string()) + .await; + let first_body = to_bytes(first_response.into_body(), usize::MAX) + .await + .expect("first body"); + let first_body_text = String::from_utf8(first_body.to_vec()).expect("utf8 body"); + assert!(first_body_text.contains("event: response.created")); + assert!(first_body_text.contains("event: response.completed")); + + first_server.send_close(1000, "done").await; + sleep(Duration::from_millis(50)).await; + + for attempt in ["first", "second"] { + let response = post_responses( + app.clone(), + json!({ + "model":"gpt-5.4", + "input":"second", + "previous_response_id":"response-1" + }), + ) + .await; + assert_eq!( + response.status(), + StatusCode::BAD_REQUEST, + "{attempt} stale continuation should fail before SSE starts" + ); + + let body = to_bytes(response.into_body(), usize::MAX) + .await + .expect("stale body"); + let payload: Value = serde_json::from_slice(&body).expect("stale json body"); + assert_eq!( + payload["error"]["code"], "previous_response_not_found", + "{attempt} stale continuation should require client replay" + ); + assert_ne!( + payload["error"]["code"], "retained_session_conflict", + "{attempt} stale continuation should release the retained lease" + ); + } + + let no_second_connect = timeout( + Duration::from_millis(250), + second_server.recv_client_message(), + ) + .await; + assert!(no_second_connect.is_err()); + + let no_third_connect = timeout( + Duration::from_millis(250), + third_server.recv_client_message(), + ) + .await; + assert!(no_third_connect.is_err()); + + let sessions = connector.recorded_sessions().await; + assert_eq!(sessions.len(), 1); +} + +#[tokio::test] +async fn live_retained_upstream_continuation_forwards_previous_response_id_without_reconnect() { + let retained_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&retained_server), + turn_state: Some("turn-state-1".to_string()), + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector.clone())); + + let first_response = + post_responses(app.clone(), json!({"model":"gpt-5.4","input":"first"})).await; + assert_eq!(first_response.status(), StatusCode::OK); + + let first_payload: Value = serde_json::from_str(&message_text( + retained_server + .recv_client_message() + .await + .expect("first request message"), + )) + .expect("first request json"); + assert_eq!(first_payload["type"], "response.create"); + + retained_server + .send_text(r#"{"type":"response.created","response":{"id":"response-1"}}"#) + .await; + retained_server + .send_text(&assistant_text_completed_event("response-1", "first completion").to_string()) + .await; + let _ = to_bytes(first_response.into_body(), usize::MAX) + .await + .expect("first body"); + + let second_response = post_responses( + app, + json!({ + "model":"gpt-5.4", + "input":"second", + "previous_response_id":"response-1" + }), + ) + .await; + assert_eq!(second_response.status(), StatusCode::OK); + + let second_payload: Value = serde_json::from_str(&message_text( + retained_server + .recv_client_message() + .await + .expect("second request message"), + )) + .expect("second request json"); + assert_eq!(second_payload["type"], "response.create"); + assert!(second_payload.get("response").is_none()); + assert_eq!(second_payload["previous_response_id"], "response-1"); + + retained_server + .send_text(&assistant_text_completed_event("response-2", "second completion").to_string()) + .await; + let _ = to_bytes(second_response.into_body(), usize::MAX) + .await + .expect("second body"); + + let sessions = connector.recorded_sessions().await; + assert_eq!(sessions.len(), 1); +} + +#[tokio::test] +async fn context_management_compaction_does_not_override_stale_marker_semantics() { + let first_server = Arc::new(ScriptedWebSocketServer::start().await); + let second_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![ + PlannedConnection { + server: Arc::clone(&first_server), + turn_state: Some("turn-state-1".to_string()), + }, + PlannedConnection { + server: Arc::clone(&second_server), + turn_state: None, + }, + ]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let first_response = + post_responses(app.clone(), json!({"model":"gpt-5.4","input":"first"})).await; + assert_eq!(first_response.status(), StatusCode::OK); + + let first_payload: Value = serde_json::from_str(&message_text( + first_server + .recv_client_message() + .await + .expect("first request message"), + )) + .expect("first request json"); + assert_eq!(first_payload["type"], "response.create"); + + first_server + .send_text(r#"{"type":"response.created","response":{"id":"response-1"}}"#) + .await; + first_server + .send_text(&assistant_text_completed_event("response-1", "first completion").to_string()) + .await; + let _ = to_bytes(first_response.into_body(), usize::MAX) + .await + .expect("first body"); + + first_server.send_close(1000, "done").await; + sleep(Duration::from_millis(50)).await; + + let second_response = post_responses( + app, + json!({ + "model":"gpt-5.4", + "input":"second", + "previous_response_id":"response-1", + "context_management": { + "type":"compaction", + "compact_threshold": 12345 + }, + "reasoning":{"effort":"high","summary":"auto"}, + "include":["reasoning.encrypted_content"], + "truncation":"auto" + }), + ) + .await; + assert_eq!(second_response.status(), StatusCode::BAD_REQUEST); + let second_body = to_bytes(second_response.into_body(), usize::MAX) + .await + .expect("second body"); + let second_payload: Value = serde_json::from_slice(&second_body).expect("second body json"); + assert_eq!( + second_payload["error"]["code"], + "previous_response_not_found" + ); + + let no_second_connect = timeout( + Duration::from_millis(250), + second_server.recv_client_message(), + ) + .await; + assert!(no_second_connect.is_err()); +} + +#[tokio::test] +async fn context_management_only_ordinary_request_remains_retained_and_normal() { + let retained_server = Arc::new(ScriptedWebSocketServer::start().await); + let auxiliary_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![ + PlannedConnection { + server: Arc::clone(&retained_server), + turn_state: None, + }, + PlannedConnection { + server: Arc::clone(&auxiliary_server), + turn_state: None, + }, + ]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector.clone())); + + let first_response = + post_responses(app.clone(), json!({"model":"gpt-5.4","input":"first"})).await; + assert_eq!(first_response.status(), StatusCode::OK); + + let _ = retained_server + .recv_client_message() + .await + .expect("first request message"); + retained_server + .send_text(r#"{"type":"response.created","response":{"id":"response-1"}}"#) + .await; + retained_server + .send_text(&assistant_text_completed_event("response-1", "first completion").to_string()) + .await; + let _ = to_bytes(first_response.into_body(), usize::MAX) + .await + .expect("first body"); + + let second_response = post_responses( + app, + json!({ + "model":"gpt-5.4", + "input":"second", + "previous_response_id":"response-1", + "context_management": { + "type":"compaction", + "compact_threshold":12345 + } + }), + ) + .await; + assert_eq!(second_response.status(), StatusCode::OK); + + let second_payload: Value = serde_json::from_str(&message_text( + timeout( + Duration::from_millis(250), + retained_server.recv_client_message(), + ) + .await + .expect("retained followup request timeout") + .expect("retained followup request"), + )) + .expect("second request json"); + assert_eq!(second_payload["type"], "response.create"); + assert_eq!(second_payload["previous_response_id"], "response-1"); + assert!(second_payload.get("context_management").is_none()); + + let no_auxiliary_connect = timeout( + Duration::from_millis(250), + auxiliary_server.recv_client_message(), + ) + .await; + assert!(no_auxiliary_connect.is_err()); + + retained_server + .send_text(&assistant_text_completed_event("response-2", "second completion").to_string()) + .await; + let _ = to_bytes(second_response.into_body(), usize::MAX) + .await + .expect("second body"); + + let requested_sessions = connector.recorded_requested_sessions().await; + assert_eq!(requested_sessions.len(), 1); +} + +#[tokio::test] +async fn missing_previous_response_id_returns_stable_not_found() { + let app = build_test_router(ThreadlineConfig::default(), Arc::new(FailingConnector)); + + let response = post_responses( + app, + json!({ + "model":"gpt-5.4", + "input":"missing", + "previous_response_id":"response-missing" + }), + ) + .await; + + assert_eq!(response.status(), StatusCode::BAD_REQUEST); + let body = to_bytes(response.into_body(), usize::MAX) + .await + .expect("body"); + let payload: Value = serde_json::from_slice(&body).expect("json body"); + assert_eq!(payload["error"]["code"], "previous_response_not_found"); +} + +#[tokio::test] +async fn previous_response_not_found_remains_stable_with_simple_history_context_and_context_management() + { + let app = build_test_router(ThreadlineConfig::default(), Arc::new(FailingConnector)); + + let response = post_responses( + app, + summary_request_with_input( + Some("response-missing"), + vec![simple_history_context_input_item()], + ), + ) + .await; + + assert_eq!(response.status(), StatusCode::BAD_REQUEST); + let body = to_bytes(response.into_body(), usize::MAX) + .await + .expect("body"); + let payload: Value = serde_json::from_slice(&body).expect("json body"); + assert_eq!(payload["error"]["code"], "previous_response_not_found"); +} + +#[tokio::test] +async fn summary_request_with_active_previous_response_id_uses_auxiliary_session() { + let retained_server = Arc::new(ScriptedWebSocketServer::start().await); + let summary_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![ + PlannedConnection { + server: Arc::clone(&retained_server), + turn_state: None, + }, + PlannedConnection { + server: Arc::clone(&summary_server), + turn_state: None, + }, + ]); + let app = build_test_router( + ThreadlineConfig { + retained_session_capacity: 1, + ..ThreadlineConfig::default() + }, + Arc::new(connector), + ); + + let initial = post_responses(app.clone(), json!({"model":"gpt-5.4","input":"seed"})).await; + assert_eq!(initial.status(), StatusCode::OK); + let _ = retained_server + .recv_client_message() + .await + .expect("seed request"); + retained_server + .send_text(&assistant_text_completed_event("response-1", "seed completion").to_string()) + .await; + let _ = to_bytes(initial.into_body(), usize::MAX) + .await + .expect("seed body"); + + let active = post_responses( + app.clone(), + json!({ + "model":"gpt-5.4", + "input":"followup", + "previous_response_id":"response-1" + }), + ) + .await; + assert_eq!(active.status(), StatusCode::OK); + let _ = retained_server + .recv_client_message() + .await + .expect("active followup request"); + + let summary = post_responses(app, auxiliary_summary_request(Some("response-1"))).await; + assert_eq!(summary.status(), StatusCode::OK); +} + +#[tokio::test] +async fn summary_request_with_unknown_previous_response_id_uses_auxiliary_session() { + let summary_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&summary_server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let response = post_responses(app, auxiliary_summary_request(Some("response-missing"))).await; + assert_eq!(response.status(), StatusCode::OK); +} + +#[tokio::test] +async fn summary_request_all_shapes_with_active_previous_response_id_use_auxiliary_session() { + for shape in [ + SummaryRequestShape::Auto, + SummaryRequestShape::ManualFull, + SummaryRequestShape::ManualSimple, + SummaryRequestShape::NewAuto, + SummaryRequestShape::NewForeground, + ] { + let retained_server = Arc::new(ScriptedWebSocketServer::start().await); + let summary_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![ + PlannedConnection { + server: Arc::clone(&retained_server), + turn_state: None, + }, + PlannedConnection { + server: Arc::clone(&summary_server), + turn_state: None, + }, + ]); + let app = build_test_router( + ThreadlineConfig { + retained_session_capacity: 1, + ..ThreadlineConfig::default() + }, + Arc::new(connector.clone()), + ); + + let initial = post_responses(app.clone(), json!({"model":"gpt-5.4","input":"seed"})).await; + assert_eq!( + initial.status(), + StatusCode::OK, + "seed status for {}", + shape.label() + ); + let _ = retained_server + .recv_client_message() + .await + .expect("seed request"); + retained_server + .send_text(&assistant_text_completed_event("response-1", "seed completion").to_string()) + .await; + let _ = to_bytes(initial.into_body(), usize::MAX) + .await + .expect("seed body"); + + let active = post_responses( + app.clone(), + json!({ + "model":"gpt-5.4", + "input":"followup", + "previous_response_id":"response-1" + }), + ) + .await; + assert_eq!( + active.status(), + StatusCode::OK, + "active status for {}", + shape.label() + ); + let _ = retained_server + .recv_client_message() + .await + .expect("active followup request"); + + let summary = + post_responses(app, summary_request_with_shape(Some("response-1"), shape)).await; + assert_eq!( + summary.status(), + StatusCode::OK, + "summary status for {}", + shape.label() + ); + + let summary_payload: Value = serde_json::from_str(&message_text( + summary_server + .recv_client_message() + .await + .expect("summary request"), + )) + .expect("summary request json"); + assert!( + summary_payload.get("previous_response_id").is_none(), + "previous_response_id should be omitted for {}", + shape.label() + ); + + let requested_sessions = connector.recorded_requested_sessions().await; + assert_eq!( + requested_sessions.len(), + 2, + "session count for {}", + shape.label() + ); + assert!( + requested_sessions[1].is_none(), + "summary transient connect should use session None for {}", + shape.label() + ); + } +} + +#[tokio::test] +async fn header_classified_summary_with_active_previous_response_id_routes_transiently_without_retained_conflict() + { + let retained_server = Arc::new(ScriptedWebSocketServer::start().await); + let summary_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![ + PlannedConnection { + server: Arc::clone(&retained_server), + turn_state: None, + }, + PlannedConnection { + server: Arc::clone(&summary_server), + turn_state: None, + }, + ]); + let app = build_test_router( + ThreadlineConfig { + retained_session_capacity: 1, + jobs_enabled: true, + ..ThreadlineConfig::default() + }, + Arc::new(connector.clone()), + ); + + let initial = post_responses(app.clone(), json!({"model":"gpt-5.4","input":"seed"})).await; + assert_eq!(initial.status(), StatusCode::OK); + let _ = retained_server + .recv_client_message() + .await + .expect("seed request"); + retained_server + .send_text(&assistant_text_completed_event("response-1", "seed completion").to_string()) + .await; + let _ = to_bytes(initial.into_body(), usize::MAX) + .await + .expect("seed body"); + + let active = post_responses( + app.clone(), + json!({ + "model":"gpt-5.4", + "input":"followup", + "previous_response_id":"response-1" + }), + ) + .await; + assert_eq!(active.status(), StatusCode::OK); + let _ = retained_server + .recv_client_message() + .await + .expect("active followup request"); + + let response = post_responses_with_headers( + app, + json!({ + "model":"gpt-5.4", + "input":"ordinary header classified summary", + "previous_response_id":"response-1", + "context_management":{ + "type":"compaction", + "compact_threshold":12345 + }, + "tools":[ + { + "type":"function", + "name":"user_tool", + "description":"User tool", + "parameters":{"type":"object"} + }, + { + "type":"function", + "name":"threadline_start_job", + "description":"Threadline job tool", + "parameters":{"type":"object"} + } + ] + }), + &[("x-interaction-type", " conversation-compaction ")], + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let body_task = tokio::spawn(async move { + to_bytes(response.into_body(), usize::MAX) + .await + .expect("body bytes") + }); + + let request_payload: Value = serde_json::from_str(&message_text( + summary_server + .recv_client_message() + .await + .expect("summary request"), + )) + .expect("summary request json"); + assert_eq!(request_payload["type"], "response.create"); + assert!(request_payload.get("previous_response_id").is_none()); + assert!( + request_payload.get("context_management").is_none(), + "header-classified AuxiliarySummary request should omit upstream context_management" + ); + let tools = request_payload["tools"].as_array().expect("tools array"); + assert!(tools.iter().any(|tool| tool["name"] == "user_tool")); + assert!(!tools.iter().any(|tool| { + tool["name"] + .as_str() + .is_some_and(|name| name.starts_with("threadline_")) + })); + + let requested_sessions = connector.recorded_requested_sessions().await; + assert_eq!(requested_sessions.len(), 2); + assert!(requested_sessions[1].is_none()); + + summary_server + .send_text( + r#"{"type":"response.output_item.done","output_index":0,"item":{"type":"function_call","call_id":"call-job","name":"threadline_start_job","arguments":"{\"command\":[\"echo\",\"hello\"]}"}}"#, + ) + .await; + summary_server + .send_text(r#"{"type":"response.completed","response":{"id":"response-header-summary"}}"#) + .await; + + let maybe_followup = tokio::time::timeout( + Duration::from_millis(100), + summary_server.recv_client_message(), + ) + .await; + assert!( + !matches!(maybe_followup, Ok(Some(_))), + "expected header-classified AuxiliarySummary request to avoid internal tool follow-up traffic" + ); + + let body = body_task.await.expect("body task"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + assert!(!body_text.contains("threadline_start_job")); +} + +#[tokio::test] +async fn summary_request_omits_previous_response_id_and_context_management_upstream() { + let summary_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&summary_server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let response = post_responses(app, auxiliary_summary_request(Some("response-1"))).await; + assert_eq!(response.status(), StatusCode::OK); + + let payload: Value = serde_json::from_str(&message_text( + summary_server + .recv_client_message() + .await + .expect("summary request"), + )) + .expect("summary request json"); + assert_eq!(payload["type"], "response.create"); + assert!(payload.get("previous_response_id").is_none()); + assert!( + payload.get("context_management").is_none(), + "AuxiliarySummary request should omit upstream context_management" + ); + let tools = payload["tools"].as_array().expect("tools array"); + assert!(tools.iter().any(|tool| tool["name"] == "user_tool")); + assert!(!tools.iter().any(|tool| tool["name"] == "threadline_echo")); +} + +#[tokio::test] +async fn summary_request_all_shapes_omit_previous_response_id_context_management_and_threadline_tools_upstream() + { + for shape in [ + SummaryRequestShape::Auto, + SummaryRequestShape::ManualFull, + SummaryRequestShape::ManualSimple, + SummaryRequestShape::NewAuto, + SummaryRequestShape::NewForeground, + ] { + let summary_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&summary_server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let response = + post_responses(app, summary_request_with_shape(Some("response-1"), shape)).await; + assert_eq!( + response.status(), + StatusCode::OK, + "response status for {}", + shape.label() + ); + + let payload: Value = serde_json::from_str(&message_text( + summary_server + .recv_client_message() + .await + .expect("summary request"), + )) + .expect("summary request json"); + assert_eq!( + payload["type"], + "response.create", + "payload type for {}", + shape.label() + ); + assert!( + payload.get("previous_response_id").is_none(), + "previous_response_id should be omitted for {}", + shape.label() + ); + assert!( + payload.get("context_management").is_none(), + "context_management should be omitted for {}", + shape.label() + ); + let tools = payload["tools"].as_array().expect("tools array"); + assert!( + tools.iter().any(|tool| tool["name"] == "user_tool"), + "user tool should remain for {}", + shape.label() + ); + assert!( + !tools.iter().any(|tool| tool["name"] == "threadline_echo"), + "threadline tool should be stripped for {}", + shape.label() + ); + } +} + +#[tokio::test] +async fn summary_request_with_context_management_strips_context_management_and_previous_response_id_upstream() + { + let summary_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&summary_server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let response = post_responses(app, auxiliary_summary_request(Some("response-1"))).await; + assert_eq!(response.status(), StatusCode::OK); + + let forwarded: Value = serde_json::from_str(&message_text( + summary_server + .recv_client_message() + .await + .expect("summary request"), + )) + .expect("summary request json"); + assert!(forwarded.get("previous_response_id").is_none()); + assert!(forwarded.get("context_management").is_none()); + let tools = forwarded["tools"].as_array().expect("tools array"); + assert!(tools.iter().any(|tool| tool["name"] == "user_tool")); + assert!(!tools.iter().any(|tool| tool["name"] == "threadline_echo")); +} + +#[tokio::test] +async fn context_management_stripped_diagnostics_are_privacy_safe() { + let trace_guard = TraceCaptureGuard::begin().await; + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + let raw_context_secret = "secret-context-123"; + let raw_prompt = "sensitive-user-prompt"; + + let response = post_responses( + app, + json!({ + "model":"gpt-5.4", + "input": raw_prompt, + "context_management": { + "type":"compaction", + "compact_threshold":12345, + "note": raw_context_secret + } + }), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let request_payload: Value = serde_json::from_str(&message_text( + server.recv_client_message().await.expect("request message"), + )) + .expect("request json"); + assert!(request_payload.get("context_management").is_none()); + + server + .send_text(&assistant_text_completed_event("response-context-stripped", "done").to_string()) + .await; + let _ = to_bytes(response.into_body(), usize::MAX) + .await + .expect("response body"); + + let logs = trace_guard.logs(); + let stripped_line = logs + .lines() + .find(|line| { + line.contains("context_management_stripped") + && line.contains("client_compaction_only=true") + && (line.contains("route_kind=\"normal\"") || line.contains("route_kind=normal")) + }) + .expect("context management stripped trace line"); + + assert!(stripped_line.contains("client_compaction_only=true")); + assert!(stripped_line.contains("route_kind=")); + assert!( + stripped_line.contains("request_class=\"normal\"") + || stripped_line.contains("request_class=normal"), + "unexpected stripped diagnostics line: {stripped_line}" + ); + assert!(!stripped_line.contains(raw_context_secret)); + assert!(!stripped_line.contains(raw_prompt)); + assert!(!stripped_line.contains("compact_threshold")); + assert!(!stripped_line.contains("note")); + assert!(!stripped_line.contains("{\"model\":\"gpt-5.4\"")); +} + +#[tokio::test] +async fn summary_request_without_previous_response_id_uses_auxiliary_session() { + let summary_server = Arc::new(ScriptedWebSocketServer::start().await); + let ordinary_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![ + PlannedConnection { + server: Arc::clone(&summary_server), + turn_state: None, + }, + PlannedConnection { + server: Arc::clone(&ordinary_server), + turn_state: None, + }, + ]); + let app = build_test_router( + ThreadlineConfig { + retained_session_capacity: 1, + ..ThreadlineConfig::default() + }, + Arc::new(connector), + ); + + let summary = post_responses(app.clone(), auxiliary_summary_request(None)).await; + assert_eq!(summary.status(), StatusCode::OK); + let summary_payload: Value = serde_json::from_str(&message_text( + summary_server + .recv_client_message() + .await + .expect("summary request"), + )) + .expect("summary request json"); + assert!(summary_payload.get("previous_response_id").is_none()); + + let ordinary = post_responses(app, json!({"model":"gpt-5.4","input":"ordinary"})).await; + assert_eq!(ordinary.status(), StatusCode::OK); +} + +#[tokio::test] +async fn request_routing_diagnostics_distinguish_summary_without_logging_raw_request_content() { + let trace_guard = TraceCaptureGuard::begin().await; + let summary_server = Arc::new(ScriptedWebSocketServer::start().await); + let normal_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![ + PlannedConnection { + server: Arc::clone(&summary_server), + turn_state: None, + }, + PlannedConnection { + server: Arc::clone(&normal_server), + turn_state: None, + }, + ]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + let raw_request_secret = "secret-123"; + let raw_request_account = "acct_123456789"; + + let summary = post_responses(app.clone(), auxiliary_summary_request(Some("response-1"))).await; + assert_eq!(summary.status(), StatusCode::OK); + let _ = summary_server + .recv_client_message() + .await + .expect("summary request"); + summary_server + .send_text( + &assistant_text_completed_event("response-summary-diagnostics", "summary completion") + .to_string(), + ) + .await; + let _ = to_bytes(summary.into_body(), usize::MAX) + .await + .expect("summary body"); + + let normal = post_responses( + app, + json!({ + "model":"gpt-5.4", + "input": format!("Account {raw_request_account} credential {raw_request_secret}") + }), + ) + .await; + assert_eq!(normal.status(), StatusCode::OK); + let _ = normal_server + .recv_client_message() + .await + .expect("normal request"); + normal_server + .send_text( + &assistant_text_completed_event("response-normal-diagnostics", "normal completion") + .to_string(), + ) + .await; + let _ = to_bytes(normal.into_body(), usize::MAX) + .await + .expect("normal body"); + + let logs = trace_guard.logs(); + let summary_line = logs + .lines() + .find(|line| { + line.contains("responses_request_routed") + && line.contains("request_class=\"auxiliary_summary\"") + && line.contains("previous_response_id_present=true") + }) + .expect("summary routing diagnostics trace line"); + assert!(summary_line.contains("context_management_present=true")); + assert!(summary_line.contains("interaction_type=\"none\"")); + assert!(summary_line.contains("interaction_type_compaction_hit=false")); + assert!(summary_line.contains("tool_choice=\"none\"")); + assert!(summary_line.contains("tools_count=2")); + assert!(summary_line.contains("input_item_count=2")); + assert!(summary_line.contains("last_input_role=\"system\"")); + assert!(summary_line.contains("last_input_type=\"message\"")); + assert!(summary_line.contains("manual_summary_prompt_hit=false")); + assert!(summary_line.contains("manual_structure_instruction_hit=false")); + assert!(summary_line.contains("manual_tool_results_instruction_hit=false")); + assert!(summary_line.contains("auto_context_too_large_hit=true")); + assert!(summary_line.contains("auto_summary_tags_hit=true")); + assert!(summary_line.contains("auto_only_task_hit=true")); + assert!(summary_line.contains("simple_history_context_hit=false")); + assert!(summary_line.contains("new_auto_detailed_summary_hit=false")); + assert!(summary_line.contains("new_auto_user_history_hit=false")); + assert!(summary_line.contains("new_auto_user_final_summary_prompt_hit=false")); + assert!(summary_line.contains("summary_instruction_like_hit=true")); + assert!(!summary_line.contains("response-1")); + assert!(!summary_line.contains(auxiliary_summary_text())); + assert!(!summary_line.contains("{\"model\":\"gpt-5.4\"")); + + let normal_line = logs + .lines() + .find(|line| { + line.contains("responses_request_routed") + && line.contains("request_class=\"normal\"") + && line.contains("previous_response_id_present=false") + && line.contains("context_management_present=false") + && line.contains("tools_count=0") + && line.contains("input_item_count=1") + && line.contains("last_input_type=\"string\"") + }) + .expect("normal routing diagnostics trace line"); + assert!(normal_line.contains("previous_response_id_present=false")); + assert!(normal_line.contains("context_management_present=false")); + assert!(normal_line.contains("interaction_type=\"none\"")); + assert!(normal_line.contains("interaction_type_compaction_hit=false")); + assert!(normal_line.contains("tool_choice=\"none\"")); + assert!(normal_line.contains("tools_count=0")); + assert!(normal_line.contains("input_item_count=1")); + assert!(normal_line.contains("last_input_role=\"none\"")); + assert!(normal_line.contains("last_input_type=\"string\"")); + assert!(normal_line.contains("manual_summary_prompt_hit=false")); + assert!(normal_line.contains("manual_structure_instruction_hit=false")); + assert!(normal_line.contains("manual_tool_results_instruction_hit=false")); + assert!(normal_line.contains("auto_context_too_large_hit=false")); + assert!(normal_line.contains("auto_summary_tags_hit=false")); + assert!(normal_line.contains("auto_only_task_hit=false")); + assert!(normal_line.contains("simple_history_context_hit=false")); + assert!(normal_line.contains("new_auto_detailed_summary_hit=false")); + assert!(normal_line.contains("new_auto_user_history_hit=false")); + assert!(normal_line.contains("new_auto_user_final_summary_prompt_hit=false")); + assert!(normal_line.contains("summary_instruction_like_hit=false")); + assert!(!normal_line.contains(raw_request_secret)); + assert!(!normal_line.contains(raw_request_account)); + assert!(!normal_line.contains("{\"model\":\"gpt-5.4\"")); +} + +#[tokio::test] +async fn header_classified_request_routing_diagnostics_are_privacy_safe() { + let trace_guard = TraceCaptureGuard::begin().await; + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + let raw_request_secret = "secret-789"; + let raw_request_header = "conversation-compaction"; + + let response = post_responses_with_headers( + app, + json!({ + "model":"gpt-5.4", + "input": format!("ordinary request body {raw_request_secret}"), + "previous_response_id":"response-1", + "context_management":{ + "type":"compaction", + "compact_threshold":12345 + } + }), + &[("x-interaction-type", raw_request_header)], + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + let _ = server.recv_client_message().await.expect("summary request"); + server + .send_text( + &assistant_text_completed_event("response-header-diagnostics", "summary completion") + .to_string(), + ) + .await; + let _ = to_bytes(response.into_body(), usize::MAX) + .await + .expect("response body"); + + let logs = trace_guard.logs(); + let routed_line = logs + .lines() + .find(|line| { + line.contains("responses_request_routed") + && line.contains("request_class=\"auxiliary_summary\"") + && line.contains("interaction_type=\"conversation_compaction\"") + }) + .expect("header routing diagnostics trace line"); + assert!(routed_line.contains("interaction_type_compaction_hit=true")); + assert!(routed_line.contains("previous_response_id_present=true")); + assert!(routed_line.contains("context_management_present=true")); + assert!(routed_line.contains("manual_summary_prompt_hit=false")); + assert!(routed_line.contains("summary_instruction_like_hit=false")); + assert!(!routed_line.contains(raw_request_secret)); + assert!(!routed_line.contains(raw_request_header)); + assert!(!routed_line.contains("response-1")); + assert!(!routed_line.contains("{\"model\":\"gpt-5.4\"")); +} + +#[tokio::test] +async fn summary_response_id_is_not_registered_as_continuation_marker() { + let summary_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&summary_server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let summary = post_responses(app.clone(), auxiliary_summary_request(Some("response-1"))).await; + assert_eq!(summary.status(), StatusCode::OK); + let _ = summary_server + .recv_client_message() + .await + .expect("summary request"); + summary_server + .send_text(r#"{"type":"response.completed","response":{"id":"response-summary"}}"#) + .await; + let _ = to_bytes(summary.into_body(), usize::MAX) + .await + .expect("summary body"); + + let rejected = post_responses( + app, + json!({ + "model":"gpt-5.4", + "input":"resume", + "previous_response_id":"response-summary" + }), + ) + .await; + assert_eq!(rejected.status(), StatusCode::BAD_REQUEST); + let body = to_bytes(rejected.into_body(), usize::MAX) + .await + .expect("rejected body"); + let payload: Value = serde_json::from_slice(&body).expect("rejected json body"); + assert_eq!(payload["error"]["code"], "previous_response_not_found"); +} + +#[tokio::test] +async fn summary_request_all_shape_response_ids_are_not_registered_as_continuation_markers() { + for shape in [ + SummaryRequestShape::Auto, + SummaryRequestShape::ManualFull, + SummaryRequestShape::ManualSimple, + SummaryRequestShape::NewAuto, + SummaryRequestShape::NewForeground, + ] { + let summary_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&summary_server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let summary = post_responses( + app.clone(), + summary_request_with_shape(Some("response-1"), shape), + ) + .await; + assert_eq!( + summary.status(), + StatusCode::OK, + "summary status for {}", + shape.label() + ); + let _ = summary_server + .recv_client_message() + .await + .expect("summary request"); + summary_server + .send_text(&format!( + "{{\"type\":\"response.completed\",\"response\":{{\"id\":\"{}\"}}}}", + shape.response_id() + )) + .await; + let _ = to_bytes(summary.into_body(), usize::MAX) + .await + .expect("summary body"); + + let rejected = post_responses( + app, + json!({ + "model":"gpt-5.4", + "input":"resume", + "previous_response_id":shape.response_id() + }), + ) + .await; + assert_eq!( + rejected.status(), + StatusCode::BAD_REQUEST, + "rejected status for {}", + shape.label() + ); + let body = to_bytes(rejected.into_body(), usize::MAX) + .await + .expect("rejected body"); + let payload: Value = serde_json::from_slice(&body).expect("rejected json body"); + assert_eq!( + payload["error"]["code"], + "previous_response_not_found", + "error code for {}", + shape.label() + ); + } +} + +#[tokio::test] +async fn summary_request_broader_shapes_with_active_previous_response_id_keep_narrow_fallback_contract() + { + for (name, input, expect_reroute) in [ + ( + "quote_only", + vec![quoted_manual_summary_prompt_input_item()], + false, + ), + ( + "simple_history_only", + vec![simple_history_context_input_item()], + false, + ), + ( + "quote_plus_simple_history", + vec![ + simple_history_context_input_item(), + quoted_manual_summary_prompt_input_item(), + ], + false, + ), + ] { + let retained_server = Arc::new(ScriptedWebSocketServer::start().await); + let summary_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![ + PlannedConnection { + server: Arc::clone(&retained_server), + turn_state: None, + }, + PlannedConnection { + server: Arc::clone(&summary_server), + turn_state: None, + }, + ]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let initial = post_responses(app.clone(), json!({"model":"gpt-5.4","input":"seed"})).await; + assert_eq!(initial.status(), StatusCode::OK, "seed status for {name}"); + let _ = retained_server + .recv_client_message() + .await + .expect("seed request"); + retained_server + .send_text(&assistant_text_completed_event("response-1", "seed completion").to_string()) + .await; + let _ = to_bytes(initial.into_body(), usize::MAX) + .await + .expect("seed body"); + + let active = post_responses( + app.clone(), + json!({ + "model":"gpt-5.4", + "input":"followup", + "previous_response_id":"response-1" + }), + ) + .await; + assert_eq!(active.status(), StatusCode::OK, "active status for {name}"); + let _ = retained_server + .recv_client_message() + .await + .expect("active followup request"); + + let response = + post_responses(app, summary_request_with_input(Some("response-1"), input)).await; + if expect_reroute { + assert_eq!( + response.status(), + StatusCode::OK, + "rerouted status for {name}" + ); + + let payload: Value = serde_json::from_str(&message_text( + summary_server + .recv_client_message() + .await + .expect("broader summary request"), + )) + .expect("broader summary request json"); + assert_eq!( + payload["type"], "response.create", + "request type for {name}" + ); + assert!( + payload.get("previous_response_id").is_none(), + "previous_response_id omitted for {name}" + ); + assert!( + payload.get("context_management").is_none(), + "context_management omitted for {name}" + ); + let tools = payload["tools"].as_array().expect("tools array"); + assert!( + tools.iter().any(|tool| tool["name"] == "user_tool"), + "user tool retained for {name}" + ); + assert!( + !tools.iter().any(|tool| tool["name"] == "threadline_echo"), + "threadline tool stripped for {name}" + ); + + summary_server + .send_text( + &assistant_text_completed_event( + "response-broader-summary", + "summary completion", + ) + .to_string(), + ) + .await; + let _ = to_bytes(response.into_body(), usize::MAX) + .await + .expect("rerouted body"); + } else { + assert_eq!( + response.status(), + StatusCode::CONFLICT, + "conflict status for {name}" + ); + let body = to_bytes(response.into_body(), usize::MAX) + .await + .expect("conflict body"); + let payload: Value = serde_json::from_slice(&body).expect("conflict json body"); + assert_eq!( + payload["error"]["code"], "retained_session_conflict", + "conflict code for {name}" + ); + } + } +} + +#[tokio::test] +async fn transient_summary_request_preserves_auxiliary_behavior_but_does_not_revive_stale_marker() { + let retained_server = Arc::new(ScriptedWebSocketServer::start().await); + let summary_server = Arc::new(ScriptedWebSocketServer::start().await); + let resumed_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![ + PlannedConnection { + server: Arc::clone(&retained_server), + turn_state: Some("turn-state-1".to_string()), + }, + PlannedConnection { + server: Arc::clone(&summary_server), + turn_state: None, + }, + PlannedConnection { + server: Arc::clone(&resumed_server), + turn_state: None, + }, + ]); + let app = build_test_router( + ThreadlineConfig { + retained_session_capacity: 1, + ..ThreadlineConfig::default() + }, + Arc::new(connector), + ); + + let seed = post_responses(app.clone(), json!({"model":"gpt-5.4","input":"seed"})).await; + assert_eq!(seed.status(), StatusCode::OK); + let _ = retained_server + .recv_client_message() + .await + .expect("seed request"); + retained_server + .send_text(&assistant_text_completed_event("response-1", "seed completion").to_string()) + .await; + let _ = to_bytes(seed.into_body(), usize::MAX) + .await + .expect("seed body"); + retained_server.send_close(1000, "seed complete").await; + sleep(Duration::from_millis(50)).await; + + let summary = post_responses(app.clone(), auxiliary_summary_request(Some("response-1"))).await; + assert_eq!(summary.status(), StatusCode::OK); + let _ = summary_server + .recv_client_message() + .await + .expect("summary request"); + summary_server + .send_text( + &assistant_text_completed_event("response-summary", "summary completion").to_string(), + ) + .await; + let _ = to_bytes(summary.into_body(), usize::MAX) + .await + .expect("summary body"); + + let resumed = post_responses( + app, + json!({ + "model":"gpt-5.4", + "input":"resume", + "previous_response_id":"response-1" + }), + ) + .await; + assert_eq!(resumed.status(), StatusCode::BAD_REQUEST); + let resumed_body = to_bytes(resumed.into_body(), usize::MAX) + .await + .expect("resumed body"); + let resumed_payload: Value = serde_json::from_slice(&resumed_body).expect("resumed body json"); + assert_eq!( + resumed_payload["error"]["code"], + "previous_response_not_found" + ); + + let no_resume_connect = timeout( + Duration::from_millis(250), + resumed_server.recv_client_message(), + ) + .await; + assert!(no_resume_connect.is_err()); +} + +#[tokio::test] +async fn transient_summary_request_uses_no_retained_capacity_after_completion() { + let summary_server = Arc::new(ScriptedWebSocketServer::start().await); + let ordinary_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![ + PlannedConnection { + server: Arc::clone(&summary_server), + turn_state: None, + }, + PlannedConnection { + server: Arc::clone(&ordinary_server), + turn_state: None, + }, + ]); + let app = build_test_router( + ThreadlineConfig { + retained_session_capacity: 1, + ..ThreadlineConfig::default() + }, + Arc::new(connector), + ); + + let summary = post_responses(app.clone(), auxiliary_summary_request(None)).await; + assert_eq!(summary.status(), StatusCode::OK); + let _ = summary_server + .recv_client_message() + .await + .expect("summary request"); + summary_server + .send_text(r#"{"type":"response.completed","response":{"id":"response-summary"}}"#) + .await; + let _ = to_bytes(summary.into_body(), usize::MAX) + .await + .expect("summary body"); + + let ordinary = post_responses(app, json!({"model":"gpt-5.4","input":"ordinary"})).await; + assert_eq!(ordinary.status(), StatusCode::OK); +} + +#[tokio::test] +async fn transient_summary_request_can_run_while_previous_marker_is_active_at_capacity() { + let retained_server = Arc::new(ScriptedWebSocketServer::start().await); + let summary_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![ + PlannedConnection { + server: Arc::clone(&retained_server), + turn_state: None, + }, + PlannedConnection { + server: Arc::clone(&summary_server), + turn_state: None, + }, + ]); + let app = build_test_router( + ThreadlineConfig { + retained_session_capacity: 1, + ..ThreadlineConfig::default() + }, + Arc::new(connector), + ); + + let seed = post_responses(app.clone(), json!({"model":"gpt-5.4","input":"seed"})).await; + assert_eq!(seed.status(), StatusCode::OK); + let _ = retained_server + .recv_client_message() + .await + .expect("seed request"); + retained_server + .send_text(&assistant_text_completed_event("response-1", "seed completion").to_string()) + .await; + let _ = to_bytes(seed.into_body(), usize::MAX) + .await + .expect("seed body"); + + let active = post_responses( + app.clone(), + json!({ + "model":"gpt-5.4", + "input":"followup", + "previous_response_id":"response-1" + }), + ) + .await; + assert_eq!(active.status(), StatusCode::OK); + let _ = retained_server + .recv_client_message() + .await + .expect("active followup request"); + + let summary = post_responses(app, auxiliary_summary_request(Some("response-1"))).await; + assert_eq!(summary.status(), StatusCode::OK); +} + +#[tokio::test] +async fn transient_summary_request_failure_or_drop_does_not_leave_capacity_blocked() { + let failed_server = Arc::new(ScriptedWebSocketServer::start().await); + let ordinary_after_failed_server = Arc::new(ScriptedWebSocketServer::start().await); + let failure_connector = RecordingConnector::new(vec![ + PlannedConnection { + server: Arc::clone(&failed_server), + turn_state: None, + }, + PlannedConnection { + server: Arc::clone(&ordinary_after_failed_server), + turn_state: None, + }, + ]); + let failure_app = build_test_router( + ThreadlineConfig { + retained_session_capacity: 1, + ..ThreadlineConfig::default() + }, + Arc::new(failure_connector), + ); + + let failed_summary = post_responses(failure_app.clone(), auxiliary_summary_request(None)).await; + assert_eq!(failed_summary.status(), StatusCode::OK); + let _ = failed_server + .recv_client_message() + .await + .expect("failed summary request"); + failed_server + .send_text(r#"{"type":"response.failed","response":{"id":"response-summary"},"error":{"code":"upstream_response_failed","message":"failed"}}"#) + .await; + let _ = to_bytes(failed_summary.into_body(), usize::MAX) + .await + .expect("failed summary body"); + + let ordinary_after_failed = post_responses( + failure_app, + json!({"model":"gpt-5.4","input":"ordinary-after-failed"}), + ) + .await; + assert_eq!(ordinary_after_failed.status(), StatusCode::OK); + + let dropped_server = Arc::new(ScriptedWebSocketServer::start().await); + let ordinary_after_drop_server = Arc::new(ScriptedWebSocketServer::start().await); + let drop_connector = RecordingConnector::new(vec![ + PlannedConnection { + server: Arc::clone(&dropped_server), + turn_state: None, + }, + PlannedConnection { + server: Arc::clone(&ordinary_after_drop_server), + turn_state: None, + }, + ]); + let drop_app = build_test_router( + ThreadlineConfig { + retained_session_capacity: 1, + ..ThreadlineConfig::default() + }, + Arc::new(drop_connector), + ); + + let dropped_summary = post_responses(drop_app.clone(), auxiliary_summary_request(None)).await; + assert_eq!(dropped_summary.status(), StatusCode::OK); + let _ = dropped_server + .recv_client_message() + .await + .expect("dropped summary request"); + drop(dropped_summary); + sleep(Duration::from_millis(50)).await; + + let ordinary_after_drop = post_responses( + drop_app, + json!({"model":"gpt-5.4","input":"ordinary-after-drop"}), + ) + .await; + assert_eq!(ordinary_after_drop.status(), StatusCode::OK); +} + +#[tokio::test] +async fn transient_summary_request_terminal_paths_close_pump_or_upstream_handle() { + let completion_server = Arc::new(ScriptedWebSocketServer::start().await); + let completion_connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&completion_server), + turn_state: None, + }]); + let completion_app = build_test_router( + ThreadlineConfig::default(), + Arc::new(completion_connector.clone()), + ); + + let completion_response = post_responses(completion_app, auxiliary_summary_request(None)).await; + assert_eq!(completion_response.status(), StatusCode::OK); + let _ = completion_server + .recv_client_message() + .await + .expect("completion summary request"); + completion_server + .send_text(r#"{"type":"response.completed","response":{"id":"response-summary"}}"#) + .await; + let _ = to_bytes(completion_response.into_body(), usize::MAX) + .await + .expect("completion summary body"); + sleep(Duration::from_millis(50)).await; + let completion_sockets = completion_connector.recorded_websockets().await; + assert!(completion_sockets[0].upgrade().is_none()); + + let failure_server = Arc::new(ScriptedWebSocketServer::start().await); + let failure_connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&failure_server), + turn_state: None, + }]); + let failure_app = build_test_router( + ThreadlineConfig::default(), + Arc::new(failure_connector.clone()), + ); + + let failure_response = post_responses(failure_app, auxiliary_summary_request(None)).await; + assert_eq!(failure_response.status(), StatusCode::OK); + let _ = failure_server + .recv_client_message() + .await + .expect("failure summary request"); + failure_server + .send_text(r#"{"type":"response.failed","response":{"id":"response-summary"},"error":{"code":"upstream_response_failed","message":"failed"}}"#) + .await; + let _ = to_bytes(failure_response.into_body(), usize::MAX) + .await + .expect("failure summary body"); + sleep(Duration::from_millis(50)).await; + let failure_sockets = failure_connector.recorded_websockets().await; + assert!(failure_sockets[0].upgrade().is_none()); + + let drop_server = Arc::new(ScriptedWebSocketServer::start().await); + let drop_connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&drop_server), + turn_state: None, + }]); + let drop_app = build_test_router( + ThreadlineConfig::default(), + Arc::new(drop_connector.clone()), + ); + + let drop_response = post_responses(drop_app, auxiliary_summary_request(None)).await; + assert_eq!(drop_response.status(), StatusCode::OK); + let _ = drop_server + .recv_client_message() + .await + .expect("drop summary request"); + drop(drop_response); + sleep(Duration::from_millis(50)).await; + let drop_sockets = drop_connector.recorded_websockets().await; + assert!(drop_sockets[0].upgrade().is_none()); +} + +#[tokio::test] +async fn concurrent_marker_reuse_returns_conflict_and_client_drop_releases_the_lease() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let initial = post_responses(app.clone(), json!({"model":"gpt-5.4","input":"seed"})).await; + let _ = server.recv_client_message().await.expect("seed request"); + server + .send_text(&assistant_text_completed_event("response-1", "seed completion").to_string()) + .await; + let _ = to_bytes(initial.into_body(), usize::MAX) + .await + .expect("seed body"); + + let active = post_responses( + app.clone(), + json!({ + "model":"gpt-5.4", + "input":"followup", + "previous_response_id":"response-1" + }), + ) + .await; + assert_eq!(active.status(), StatusCode::OK); + let _ = server + .recv_client_message() + .await + .expect("active followup request"); + + let conflict = post_responses( + app.clone(), + json!({ + "model":"gpt-5.4", + "input":"conflict", + "previous_response_id":"response-1" + }), + ) + .await; + assert_eq!(conflict.status(), StatusCode::CONFLICT); + + drop(active); + sleep(Duration::from_millis(50)).await; + + let retried = post_responses( + app, + json!({ + "model":"gpt-5.4", + "input":"retry", + "previous_response_id":"response-1" + }), + ) + .await; + assert_eq!(retried.status(), StatusCode::OK); +} + +#[tokio::test] +async fn retained_session_conflict_fallback_summary_request_reroutes_transiently() { + let retained_server = Arc::new(ScriptedWebSocketServer::start().await); + let summary_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![ + PlannedConnection { + server: Arc::clone(&retained_server), + turn_state: None, + }, + PlannedConnection { + server: Arc::clone(&summary_server), + turn_state: None, + }, + ]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let initial = post_responses(app.clone(), json!({"model":"gpt-5.4","input":"seed"})).await; + assert_eq!(initial.status(), StatusCode::OK); + let _ = retained_server + .recv_client_message() + .await + .expect("seed request"); + retained_server + .send_text(&assistant_text_completed_event("response-1", "seed completion").to_string()) + .await; + let _ = to_bytes(initial.into_body(), usize::MAX) + .await + .expect("seed body"); + + let active = post_responses( + app.clone(), + json!({ + "model":"gpt-5.4", + "input":"followup", + "previous_response_id":"response-1" + }), + ) + .await; + assert_eq!(active.status(), StatusCode::OK); + let _ = retained_server + .recv_client_message() + .await + .expect("active followup request"); + + let summary = post_responses( + app, + retained_conflict_fallback_summary_request(Some("response-1")), + ) + .await; + assert_eq!(summary.status(), StatusCode::OK); + + let payload: Value = serde_json::from_str(&message_text( + summary_server + .recv_client_message() + .await + .expect("fallback summary request"), + )) + .expect("fallback summary request json"); + assert_eq!(payload["type"], "response.create"); + assert!(payload.get("previous_response_id").is_none()); + let tools = payload["tools"].as_array().expect("tools array"); + assert!(tools.iter().any(|tool| tool["name"] == "user_tool")); + assert!(!tools.iter().any(|tool| tool["name"] == "threadline_echo")); + + summary_server + .send_text( + &assistant_text_completed_event("response-fallback-summary", "summary completion") + .to_string(), + ) + .await; + let _ = to_bytes(summary.into_body(), usize::MAX) + .await + .expect("summary body"); +} + +#[tokio::test] +async fn retained_session_conflict_context_management_without_summary_input_remains_conflict() { + let retained_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&retained_server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let initial = post_responses(app.clone(), json!({"model":"gpt-5.4","input":"seed"})).await; + assert_eq!(initial.status(), StatusCode::OK); + let _ = retained_server + .recv_client_message() + .await + .expect("seed request"); + retained_server + .send_text(&assistant_text_completed_event("response-1", "seed completion").to_string()) + .await; + let _ = to_bytes(initial.into_body(), usize::MAX) + .await + .expect("seed body"); + + let active = post_responses( + app.clone(), + json!({ + "model":"gpt-5.4", + "input":"followup", + "previous_response_id":"response-1" + }), + ) + .await; + assert_eq!(active.status(), StatusCode::OK); + let _ = retained_server + .recv_client_message() + .await + .expect("active followup request"); + + let conflict = post_responses( + app, + summary_request_with_input( + Some("response-1"), + vec![json!({ + "type": "message", + "role": "user", + "content": [ + { + "type": "input_text", + "text": "This is ordinary context management content without a summary request." + } + ] + })], + ), + ) + .await; + assert_eq!(conflict.status(), StatusCode::CONFLICT); + let body = to_bytes(conflict.into_body(), usize::MAX) + .await + .expect("conflict body"); + let payload: Value = serde_json::from_slice(&body).expect("conflict json body"); + assert_eq!(payload["error"]["code"], "retained_session_conflict"); +} + +#[tokio::test] +async fn retained_session_conflict_tool_choice_none_without_summary_fingerprint_remains_conflict() { + let retained_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&retained_server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let initial = post_responses(app.clone(), json!({"model":"gpt-5.4","input":"seed"})).await; + assert_eq!(initial.status(), StatusCode::OK); + let _ = retained_server + .recv_client_message() + .await + .expect("seed request"); + retained_server + .send_text(&assistant_text_completed_event("response-1", "seed completion").to_string()) + .await; + let _ = to_bytes(initial.into_body(), usize::MAX) + .await + .expect("seed body"); + + let active = post_responses( + app.clone(), + json!({ + "model":"gpt-5.4", + "input":"followup", + "previous_response_id":"response-1" + }), + ) + .await; + assert_eq!(active.status(), StatusCode::OK); + let _ = retained_server + .recv_client_message() + .await + .expect("active followup request"); + + let conflict = post_responses( + app, + json!({ + "model": "gpt-5.4", + "previous_response_id": "response-1", + "context_management": { + "type": "compaction", + "compact_threshold": 12345 + }, + "tool_choice": "none", + "tools": [ + { + "type": "function", + "name": "user_tool", + "description": "User-defined tool", + "parameters": { + "type": "object", + "properties": {}, + "additionalProperties": false + } + }, + { + "type": "function", + "name": "threadline_echo", + "description": "Threadline internal tool", + "parameters": { + "type": "object", + "properties": { + "value": { + "type": "string" + } + }, + "required": ["value"], + "additionalProperties": false + } + } + ], + "input": [ + { + "type": "input_text", + "text": "Do not summarize this request; continue normal work." + } + ] + }), + ) + .await; + assert_eq!(conflict.status(), StatusCode::CONFLICT); + let body = to_bytes(conflict.into_body(), usize::MAX) + .await + .expect("conflict body"); + let payload: Value = serde_json::from_slice(&body).expect("conflict json body"); + assert_eq!(payload["error"]["code"], "retained_session_conflict"); +} + +#[tokio::test] +async fn retained_session_conflict_rerouted_diagnostics_are_privacy_safe() { + let trace_guard = TraceCaptureGuard::begin().await; + let raw_request_secret = "secret-456"; + let raw_request_account = "acct_987654321"; + let retained_server = Arc::new(ScriptedWebSocketServer::start().await); + let summary_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![ + PlannedConnection { + server: Arc::clone(&retained_server), + turn_state: None, + }, + PlannedConnection { + server: Arc::clone(&summary_server), + turn_state: None, + }, + ]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let initial = post_responses(app.clone(), json!({"model":"gpt-5.4","input":"seed"})).await; + assert_eq!(initial.status(), StatusCode::OK); + let _ = retained_server + .recv_client_message() + .await + .expect("seed request"); + retained_server + .send_text(&assistant_text_completed_event("response-1", "seed completion").to_string()) + .await; + let _ = to_bytes(initial.into_body(), usize::MAX) + .await + .expect("seed body"); + + let active = post_responses( + app.clone(), + json!({ + "model":"gpt-5.4", + "input":"followup", + "previous_response_id":"response-1" + }), + ) + .await; + assert_eq!(active.status(), StatusCode::OK); + let _ = retained_server + .recv_client_message() + .await + .expect("active followup request"); + + let rerouted = post_responses( + app, + summary_request_with_input( + Some("response-1"), + vec![ + json!({ + "type": "message", + "role": "user", + "content": [ + { + "type": "input_text", + "text": format!("Account {raw_request_account} credential {raw_request_secret}") + } + ] + }), + retained_conflict_fallback_summary_input_item(), + ], + ), + ) + .await; + assert_eq!(rerouted.status(), StatusCode::OK); + let _ = summary_server + .recv_client_message() + .await + .expect("fallback summary request"); + summary_server + .send_text( + &assistant_text_completed_event("response-fallback-diagnostics", "summary completion") + .to_string(), + ) + .await; + let _ = to_bytes(rerouted.into_body(), usize::MAX) + .await + .expect("rerouted body"); + + let logs = trace_guard.logs(); + let rerouted_line = logs + .lines() + .rev() + .find(|line| { + line.contains("retained_session_conflict_rerouted") + && line.contains("fallback_summary_input_hit=true") + && line.contains("tools_count=2") + && line.contains("input_item_count=2") + }) + .expect("rerouted diagnostics trace line"); + assert!(rerouted_line.contains("request_class=\"normal\"")); + assert!(rerouted_line.contains("interaction_type=\"none\"")); + assert!(rerouted_line.contains("interaction_type_compaction_hit=false")); + assert!(rerouted_line.contains("previous_response_id_present=true")); + assert!(rerouted_line.contains("context_management_present=true")); + assert!(rerouted_line.contains("manual_summary_prompt_hit=true")); + assert!(rerouted_line.contains("manual_structure_instruction_hit=true")); + assert!(rerouted_line.contains("manual_tool_results_instruction_hit=true")); + assert!(rerouted_line.contains("auto_context_too_large_hit=false")); + assert!(rerouted_line.contains("auto_summary_tags_hit=false")); + assert!(rerouted_line.contains("auto_only_task_hit=false")); + assert!(rerouted_line.contains("simple_history_context_hit=false")); + assert!(rerouted_line.contains("new_auto_detailed_summary_hit=false")); + assert!(rerouted_line.contains("new_auto_user_history_hit=false")); + assert!(rerouted_line.contains("new_auto_user_final_summary_prompt_hit=false")); + assert!(rerouted_line.contains("summary_instruction_like_hit=false")); + assert!(rerouted_line.contains("fallback_summary_input_hit=true")); + assert!(rerouted_line.contains("reroute_reason=\"fallback_summary_input\"")); + assert!(rerouted_line.contains("tool_choice=\"none\"")); + assert!(rerouted_line.contains("tools_count=2")); + assert!(rerouted_line.contains("input_item_count=2")); + assert!(rerouted_line.contains("last_input_role=\"none\"")); + assert!(rerouted_line.contains("last_input_type=\"input_text\"")); + assert!(!rerouted_line.contains(raw_request_secret)); + assert!(!rerouted_line.contains(raw_request_account)); + assert!(!rerouted_line.contains(manual_summary_text())); + assert!(!rerouted_line.contains("response-1")); + assert!(!rerouted_line.contains("{\"model\":\"gpt-5.4\"")); +} + +#[tokio::test] +async fn retained_session_conflict_fallback_empty_completed_output_preserves_auxiliary_behavior() { + let retained_server = Arc::new(ScriptedWebSocketServer::start().await); + let summary_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![ + PlannedConnection { + server: Arc::clone(&retained_server), + turn_state: None, + }, + PlannedConnection { + server: Arc::clone(&summary_server), + turn_state: None, + }, + ]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let initial = post_responses(app.clone(), json!({"model":"gpt-5.4","input":"seed"})).await; + assert_eq!(initial.status(), StatusCode::OK); + let _ = retained_server + .recv_client_message() + .await + .expect("seed request"); + retained_server + .send_text(&assistant_text_completed_event("response-1", "seed completion").to_string()) + .await; + let _ = to_bytes(initial.into_body(), usize::MAX) + .await + .expect("seed body"); + + let active = post_responses( + app.clone(), + json!({ + "model":"gpt-5.4", + "input":"followup", + "previous_response_id":"response-1" + }), + ) + .await; + assert_eq!(active.status(), StatusCode::OK); + let _ = retained_server + .recv_client_message() + .await + .expect("active followup request"); + + let response = post_responses( + app.clone(), + retained_conflict_fallback_summary_request(Some("response-1")), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + let forwarded: Value = serde_json::from_str(&message_text( + summary_server + .recv_client_message() + .await + .expect("fallback summary request"), + )) + .expect("fallback summary request json"); + assert!(forwarded.get("previous_response_id").is_none()); + let tools = forwarded["tools"].as_array().expect("tools array"); + assert!(tools.iter().any(|tool| tool["name"] == "user_tool")); + assert!(!tools.iter().any(|tool| tool["name"] == "threadline_echo")); + + let completed_event = json!({ + "type": "response.completed", + "response": { + "id": "response-fallback-empty" + } + }); + summary_server.send_text(&completed_event.to_string()).await; + + let body = timeout( + Duration::from_secs(2), + to_bytes(response.into_body(), usize::MAX), + ) + .await + .expect("fallback summary body timeout") + .expect("fallback summary body"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + let (event, data) = sse_event_and_data(frames.first().expect("summary completed frame")); + let payload: Value = serde_json::from_str(data).expect("summary completed json"); + + assert_eq!(frames.len(), 2); + assert_eq!(event, "response.completed"); + assert_eq!(payload, completed_event); + assert_done_frame(frames[1]); + + let rejected = post_responses( + app, + json!({ + "model":"gpt-5.4", + "input":"resume", + "previous_response_id":"response-fallback-empty" + }), + ) + .await; + assert_eq!(rejected.status(), StatusCode::BAD_REQUEST); + let rejected_body = to_bytes(rejected.into_body(), usize::MAX) + .await + .expect("rejected body"); + let rejected_payload: Value = serde_json::from_slice(&rejected_body).expect("rejected json"); + assert_eq!( + rejected_payload["error"]["code"], + "previous_response_not_found" + ); +} + +#[tokio::test] +async fn retained_session_capacity_exhaustion_returns_503() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server, + turn_state: None, + }]); + let app = build_test_router( + ThreadlineConfig { + retained_session_capacity: 1, + ..ThreadlineConfig::default() + }, + Arc::new(connector), + ); + + let active = post_responses(app.clone(), json!({"model":"gpt-5.4","input":"first"})).await; + assert_eq!(active.status(), StatusCode::OK); + + let exhausted = post_responses(app, json!({"model":"gpt-5.4","input":"second"})).await; + assert_eq!(exhausted.status(), StatusCode::SERVICE_UNAVAILABLE); + let body = to_bytes(exhausted.into_body(), usize::MAX) + .await + .expect("body"); + let payload: Value = serde_json::from_slice(&body).expect("json body"); + assert_eq!( + payload["error"]["code"], + "retained_session_capacity_exceeded" + ); + + drop(active); +} + +#[tokio::test] +async fn upstream_connect_failure_returns_502() { + let app = build_test_router(ThreadlineConfig::default(), Arc::new(FailingConnector)); + + let response = post_responses(app, json!({"model":"gpt-5.4","input":"connect"})).await; + + assert_eq!(response.status(), StatusCode::BAD_GATEWAY); + let body = to_bytes(response.into_body(), usize::MAX) + .await + .expect("body"); + let payload: Value = serde_json::from_slice(&body).expect("json body"); + assert_eq!( + payload["error"]["code"], + "upstream_websocket_connect_failed" + ); +} + +#[tokio::test] +async fn upstream_pretty_json_is_compacted_before_downstream_sse() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let response = post_responses(app, json!({"model":"gpt-5.4","input":"pretty-delta"})).await; + assert_eq!(response.status(), StatusCode::OK); + let _ = server + .recv_client_message() + .await + .expect("pretty delta request"); + server + .send_text("{\n \"type\": \"response.output_text.delta\",\n \"delta\": \"hello\"\n}") + .await; + server + .send_text(&assistant_text_completed_event("response-1", "hello").to_string()) + .await; + + let body = to_bytes(response.into_body(), usize::MAX) + .await + .expect("body"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + assert_eq!( + frames.len(), + 3, + "expected delta, completed, and bare DONE SSE frames, got body: {body_text}" + ); + + let (event, data) = sse_event_and_data(frames[0]); + let payload: Value = serde_json::from_str(data).expect("delta json"); + let (completed_event, completed_data) = sse_event_and_data(frames[1]); + let completed_payload: Value = serde_json::from_str(completed_data).expect("completed json"); + + assert_eq!(event, "response.output_text.delta"); + assert_eq!( + payload, + json!({"type":"response.output_text.delta","delta":"hello"}) + ); + assert_eq!(completed_event, "response.completed"); + assert_eq!( + completed_payload, + assistant_text_completed_event("response-1", "hello") + ); + + assert_done_frame(frames[2]); +} + +#[tokio::test] +async fn downstream_body_stream_exposes_route_chunk_boundaries() { + let app = axum::Router::new().route( + "/chunks", + axum::routing::get(|| async { + Body::from_stream(stream::iter([ + Ok::<_, std::convert::Infallible>(Bytes::from_static(b"first")), + Ok::<_, std::convert::Infallible>(Bytes::from_static(b"second")), + ])) + }), + ); + + let response = app + .oneshot( + Request::builder() + .method("GET") + .uri("/chunks") + .body(Body::empty()) + .expect("request"), + ) + .await + .expect("response"); + + assert_eq!(response.status(), StatusCode::OK); + + let mut body_stream = response.into_body().into_data_stream(); + let first = next_body_chunk(&mut body_stream).await; + let second = next_body_chunk(&mut body_stream).await; + let third = body_stream.next().await; + + assert_eq!(first, Bytes::from_static(b"first")); + assert_eq!(second, Bytes::from_static(b"second")); + assert!(third.is_none(), "expected EOF after the second body chunk"); +} + +#[tokio::test] +async fn upstream_pretty_response_completed_is_compacted_before_downstream_sse() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let response = post_responses(app, json!({"model":"gpt-5.4","input":"pretty-completed"})).await; + assert_eq!(response.status(), StatusCode::OK); + let _ = server + .recv_client_message() + .await + .expect("pretty completed request"); + let completed_event = assistant_text_completed_event("response-1", "hello from completed"); + server + .send_text(&serde_json::to_string_pretty(&completed_event).expect("pretty completed json")) + .await; + + let body = to_bytes(response.into_body(), usize::MAX) + .await + .expect("body"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + assert_eq!( + frames.len(), + 3, + "expected synthetic delta, completed SSE, and bare DONE frame, got body: {body_text}" + ); + + let (delta_event, delta_data) = sse_event_and_data(frames[0]); + let delta_payload: Value = serde_json::from_str(delta_data).expect("delta json"); + let (event, data) = sse_event_and_data(frames[1]); + let payload: Value = serde_json::from_str(data).expect("completed json"); + + assert_eq!(delta_event, "response.output_text.delta"); + assert_eq!( + delta_payload, + json!({ + "type":"response.output_text.delta", + "delta":"hello from completed", + "output_index":0, + "content_index":0 + }) + ); + assert_eq!(event, "response.completed"); + assert_eq!(payload, completed_event); + + assert_done_frame(frames[2]); +} + +#[tokio::test] +async fn downstream_completed_and_done_are_separate_body_chunks_before_eof() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let response = post_responses(app, json!({"model":"gpt-5.4","input":"chunk-boundary"})).await; + assert_eq!(response.status(), StatusCode::OK); + let _ = server + .recv_client_message() + .await + .expect("chunk-boundary request"); + let completed_event = assistant_text_completed_event("response-1", "hello from completed"); + server + .send_text(&serde_json::to_string_pretty(&completed_event).expect("pretty completed json")) + .await; + + let mut body_stream = response.into_body().into_data_stream(); + let first = next_body_chunk(&mut body_stream).await; + let first_text = String::from_utf8(first.to_vec()).expect("utf8 first chunk"); + assert!( + !first_text.contains("data: [DONE]"), + "expected the first synthetic delta chunk to exclude the bare DONE sentinel" + ); + let (event, data) = sse_event_and_data(first_text.trim_end()); + let payload: Value = serde_json::from_str(data).expect("delta json"); + assert_eq!(event, "response.output_text.delta"); + assert_eq!( + payload, + json!({ + "type":"response.output_text.delta", + "delta":"hello from completed", + "output_index":0, + "content_index":0 + }), + "expected the first chunk to contain only the synthetic response.output_text.delta SSE frame" + ); + + let second = match body_stream.next().await { + Some(Ok(chunk)) => chunk, + Some(Err(error)) => panic!("expected a completed chunk, got body error: {error}"), + None => panic!( + "expected a separate completed chunk after the synthetic delta chunk, but reached EOF after first chunk: {first_text:?}" + ), + }; + let second_text = String::from_utf8(second.to_vec()).expect("utf8 second chunk"); + let (second_event, second_data) = sse_event_and_data(second_text.trim_end()); + let second_payload: Value = serde_json::from_str(second_data).expect("completed json"); + assert_eq!(second_event, "response.completed"); + assert_eq!(second_payload, completed_event); + + let third = match body_stream.next().await { + Some(Ok(chunk)) => chunk, + Some(Err(error)) => panic!("expected a bare DONE chunk, got body error: {error}"), + None => panic!( + "expected a separate bare DONE chunk after the completed chunk, but reached EOF after second chunk: {second_text:?}" + ), + }; + let fourth = body_stream.next().await; + + assert_eq!( + third, + Bytes::from_static(b"data: [DONE]\n\n"), + "expected the third chunk to be exactly the bare downstream DONE sentinel" + ); + assert!(fourth.is_none(), "expected EOF after the bare DONE chunk"); +} + +#[tokio::test] +async fn completed_marker_can_be_reused_after_completed_chunk_before_done_or_eof() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let seed = post_responses(app.clone(), json!({"model":"gpt-5.4","input":"seed"})).await; + assert_eq!(seed.status(), StatusCode::OK); + let _ = server.recv_client_message().await.expect("seed request"); + server + .send_text(&assistant_text_completed_event("response-1", "seed completion").to_string()) + .await; + let _ = to_bytes(seed.into_body(), usize::MAX) + .await + .expect("seed body"); + + let active = post_responses( + app.clone(), + json!({ + "model":"gpt-5.4", + "input":"followup", + "previous_response_id":"response-1" + }), + ) + .await; + assert_eq!(active.status(), StatusCode::OK); + let active_payload: Value = serde_json::from_str(&message_text( + server.recv_client_message().await.expect("active request"), + )) + .expect("active request json"); + assert_eq!(active_payload["previous_response_id"], "response-1"); + server + .send_text(&assistant_text_completed_event("response-2", "followup completion").to_string()) + .await; + + let mut active_body = active.into_body().into_data_stream(); + let first_chunk = next_body_chunk(&mut active_body).await; + let first_text = String::from_utf8(first_chunk.to_vec()).expect("utf8 first chunk"); + let (event, data) = sse_event_and_data(first_text.trim_end()); + let payload: Value = serde_json::from_str(data).expect("delta json"); + assert_eq!(event, "response.output_text.delta"); + assert_eq!(payload["delta"], "followup completion"); + + let completed_chunk = next_body_chunk(&mut active_body).await; + let completed_text = String::from_utf8(completed_chunk.to_vec()).expect("utf8 completed chunk"); + let (completed_event, completed_data) = sse_event_and_data(completed_text.trim_end()); + let completed_payload: Value = serde_json::from_str(completed_data).expect("completed json"); + assert_eq!(completed_event, "response.completed"); + assert_eq!( + completed_payload, + assistant_text_completed_event("response-2", "followup completion") + ); + + let resumed = post_responses( + app.clone(), + json!({ + "model":"gpt-5.4", + "input":"resume-before-done", + "previous_response_id":"response-2" + }), + ) + .await; + assert_eq!(resumed.status(), StatusCode::OK); + let resumed_payload: Value = serde_json::from_str(&message_text( + server.recv_client_message().await.expect("resumed request"), + )) + .expect("resumed request json"); + assert_eq!(resumed_payload["previous_response_id"], "response-2"); + + let done_chunk = next_body_chunk(&mut active_body).await; + assert_eq!(done_chunk, Bytes::from_static(b"data: [DONE]\n\n")); + assert!( + active_body.next().await.is_none(), + "expected EOF after DONE" + ); + + server + .send_text(&assistant_text_completed_event("response-3", "resume completion").to_string()) + .await; + let _ = to_bytes(resumed.into_body(), usize::MAX) + .await + .expect("resumed body"); +} + +#[tokio::test] +async fn internal_tool_followup_strips_context_management_from_initial_and_followup_response_create() + { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let response = post_responses( + app, + json!({ + "model":"gpt-5.4", + "input":"internal-tool-followup", + "context_management":{ + "type":"compaction", + "compact_threshold":12345 + } + }), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let response_body_task = tokio::spawn(async move { + timeout( + Duration::from_secs(2), + to_bytes(response.into_body(), usize::MAX), + ) + .await + .expect("response body timeout") + .expect("response body") + }); + + let initial_payload: Value = serde_json::from_str(&message_text( + server.recv_client_message().await.expect("initial request"), + )) + .expect("initial request json"); + assert!(initial_payload.get("context_management").is_none()); + + server + .send_text(r#"{"type":"response.created","response":{"id":"response-intermediate"}}"#) + .await; + server + .send_text( + r#"{"type":"response.output_item.done","output_index":0,"item":{"type":"function_call","call_id":"call-1","name":"threadline_echo","arguments":"{\"value\":\"alpha\"}"}}"#, + ) + .await; + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-intermediate"}}"#) + .await; + + let followup_payload: Value = serde_json::from_str(&message_text( + timeout(Duration::from_secs(2), server.recv_client_message()) + .await + .expect("followup request timeout") + .expect("followup request"), + )) + .expect("followup request json"); + assert!(followup_payload.get("context_management").is_none()); + assert_eq!( + followup_payload["previous_response_id"], + "response-intermediate" + ); + assert_eq!(followup_payload["input"][0]["type"], "function_call_output"); + assert_eq!(followup_payload["input"][0]["call_id"], "call-1"); + + server + .send_text( + &assistant_text_completed_event("response-final", "tool followup complete").to_string(), + ) + .await; + let _ = response_body_task.await.expect("response body task"); +} + +#[tokio::test] +async fn recoverable_upstream_close_releases_prior_marker_after_completed_chunk_before_body_drop() { + let first_server = Arc::new(ScriptedWebSocketServer::start().await); + let reconnect_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![ + PlannedConnection { + server: Arc::clone(&first_server), + turn_state: Some("turn-state-1".to_string()), + }, + PlannedConnection { + server: Arc::clone(&reconnect_server), + turn_state: None, + }, + ]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let initial = post_responses(app.clone(), json!({"model":"gpt-5.4","input":"seed"})).await; + assert_eq!(initial.status(), StatusCode::OK); + let _ = first_server + .recv_client_message() + .await + .expect("seed request"); + first_server + .send_text(&assistant_text_completed_event("response-1", "seed completion").to_string()) + .await; + + let mut initial_body = initial.into_body().into_data_stream(); + let first_chunk = next_body_chunk(&mut initial_body).await; + let first_text = String::from_utf8(first_chunk.to_vec()).expect("utf8 first chunk"); + let (event, data) = sse_event_and_data(first_text.trim_end()); + let payload: Value = serde_json::from_str(data).expect("delta json"); + assert_eq!(event, "response.output_text.delta"); + assert_eq!(payload["delta"], "seed completion"); + + first_server.send_close(1000, "done").await; + sleep(Duration::from_millis(50)).await; + + let completed_chunk = next_body_chunk(&mut initial_body).await; + let completed_text = String::from_utf8(completed_chunk.to_vec()).expect("utf8 completed chunk"); + let (completed_event, completed_data) = sse_event_and_data(completed_text.trim_end()); + let completed_payload: Value = serde_json::from_str(completed_data).expect("completed json"); + assert_eq!(completed_event, "response.completed"); + assert_eq!( + completed_payload, + assistant_text_completed_event("response-1", "seed completion") + ); + + let resumed = post_responses( + app.clone(), + json!({ + "model":"gpt-5.4", + "input":"resume-after-close", + "previous_response_id":"response-1" + }), + ) + .await; + assert_eq!(resumed.status(), StatusCode::BAD_REQUEST); + let resumed_body = to_bytes(resumed.into_body(), usize::MAX) + .await + .expect("resumed body"); + let resumed_payload: Value = serde_json::from_slice(&resumed_body).expect("resumed body json"); + assert_eq!( + resumed_payload["error"]["code"], + "previous_response_not_found" + ); + + let no_resume_connect = timeout( + Duration::from_millis(250), + reconnect_server.recv_client_message(), + ) + .await; + assert!(no_resume_connect.is_err()); + + let done_chunk = next_body_chunk(&mut initial_body).await; + assert_eq!(done_chunk, Bytes::from_static(b"data: [DONE]\n\n")); + assert!( + initial_body.next().await.is_none(), + "expected EOF after DONE" + ); +} + +#[tokio::test] +async fn live_shaped_response_completed_with_internal_tool_name_still_reaches_done_and_eof() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let response = post_responses( + app, + json!({"model":"gpt-5.4","input":"live-shaped-completed"}), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + let _ = server + .recv_client_message() + .await + .expect("live-shaped-completed request"); + server + .send_text( + r#"{"type":"response.completed","response":{"id":"response-1","output":[{"type":"function_call","name":"threadline_echo","call_id":"call-1"},{"type":"message","role":"assistant","content":[{"type":"output_text","text":"done"}]}]}}"#, + ) + .await; + + let body = to_bytes(response.into_body(), usize::MAX) + .await + .expect("body"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + + assert_eq!( + frames.len(), + 3, + "expected synthetic delta, completed SSE, and bare DONE frame, got body: {body_text}" + ); + + let (delta_event, delta_data) = sse_event_and_data(frames[0]); + let delta_payload: Value = serde_json::from_str(delta_data).expect("delta json"); + assert_eq!(delta_event, "response.output_text.delta"); + assert_eq!( + delta_payload, + json!({ + "type": "response.output_text.delta", + "delta": "done", + "output_index": 1, + "content_index": 0 + }) + ); + + let (event, data) = sse_event_and_data(frames[1]); + let payload: Value = serde_json::from_str(data).expect("completed json"); + assert_eq!(event, "response.completed"); + assert_eq!(payload["response"]["id"], "response-1"); + assert_eq!( + payload["response"]["output"], + json!([ + { + "type": "message", + "role": "assistant", + "content": [ + { + "type": "output_text", + "text": "done" + } + ] + } + ]) + ); + assert_done_frame(frames[2]); +} + +#[tokio::test] +async fn completed_with_internal_function_call_and_assistant_text_synthesizes_delta() { + let completed_event = json!({ + "type": "response.completed", + "response": { + "id": "response-internal-visible-text", + "output": [ + { + "type": "function_call", + "name": "threadline_echo", + "call_id": "call-internal" + }, + { + "id": "assistant-item-internal-visible", + "type": "message", + "role": "assistant", + "content": [ + { + "type": "output_text", + "text": "visible completed text" + } + ] + } + ] + } + }); + + let capture = capture_completed_output_stream(vec![completed_event.clone()]).await; + + assert_eq!(capture.downstream_events.len(), 2); + assert_eq!( + capture.downstream_events[0].event, + "response.output_text.delta" + ); + assert_eq!( + capture.downstream_events[0].payload, + json!({ + "type": "response.output_text.delta", + "delta": "visible completed text", + "item_id": "assistant-item-internal-visible", + "output_index": 1, + "content_index": 0 + }) + ); + assert_eq!(capture.downstream_events[1].event, "response.completed"); + assert_eq!( + capture.downstream_events[1].payload, + json!({ + "type": "response.completed", + "response": { + "id": "response-internal-visible-text", + "output": [ + { + "id": "assistant-item-internal-visible", + "type": "message", + "role": "assistant", + "content": [ + { + "type": "output_text", + "text": "visible completed text" + } + ] + } + ] + } + }) + ); + assert_eq!(capture.done_frame, "data: [DONE]"); +} + +#[tokio::test] +async fn upstream_response_failed_emits_response_failed_terminal_event() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let response = post_responses(app, json!({"model":"gpt-5.4","input":"failure"})).await; + assert_eq!(response.status(), StatusCode::OK); + let _ = server.recv_client_message().await.expect("failure request"); + server + .send_text(r#"{"type":"response.failed","response":{"id":"response-1"},"error":{"code":"upstream_response_failed","message":"failed"}}"#) + .await; + + let body = to_bytes(response.into_body(), usize::MAX) + .await + .expect("body"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + let (event, data) = sse_event_and_data(frames.first().expect("failed frame")); + let payload: Value = serde_json::from_str(data).expect("failed json"); + + assert_eq!(frames.len(), 2); + assert_eq!(event, "response.failed"); + assert_eq!(payload["type"], "response.failed"); + assert_eq!(payload["response"]["id"], "response-1"); + assert_eq!(payload["response"]["status"], "failed"); + assert_eq!( + payload["response"]["error"]["code"], + "upstream_response_failed" + ); + assert_eq!(payload["response"]["error"]["message"], "failed"); + assert_done_frame(frames[1]); +} + +#[tokio::test] +async fn terminal_failed_and_incomplete_payloads_preserve_vscode_terminal_fields() { + let failed_server = Arc::new(ScriptedWebSocketServer::start().await); + let failed_connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&failed_server), + turn_state: None, + }]); + let failed_app = build_test_router(ThreadlineConfig::default(), Arc::new(failed_connector)); + + let failed_response = post_responses( + failed_app, + json!({"model":"gpt-5.4","input":"failed-terminal-fields"}), + ) + .await; + assert_eq!(failed_response.status(), StatusCode::OK); + let _ = failed_server + .recv_client_message() + .await + .expect("failed request"); + failed_server + .send_text( + r#"{"type":"response.failed","response":{"id":"response-failed-fields","model":"gpt-5.4","usage":{"input_tokens":10,"output_tokens":4,"total_tokens":14},"output":[{"id":"assistant-visible","type":"message","role":"assistant","content":[{"type":"output_text","text":"visible failed text"}]}]},"error":{"code":"upstream_response_failed","message":"failed"}}"#, + ) + .await; + + let failed_body = timeout( + Duration::from_secs(2), + to_bytes(failed_response.into_body(), usize::MAX), + ) + .await + .expect("failed body timeout") + .expect("failed body"); + let failed_text = String::from_utf8(failed_body.to_vec()).expect("utf8 failed body"); + let failed_frames = split_sse_frames(&failed_text); + let (failed_event, failed_data) = + sse_event_and_data(failed_frames.first().expect("failed frame")); + let failed_payload: Value = serde_json::from_str(failed_data).expect("failed payload json"); + + assert_eq!(failed_event, "response.failed"); + assert_eq!(failed_payload["response"]["id"], "response-failed-fields"); + assert_eq!(failed_payload["response"]["model"], "gpt-5.4"); + assert_eq!(failed_payload["response"]["usage"]["total_tokens"], 14); + assert_eq!( + assistant_output_text_from_completed( + &json!({"response": failed_payload["response"].clone()}) + ), + "visible failed text" + ); + assert_done_frame(failed_frames[1]); + + let incomplete_server = Arc::new(ScriptedWebSocketServer::start().await); + let incomplete_connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&incomplete_server), + turn_state: None, + }]); + let incomplete_app = + build_test_router(ThreadlineConfig::default(), Arc::new(incomplete_connector)); + + let incomplete_response = post_responses( + incomplete_app, + json!({"model":"gpt-5.4","input":"incomplete-terminal-fields"}), + ) + .await; + assert_eq!(incomplete_response.status(), StatusCode::OK); + let _ = incomplete_server + .recv_client_message() + .await + .expect("incomplete request"); + incomplete_server + .send_text( + r#"{"type":"response.incomplete","response":{"id":"response-incomplete-fields","model":"gpt-5.4","usage":{"input_tokens":8,"output_tokens":3,"total_tokens":11},"output":[{"id":"assistant-partial","type":"message","role":"assistant","content":[{"type":"output_text","text":"visible partial text"}]}],"incomplete_details":{"reason":"max_output_tokens"}}}"#, + ) + .await; + incomplete_server.send_close(1000, "incomplete").await; + + let incomplete_body = timeout( + Duration::from_secs(2), + to_bytes(incomplete_response.into_body(), usize::MAX), + ) + .await + .expect("incomplete body timeout") + .expect("incomplete body"); + let incomplete_text = + String::from_utf8(incomplete_body.to_vec()).expect("utf8 incomplete body"); + let incomplete_frames = split_sse_frames(&incomplete_text); + let (incomplete_event, incomplete_data) = + sse_event_and_data(incomplete_frames.first().expect("incomplete frame")); + let incomplete_payload: Value = + serde_json::from_str(incomplete_data).expect("incomplete payload json"); + + assert_eq!(incomplete_event, "response.incomplete"); + assert_eq!( + incomplete_payload["response"]["id"], + "response-incomplete-fields" + ); + assert_eq!(incomplete_payload["response"]["model"], "gpt-5.4"); + assert_eq!(incomplete_payload["response"]["usage"]["total_tokens"], 11); + assert_eq!( + incomplete_payload["response"]["incomplete_details"]["reason"], + "max_output_tokens" + ); + assert_eq!( + assistant_output_text_from_completed( + &json!({"response": incomplete_payload["response"].clone()}) + ), + "visible partial text" + ); + assert_done_frame(incomplete_frames[1]); +} + +#[tokio::test] +async fn upstream_incomplete_emits_terminal_response_incomplete_without_marker() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let response = post_responses( + app.clone(), + json!({"model":"gpt-5.4","input":"terminal-incomplete"}), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + let _ = server + .recv_client_message() + .await + .expect("incomplete request"); + server + .send_text( + r#"{"type":"response.incomplete","response":{"id":"response-incomplete","model":"gpt-5.4","usage":{"input_tokens":3,"output_tokens":2,"total_tokens":5},"output":[{"id":"assistant-partial","type":"message","role":"assistant","content":[{"type":"output_text","text":"partial answer"}]}],"incomplete_details":{"reason":"max_output_tokens"}}}"#, + ) + .await; + server.send_close(1000, "incomplete").await; + + let body = timeout( + Duration::from_secs(2), + to_bytes(response.into_body(), usize::MAX), + ) + .await + .expect("incomplete body timeout") + .expect("incomplete body"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + let (event, data) = sse_event_and_data(frames.first().expect("incomplete frame")); + let payload: Value = serde_json::from_str(data).expect("incomplete json"); + + assert_eq!(frames.len(), 2); + assert_eq!(event, "response.incomplete"); + assert_eq!(payload["response"]["id"], "response-incomplete"); + assert_eq!( + payload["response"]["incomplete_details"]["reason"], + "max_output_tokens" + ); + assert_done_frame(frames[1]); + + let rejected = post_responses( + app, + json!({ + "model":"gpt-5.4", + "input":"invalid-incomplete-resume", + "previous_response_id":"response-incomplete" + }), + ) + .await; + + assert_eq!(rejected.status(), StatusCode::BAD_REQUEST); + let rejected_body = to_bytes(rejected.into_body(), usize::MAX) + .await + .expect("rejected body"); + let rejected_payload: Value = serde_json::from_slice(&rejected_body).expect("rejected json"); + assert_eq!( + rejected_payload["error"]["code"], + "previous_response_not_found" + ); +} + +#[tokio::test] +async fn response_failed_releases_prior_completed_marker_after_recoverable_close() { + let first_server = Arc::new(ScriptedWebSocketServer::start().await); + let reconnect_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![ + PlannedConnection { + server: Arc::clone(&first_server), + turn_state: Some("turn-state-1".to_string()), + }, + PlannedConnection { + server: Arc::clone(&reconnect_server), + turn_state: None, + }, + ]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let initial = post_responses(app.clone(), json!({"model":"gpt-5.4","input":"seed"})).await; + assert_eq!(initial.status(), StatusCode::OK); + let _ = first_server + .recv_client_message() + .await + .expect("seed request"); + first_server + .send_text(&assistant_text_completed_event("response-1", "seed completion").to_string()) + .await; + let _ = to_bytes(initial.into_body(), usize::MAX) + .await + .expect("seed body"); + + let failed = post_responses( + app.clone(), + json!({ + "model":"gpt-5.4", + "input":"failure", + "previous_response_id":"response-1" + }), + ) + .await; + assert_eq!(failed.status(), StatusCode::OK); + let failed_payload: Value = serde_json::from_str(&message_text( + first_server + .recv_client_message() + .await + .expect("failed request message"), + )) + .expect("failed request json"); + assert!(failed_payload.get("response").is_none()); + assert_eq!(failed_payload["previous_response_id"], "response-1"); + first_server + .send_text(r#"{"type":"response.failed","response":{"id":"response-failed"},"error":{"code":"upstream_response_failed","message":"failed"}}"#) + .await; + let _ = to_bytes(failed.into_body(), usize::MAX) + .await + .expect("failed body"); + + first_server.send_close(1000, "failed turn complete").await; + sleep(Duration::from_millis(50)).await; + + let resumed = post_responses( + app, + json!({ + "model":"gpt-5.4", + "input":"resume", + "previous_response_id":"response-1" + }), + ) + .await; + assert_eq!(resumed.status(), StatusCode::BAD_REQUEST); + let resumed_body = to_bytes(resumed.into_body(), usize::MAX) + .await + .expect("resumed body"); + let resumed_payload: Value = serde_json::from_slice(&resumed_body).expect("resumed body json"); + assert_eq!( + resumed_payload["error"]["code"], + "previous_response_not_found" + ); + + let no_resume_connect = timeout( + Duration::from_millis(250), + reconnect_server.recv_client_message(), + ) + .await; + assert!(no_resume_connect.is_err()); +} + +#[tokio::test] +async fn failed_turn_releases_prior_marker_before_body_drop_and_blocks_resume() { + let first_server = Arc::new(ScriptedWebSocketServer::start().await); + let reconnect_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![ + PlannedConnection { + server: Arc::clone(&first_server), + turn_state: Some("turn-state-1".to_string()), + }, + PlannedConnection { + server: Arc::clone(&reconnect_server), + turn_state: None, + }, + ]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let initial = post_responses(app.clone(), json!({"model":"gpt-5.4","input":"seed"})).await; + assert_eq!(initial.status(), StatusCode::OK); + let _ = first_server + .recv_client_message() + .await + .expect("seed request"); + first_server + .send_text(&assistant_text_completed_event("response-1", "seed completion").to_string()) + .await; + let _ = to_bytes(initial.into_body(), usize::MAX) + .await + .expect("seed body"); + + let failed = post_responses( + app.clone(), + json!({ + "model":"gpt-5.4", + "input":"failure", + "previous_response_id":"response-1" + }), + ) + .await; + assert_eq!(failed.status(), StatusCode::OK); + let failed_payload: Value = serde_json::from_str(&message_text( + first_server + .recv_client_message() + .await + .expect("failed request message"), + )) + .expect("failed request json"); + assert_eq!(failed_payload["previous_response_id"], "response-1"); + first_server + .send_text(r#"{"type":"response.failed","response":{"id":"response-failed"},"error":{"code":"upstream_response_failed","message":"failed"}}"#) + .await; + + let mut failed_body = failed.into_body().into_data_stream(); + let failed_chunk = next_body_chunk(&mut failed_body).await; + let failed_text = String::from_utf8(failed_chunk.to_vec()).expect("utf8 failed chunk"); + let (event, data) = sse_event_and_data(failed_text.trim_end()); + let failed_event: Value = serde_json::from_str(data).expect("failed event json"); + assert_eq!(event, "response.failed"); + assert_eq!(failed_event["response"]["id"], "response-failed"); + + let resumed = post_responses( + app.clone(), + json!({ + "model":"gpt-5.4", + "input":"resume-before-failed-body-drop", + "previous_response_id":"response-1" + }), + ) + .await; + assert_eq!(resumed.status(), StatusCode::BAD_REQUEST); + let resumed_body = to_bytes(resumed.into_body(), usize::MAX) + .await + .expect("resumed body"); + let resumed_payload: Value = serde_json::from_slice(&resumed_body).expect("resumed body json"); + assert_eq!( + resumed_payload["error"]["code"], + "previous_response_not_found" + ); + + let no_resume_connect = timeout( + Duration::from_millis(250), + reconnect_server.recv_client_message(), + ) + .await; + assert!(no_resume_connect.is_err()); + + let done_chunk = next_body_chunk(&mut failed_body).await; + assert_eq!(done_chunk, Bytes::from_static(b"data: [DONE]\n\n")); + assert!( + failed_body.next().await.is_none(), + "expected EOF after DONE" + ); +} + +#[tokio::test] +async fn response_failed_id_is_not_a_continuation_marker() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let initial = post_responses(app.clone(), json!({"model":"gpt-5.4","input":"seed"})).await; + assert_eq!(initial.status(), StatusCode::OK); + let _ = server.recv_client_message().await.expect("seed request"); + server + .send_text(&assistant_text_completed_event("response-1", "seed completion").to_string()) + .await; + let _ = to_bytes(initial.into_body(), usize::MAX) + .await + .expect("seed body"); + + let failed = post_responses( + app.clone(), + json!({ + "model":"gpt-5.4", + "input":"failure", + "previous_response_id":"response-1" + }), + ) + .await; + assert_eq!(failed.status(), StatusCode::OK); + let _ = server.recv_client_message().await.expect("failed request"); + server + .send_text(r#"{"type":"response.failed","response":{"id":"response-failed"},"error":{"code":"upstream_response_failed","message":"failed"}}"#) + .await; + let _ = to_bytes(failed.into_body(), usize::MAX) + .await + .expect("failed body"); + + let rejected = post_responses( + app, + json!({ + "model":"gpt-5.4", + "input":"invalid-resume", + "previous_response_id":"response-failed" + }), + ) + .await; + + assert_eq!(rejected.status(), StatusCode::BAD_REQUEST); + let body = to_bytes(rejected.into_body(), usize::MAX) + .await + .expect("rejected body"); + let payload: Value = serde_json::from_slice(&body).expect("rejected json body"); + assert_eq!(payload["error"]["code"], "previous_response_not_found"); +} + +#[tokio::test] +async fn upstream_error_event_with_previous_response_not_found_code_emits_previous_response_not_found() + { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let response = post_responses(app, json!({"model":"gpt-5.4","input":"error-code"})).await; + assert_eq!(response.status(), StatusCode::OK); + let _ = server.recv_client_message().await.expect("error request"); + server + .send_text( + r#"{"type":"error","error":{"code":"previous_response_not_found","message":"unrelated upstream text"},"status":404}"#, + ) + .await; + + let body = to_bytes(response.into_body(), usize::MAX) + .await + .expect("body"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + let (event, data) = sse_event_and_data(frames.first().expect("failed frame")); + let payload: Value = serde_json::from_str(data).expect("failed json"); + + assert_eq!(frames.len(), 2); + assert_eq!(event, "response.failed"); + assert_eq!(payload["type"], "response.failed"); + assert_eq!(payload["response"]["status"], "failed"); + assert_eq!( + payload["response"]["error"]["code"], + "previous_response_not_found" + ); + assert_eq!( + payload["response"]["error"]["message"], + "Threadline could not find the retained session for that previous_response_id." + ); + assert!( + !body_text.contains("event: error\n"), + "raw upstream error must not be forwarded as a raw error event: {body_text}" + ); + assert_done_frame(frames[1]); +} + +#[tokio::test] +async fn upstream_error_event_with_previous_response_not_found_message_emits_previous_response_not_found() + { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let response = post_responses(app, json!({"model":"gpt-5.4","input":"error-message"})).await; + assert_eq!(response.status(), StatusCode::OK); + let _ = server.recv_client_message().await.expect("error request"); + server + .send_text( + r#"{"type":"error","error":{"code":"upstream_boom","message":"Previous response with id 'resp_123' not found."},"status":404}"#, + ) + .await; + + let body = to_bytes(response.into_body(), usize::MAX) + .await + .expect("body"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + let (event, data) = sse_event_and_data(frames.first().expect("failed frame")); + let payload: Value = serde_json::from_str(data).expect("failed json"); + + assert_eq!(frames.len(), 2); + assert_eq!(event, "response.failed"); + assert_eq!( + payload["response"]["error"]["code"], + "previous_response_not_found" + ); + assert_done_frame(frames[1]); +} + +#[tokio::test] +async fn upstream_error_event_partial_previous_response_messages_remain_upstream_error_event() { + for (case_name, error_message) in [ + ( + "previous-response-only", + "Previous response with id 'resp_123' expired.", + ), + ("not-found-only", "Session marker not found."), + ] { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let response = post_responses( + app, + json!({"model":"gpt-5.4","input":format!("partial-{case_name}")}), + ) + .await; + assert_eq!(response.status(), StatusCode::OK, "status for {case_name}"); + let _ = server.recv_client_message().await.expect("error request"); + server + .send_text( + &json!({ + "type": "error", + "error": { + "code": "upstream_boom", + "message": error_message, + }, + "status": 404, + }) + .to_string(), + ) + .await; + + let body = to_bytes(response.into_body(), usize::MAX) + .await + .expect("body"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + let (event, data) = sse_event_and_data(frames.first().expect("failed frame")); + let payload: Value = serde_json::from_str(data).expect("failed json"); + + assert_eq!(frames.len(), 2, "frame count for {case_name}"); + assert_eq!(event, "response.failed", "event for {case_name}"); + assert_eq!( + payload["response"]["error"]["code"], "upstream_error_event", + "error code for {case_name}" + ); + assert_done_frame(frames[1]); + } +} + +#[tokio::test] +async fn classified_upstream_previous_response_not_found_releases_prior_marker_without_reconnect() { + let first_server = Arc::new(ScriptedWebSocketServer::start().await); + let reconnect_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![ + PlannedConnection { + server: Arc::clone(&first_server), + turn_state: Some("turn-state-1".to_string()), + }, + PlannedConnection { + server: Arc::clone(&reconnect_server), + turn_state: None, + }, + ]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let initial = post_responses(app.clone(), json!({"model":"gpt-5.4","input":"seed"})).await; + assert_eq!(initial.status(), StatusCode::OK); + let _ = first_server + .recv_client_message() + .await + .expect("seed request"); + first_server + .send_text(&assistant_text_completed_event("response-1", "seed completion").to_string()) + .await; + let _ = to_bytes(initial.into_body(), usize::MAX) + .await + .expect("seed body"); + + let failed = post_responses( + app.clone(), + json!({ + "model":"gpt-5.4", + "input":"failure", + "previous_response_id":"response-1" + }), + ) + .await; + assert_eq!(failed.status(), StatusCode::OK); + let failed_payload: Value = serde_json::from_str(&message_text( + first_server + .recv_client_message() + .await + .expect("failed request message"), + )) + .expect("failed request json"); + assert_eq!(failed_payload["previous_response_id"], "response-1"); + first_server + .send_text( + r#"{"type":"error","error":{"code":"previous_response_not_found","message":"Previous response with id 'response-1' not found."},"status":404}"#, + ) + .await; + let _ = to_bytes(failed.into_body(), usize::MAX) + .await + .expect("failed body"); + + first_server + .send_close(1000, "classified failure complete") + .await; + sleep(Duration::from_millis(50)).await; + + let resumed = post_responses( + app, + json!({ + "model":"gpt-5.4", + "input":"resume", + "previous_response_id":"response-1" + }), + ) + .await; + assert_eq!(resumed.status(), StatusCode::BAD_REQUEST); + let resumed_body = to_bytes(resumed.into_body(), usize::MAX) + .await + .expect("resumed body"); + let resumed_payload: Value = serde_json::from_slice(&resumed_body).expect("resumed body json"); + assert_eq!( + resumed_payload["error"]["code"], + "previous_response_not_found" + ); + + let no_resume_connect = timeout( + Duration::from_millis(250), + reconnect_server.recv_client_message(), + ) + .await; + assert!(no_resume_connect.is_err()); +} + +#[tokio::test] +async fn upstream_error_event_emits_response_failed_and_done_without_successful_completion() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let response = post_responses(app, json!({"model":"gpt-5.4","input":"error"})).await; + assert_eq!(response.status(), StatusCode::OK); + let _ = server.recv_client_message().await.expect("error request"); + server + .send_text( + r#"{"type":"error","error":{"code":"upstream_boom","message":"boom"},"status":502}"#, + ) + .await; + + let body = to_bytes(response.into_body(), usize::MAX) + .await + .expect("body"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + + assert!( + !body_text.contains("event: error\n"), + "raw upstream error must not be forwarded as a raw error event: {body_text}" + ); + + let (event, data) = sse_event_and_data(frames.first().expect("failed frame")); + let payload: Value = serde_json::from_str(data).expect("failed json"); + + assert_eq!( + frames.len(), + 2, + "raw upstream error must be normalized into downstream response.failed plus DONE frames: {body_text}" + ); + assert_eq!(event, "response.failed"); + assert_eq!(payload["type"], "response.failed"); + assert_eq!(payload["response"]["status"], "failed"); + assert_eq!(payload["response"]["error"]["code"], "upstream_error_event"); + assert!( + payload["response"]["error"]["message"] + .as_str() + .is_some_and(|message| !message.is_empty()), + "raw upstream error must surface a stable Threadline error message: {payload:?}" + ); + assert!( + !body_text.contains("event: response.completed\n"), + "raw upstream error must not emit successful completion semantics: {body_text}" + ); + assert_done_frame(frames[1]); +} + +#[tokio::test] +async fn upstream_done_or_eof_without_completed_emits_response_failed_not_done_only() { + for case_name in ["done", "eof"] { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let response = post_responses( + app, + json!({"model":"gpt-5.4","input":format!("terminal-{case_name}")}), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + let _ = server + .recv_client_message() + .await + .expect("terminal request"); + + match case_name { + "done" => { + server.send_text("[DONE]").await; + server.send_close(1000, "done before completed").await; + } + "eof" => { + server.abort_connection().await; + } + _ => unreachable!("unexpected terminal case"), + } + + let body = timeout( + Duration::from_secs(2), + to_bytes(response.into_body(), usize::MAX), + ) + .await + .expect("terminal body timeout") + .expect("terminal body"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + let (event, data) = sse_event_and_data(frames.first().expect("failed frame")); + let payload: Value = serde_json::from_str(data).expect("failed json"); + + assert_eq!( + frames.len(), + 2, + "expected terminal failed event plus DONE for {case_name}: {body_text}" + ); + assert_eq!(event, "response.failed"); + assert_eq!(payload["type"], "response.failed"); + assert_eq!(payload["response"]["status"], "failed"); + assert!( + payload["response"]["error"]["code"] + .as_str() + .is_some_and(|code| !code.is_empty()), + "expected a stable failure code for {case_name}: {payload:?}" + ); + assert_done_frame(frames[1]); + } +} + +#[tokio::test] +async fn malformed_upstream_json_emits_a_stable_sse_error_and_releases_the_marker() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let initial = post_responses(app.clone(), json!({"model":"gpt-5.4","input":"seed"})).await; + let _ = server.recv_client_message().await.expect("seed request"); + server + .send_text(&assistant_text_completed_event("response-1", "seed completion").to_string()) + .await; + let _ = to_bytes(initial.into_body(), usize::MAX) + .await + .expect("seed body"); + + let response = post_responses( + app.clone(), + json!({ + "model":"gpt-5.4", + "input":"malformed", + "previous_response_id":"response-1" + }), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + let _ = server + .recv_client_message() + .await + .expect("malformed request"); + server.send_text("not-json").await; + + let body = to_bytes(response.into_body(), usize::MAX) + .await + .expect("body"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + let (event, data) = sse_event_and_data(frames.first().expect("error frame")); + let payload: Value = serde_json::from_str(data).expect("error json"); + + assert_eq!(event, "error"); + assert_eq!(payload["error"]["code"], "upstream_invalid_json"); + + sleep(Duration::from_millis(50)).await; + let retried = post_responses( + app, + json!({ + "model":"gpt-5.4", + "input":"retry", + "previous_response_id":"response-1" + }), + ) + .await; + assert_eq!(retried.status(), StatusCode::BAD_REQUEST); + let body = to_bytes(retried.into_body(), usize::MAX) + .await + .expect("retry body"); + let payload: Value = serde_json::from_slice(&body).expect("retry json body"); + assert_eq!(payload["error"]["code"], "previous_response_not_found"); +} + +#[tokio::test] +async fn nested_response_markers_remain_reusable_without_main_agent_assumptions() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector.clone())); + + let first = post_responses(app.clone(), json!({"model":"gpt-5.4","input":"first"})).await; + let _ = server.recv_client_message().await.expect("first request"); + server + .send_text( + &assistant_text_completed_event("response-parent", "parent completion").to_string(), + ) + .await; + let _ = to_bytes(first.into_body(), usize::MAX) + .await + .expect("first body"); + + let second = post_responses( + app.clone(), + json!({ + "model":"gpt-5.4", + "input":"second", + "previous_response_id":"response-parent" + }), + ) + .await; + let second_payload: Value = serde_json::from_str(&message_text( + server.recv_client_message().await.expect("second request"), + )) + .expect("second request json"); + assert!(second_payload.get("response").is_none()); + assert_eq!(second_payload["previous_response_id"], "response-parent"); + server + .send_text( + &assistant_text_completed_event("response-child", "child completion").to_string(), + ) + .await; + let _ = to_bytes(second.into_body(), usize::MAX) + .await + .expect("second body"); + + let third = post_responses( + app.clone(), + json!({ + "model":"gpt-5.4", + "input":"third", + "previous_response_id":"response-parent" + }), + ) + .await; + let third_payload: Value = serde_json::from_str(&message_text( + server.recv_client_message().await.expect("third request"), + )) + .expect("third request json"); + assert!(third_payload.get("response").is_none()); + assert_eq!(third_payload["previous_response_id"], "response-parent"); + server + .send_text( + &assistant_text_completed_event("response-third", "third completion").to_string(), + ) + .await; + let _ = to_bytes(third.into_body(), usize::MAX) + .await + .expect("third body"); + + let fourth = post_responses( + app, + json!({ + "model":"gpt-5.4", + "input":"fourth", + "previous_response_id":"response-child" + }), + ) + .await; + let fourth_payload: Value = serde_json::from_str(&message_text( + server.recv_client_message().await.expect("fourth request"), + )) + .expect("fourth request json"); + assert!(fourth_payload.get("response").is_none()); + assert_eq!(fourth_payload["previous_response_id"], "response-child"); + server + .send_text( + &assistant_text_completed_event("response-fourth", "fourth completion").to_string(), + ) + .await; + let _ = to_bytes(fourth.into_body(), usize::MAX) + .await + .expect("fourth body"); + + let sessions = connector.recorded_sessions().await; + assert_eq!(sessions.len(), 1); +} + +#[tokio::test] +async fn supported_request_fields_are_preserved_while_codex_unsupported_fields_are_omitted() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let response = post_responses( + app, + json!({ + "type":"wrong.type", + "model":"gpt-5.4", + "input":[{"role":"user","content":[{"type":"input_text","text":"hello"}]}], + "tools":[{ + "type":"function", + "name":"user_tool", + "description":"User-defined tool", + "parameters":{"type":"object","properties":{},"additionalProperties":false} + }], + "tool_choice":{"type":"function","name":"user_tool"}, + "parallel_tool_calls":false, + "reasoning":{"effort":"high","summary":"auto"}, + "include":["reasoning.encrypted_content"], + "store":true, + "prompt_cache_key":"cache-key-1", + "max_output_tokens":321, + "max_tokens":654, + "max_completion_tokens":987, + "truncation":"auto" + }), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let request_payload: Value = serde_json::from_str(&message_text( + server.recv_client_message().await.expect("request message"), + )) + .expect("request json"); + assert_eq!(request_payload["type"], "response.create"); + assert!(request_payload.get("response").is_none()); + let response_payload = &request_payload; + let tools = response_payload["tools"].as_array().expect("tools array"); + + assert!(tools.iter().any(|tool| tool["name"] == "user_tool")); + assert_eq!( + response_payload["tool_choice"], + json!({"type":"function","name":"user_tool"}) + ); + assert_eq!(response_payload["parallel_tool_calls"], Value::Bool(false)); + assert_eq!( + response_payload["reasoning"], + json!({"effort":"high","summary":"auto"}) + ); + assert_eq!( + response_payload["include"], + json!(["reasoning.encrypted_content"]) + ); + assert_eq!(response_payload["store"], Value::Bool(false)); + assert_eq!( + response_payload["prompt_cache_key"], + Value::String("cache-key-1".to_string()) + ); + assert_codex_unsupported_response_fields_are_absent(response_payload); + + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-1"}}"#) + .await; + let _ = to_bytes(response.into_body(), usize::MAX) + .await + .expect("body"); +} + +#[tokio::test] +async fn utility_model_alias_rewrites_upstream_model() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router( + ThreadlineConfig { + profile: RouteProfile::Utility, + ..ThreadlineConfig::default() + }, + Arc::new(connector), + ); + + let response = post_responses( + app, + json!({ + "model":"threadline-utility-gpt-5.4-mini", + "input":"utility-alias" + }), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let request_payload: Value = serde_json::from_str(&message_text( + server.recv_client_message().await.expect("request message"), + )) + .expect("request json"); + assert_eq!(request_payload["type"], "response.create"); + assert_eq!(request_payload["model"], "gpt-5.4-mini"); + + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-utility-alias"}}"#) + .await; + let _ = to_bytes(response.into_body(), usize::MAX) + .await + .expect("body"); +} + +#[tokio::test] +async fn utility_reasoning_effort_is_preserved() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router( + ThreadlineConfig { + profile: RouteProfile::Utility, + ..ThreadlineConfig::default() + }, + Arc::new(connector), + ); + + let response = post_responses( + app, + json!({ + "model":"threadline-utility-gpt-5.4-mini", + "input":"utility-reasoning", + "reasoning":{"effort":"high","summary":"auto"} + }), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let request_payload: Value = serde_json::from_str(&message_text( + server.recv_client_message().await.expect("request message"), + )) + .expect("request json"); + assert_eq!(request_payload["model"], "gpt-5.4-mini"); + assert_eq!( + request_payload["reasoning"], + json!({"effort":"high","summary":"auto"}) + ); + + server + .send_text( + r#"{"type":"response.completed","response":{"id":"response-utility-reasoning"}}"#, + ) + .await; + let _ = to_bytes(response.into_body(), usize::MAX) + .await + .expect("body"); +} + +#[tokio::test] +async fn utility_reasoning_all_turns_and_encrypted_content_are_preserved_for_supported_model() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router( + ThreadlineConfig { + profile: RouteProfile::Utility, + ..ThreadlineConfig::default() + }, + Arc::new(connector), + ); + + let response = post_responses( + app, + json!({ + "model":"threadline-utility-gpt-5.4-mini", + "input":"utility-all-turns-supported", + "reasoning":{"context":"all_turns","effort":"high"}, + "include":["reasoning.encrypted_content"] + }), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let request_payload: Value = serde_json::from_str(&message_text( + server.recv_client_message().await.expect("request message"), + )) + .expect("request json"); + assert_eq!(request_payload["type"], "response.create"); + assert_eq!(request_payload["model"], "gpt-5.4-mini"); + assert_eq!( + request_payload["reasoning"], + json!({"context":"all_turns","effort":"high"}) + ); + assert_eq!( + request_payload["include"], + json!(["reasoning.encrypted_content"]) + ); + + server + .send_text( + r#"{"type":"response.completed","response":{"id":"response-utility-all-turns"}}"#, + ) + .await; + let _ = to_bytes(response.into_body(), usize::MAX) + .await + .expect("body"); +} + +#[tokio::test] +async fn utility_request_omits_previous_response_id_context_management_and_threadline_tools_upstream() + { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router( + ThreadlineConfig { + profile: RouteProfile::Utility, + retained_session_capacity: 1, + jobs_enabled: true, + ..ThreadlineConfig::default() + }, + Arc::new(connector), + ); + + let response = post_responses( + app, + json!({ + "model":"threadline-utility-gpt-5.4-mini", + "input":"utility-upstream-normalization", + "previous_response_id":"response-stale", + "context_management":{ + "type":"compaction", + "compact_threshold":12345 + }, + "reasoning":{"effort":"high","summary":"auto"}, + "tools":[ + { + "type":"function", + "name":"user_tool", + "description":"User tool", + "parameters":{"type":"object"} + }, + { + "type":"function", + "name":"threadline_echo", + "description":"Threadline internal tool", + "parameters":{"type":"object"} + }, + { + "type":"function", + "name":"threadline_start_job", + "description":"Threadline job tool", + "parameters":{"type":"object"} + } + ] + }), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let request_payload: Value = serde_json::from_str(&message_text( + server.recv_client_message().await.expect("request message"), + )) + .expect("request json"); + assert_eq!(request_payload["type"], "response.create"); + assert_eq!(request_payload["model"], "gpt-5.4-mini"); + assert!(request_payload.get("previous_response_id").is_none()); + assert!(request_payload.get("context_management").is_none()); + assert_eq!( + request_payload["reasoning"], + json!({"effort":"high","summary":"auto"}) + ); + let tools = request_payload["tools"].as_array().expect("tools array"); + assert!(tools.iter().any(|tool| tool["name"] == "user_tool")); + assert!(!tools.iter().any(|tool| { + tool["name"] + .as_str() + .is_some_and(|name| name.starts_with("threadline_")) + })); + + server + .send_text( + r#"{"type":"response.completed","response":{"id":"response-utility-normalized"}}"#, + ) + .await; + let _ = to_bytes(response.into_body(), usize::MAX) + .await + .expect("body"); +} + +#[tokio::test] +async fn utility_requests_ignore_retained_capacity_and_open_fresh_sessions() { + let first_server = Arc::new(ScriptedWebSocketServer::start().await); + let second_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![ + PlannedConnection { + server: Arc::clone(&first_server), + turn_state: None, + }, + PlannedConnection { + server: Arc::clone(&second_server), + turn_state: None, + }, + ]); + let app = build_test_router( + ThreadlineConfig { + profile: RouteProfile::Utility, + retained_session_capacity: 1, + ..ThreadlineConfig::default() + }, + Arc::new(connector.clone()), + ); + + let first = post_responses( + app.clone(), + json!({ + "model":"threadline-utility-gpt-5.4-mini", + "input":"first utility request" + }), + ) + .await; + assert_eq!(first.status(), StatusCode::OK); + + let first_payload: Value = serde_json::from_str(&message_text( + first_server + .recv_client_message() + .await + .expect("first request message"), + )) + .expect("first request json"); + assert!(first_payload.get("previous_response_id").is_none()); + + let second = post_responses( + app.clone(), + json!({ + "model":"threadline-utility-gpt-5.4-mini", + "input":"second utility request", + "previous_response_id":"response-utility-stale" + }), + ) + .await; + assert_eq!(second.status(), StatusCode::OK); + + let second_payload: Value = serde_json::from_str(&message_text( + second_server + .recv_client_message() + .await + .expect("second request message"), + )) + .expect("second request json"); + assert!(second_payload.get("previous_response_id").is_none()); + + first_server + .send_text(r#"{"type":"response.completed","response":{"id":"response-utility-1"}}"#) + .await; + second_server + .send_text(r#"{"type":"response.completed","response":{"id":"response-utility-2"}}"#) + .await; + + let _ = to_bytes(first.into_body(), usize::MAX) + .await + .expect("first body"); + let _ = to_bytes(second.into_body(), usize::MAX) + .await + .expect("second body"); + + let sessions = connector.recorded_sessions().await; + assert_eq!(sessions.len(), 2); +} + +#[tokio::test] +async fn utility_does_not_execute_upstream_threadline_job_calls_locally() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router( + ThreadlineConfig { + profile: RouteProfile::Utility, + jobs_enabled: true, + ..ThreadlineConfig::default() + }, + Arc::new(connector), + ); + + let response = post_responses( + app, + json!({ + "model":"threadline-utility-gpt-5.4-mini", + "input":"utility-job-tool-call" + }), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let body_task = tokio::spawn(async move { + to_bytes(response.into_body(), usize::MAX) + .await + .expect("body bytes") + }); + + let request_payload: Value = serde_json::from_str(&message_text( + server.recv_client_message().await.expect("initial request"), + )) + .expect("initial request json"); + let tools = request_payload + .get("tools") + .and_then(Value::as_array) + .cloned() + .unwrap_or_default(); + assert!( + !tools.iter().any(|tool| { + tool["name"] + .as_str() + .is_some_and(|name| name.starts_with("threadline_")) + }), + "expected Utility request to exclude internal tools before streaming" + ); + + server + .send_text( + r#"{"type":"response.output_item.done","output_index":0,"item":{"type":"function_call","call_id":"call-job","name":"threadline_start_job","arguments":"{\"command\":[\"echo\",\"hello\"]}"}}"#, + ) + .await; + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-utility-job"}}"#) + .await; + + let maybe_followup = + tokio::time::timeout(Duration::from_millis(100), server.recv_client_message()).await; + assert!( + !matches!(maybe_followup, Ok(Some(_))), + "expected Utility request to avoid internal job-tool follow-up traffic" + ); + + let body = body_task.await.expect("body task"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + let (event, data) = sse_event_and_data(frames.first().expect("terminal frame")); + let payload: Value = serde_json::from_str(data).expect("terminal json"); + + assert_eq!(event, "response.completed"); + assert_eq!(payload["response"]["id"], "response-utility-job"); + assert!(!body_text.contains("threadline_start_job")); + assert_done_frame(frames[1]); +} + +#[tokio::test] +async fn utility_terminal_completion_drops_transient_upstream_handle() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router( + ThreadlineConfig { + profile: RouteProfile::Utility, + retained_session_capacity: 1, + ..ThreadlineConfig::default() + }, + Arc::new(connector.clone()), + ); + + let response = post_responses( + app, + json!({ + "model":"threadline-utility-gpt-5.4-mini", + "input":"utility-cleanup" + }), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let _ = server + .recv_client_message() + .await + .expect("utility cleanup request"); + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-utility-cleanup"}}"#) + .await; + let _ = to_bytes(response.into_body(), usize::MAX) + .await + .expect("cleanup body"); + + sleep(Duration::from_millis(50)).await; + let sockets = connector.recorded_websockets().await; + assert!(sockets[0].upgrade().is_none()); +} + +#[tokio::test] +async fn missing_or_null_instructions_are_normalized_for_upstream_response_create() { + let missing_server = Arc::new(ScriptedWebSocketServer::start().await); + let null_server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![ + PlannedConnection { + server: Arc::clone(&missing_server), + turn_state: None, + }, + PlannedConnection { + server: Arc::clone(&null_server), + turn_state: None, + }, + ]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let missing_response = post_responses( + app.clone(), + json!({ + "type":"wrong.type", + "model":"gpt-5.4", + "input":[{"role":"user","content":[{"type":"input_text","text":"hello"}]}], + "max_output_tokens":321, + "max_tokens":654, + "max_completion_tokens":987, + "truncation":"auto" + }), + ) + .await; + assert_eq!(missing_response.status(), StatusCode::OK); + + let missing_payload: Value = serde_json::from_str(&message_text( + missing_server + .recv_client_message() + .await + .expect("missing request message"), + )) + .expect("missing request json"); + assert_eq!(missing_payload["type"], "response.create"); + assert_eq!( + missing_payload["instructions"], + Value::String(String::new()) + ); + assert_eq!(missing_payload["store"], Value::Bool(false)); + assert_codex_unsupported_response_fields_are_absent(&missing_payload); + + missing_server + .send_text(r#"{"type":"response.completed","response":{"id":"response-1"}}"#) + .await; + let _ = to_bytes(missing_response.into_body(), usize::MAX) + .await + .expect("missing body"); + + let null_response = post_responses( + app, + json!({ + "type":"wrong.type", + "model":"gpt-5.4", + "input":[{"role":"user","content":[{"type":"input_text","text":"hello again"}]}], + "instructions":null, + "max_output_tokens":654, + "max_tokens":321, + "max_completion_tokens":111, + "truncation":"disabled" + }), + ) + .await; + assert_eq!(null_response.status(), StatusCode::OK); + + let null_payload: Value = serde_json::from_str(&message_text( + null_server + .recv_client_message() + .await + .expect("null request message"), + )) + .expect("null request json"); + assert_eq!(null_payload["type"], "response.create"); + assert_eq!(null_payload["instructions"], Value::String(String::new())); + assert_eq!(null_payload["store"], Value::Bool(false)); + assert_codex_unsupported_response_fields_are_absent(&null_payload); + + null_server + .send_text(r#"{"type":"response.completed","response":{"id":"response-2"}}"#) + .await; + let _ = to_bytes(null_response.into_body(), usize::MAX) + .await + .expect("null body"); +} + +#[tokio::test] +async fn explicit_instructions_are_preserved_in_upstream_response_create() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let response = post_responses( + app, + json!({ + "type":"wrong.type", + "model":"gpt-5.4", + "input":[{"role":"user","content":[{"type":"input_text","text":"preserve me"}]}], + "instructions":"explicit downstream instructions", + "max_output_tokens":987, + "max_tokens":654, + "max_completion_tokens":321, + "truncation":"auto" + }), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let payload: Value = serde_json::from_str(&message_text( + server + .recv_client_message() + .await + .expect("explicit instructions request message"), + )) + .expect("explicit instructions request json"); + assert_eq!(payload["type"], "response.create"); + assert_eq!( + payload["instructions"], + Value::String("explicit downstream instructions".to_string()) + ); + assert_codex_unsupported_response_fields_are_absent(&payload); + + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-3"}}"#) + .await; + let _ = to_bytes(response.into_body(), usize::MAX) + .await + .expect("explicit instructions body"); +} + +struct DownstreamSseEvent { + event: String, + payload: Value, +} + +struct ApplyPatchStreamCapture { + upstream_events: Vec, + downstream_events: Vec, + done_frame: String, +} + +struct CompactionStreamCapture { + upstream_events: Vec, + downstream_events: Vec, + done_frame: String, +} + +struct CompletedOutputStreamCapture { + downstream_events: Vec, + done_frame: String, +} + +type SharedTraceBytes = Arc>>; +type ActiveTraceBytes = StdMutex>; + +struct SharedLogBuffer { + bytes: SharedTraceBytes, +} + +impl SharedLogBuffer { + fn new() -> Self { + Self { + bytes: Arc::new(StdMutex::new(Vec::new())), + } + } + + fn logs(&self) -> String { + String::from_utf8(self.bytes.lock().expect("log buffer lock").clone()) + .expect("utf8 trace logs") + } +} + +static TRACE_CAPTURE_LOCK: OnceLock> = OnceLock::new(); +static ACTIVE_TRACE_BUFFER: OnceLock = OnceLock::new(); +static TRACE_SUBSCRIBER_INIT: OnceLock<()> = OnceLock::new(); + +fn trace_capture_lock() -> &'static Mutex<()> { + TRACE_CAPTURE_LOCK.get_or_init(|| Mutex::new(())) +} + +fn active_trace_buffer() -> &'static ActiveTraceBytes { + ACTIVE_TRACE_BUFFER.get_or_init(|| StdMutex::new(None)) +} + +fn ensure_test_trace_subscriber() { + TRACE_SUBSCRIBER_INIT.get_or_init(|| { + let subscriber = tracing_subscriber::fmt() + .with_max_level(tracing::Level::TRACE) + .without_time() + .with_ansi(false) + .with_writer(GlobalTraceCapture) + .finish(); + tracing::subscriber::set_global_default(subscriber) + .expect("global trace subscriber should only initialize once"); + }); +} + +#[derive(Clone, Copy)] +struct GlobalTraceCapture; + +struct GlobalTraceWriter; + +impl Write for GlobalTraceWriter { + fn write(&mut self, buf: &[u8]) -> io::Result { + if let Some(bytes) = active_trace_buffer() + .lock() + .expect("active trace buffer lock") + .as_ref() + { + bytes + .lock() + .expect("log buffer lock") + .extend_from_slice(buf); + } + Ok(buf.len()) + } + + fn flush(&mut self) -> io::Result<()> { + Ok(()) + } +} + +impl<'a> MakeWriter<'a> for GlobalTraceCapture { + type Writer = GlobalTraceWriter; + + fn make_writer(&'a self) -> Self::Writer { + GlobalTraceWriter + } +} + +struct TraceCaptureGuard { + _lock: tokio::sync::MutexGuard<'static, ()>, + log_buffer: SharedLogBuffer, +} + +impl TraceCaptureGuard { + async fn begin() -> Self { + let lock = trace_capture_lock().lock().await; + ensure_test_trace_subscriber(); + let log_buffer = SharedLogBuffer::new(); + *active_trace_buffer() + .lock() + .expect("active trace buffer lock") = Some(Arc::clone(&log_buffer.bytes)); + Self { + _lock: lock, + log_buffer, + } + } + + fn logs(&self) -> String { + self.log_buffer.logs() + } +} + +impl Drop for TraceCaptureGuard { + fn drop(&mut self) { + *active_trace_buffer() + .lock() + .expect("active trace buffer lock") = None; + } +} + +async fn capture_visible_apply_patch_stream() -> ApplyPatchStreamCapture { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let response = + post_responses(app, json!({"model":"gpt-5.4","input":"apply-patch-stream"})).await; + assert_eq!(response.status(), StatusCode::OK); + + let _ = server + .recv_client_message() + .await + .expect("apply patch stream request"); + + let mut body_stream = response.into_body().into_data_stream(); + + let added_event = json!({ + "type": "response.output_item.added", + "output_index": 0, + "item": { + "id": "fc_apply_patch_1", + "type": "function_call", + "call_id": "call-apply-patch", + "name": "apply_patch", + "arguments": "" + } + }); + server.send_text(&added_event.to_string()).await; + + let added_chunk = next_body_chunk(&mut body_stream).await; + let added_text = String::from_utf8(added_chunk.to_vec()).expect("utf8 added chunk"); + let (added_sse_event, added_sse_data) = sse_event_and_data(added_text.trim_end()); + let added_payload: Value = serde_json::from_str(added_sse_data).expect("added payload json"); + + let first_delta_event = json!({ + "type": "response.function_call_arguments.delta", + "output_index": 0, + "item_id": "fc_apply_patch_1", + "delta": "{\"input\":\"*** Begin Patch" + }); + server.send_text(&first_delta_event.to_string()).await; + + let first_delta_chunk = next_body_chunk(&mut body_stream).await; + let first_delta_text = + String::from_utf8(first_delta_chunk.to_vec()).expect("utf8 first delta chunk"); + let (first_delta_sse_event, first_delta_sse_data) = + sse_event_and_data(first_delta_text.trim_end()); + let first_delta_payload: Value = + serde_json::from_str(first_delta_sse_data).expect("first delta payload json"); + + let second_delta_event = json!({ + "type": "response.function_call_arguments.delta", + "output_index": 0, + "item_id": "fc_apply_patch_1", + "delta": "\n*** End Patch\"}" + }); + server.send_text(&second_delta_event.to_string()).await; + + let second_delta_chunk = next_body_chunk(&mut body_stream).await; + let second_delta_text = + String::from_utf8(second_delta_chunk.to_vec()).expect("utf8 second delta chunk"); + let (second_delta_sse_event, second_delta_sse_data) = + sse_event_and_data(second_delta_text.trim_end()); + let second_delta_payload: Value = + serde_json::from_str(second_delta_sse_data).expect("second delta payload json"); + + let arguments_done_event = json!({ + "type": "response.function_call_arguments.done", + "output_index": 0, + "item_id": "fc_apply_patch_1", + "arguments": "{\"input\":\"*** Begin Patch\n*** End Patch\"}" + }); + server.send_text(&arguments_done_event.to_string()).await; + + let arguments_done_chunk = next_body_chunk(&mut body_stream).await; + let arguments_done_text = + String::from_utf8(arguments_done_chunk.to_vec()).expect("utf8 arguments done chunk"); + let (arguments_done_sse_event, arguments_done_sse_data) = + sse_event_and_data(arguments_done_text.trim_end()); + let arguments_done_payload: Value = + serde_json::from_str(arguments_done_sse_data).expect("arguments done payload json"); + + let done_event = json!({ + "type": "response.output_item.done", + "output_index": 0, + "item": { + "id": "fc_apply_patch_1", + "type": "function_call", + "call_id": "call-apply-patch", + "name": "apply_patch", + "arguments": "{\"input\":\"*** Begin Patch\n*** End Patch\"}" + } + }); + server.send_text(&done_event.to_string()).await; + + let done_chunk = next_body_chunk(&mut body_stream).await; + let done_text = String::from_utf8(done_chunk.to_vec()).expect("utf8 done chunk"); + let (done_sse_event, done_sse_data) = sse_event_and_data(done_text.trim_end()); + let done_payload: Value = serde_json::from_str(done_sse_data).expect("done payload json"); + + let completed_event = + json!({"type": "response.completed", "response": {"id": "response-apply-patch"}}); + server.send_text(&completed_event.to_string()).await; + + let completed_chunk = next_body_chunk(&mut body_stream).await; + let completed_text = String::from_utf8(completed_chunk.to_vec()).expect("utf8 completed chunk"); + let (completed_sse_event, completed_sse_data) = sse_event_and_data(completed_text.trim_end()); + let completed_payload: Value = + serde_json::from_str(completed_sse_data).expect("completed payload json"); + + let done_sentinel_chunk = next_body_chunk(&mut body_stream).await; + let done_sentinel_text = + String::from_utf8(done_sentinel_chunk.to_vec()).expect("utf8 done sentinel chunk"); + assert_done_frame(done_sentinel_text.trim_end()); + assert!( + body_stream.next().await.is_none(), + "expected EOF after downstream DONE sentinel" + ); + + ApplyPatchStreamCapture { + upstream_events: vec![ + added_event, + first_delta_event, + second_delta_event, + arguments_done_event, + done_event, + completed_event, + ], + downstream_events: vec![ + DownstreamSseEvent { + event: added_sse_event.to_string(), + payload: added_payload, + }, + DownstreamSseEvent { + event: first_delta_sse_event.to_string(), + payload: first_delta_payload, + }, + DownstreamSseEvent { + event: second_delta_sse_event.to_string(), + payload: second_delta_payload, + }, + DownstreamSseEvent { + event: arguments_done_sse_event.to_string(), + payload: arguments_done_payload, + }, + DownstreamSseEvent { + event: done_sse_event.to_string(), + payload: done_payload, + }, + DownstreamSseEvent { + event: completed_sse_event.to_string(), + payload: completed_payload, + }, + ], + done_frame: done_sentinel_text.trim_end().to_string(), + } +} + +async fn capture_compaction_stream(compaction_name_field: &str) -> CompactionStreamCapture { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let response = + post_responses(app, json!({"model":"gpt-5.4","input":"compaction-stream"})).await; + assert_eq!(response.status(), StatusCode::OK); + + let _ = server + .recv_client_message() + .await + .expect("compaction stream request"); + + let mut body_stream = response.into_body().into_data_stream(); + + let added_event = json!({ + "type": "response.output_item.added", + "output_index": 0, + "item": { + "id": "cmp_1", + "type": "compaction", + compaction_name_field: "threadline_echo", + "encrypted_content": "opaque-added" + } + }); + server.send_text(&added_event.to_string()).await; + + let added_chunk = next_body_chunk(&mut body_stream).await; + let added_text = String::from_utf8(added_chunk.to_vec()).expect("utf8 added chunk"); + let (added_sse_event, added_sse_data) = sse_event_and_data(added_text.trim_end()); + let added_payload: Value = serde_json::from_str(added_sse_data).expect("added payload json"); + + let done_event = json!({ + "type": "response.output_item.done", + "output_index": 0, + "item": { + "id": "cmp_1", + "type": "compaction", + compaction_name_field: "threadline_echo", + "encrypted_content": "opaque-done" + } + }); + server.send_text(&done_event.to_string()).await; + + let done_chunk = next_body_chunk(&mut body_stream).await; + let done_text = String::from_utf8(done_chunk.to_vec()).expect("utf8 done chunk"); + let (done_sse_event, done_sse_data) = sse_event_and_data(done_text.trim_end()); + let done_payload: Value = serde_json::from_str(done_sse_data).expect("done payload json"); + + let completed_event = json!({ + "type": "response.completed", + "response": { + "id": "response-compaction", + "output": [ + { + "id": "cmp_1", + "type": "compaction", + compaction_name_field: "threadline_echo", + "encrypted_content": "opaque-completed" + }, + { + "type": "message", + "role": "assistant", + "content": [ + { + "type": "output_text", + "text": "done" + } + ] + } + ] + } + }); + server.send_text(&completed_event.to_string()).await; + + let delta_chunk = next_body_chunk(&mut body_stream).await; + let delta_text = String::from_utf8(delta_chunk.to_vec()).expect("utf8 delta chunk"); + let (delta_sse_event, delta_sse_data) = sse_event_and_data(delta_text.trim_end()); + let delta_payload: Value = serde_json::from_str(delta_sse_data).expect("delta payload json"); + + let completed_chunk = next_body_chunk(&mut body_stream).await; + let completed_text = String::from_utf8(completed_chunk.to_vec()).expect("utf8 completed chunk"); + let (completed_sse_event, completed_sse_data) = sse_event_and_data(completed_text.trim_end()); + let completed_payload: Value = + serde_json::from_str(completed_sse_data).expect("completed payload json"); + + let done_sentinel_chunk = next_body_chunk(&mut body_stream).await; + let done_sentinel_text = + String::from_utf8(done_sentinel_chunk.to_vec()).expect("utf8 done sentinel chunk"); + assert_done_frame(done_sentinel_text.trim_end()); + assert!( + body_stream.next().await.is_none(), + "expected EOF after downstream DONE sentinel" + ); + + CompactionStreamCapture { + upstream_events: vec![added_event, done_event, completed_event], + downstream_events: vec![ + DownstreamSseEvent { + event: added_sse_event.to_string(), + payload: added_payload, + }, + DownstreamSseEvent { + event: done_sse_event.to_string(), + payload: done_payload, + }, + DownstreamSseEvent { + event: delta_sse_event.to_string(), + payload: delta_payload, + }, + DownstreamSseEvent { + event: completed_sse_event.to_string(), + payload: completed_payload, + }, + ], + done_frame: done_sentinel_text.trim_end().to_string(), + } +} + +async fn capture_completed_output_stream( + upstream_events: Vec, +) -> CompletedOutputStreamCapture { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let response = post_responses( + app, + json!({"model":"gpt-5.4","input":"completed-output-stream"}), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let _ = server + .recv_client_message() + .await + .expect("completed output stream request"); + + let mut body_stream = response.into_body().into_data_stream(); + for upstream_event in upstream_events { + server.send_text(&upstream_event.to_string()).await; + } + + let mut downstream_events = Vec::new(); + let done_frame = loop { + let chunk = match body_stream.next().await { + Some(Ok(chunk)) => chunk, + Some(Err(error)) => panic!("expected SSE chunk, got body error: {error}"), + None => panic!("expected downstream DONE sentinel before EOF"), + }; + + let chunk_text = String::from_utf8(chunk.to_vec()).expect("utf8 SSE chunk"); + let frame = chunk_text.trim_end(); + if frame == "data: [DONE]" { + break frame.to_string(); + } + + let (event, data) = sse_event_and_data(frame); + let payload: Value = serde_json::from_str(data).expect("SSE payload json"); + downstream_events.push(DownstreamSseEvent { + event: event.to_string(), + payload, + }); + }; + + assert!( + body_stream.next().await.is_none(), + "expected EOF after downstream DONE sentinel" + ); + + CompletedOutputStreamCapture { + downstream_events, + done_frame, + } +} + +fn assistant_output_text_from_completed(payload: &Value) -> String { + let mut text = String::new(); + + let Some(output) = payload["response"]["output"].as_array() else { + return text; + }; + + for item in output { + if item["type"] != "message" || item["role"] != "assistant" { + continue; + } + + let Some(content) = item["content"].as_array() else { + continue; + }; + + for part in content { + if part["type"] != "output_text" { + continue; + } + + if let Some(segment) = part["text"].as_str() { + text.push_str(segment); + } + } + } + + text +} + +fn output_text_delta_strings(events: &[DownstreamSseEvent]) -> Vec { + events + .iter() + .filter(|event| event.event == "response.output_text.delta") + .filter_map(|event| { + event.payload["delta"] + .as_str() + .map(|delta| delta.to_string()) + }) + .collect() +} + +#[tokio::test] +async fn responses_bridge_apply_patch_added_precedes_delta_with_vs_code_required_metadata() { + let capture = capture_visible_apply_patch_stream().await; + + let added_index = capture + .downstream_events + .iter() + .position(|event| event.event == "response.output_item.added") + .expect("added event"); + let first_delta_index = capture + .downstream_events + .iter() + .position(|event| event.event == "response.function_call_arguments.delta") + .expect("first delta event"); + + assert!( + added_index < first_delta_index, + "expected visible function call added event before argument deltas" + ); + + let added_payload = &capture.downstream_events[added_index].payload; + assert_eq!(added_payload["type"], "response.output_item.added"); + assert_eq!(added_payload["output_index"], 0); + assert_eq!(added_payload["item"]["type"], "function_call"); + assert_eq!(added_payload["item"]["name"], "apply_patch"); + assert_eq!(added_payload["item"]["call_id"], "call-apply-patch"); + assert_eq!(added_payload["item"]["id"], "fc_apply_patch_1"); +} + +#[tokio::test] +async fn responses_bridge_apply_patch_delta_matches_added_output_index() { + let capture = capture_visible_apply_patch_stream().await; + + let added_payload = &capture.downstream_events[0].payload; + let added_output_index = added_payload["output_index"].clone(); + let delta_events: Vec<&DownstreamSseEvent> = capture + .downstream_events + .iter() + .filter(|event| event.event == "response.function_call_arguments.delta") + .collect(); + + assert_eq!( + delta_events.len(), + 2, + "expected two visible argument deltas" + ); + for delta_event in delta_events { + assert_eq!( + delta_event.payload["output_index"], added_output_index, + "expected visible argument delta to preserve added output_index" + ); + assert_eq!(delta_event.payload["item_id"], "fc_apply_patch_1"); + } +} + +#[tokio::test] +async fn responses_bridge_apply_patch_done_preserves_complete_arguments() { + let capture = capture_visible_apply_patch_stream().await; + + let arguments_done_payload = &capture.downstream_events[3].payload; + assert_eq!( + arguments_done_payload["type"], + "response.function_call_arguments.done" + ); + assert_eq!(arguments_done_payload["output_index"], 0); + assert_eq!(arguments_done_payload["item_id"], "fc_apply_patch_1"); + assert_eq!( + arguments_done_payload["arguments"], + "{\"input\":\"*** Begin Patch\n*** End Patch\"}" + ); + + let done_payload = &capture.downstream_events[4].payload; + assert_eq!(done_payload["type"], "response.output_item.done"); + assert_eq!(done_payload["output_index"], 0); + assert_eq!(done_payload["item"]["id"], "fc_apply_patch_1"); + assert_eq!(done_payload["item"]["call_id"], "call-apply-patch"); + assert_eq!(done_payload["item"]["name"], "apply_patch"); + assert_eq!( + done_payload["item"]["arguments"], + "{\"input\":\"*** Begin Patch\n*** End Patch\"}" + ); +} + +#[tokio::test] +async fn responses_bridge_visible_function_call_payloads_are_forwarded_without_mutation() { + let capture = capture_visible_apply_patch_stream().await; + + for (index, upstream_event) in capture.upstream_events[..capture.upstream_events.len() - 1] + .iter() + .enumerate() + { + assert_eq!( + capture.downstream_events[index].payload, *upstream_event, + "expected downstream SSE payload to match upstream event for index {index}" + ); + assert_eq!( + capture.downstream_events[index].event, + upstream_event["type"] + .as_str() + .expect("upstream event type"), + "expected downstream SSE event name to match upstream event type for index {index}" + ); + } + assert_eq!(capture.downstream_events[5].event, "response.completed"); + assert_eq!( + capture.downstream_events[5].payload, + capture.upstream_events[5] + ); + assert_eq!(capture.done_frame, "data: [DONE]"); +} + +#[tokio::test] +async fn compaction_output_item_added_is_forwarded_downstream() { + let capture = capture_compaction_stream("name").await; + + assert_eq!( + capture.downstream_events[0].event, + "response.output_item.added" + ); + assert_eq!( + capture.downstream_events[0].payload, + capture.upstream_events[0] + ); + assert_eq!( + capture.downstream_events[0].payload["item"]["type"], + "compaction" + ); + assert_eq!( + capture.downstream_events[0].payload["item"]["encrypted_content"], + "opaque-added" + ); +} + +#[tokio::test] +async fn compaction_output_item_done_is_forwarded_downstream() { + let capture = capture_compaction_stream("tool_name").await; + + assert_eq!( + capture.downstream_events[1].event, + "response.output_item.done" + ); + assert_eq!( + capture.downstream_events[1].payload, + capture.upstream_events[1] + ); + assert_eq!( + capture.downstream_events[1].payload["item"]["type"], + "compaction" + ); + assert_eq!( + capture.downstream_events[1].payload["item"]["encrypted_content"], + "opaque-done" + ); +} + +#[tokio::test] +async fn completed_response_preserves_assistant_text_and_compaction_output() { + let capture = capture_compaction_stream("name").await; + + assert_eq!( + capture.downstream_events[2].event, + "response.output_text.delta" + ); + assert_eq!( + capture.downstream_events[2].payload, + json!({ + "type": "response.output_text.delta", + "delta": "done", + "output_index": 1, + "content_index": 0 + }) + ); + assert_eq!(capture.downstream_events[3].event, "response.completed"); + assert_eq!( + capture.downstream_events[3].payload, + json!({ + "type": "response.completed", + "response": { + "id": "response-compaction", + "output": [ + { + "id": "cmp_1", + "type": "compaction", + "name": "threadline_echo", + "encrypted_content": "opaque-completed" + }, + { + "type": "message", + "role": "assistant", + "content": [ + { + "type": "output_text", + "text": "done" + } + ] + } + ] + } + }) + ); + assert_eq!(capture.done_frame, "data: [DONE]"); +} + +#[tokio::test] +async fn completed_response_preserves_compaction_output() { + let completed_event = json!({ + "type": "response.completed", + "response": { + "id": "response-completed-compaction-only", + "output": [ + { + "id": "cmp-completed-only", + "type": "compaction", + "tool_name": "threadline_echo", + "encrypted_content": "opaque-completed-only" + } + ] + } + }); + + let capture = capture_completed_output_stream(vec![completed_event.clone()]).await; + + assert_eq!(capture.downstream_events.len(), 1); + assert!(output_text_delta_strings(&capture.downstream_events).is_empty()); + assert_eq!(capture.downstream_events[0].event, "response.completed"); + assert_eq!(capture.downstream_events[0].payload, completed_event); + assert_eq!(capture.done_frame, "data: [DONE]"); +} + +#[tokio::test] +async fn completed_response_preserves_compaction_marker_items_while_hiding_internal_function_calls() +{ + let capture = capture_completed_output_stream(vec![json!({ + "type": "response.completed", + "response": { + "id": "response-completed-compaction-sanitized", + "output": [ + { + "type": "function_call", + "name": "threadline_echo", + "call_id": "call-1", + "arguments": "{\"value\":\"alpha\"}" + }, + { + "id": "cmp-preserved", + "type": "compaction", + "tool_name": "threadline_echo", + "encrypted_content": "opaque-compaction" + }, + { + "id": "ctx-preserved", + "type": "context", + "encrypted_content": "opaque-context" + } + ] + } + })]) + .await; + + assert_eq!(capture.downstream_events.len(), 1); + assert!(output_text_delta_strings(&capture.downstream_events).is_empty()); + assert_eq!(capture.downstream_events[0].event, "response.completed"); + assert_eq!( + capture.downstream_events[0].payload, + json!({ + "type": "response.completed", + "response": { + "id": "response-completed-compaction-sanitized", + "output": [ + { + "id": "cmp-preserved", + "type": "compaction", + "tool_name": "threadline_echo", + "encrypted_content": "opaque-compaction" + }, + { + "id": "ctx-preserved", + "type": "context", + "encrypted_content": "opaque-context" + } + ] + } + }) + ); + assert_eq!(capture.done_frame, "data: [DONE]"); +} + +#[tokio::test] +async fn compaction_output_item_done_counts_as_observable_output_when_forwarded() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let response = post_responses(app, json!({"model":"gpt-5.4","input":"compaction-only"})).await; + assert_eq!(response.status(), StatusCode::OK); + + let _ = server + .recv_client_message() + .await + .expect("compaction-only request"); + + server + .send_text( + r#"{"type":"response.output_item.done","output_index":0,"item":{"id":"cmp_1","type":"compaction","tool_name":"threadline_echo","encrypted_content":"opaque-done"}}"#, + ) + .await; + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-compaction-only"}}"#) + .await; + + let body = timeout( + Duration::from_secs(2), + to_bytes(response.into_body(), usize::MAX), + ) + .await + .expect("compaction-only body timeout") + .expect("compaction-only body"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + + assert_eq!(frames.len(), 3); + let done_frame = sse_event_and_data(frames[0]); + let completed_frame = sse_event_and_data(frames[1]); + + assert_eq!(done_frame.0, "response.output_item.done"); + assert_eq!( + serde_json::from_str::(done_frame.1).expect("compaction done json"), + json!({ + "type": "response.output_item.done", + "output_index": 0, + "item": { + "id": "cmp_1", + "type": "compaction", + "tool_name": "threadline_echo", + "encrypted_content": "opaque-done" + } + }) + ); + assert_eq!(completed_frame.0, "response.completed"); + assert_eq!( + serde_json::from_str::(completed_frame.1).expect("compaction completed json"), + json!({ + "type": "response.completed", + "response": { + "id": "response-compaction-only" + } + }) + ); + assert_done_frame(frames[2]); +} + +#[tokio::test] +async fn compaction_only_completed_output_counts_as_observable_output() { + let completed_event = json!({ + "type": "response.completed", + "response": { + "id": "response-compaction-observable-only", + "output": [ + { + "id": "cmp-observable-only", + "type": "compaction", + "name": "threadline_echo", + "encrypted_content": "opaque-observable-only" + } + ] + } + }); + + let capture = capture_completed_output_stream(vec![completed_event.clone()]).await; + + assert_eq!(capture.downstream_events.len(), 1); + assert_eq!(capture.downstream_events[0].event, "response.completed"); + assert_eq!(capture.downstream_events[0].payload, completed_event); + assert_ne!(capture.downstream_events[0].event, "response.failed"); + assert_eq!(capture.done_frame, "data: [DONE]"); +} + +#[tokio::test] +async fn completed_only_assistant_output_text_is_synthesized_as_delta() { + let completed_event = json!({ + "type": "response.completed", + "response": { + "id": "response-completed-only", + "output": [ + { + "id": "assistant-item-1", + "type": "message", + "role": "assistant", + "content": [ + { + "type": "output_text", + "text": "hello from completed" + } + ] + } + ] + } + }); + + let capture = capture_completed_output_stream(vec![completed_event.clone()]).await; + + assert_eq!(capture.downstream_events.len(), 2); + assert_eq!( + capture.downstream_events[0].event, + "response.output_text.delta" + ); + assert_eq!( + capture.downstream_events[0].payload, + json!({ + "type": "response.output_text.delta", + "delta": "hello from completed", + "item_id": "assistant-item-1", + "output_index": 0, + "content_index": 0 + }) + ); + assert_eq!(capture.downstream_events[1].event, "response.completed"); + assert_eq!(capture.downstream_events[1].payload, completed_event); + assert_eq!(capture.done_frame, "data: [DONE]"); +} + +#[tokio::test] +async fn output_text_done_only_text_is_synthesized_as_delta() { + let output_text_done_event = json!({ + "type": "response.output_text.done", + "item_id": "assistant-item-done-only", + "output_index": 0, + "content_index": 0, + "text": "hello from output_text.done" + }); + let completed_event = json!({ + "type": "response.completed", + "response": { + "id": "response-output-text-done-only" + } + }); + + let capture = capture_completed_output_stream(vec![ + output_text_done_event.clone(), + completed_event.clone(), + ]) + .await; + + assert_eq!(capture.downstream_events.len(), 3); + assert_eq!( + capture.downstream_events[0].event, + "response.output_text.delta" + ); + assert_eq!( + capture.downstream_events[0].payload, + json!({ + "type": "response.output_text.delta", + "delta": "hello from output_text.done", + "item_id": "assistant-item-done-only", + "output_index": 0, + "content_index": 0 + }) + ); + assert_eq!( + capture.downstream_events[1].event, + "response.output_text.done" + ); + assert_eq!(capture.downstream_events[1].payload, output_text_done_event); + assert_eq!(capture.downstream_events[2].event, "response.completed"); + assert_eq!( + capture.downstream_events[2].payload, + json!({ + "type": "response.completed", + "response": { + "id": "response-output-text-done-only", + "output": [ + { + "id": "assistant-item-done-only", + "type": "message", + "role": "assistant", + "content": [ + { + "type": "output_text", + "text": "hello from output_text.done", + "annotations": [] + } + ] + } + ] + } + }) + ); + assert_eq!(capture.done_frame, "data: [DONE]"); +} + +#[tokio::test] +async fn completed_without_visible_message_inserts_synthetic_assistant_message_from_done_text() { + let output_text_done_event = json!({ + "type": "response.output_text.done", + "item_id": "assistant-item-done-only", + "output_index": 0, + "content_index": 0, + "text": "hello from output_text.done" + }); + let completed_event = json!({ + "type": "response.completed", + "response": { + "id": "response-synthetic-completed-message", + "output": [ + { + "type": "function_call", + "name": "threadline_echo", + "call_id": "call-1" + }, + { + "id": "cmp-1", + "type": "compaction", + "encrypted_content": "opaque" + } + ] + } + }); + + let capture = + capture_completed_output_stream(vec![output_text_done_event.clone(), completed_event]) + .await; + + assert_eq!(capture.downstream_events.len(), 3); + assert_eq!( + capture.downstream_events[0].event, + "response.output_text.delta" + ); + assert_eq!( + capture.downstream_events[0].payload["delta"], + "hello from output_text.done" + ); + assert_eq!( + capture.downstream_events[1].event, + "response.output_text.done" + ); + assert_eq!(capture.downstream_events[1].payload, output_text_done_event); + assert_eq!(capture.downstream_events[2].event, "response.completed"); + assert_eq!( + capture.downstream_events[2].payload, + json!({ + "type": "response.completed", + "response": { + "id": "response-synthetic-completed-message", + "output": [ + { + "id": "cmp-1", + "type": "compaction", + "encrypted_content": "opaque" + }, + { + "id": "assistant-item-done-only", + "type": "message", + "role": "assistant", + "content": [ + { + "type": "output_text", + "text": "hello from output_text.done", + "annotations": [] + } + ] + } + ] + } + }) + ); + assert_eq!(capture.done_frame, "data: [DONE]"); +} + +#[tokio::test] +async fn codex_output_item_done_message_becomes_vscode_completed_output() { + let output_item_done_event = json!({ + "type": "response.output_item.done", + "output_index": 0, + "item": { + "id": "assistant-item-done-message", + "type": "message", + "role": "assistant", + "content": [ + { + "type": "output_text", + "text": "hello from output_item.done" + } + ] + } + }); + let completed_event = json!({ + "type": "response.completed", + "response": { + "id": "response-output-item-done-message" + } + }); + + let capture = capture_completed_output_stream(vec![ + output_item_done_event.clone(), + completed_event.clone(), + ]) + .await; + + assert_eq!(capture.downstream_events.len(), 3); + assert_eq!( + capture.downstream_events[0].event, + "response.output_text.delta" + ); + assert_eq!( + capture.downstream_events[0].payload, + json!({ + "type": "response.output_text.delta", + "delta": "hello from output_item.done", + "item_id": "assistant-item-done-message", + "output_index": 0, + "content_index": 0 + }) + ); + assert_eq!( + capture.downstream_events[1].event, + "response.output_item.done" + ); + assert_eq!(capture.downstream_events[1].payload, output_item_done_event); + assert_eq!(capture.downstream_events[2].event, "response.completed"); + assert_eq!( + capture.downstream_events[2].payload, + json!({ + "type": "response.completed", + "response": { + "id": "response-output-item-done-message", + "output": [ + { + "id": "assistant-item-done-message", + "type": "message", + "role": "assistant", + "content": [ + { + "type": "output_text", + "text": "hello from output_item.done", + "annotations": [] + } + ] + } + ] + } + }) + ); + assert_eq!(capture.done_frame, "data: [DONE]"); +} + +#[tokio::test] +async fn direct_output_text_delta_backfills_empty_completed_output() { + let delta_event = json!({ + "type": "response.output_text.delta", + "delta": "hello from direct delta", + "item_id": "assistant-item-direct-delta", + "output_index": 0, + "content_index": 0 + }); + let completed_event = json!({ + "type": "response.completed", + "response": { + "id": "response-direct-delta-backfill" + } + }); + + let capture = + capture_completed_output_stream(vec![delta_event.clone(), completed_event.clone()]).await; + + assert_eq!(capture.downstream_events.len(), 2); + assert_eq!( + output_text_delta_strings(&capture.downstream_events), + vec!["hello from direct delta"] + ); + assert_eq!(capture.downstream_events[0].payload, delta_event); + assert_eq!(capture.downstream_events[1].event, "response.completed"); + assert_eq!( + assistant_output_text_from_completed(&capture.downstream_events[1].payload), + "hello from direct delta" + ); + assert_eq!(capture.done_frame, "data: [DONE]"); +} + +#[tokio::test] +async fn visible_text_sources_are_not_duplicated_across_delta_done_item_and_completed() { + let delta_event = json!({ + "type": "response.output_text.delta", + "delta": "hello from every source", + "item_id": "assistant-item-shared", + "output_index": 0, + "content_index": 0 + }); + let output_text_done_event = json!({ + "type": "response.output_text.done", + "item_id": "assistant-item-shared", + "output_index": 0, + "content_index": 0, + "text": "hello from every source" + }); + let output_item_done_event = json!({ + "type": "response.output_item.done", + "output_index": 0, + "item": { + "id": "assistant-item-shared", + "type": "message", + "role": "assistant", + "content": [ + { + "type": "output_text", + "text": "hello from every source" + } + ] + } + }); + let completed_event = json!({ + "type": "response.completed", + "response": { + "id": "response-visible-dedupe", + "output": [ + { + "id": "assistant-item-shared", + "type": "message", + "role": "assistant", + "content": [ + { + "type": "output_text", + "text": "hello from every source" + } + ] + } + ] + } + }); + + let capture = capture_completed_output_stream(vec![ + delta_event.clone(), + output_text_done_event.clone(), + output_item_done_event.clone(), + completed_event.clone(), + ]) + .await; + + assert_eq!( + output_text_delta_strings(&capture.downstream_events), + vec!["hello from every source"] + ); + assert_eq!(capture.downstream_events.len(), 4); + assert_eq!(capture.downstream_events[0].payload, delta_event); + assert_eq!(capture.downstream_events[1].payload, output_text_done_event); + assert_eq!(capture.downstream_events[2].payload, output_item_done_event); + assert_eq!(capture.downstream_events[3].event, "response.completed"); + assert_eq!( + assistant_output_text_from_completed(&capture.downstream_events[3].payload), + "hello from every source" + ); + assert_eq!(capture.done_frame, "data: [DONE]"); +} + +#[tokio::test] +async fn empty_response_completed_emits_no_observable_output_failure() { + let capture = capture_completed_output_stream(vec![json!({ + "type": "response.completed", + "response": { + "id": "response-empty-terminal" + } + })]) + .await; + + assert_eq!(capture.downstream_events.len(), 1); + assert_eq!(capture.downstream_events[0].event, "response.failed"); + assert_eq!( + capture.downstream_events[0].payload, + no_observable_output_failed_event("response-empty-terminal") + ); + assert_eq!(capture.done_frame, "data: [DONE]"); +} + +#[tokio::test] +async fn no_observable_output_diagnostics_do_not_log_arguments_or_encrypted_content() { + let trace_guard = TraceCaptureGuard::begin().await; + let response_id = "response-no-observable-diagnostics"; + let raw_arguments = "{\"api_key\":\"secret-123\"}"; + let encrypted_content = "opaque-encrypted-payload"; + let capture = capture_completed_output_stream(vec![json!({ + "type": "response.completed", + "response": { + "id": response_id, + "output": [ + { + "id": "fc-internal-only", + "type": "function_call", + "call_id": "call-internal-only", + "name": "threadline_echo", + "arguments": raw_arguments + }, + { + "id": "state-marker-internal-only", + "type": "state_marker", + "encrypted_content": encrypted_content + } + ] + } + })]) + .await; + + assert_eq!(capture.downstream_events.len(), 1); + assert_eq!(capture.downstream_events[0].event, "response.failed"); + assert_eq!( + capture.downstream_events[0].payload, + no_observable_output_failed_event(response_id) + ); + + let logs = trace_guard.logs(); + let guard_line = logs + .lines() + .find(|line| { + line.contains("responses_translation_no_observable_output_guard") + && line.contains(response_id) + }) + .expect("guard diagnostics trace line"); + + assert!(guard_line.contains(response_id)); + assert!(guard_line.contains("completed_output_item_types")); + assert!(guard_line.contains("function_call")); + assert!(guard_line.contains("state_marker")); + assert!(!guard_line.contains(raw_arguments)); + assert!(!guard_line.contains(encrypted_content)); + assert!(!guard_line.contains("arguments=")); + assert!(!guard_line.contains("encrypted_content=")); +} + +#[tokio::test] +async fn external_function_call_completed_output_remains_visible() { + let tool_done_event = json!({ + "type": "response.output_item.done", + "output_index": 0, + "item": { + "id": "fc-visible-only", + "type": "function_call", + "call_id": "call-visible-only", + "name": "apply_patch", + "arguments": "{\"input\":\"*** Begin Patch\\n*** End Patch\"}" + } + }); + let completed_event = json!({ + "type": "response.completed", + "response": { + "id": "response-visible-tool-only", + "output": [ + { + "id": "fc-visible-only", + "type": "function_call", + "call_id": "call-visible-only", + "name": "apply_patch", + "arguments": "{\"input\":\"*** Begin Patch\\n*** End Patch\"}" + } + ] + } + }); + + let capture = + capture_completed_output_stream(vec![tool_done_event.clone(), completed_event.clone()]) + .await; + + assert_eq!(capture.downstream_events.len(), 2); + assert_eq!( + capture.downstream_events[0].event, + "response.output_item.done" + ); + assert_eq!(capture.downstream_events[0].payload, tool_done_event); + assert_eq!(capture.downstream_events[1].event, "response.completed"); + assert_eq!(capture.downstream_events[1].payload, completed_event); + assert_eq!(capture.done_frame, "data: [DONE]"); +} + +#[tokio::test] +async fn unknown_marker_like_completed_output_remains_non_observable() { + let capture = capture_completed_output_stream(vec![json!({ + "type": "response.completed", + "response": { + "id": "response-unknown-marker", + "output": [ + { + "id": "marker-1", + "type": "state_marker", + "encrypted_content": "opaque-marker" + } + ] + } + })]) + .await; + + assert_eq!(capture.downstream_events.len(), 1); + assert_eq!(capture.downstream_events[0].event, "response.failed"); + assert_eq!( + capture.downstream_events[0].payload, + no_observable_output_failed_event("response-unknown-marker") + ); + assert_eq!(capture.done_frame, "data: [DONE]"); + assert!(output_text_delta_strings(&capture.downstream_events).is_empty()); +} + +#[tokio::test] +async fn internal_function_call_completed_output_remains_sanitized_and_non_observable() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let response = post_responses( + app.clone(), + json!({"model":"gpt-5.4","input":"internal-only-completed"}), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + let _ = server + .recv_client_message() + .await + .expect("internal-only request"); + + let completed_event = json!({ + "type": "response.completed", + "response": { + "id": "response-internal-only", + "output": [ + { + "type": "function_call", + "name": "threadline_echo", + "call_id": "call-1", + "arguments": "{\"value\":\"alpha\"}" + } + ] + } + }); + server.send_text(&completed_event.to_string()).await; + + let body = timeout( + Duration::from_secs(2), + to_bytes(response.into_body(), usize::MAX), + ) + .await + .expect("internal-only body timeout") + .expect("internal-only body"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + let (event, data) = sse_event_and_data(frames.first().expect("terminal failed frame")); + let payload: Value = serde_json::from_str(data).expect("terminal failed json"); + + assert_eq!(frames.len(), 2); + assert_eq!(event, "response.failed"); + assert_eq!(payload["type"], "response.failed"); + assert_eq!( + payload["response"]["error"]["code"], + "threadline_no_observable_output" + ); + assert_done_frame(frames[1]); + + let rejected = post_responses( + app, + json!({ + "model":"gpt-5.4", + "input":"invalid-internal-only-resume", + "previous_response_id":"response-internal-only" + }), + ) + .await; + + assert_eq!(rejected.status(), StatusCode::BAD_REQUEST); + let rejected_body = to_bytes(rejected.into_body(), usize::MAX) + .await + .expect("rejected body"); + let rejected_payload: Value = serde_json::from_slice(&rejected_body).expect("rejected json"); + assert_eq!( + rejected_payload["error"]["code"], + "previous_response_not_found" + ); +} + +#[tokio::test] +async fn auxiliary_summary_compaction_only_completed_preserves_transient_behavior() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let response = post_responses(app.clone(), auxiliary_summary_request(None)).await; + assert_eq!(response.status(), StatusCode::OK); + let _ = server.recv_client_message().await.expect("summary request"); + + let completed_event = json!({ + "type": "response.completed", + "response": { + "id": "response-summary", + "output": [ + { + "id": "cmp-1", + "type": "compaction", + "name": "threadline_echo", + "encrypted_content": "opaque-summary" + } + ] + } + }); + server.send_text(&completed_event.to_string()).await; + + let body = timeout( + Duration::from_secs(2), + to_bytes(response.into_body(), usize::MAX), + ) + .await + .expect("summary body timeout") + .expect("summary body"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + let (event, data) = sse_event_and_data(frames.first().expect("summary completed frame")); + let payload: Value = serde_json::from_str(data).expect("summary completed json"); + + assert_eq!(frames.len(), 2); + assert_eq!(event, "response.completed"); + assert_eq!(payload, completed_event); + assert_done_frame(frames[1]); + + let rejected = post_responses( + app, + json!({ + "model":"gpt-5.4", + "input":"invalid-summary-resume", + "previous_response_id":"response-summary" + }), + ) + .await; + + assert_eq!(rejected.status(), StatusCode::BAD_REQUEST); + let rejected_body = to_bytes(rejected.into_body(), usize::MAX) + .await + .expect("rejected body"); + let rejected_payload: Value = serde_json::from_slice(&rejected_body).expect("rejected json"); + assert_eq!( + rejected_payload["error"]["code"], + "previous_response_not_found" + ); +} + +#[tokio::test] +async fn image_generation_completed_output_remains_successful_without_text() { + let completed_event = json!({ + "type": "response.completed", + "response": { + "id": "response-image-generation", + "output": [ + { + "id": "img-1", + "type": "image_generation_call", + "result": "image-asset-1" + } + ] + } + }); + + let capture = capture_completed_output_stream(vec![completed_event.clone()]).await; + + assert_eq!(capture.downstream_events.len(), 1); + assert!(output_text_delta_strings(&capture.downstream_events).is_empty()); + assert_eq!(capture.downstream_events[0].event, "response.completed"); + assert_eq!(capture.downstream_events[0].payload, completed_event); + assert_eq!(capture.done_frame, "data: [DONE]"); +} + +#[tokio::test] +async fn forwarded_tool_event_does_not_hide_upstream_response_failed() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let response = post_responses( + app, + json!({"model":"gpt-5.4","input":"visible-tool-then-upstream-failed"}), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let _ = server + .recv_client_message() + .await + .expect("visible tool request"); + + let tool_done_event = json!({ + "type": "response.output_item.done", + "output_index": 0, + "item": { + "id": "fc-visible-failed", + "type": "function_call", + "call_id": "call-visible-failed", + "name": "apply_patch", + "arguments": "{\"input\":\"*** Begin Patch\\n*** End Patch\"}" + } + }); + let failed_event = json!({ + "type": "response.failed", + "response": { + "id": "response-visible-tool-failed" + }, + "error": { + "code": "upstream_response_failed", + "message": "failed" + } + }); + server.send_text(&tool_done_event.to_string()).await; + server.send_text(&failed_event.to_string()).await; + + let body = timeout( + Duration::from_secs(2), + to_bytes(response.into_body(), usize::MAX), + ) + .await + .expect("body timeout") + .expect("body bytes"); + let body_text = String::from_utf8(body.to_vec()).expect("utf8 body"); + let frames = split_sse_frames(&body_text); + + assert_eq!(frames.len(), 3); + let tool_frame = sse_event_and_data(frames[0]); + let tool_payload: Value = serde_json::from_str(tool_frame.1).expect("tool json"); + assert_eq!(tool_frame.0, "response.output_item.done"); + assert_eq!(tool_payload, tool_done_event); + + let failed_frame = sse_event_and_data(frames[1]); + let failed_payload: Value = serde_json::from_str(failed_frame.1).expect("failed json"); + assert_eq!(failed_frame.0, "response.failed"); + assert_eq!(failed_payload["type"], "response.failed"); + assert_eq!( + failed_payload["response"]["id"], + "response-visible-tool-failed" + ); + assert_eq!( + failed_payload["response"]["error"]["code"], + "upstream_response_failed" + ); + assert_eq!(failed_payload["response"]["error"]["message"], "failed"); + assert_done_frame(frames[2]); +} + +#[tokio::test] +async fn missing_visible_text_identity_fields_do_not_duplicate_or_drop_distinct_text() { + let delta_event = json!({ + "type": "response.output_text.delta", + "delta": "repeat" + }); + let output_text_done_event = json!({ + "type": "response.output_text.done", + "text": "repeat" + }); + let output_item_done_event = json!({ + "type": "response.output_item.done", + "item": { + "type": "message", + "role": "assistant", + "content": [ + { + "type": "output_text", + "text": " and distinct" + } + ] + } + }); + let completed_event = json!({ + "type": "response.completed", + "response": { + "id": "response-missing-identity", + "output": [ + { + "type": "message", + "role": "assistant", + "content": [ + { + "type": "output_text", + "text": "repeat and distinct" + } + ] + } + ] + } + }); + + let capture = capture_completed_output_stream(vec![ + delta_event, + output_text_done_event, + output_item_done_event, + completed_event, + ]) + .await; + + assert_eq!( + output_text_delta_strings(&capture.downstream_events), + vec!["repeat", " and distinct"] + ); + assert_eq!( + assistant_output_text_from_completed( + &capture + .downstream_events + .last() + .expect("completed event") + .payload + ), + "repeat and distinct" + ); + assert_eq!(capture.done_frame, "data: [DONE]"); +} + +#[tokio::test] +async fn completed_only_synthetic_delta_precedes_completed_and_done_chunks() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let response = post_responses( + app, + json!({"model":"gpt-5.4","input":"completed-output-order"}), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + + let _ = server + .recv_client_message() + .await + .expect("completed output ordering request"); + + let completed_event = json!({ + "type": "response.completed", + "response": { + "id": "response-ordering", + "output": [ + { + "id": "assistant-item-4", + "type": "message", + "role": "assistant", + "content": [ + { + "type": "output_text", + "text": "ordered text" + } + ] + } + ] + } + }); + server.send_text(&completed_event.to_string()).await; + + let mut body_stream = response.into_body().into_data_stream(); + + let first_chunk = next_body_chunk(&mut body_stream).await; + let first_text = String::from_utf8(first_chunk.to_vec()).expect("utf8 first chunk"); + let (first_event, first_data) = sse_event_and_data(first_text.trim_end()); + let first_payload: Value = serde_json::from_str(first_data).expect("first payload json"); + assert_eq!(first_event, "response.output_text.delta"); + assert_eq!(first_payload["delta"], "ordered text"); + + let second_chunk = next_body_chunk(&mut body_stream).await; + let second_text = String::from_utf8(second_chunk.to_vec()).expect("utf8 second chunk"); + let (second_event, second_data) = sse_event_and_data(second_text.trim_end()); + let second_payload: Value = serde_json::from_str(second_data).expect("second payload json"); + assert_eq!(second_event, "response.completed"); + assert_eq!(second_payload, completed_event); + + let third_chunk = next_body_chunk(&mut body_stream).await; + assert_eq!(third_chunk, Bytes::from_static(b"data: [DONE]\n\n")); + assert!( + body_stream.next().await.is_none(), + "expected EOF after downstream DONE sentinel" + ); +} + +#[tokio::test] +async fn completed_output_marker_is_reusable_after_completed_before_done() { + let server = Arc::new(ScriptedWebSocketServer::start().await); + let connector = RecordingConnector::new(vec![PlannedConnection { + server: Arc::clone(&server), + turn_state: None, + }]); + let app = build_test_router(ThreadlineConfig::default(), Arc::new(connector)); + + let seed = post_responses(app.clone(), json!({"model":"gpt-5.4","input":"seed"})).await; + assert_eq!(seed.status(), StatusCode::OK); + let _ = server.recv_client_message().await.expect("seed request"); + server + .send_text(&assistant_text_completed_event("response-1", "seed completion").to_string()) + .await; + let _ = to_bytes(seed.into_body(), usize::MAX) + .await + .expect("seed body"); + + let active = post_responses( + app.clone(), + json!({ + "model":"gpt-5.4", + "input":"completed-output-order", + "previous_response_id":"response-1" + }), + ) + .await; + assert_eq!(active.status(), StatusCode::OK); + let active_payload: Value = serde_json::from_str(&message_text( + server.recv_client_message().await.expect("active request"), + )) + .expect("active request json"); + assert_eq!(active_payload["previous_response_id"], "response-1"); + + let completed_event = json!({ + "type": "response.completed", + "response": { + "id": "response-ordering", + "output": [ + { + "id": "assistant-item-4", + "type": "message", + "role": "assistant", + "content": [ + { + "type": "output_text", + "text": "ordered text" + } + ] + } + ] + } + }); + server.send_text(&completed_event.to_string()).await; + + let mut active_body = active.into_body().into_data_stream(); + let first_chunk = next_body_chunk(&mut active_body).await; + let first_text = String::from_utf8(first_chunk.to_vec()).expect("utf8 first chunk"); + let (event, data) = sse_event_and_data(first_text.trim_end()); + let payload: Value = serde_json::from_str(data).expect("synthetic delta json"); + assert_eq!(event, "response.output_text.delta"); + assert_eq!(payload["delta"], "ordered text"); + + let second_chunk = next_body_chunk(&mut active_body).await; + let second_text = String::from_utf8(second_chunk.to_vec()).expect("utf8 second chunk"); + let (second_event, second_data) = sse_event_and_data(second_text.trim_end()); + let second_payload: Value = serde_json::from_str(second_data).expect("completed json"); + assert_eq!(second_event, "response.completed"); + assert_eq!(second_payload, completed_event); + + let resumed = post_responses( + app.clone(), + json!({ + "model":"gpt-5.4", + "input":"resume-before-queued-completed", + "previous_response_id":"response-ordering" + }), + ) + .await; + assert_eq!(resumed.status(), StatusCode::OK); + let resumed_payload: Value = serde_json::from_str(&message_text( + server.recv_client_message().await.expect("resumed request"), + )) + .expect("resumed request json"); + assert_eq!(resumed_payload["previous_response_id"], "response-ordering"); + + let third_chunk = next_body_chunk(&mut active_body).await; + assert_eq!(third_chunk, Bytes::from_static(b"data: [DONE]\n\n")); + assert!( + active_body.next().await.is_none(), + "expected EOF after DONE" + ); + + server + .send_text(r#"{"type":"response.completed","response":{"id":"response-3"}}"#) + .await; + let _ = to_bytes(resumed.into_body(), usize::MAX) + .await + .expect("resumed body"); +} + +#[tokio::test] +async fn malformed_completed_output_does_not_panic_or_synthesize_delta() { + let completed_cases = vec![ + ( + "missing-output", + json!({ + "type": "response.completed", + "response": { + "id": "response-missing-output" + } + }), + ), + ( + "output-not-array", + json!({ + "type": "response.completed", + "response": { + "id": "response-output-not-array", + "output": {} + } + }), + ), + ( + "content-missing", + json!({ + "type": "response.completed", + "response": { + "id": "response-content-missing", + "output": [ + { + "type": "message", + "role": "assistant" + } + ] + } + }), + ), + ( + "content-not-array", + json!({ + "type": "response.completed", + "response": { + "id": "response-content-not-array", + "output": [ + { + "type": "message", + "role": "assistant", + "content": {} + } + ] + } + }), + ), + ( + "non-string-text", + json!({ + "type": "response.completed", + "response": { + "id": "response-non-string-text", + "output": [ + { + "type": "message", + "role": "assistant", + "content": [ + { + "type": "output_text", + "text": 42 + } + ] + } + ] + } + }), + ), + ]; + + for (case_name, completed_event) in completed_cases { + let capture = capture_completed_output_stream(vec![completed_event.clone()]).await; + let response_id = completed_event["response"]["id"] + .as_str() + .expect("completed response id"); + assert_eq!( + capture.downstream_events.len(), + 1, + "expected malformed case {case_name} to emit only the terminal failure without a synthetic delta" + ); + assert_eq!( + capture.downstream_events[0].event, "response.failed", + "expected malformed case {case_name} to downgrade the malformed completion into response.failed" + ); + assert_eq!( + capture.downstream_events[0].payload, + no_observable_output_failed_event(response_id), + "expected malformed case {case_name} to emit the stable no-observable-output failure payload" + ); + assert_eq!(capture.done_frame, "data: [DONE]"); + } +} + +#[tokio::test] +async fn multi_part_assistant_output_text_is_synthesized_as_single_delta_from_first_contributing_part_metadata() + { + let completed_event = json!({ + "type": "response.completed", + "response": { + "id": "response-multi-part", + "output": [ + { + "type": "function_call", + "name": "apply_patch", + "call_id": "call-metadata-anchor" + }, + { + "id": "assistant-item-5", + "type": "message", + "role": "assistant", + "content": [ + { + "type": "output_text", + "text": " " + }, + { + "type": "output_text", + "text": "Hello" + }, + { + "type": "output_text", + "text": "" + }, + { + "type": "output_text", + "text": "\n" + }, + { + "type": "output_text", + "text": " world" + } + ] + }, + { + "id": "assistant-item-6", + "type": "message", + "role": "assistant", + "content": [ + { + "type": "output_text", + "text": "!" + } + ] + } + ] + } + }); + + let capture = capture_completed_output_stream(vec![completed_event.clone()]).await; + + assert_eq!(capture.downstream_events.len(), 2); + assert_eq!( + capture.downstream_events[0].payload, + json!({ + "type": "response.output_text.delta", + "delta": "Hello world!", + "item_id": "assistant-item-5", + "output_index": 1, + "content_index": 1 + }) + ); + assert_eq!(capture.downstream_events[1].event, "response.completed"); + assert_eq!(capture.downstream_events[1].payload, completed_event); + assert_eq!(capture.done_frame, "data: [DONE]"); +} diff --git a/tests/support/scripted_ws.rs b/tests/support/scripted_ws.rs new file mode 100644 index 0000000..71f7b6d --- /dev/null +++ b/tests/support/scripted_ws.rs @@ -0,0 +1,181 @@ +use std::sync::Arc; +use std::sync::atomic::{AtomicBool, Ordering}; + +use futures_util::{SinkExt, StreamExt}; +use tokio::net::TcpListener; +use tokio::sync::{Mutex, Notify, mpsc}; +use tokio::task::JoinHandle; +use tokio_tungstenite::accept_async; +use tokio_tungstenite::tungstenite::Message; + +type ServerSink = futures_util::stream::SplitSink< + tokio_tungstenite::WebSocketStream, + Message, +>; + +#[derive(Clone, Copy)] +enum StartupBehavior { + KeepAlive, + DisconnectAfterHandshake, +} + +pub struct ScriptedWebSocketServer { + url: String, + writer: Arc>>, + incoming_rx: Mutex>>, + connected: Arc, + is_connected: Arc, + accept_task: JoinHandle<()>, + reader_task: Arc>>>, +} + +#[allow(dead_code)] +impl ScriptedWebSocketServer { + pub async fn start() -> Self { + Self::start_with_behavior(StartupBehavior::KeepAlive).await + } + + pub async fn start_disconnect_after_handshake() -> Self { + Self::start_with_behavior(StartupBehavior::DisconnectAfterHandshake).await + } + + async fn start_with_behavior(startup_behavior: StartupBehavior) -> Self { + let listener = TcpListener::bind("127.0.0.1:0") + .await + .expect("bind listener"); + let address = listener.local_addr().expect("local addr"); + let url = format!("ws://{address}"); + let writer = Arc::new(Mutex::new(None)); + let reader_task = Arc::new(Mutex::new(None)); + let (incoming_tx, incoming_rx) = mpsc::unbounded_channel(); + let connected = Arc::new(Notify::new()); + let is_connected = Arc::new(AtomicBool::new(false)); + + let accept_writer = Arc::clone(&writer); + let accept_reader_task = Arc::clone(&reader_task); + let accept_connected = Arc::clone(&connected); + let accept_is_connected = Arc::clone(&is_connected); + let accept_task = tokio::spawn(async move { + let (stream, _) = listener.accept().await.expect("accept client"); + let websocket = accept_async(stream).await.expect("accept websocket"); + + if matches!(startup_behavior, StartupBehavior::DisconnectAfterHandshake) { + accept_is_connected.store(true, Ordering::SeqCst); + accept_connected.notify_waiters(); + drop(websocket); + return; + } + + let (sink, mut stream) = websocket.split(); + *accept_writer.lock().await = Some(sink); + accept_is_connected.store(true, Ordering::SeqCst); + accept_connected.notify_waiters(); + + let reader = tokio::spawn(async move { + while let Some(message) = stream.next().await { + match message { + Ok(message) => { + if incoming_tx.send(message).is_err() { + break; + } + } + Err(_) => break, + } + } + }); + *accept_reader_task.lock().await = Some(reader); + }); + + Self { + url, + writer, + incoming_rx: Mutex::new(Some(incoming_rx)), + connected, + is_connected, + accept_task, + reader_task, + } + } + + pub fn url(&self) -> &str { + &self.url + } + + pub async fn send_text(&self, text: &str) { + self.send(Message::Text(text.to_string())).await; + } + + pub async fn send_binary(&self, payload: Vec) { + self.send(Message::Binary(payload)).await; + } + + pub async fn send_ping(&self, payload: &[u8]) { + self.send(Message::Ping(payload.to_vec())).await; + } + + pub async fn send_close(&self, code: u16, reason: &str) { + self.send(Message::Close(Some( + tokio_tungstenite::tungstenite::protocol::CloseFrame { + code: tokio_tungstenite::tungstenite::protocol::frame::coding::CloseCode::from( + code, + ), + reason: reason.to_string().into(), + }, + ))) + .await; + } + + pub async fn recv_client_message(&self) -> Option { + self.wait_until_connected().await; + let mut incoming_rx = self.incoming_rx.lock().await; + let receiver = incoming_rx + .as_mut() + .expect("incoming receiver should remain available"); + receiver.recv().await + } + + pub async fn take_pending_client_messages(&self) -> Vec { + self.wait_until_connected().await; + let mut incoming_rx = self.incoming_rx.lock().await; + let receiver = incoming_rx + .as_mut() + .expect("incoming receiver should remain available"); + let mut messages = Vec::new(); + while let Ok(message) = receiver.try_recv() { + messages.push(message); + } + messages + } + + pub async fn abort_connection(&self) { + self.wait_until_connected().await; + self.writer.lock().await.take(); + if let Some(task) = self.reader_task.lock().await.take() { + task.abort(); + } + } + + async fn send(&self, message: Message) { + self.wait_until_connected().await; + let mut writer = self.writer.lock().await; + let sink = writer.as_mut().expect("client connected"); + sink.send(message).await.expect("send scripted message"); + } + + async fn wait_until_connected(&self) { + while !self.is_connected.load(Ordering::SeqCst) { + self.connected.notified().await; + } + } +} + +impl Drop for ScriptedWebSocketServer { + fn drop(&mut self) { + self.accept_task.abort(); + if let Ok(mut guard) = self.reader_task.try_lock() + && let Some(task) = guard.take() + { + task.abort(); + } + } +} diff --git a/tests/ws_pump.rs b/tests/ws_pump.rs new file mode 100644 index 0000000..88636f2 --- /dev/null +++ b/tests/ws_pump.rs @@ -0,0 +1,157 @@ +use std::time::Duration; + +#[path = "support/scripted_ws.rs"] +mod scripted_ws; + +use scripted_ws::ScriptedWebSocketServer; +use threadline::ws_pump::{LiveUpstreamWebSocket, UpstreamCloseMetadata}; +use tokio::time::{sleep, timeout}; +use tokio_tungstenite::connect_async; +use tokio_tungstenite::tungstenite::Message; + +async fn connect_pump(server: &ScriptedWebSocketServer) -> LiveUpstreamWebSocket { + let (stream, _) = connect_async(server.url()) + .await + .expect("connect client websocket"); + LiveUpstreamWebSocket::from_stream(stream) +} + +async fn wait_for_closed(pump: &LiveUpstreamWebSocket) -> UpstreamCloseMetadata { + timeout(Duration::from_secs(2), async { + loop { + if pump.is_closed() + && let Some(metadata) = pump.close_metadata().await + { + break metadata; + } + sleep(Duration::from_millis(10)).await; + } + }) + .await + .expect("pump should close") +} + +#[tokio::test] +async fn websocket_pump_replies_to_server_ping_while_idle() { + let server = ScriptedWebSocketServer::start().await; + let pump = connect_pump(&server).await; + + server.send_ping(b"idle-check").await; + + let message = timeout(Duration::from_secs(1), server.recv_client_message()) + .await + .expect("pong timeout") + .expect("client message"); + + match message { + Message::Pong(payload) => assert_eq!(payload.as_slice(), b"idle-check"), + other => panic!("expected pong, got {other:?}"), + } + assert!(!pump.is_closed()); +} + +#[tokio::test] +async fn websocket_pump_replies_to_server_ping_when_inbound_messages_are_not_consumed() { + let server = ScriptedWebSocketServer::start().await; + let pump = connect_pump(&server).await; + + server.send_text("queued-text").await; + server.send_binary(b"queued-binary".to_vec()).await; + server.send_ping(b"still-alive").await; + + let message = timeout(Duration::from_secs(1), server.recv_client_message()) + .await + .expect("pong timeout") + .expect("client message"); + match message { + Message::Pong(payload) => assert_eq!(payload.as_slice(), b"still-alive"), + other => panic!("expected pong, got {other:?}"), + } + + assert_eq!( + pump.recv_text().await.expect("recv text"), + Some("queued-text".to_string()) + ); + assert_eq!( + pump.recv_text().await.expect("recv binary as text"), + Some("queued-binary".to_string()) + ); +} + +#[tokio::test] +async fn websocket_pump_send_text_only_queues_outbound_messages() { + let server = ScriptedWebSocketServer::start().await; + let pump = connect_pump(&server).await; + + pump.send_text("from-threadline").await.expect("send text"); + + let message = timeout(Duration::from_secs(1), server.recv_client_message()) + .await + .expect("text timeout") + .expect("client message"); + match message { + Message::Text(text) => assert_eq!(text.as_str(), "from-threadline"), + other => panic!("expected text, got {other:?}"), + } + + assert!( + timeout(Duration::from_millis(100), pump.recv_text()) + .await + .is_err() + ); +} + +#[tokio::test] +async fn websocket_pump_records_close_metadata_without_panicking() { + let server = ScriptedWebSocketServer::start().await; + let pump = connect_pump(&server).await; + + server.send_close(1000, "done").await; + + let metadata = wait_for_closed(&pump).await; + + assert_eq!(metadata.code, Some(1000)); + assert_eq!(metadata.reason.as_deref(), Some("done")); + assert_eq!(metadata.error, None); +} + +#[tokio::test] +async fn websocket_pump_records_error_metadata_when_connection_drops() { + let server = ScriptedWebSocketServer::start().await; + let pump = connect_pump(&server).await; + + server.abort_connection().await; + + let metadata = wait_for_closed(&pump).await; + + assert_eq!(metadata.code, None); + assert!(metadata.reason.is_none()); + assert!(metadata.error.is_some()); +} + +#[tokio::test] +async fn websocket_pump_replies_to_server_ping_after_a_retained_idle_gap() { + let server = ScriptedWebSocketServer::start().await; + let pump = connect_pump(&server).await; + + server.send_text("response-completed").await; + assert_eq!( + pump.recv_text().await.expect("recv completed event"), + Some("response-completed".to_string()) + ); + + sleep(Duration::from_millis(100)).await; + server.send_ping(b"retained-idle-check").await; + + let message = timeout(Duration::from_secs(1), server.recv_client_message()) + .await + .expect("pong timeout") + .expect("client message"); + + match message { + Message::Pong(payload) => assert_eq!(payload.as_slice(), b"retained-idle-check"), + other => panic!("expected pong, got {other:?}"), + } + + assert!(!pump.is_closed()); +}