Skip to content

runtime-notes.md

Latest commit

 

History

History
85 lines (72 loc) · 5.53 KB

runtime-notes.md

File metadata and controls

85 lines (72 loc) · 5.53 KB
Raw

Codex hook runtime integration and verification

Technical details for the Codex Effort Hooks experiment. Start with the README for the policy example, quick start and compatibility summary.

Native integration

  1. runtime_lab.py starts codex app-server --stdio with a disposable child profile and a local deterministic Responses service. It enables step_model_switching only in that profile for the positive cases.
  2. It discovers the generated hooks using hooks/list, checks their source and exact command, then trusts their current hashes with config/batchWrite. The untrusted case skips this step and asserts that no hooks run.
  3. Native PreToolUse / PostToolUse Bash hooks execute hook_client.py. The client sends the event to an authenticated loopback controller. It refuses proxies, redirects, non-loopback URLs and unexpected endpoint shapes.
  4. Post-tool events are joined to app-server item/completed commandExecution events using thread, turn and tool-use IDs. The structured exitCode supplies the outcome: Bash hook tool_response itself contains text rather than a structured exit status in the inspected version.
  5. effort_policy.py proposes a change. The controller allows only medium and ultra, requests turn/settings/update, and adopts its new policy state only after the server reports applied. It changes no command permissions.
  6. The local Responses fixture records the actual continuation requests. Native ultra is acknowledged, while the bundled gpt-6-astra catalog resolves its reasoning effort to wire value xhigh; the test checks that normalization.

Native negative probes cover a numeric effort value, an unsupported settings field, and a wrong turn ID. The protocol's effort type is a string; arbitrary unsupported strings are not reliably rejected by that layer. The host-side allowlist therefore rejects unknown effort names before issuing an RPC.

Git processes use isolated configuration and empty init templates, so user hooks and remote URL rewrites do not affect the fixtures. The three Git-index write approvals are one-time decisions for the exact fixture commands, workspace, thread and turn. They do not grant approval for real model commands or deploy operations. Hook trust uses the normal hash-based mechanism.

Boundaries

  • This proves native hook execution, runtime setting changes and subsequent request serialization. Model responses are deterministic local fixtures: no OpenAI inference, reasoning quality, billing or cache-preservation claim is made.
  • multi_agent is disabled in the experiment. The test does not reproduce every desktop behavior associated with the Ultra preset.
  • Running the desktop's binary separately does not attach to a desktop task. A supported desktop hook/settings bridge is still needed for this workflow inside an existing app task. No global feature flag or hook was changed.
  • Commands are an exact, sequential manifest. Generic shell parsing, simultaneous tool execution, recovery verification across unrelated operations, user effort overrides and production rollout are outside this proof.
  • Remote deployment submission is not proof of success. A real integration needs a provider-specific completion check before it can return to medium.
  • Bash is the tested hook adapter. Other tools and providers need their own structured outcome contract and tests.

Failure, cleanup and reruns

verify.py starts by invalidating the prior report (passed: false) and exits nonzero on failure. Both verification entrypoints reject optimized Python, which would disable their assertions. Fix the failure, then run python3 verify.py again; it creates fresh fixtures and does not resume a partially mutated Git repo.

Every normal runtime exit closes its app-server process and local HTTP server. Temporary fixture directories and their logs remain at the paths in the report for inspection. Delete only those reported directories when no longer needed. If the runner is forcibly killed, check for a remaining child process before removing its directory. Never retry an effort RPC after an acknowledgement timeout: its native state is unknown. End that fixture and start a fresh run.

Child-only environment variables are CODEX_HOME (disposable profile), EFFORT_LAB_ENDPOINT (loopback hook URL) and EFFORT_LAB_TOKEN (ephemeral authentication token, not written to logs). The parent environment and user configuration remain unchanged. Git uses child-only isolated configuration. Generated evidence, logs and caches are ignored by Git. Raw request evidence contains only fixture data; do not reuse this logging setup for real prompts or credentials.

Inspected upstream sources