Skip to content

Generalize Foreman decision models and add Jev/OpenAI adapters - #54

Merged
JARosen merged 2 commits into
mainfrom
codex/decision-model-adapters
Oct 15, 2026
Merged

JARosen merged 2 commits into
mainfrom
codex/decision-model-adapters

Conversation

@JARosen

@JARosen JARosen commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Foreman currently couples both responsibility routing and recurring assessment to Jev. Introduce a provider-independent decision-model contract and shared assessment/router implementations, then supply Jev and OpenAI native Decisions adapters. Worker backends and Python policy remain independent of the provider. Update the architecture, setup, configuration, evidence, and integration docs around decision models.

Jev remains the default. Existing Jev constructors, timeout/budget environment settings, and Python configuration inputs remain compatible. OpenAI is an optional foreman-core[openai] dependency, selectable with FOREMAN_DECISION_PROVIDER=openai; it submits named predicate questions to /v1/decisions using gpt-6-luna by default. Refusals and malformed probabilities reject the call through existing failure handling. Both paths share provider-aware budgets and metadata-only telemetry in decision-usage.jsonl.

Validation:

  • GitHub CI passed at commit 6c0bb97 for Python 3.11, 3.12, 3.13 and Pi: https://github.com/thruwire/foreman/actions/runs/38444549226.
  • 466 Python tests passed from source and from a wheel installed in a fresh Python 3.12 environment; Ruff passed.
  • All 27 Pi/Pi Durable integration tests passed against the installed wheel.
  • Local documentation links and git diff --check passed.
  • Live Jev (jev-1.13.0) on isolated copies of foreman-test-app distinguished incomplete, verified, and intentionally regressed implementations: requirements-satisfied scores 0.04, 0.91, and 0.03. The original repo's files and existing changes were preserved.
  • A separate managed Jev/Codex run completed coding and independent verification with 33 passing tests, but the factory escalated at its worker limit when final scores remained below policy thresholds. Native Foreman hooks were also active inside those worker sessions; this is not a successful FINISHED validation. No thresholds were adjusted to force success.

Live OpenAI validation now passed using the native /v1/decisions API and gpt-6-luna for both routing and checks. On the exact same frozen calculator observations, requirements-satisfied scores were 0.00 before implementation, 0.95 after verification, and 0.00 after an intentional regression. Documentation routing scored 1.00. The key was loaded only into the test process from a user-authorized location; no credentials were copied or committed. The original test repo is unchanged.

FOREMAN_DECISION_MODEL selects only the router/check decision model. The PR contains no worker-model overrides or changes to worker-model fields or runtime worker selection. Temporary worker overrides used by the separate managed test were outside this repository and have been removed. Managed-run loading of the central .env is limited to decision settings and scoped provider credentials; three additional regression cases verify that it cannot select a worker backend/model or overwrite a worker's existing OpenAI credentials or endpoint. Full evidence and limitations are in docs/decision-model-validation.md.

Leave this PR unmerged for the user's review. ThruWire Dev currently has only the merge-gated FeatureComplete collection, with no appropriate pre-merge progress record. Its release receipt remains pending merge; no completed receipt was published and no new block was created.

@JARosen
JARosen marked this pull request as ready for review October 15, 2026 12:22
@JARosen
JARosen merged commit cfe4ad5 into main Oct 15, 2026
4 checks passed
@JARosen
JARosen deleted the codex/decision-model-adapters branch October 15, 2026 12:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant