Repository navigation
Generalize Foreman decision models and add Jev/OpenAI adapters - #54
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Foreman currently couples both responsibility routing and recurring assessment to Jev. Introduce a provider-independent decision-model contract and shared assessment/router implementations, then supply Jev and OpenAI native Decisions adapters. Worker backends and Python policy remain independent of the provider. Update the architecture, setup, configuration, evidence, and integration docs around decision models.
Jev remains the default. Existing Jev constructors, timeout/budget environment settings, and Python configuration inputs remain compatible. OpenAI is an optional
foreman-core[openai]dependency, selectable withFOREMAN_DECISION_PROVIDER=openai; it submits named predicate questions to/v1/decisionsusinggpt-6-lunaby default. Refusals and malformed probabilities reject the call through existing failure handling. Both paths share provider-aware budgets and metadata-only telemetry indecision-usage.jsonl.Validation:
git diff --checkpassed.jev-1.13.0) on isolated copies offoreman-test-appdistinguished incomplete, verified, and intentionally regressed implementations: requirements-satisfied scores 0.04, 0.91, and 0.03. The original repo's files and existing changes were preserved.FINISHEDvalidation. No thresholds were adjusted to force success.Live OpenAI validation now passed using the native
/v1/decisionsAPI andgpt-6-lunafor both routing and checks. On the exact same frozen calculator observations, requirements-satisfied scores were 0.00 before implementation, 0.95 after verification, and 0.00 after an intentional regression. Documentation routing scored 1.00. The key was loaded only into the test process from a user-authorized location; no credentials were copied or committed. The original test repo is unchanged.FOREMAN_DECISION_MODELselects only the router/check decision model. The PR contains no worker-model overrides or changes to worker-model fields or runtime worker selection. Temporary worker overrides used by the separate managed test were outside this repository and have been removed. Managed-run loading of the central.envis limited to decision settings and scoped provider credentials; three additional regression cases verify that it cannot select a worker backend/model or overwrite a worker's existing OpenAI credentials or endpoint. Full evidence and limitations are indocs/decision-model-validation.md.Leave this PR unmerged for the user's review. ThruWire Dev currently has only the merge-gated FeatureComplete collection, with no appropriate pre-merge progress record. Its release receipt remains pending merge; no completed receipt was published and no new block was created.