Skip to content

Add an LLM steering brain behind the DecidePort seam#546

Merged
xmap merged 1 commit into
mainfrom
worktree-llm-decide-port
Jul 6, 2026
Merged

Add an LLM steering brain behind the DecidePort seam#546
xmap merged 1 commit into
mainfrom
worktree-llm-decide-port

Conversation

@xmap

@xmap xmap commented Jul 6, 2026

Copy link
Copy Markdown
Owner

Summary

  • Adds LlmDecidePort, a fifth DecidePort substrate that asks an LLM for the next autonomous-experiment action (Measure a next point, or Stop) from the full SteeringEvidence. The DecidePort docstring already named an LLM as an intended brain and the LLM port named a Strategy consumer; this fills that seam.
  • Purely additive: no new aggregate, agent seed, or subscriber, so it does not touch the agent-subscriber Reaction-widening trigger.
  • Homed in cora.operation.adapters (not cora.agent.prompts) because tach forbids cora.operation -> cora.agent; consumes only the LLM port + shared steering value types.

Design

  • Stateless (full evidence handed over each call, fit for replay).
  • Translates the LLMError family into the Decide*Error taxonomy the conduct loop already folds into a deferred steering decision (never crashes the loop).
  • Guards against a hallucinated next_point naming an axis outside the SteeringSpace; SteeringAdvice self-validation catches a malformed verdict/point pairing and out-of-range confidence.
  • Evidence serialized via canonical_json_bytes so the prompt-cache prefix stays byte-stable across write/replay.
  • Wired as the llm substrate in build_decide_port via an injected llm port (ValueError when absent, mirroring the bo-missing guard); both conduct_until_advised handlers pass deps.llm through.

Scope (Stage 0)

  • Kept OFF the wire (WireDecideSubstrate unchanged) and NOT yet the ExperimentSteerer default; those are deliberate later stages.

Test plan

  • 15 new unit tests (test_llm_decide_port.py): Measure/Stop, full-error-mapping table, unknown verdict, Measure-without-point, hallucinated-axis guard, factory arm + llm-required guard.
  • Full operation unit slice green (1170 passed).
  • Architecture/tach green (proves operation -> infrastructure.ports.llm import legal + decide-port purity holds + canonical-json single-source rule).
  • pyright + ruff clean.
  • End-to-end drive through build_decide_port(..., llm=FakeLLM()) + advise_next.

🤖 Generated with Claude Code

Introduce LlmDecidePort, a fifth DecidePort substrate that asks an LLM for
the next autonomous-experiment action (Measure a next point, or Stop) given
the full SteeringEvidence. The DecidePort docstring already names an LLM as
an intended brain and the LLM port names a Strategy consumer; this fills that
seam without a new aggregate, agent seed, or subscriber, so it does not touch
the agent-subscriber Reaction-widening trigger.

The adapter is homed in cora.operation.adapters (not cora.agent.prompts)
because tach forbids cora.operation from importing cora.agent; it consumes
only the LLM port and the shared steering value types. It is stateless (full
evidence handed over each call, fit for replay), translates the LLMError
family into the Decide*Error taxonomy the conduct loop already folds into a
deferred steering decision, and guards against a hallucinated next_point that
names an axis outside the SteeringSpace. SteeringAdvice self-validation
catches a malformed verdict/point pairing and out-of-range confidence. The
evidence is serialized via canonical_json_bytes so the prompt-cache prefix
stays byte-stable across write-time and replay-time.

Wired as the "llm" substrate in build_decide_port via an injected llm port
(ValueError when absent, mirroring the bo-missing guard); both
conduct_until_advised handlers pass deps.llm through. Kept off the wire
(WireDecideSubstrate unchanged) and not yet the ExperimentSteerer default;
those are deliberate later stages.
@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  apps/api/src/cora/operation/adapters
  _llm_decide_prompt.py
  decide_port_config.py
  llm_decide_port.py
  apps/api/src/cora/operation/features/conduct_until_advised
  handler.py
  apps/api/src/cora/operation/features/conduct_until_advised_from
  handler.py
  apps/api/src/cora/operation/ports
  decide_port.py
Project Total  

This report was generated by python-coverage-comment-action

@xmap xmap merged commit bb69534 into main Jul 6, 2026
16 checks passed
@xmap xmap deleted the worktree-llm-decide-port branch July 6, 2026 02:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant