At the step limit, one request with no tools, and the prose is the answer

CHANGELOG.md

@@ -17,6 +17,20 @@ User-visible changes to Imp are recorded here.
   those fields, and so does a step that says nothing at all. The new option
   `prose: :forced_submit` keeps the old behaviour for a single-output
   signature.
+- `ReActV2` gains `on_max_iters`, what the step limit does. The default,
+  `:forced_submit`, is what it did before: one more request with `tool_choice`
+  naming `submit`. `:last_prose` makes one more request with no tools in it at
+  all, so the only thing the model can do is speak, and that prose is the
+  single text output, with `termination_reason: :last_prose`. This is the
+  ending that fits a host whose model already finishes turns by writing prose:
+  it is never asked to call a tool it did not choose. A completion that says
+  nothing finishes with an empty answer rather than an error. `:last_prose`
+  needs a signature with exactly one output of type `:string`, and is refused
+  at construction otherwise. The companion option `last_prose_note`, a string,
+  puts one line of host text in front of that request as a user message and
+  keeps it in the returned history; Imp writes no sentence of its own. Both
+  options persist through `dump`/`load`, and a dump written before them loads
+  as `:forced_submit`.
 - `ReActV2` gains `finish_on`, a map from tool name to
   `fn arguments, result, inputs -> {:finish, outputs} | :continue end`. A tool
   named there ends the turn with the outputs the function returns, which are

docs/differentials/REACT_V2_FIDELITY.md

@@ -43,6 +43,7 @@ existing fail-fast `Imp.Predict.ReAct`.
 | Unknown tools and execution failures become observations | ReActV2 records error results and continues; existing ReAct remains fail-fast | recovery test |
 | `submit` is reserved and validates final outputs | Constructor rejects user `submit`; the generated submit tool uses the task JSON schema | reserved-submit and missing-output tests |
 | Empty calls, parse failure, context exhaustion, or budget exhaustion force one submit call | Parse failure, context exhaustion and budget exhaustion still do: the final predictor call pins provider `tool_choice` to `submit` and clears `reasoning_effort`, matching the pinned call configuration. A step of prose with no tool call does not, when the task declares exactly one text output: that prose is the output and the turn is over, as it is in Anthropic's tool runner, the OpenAI Agents SDK, LangGraph's ReAct and Pydantic AI. `prose: :forced_submit` restores the upstream shape | forced-submit test, prose-answer test |
+| No upstream equivalent | `on_max_iters: :last_prose` ends a turn that reaches the step limit with one request that carries no tools, so the model can only speak; that prose is the single text output, with `termination_reason: :last_prose`, and an empty completion is an empty answer. `:last_prose_note` puts one line of host text in front of that request. The default stays the forced submit | `test/react_v2_last_prose_test.exs` |
 | No upstream equivalent | `finish_on` names tools that end the turn with the outputs they carry, the shape Pydantic AI calls an output tool | `finish_on` tests |
 | Prior calls replay as native assistant/tool messages | Chat adapter emits assistant `tool_calls` and matching tool-result messages by call ID | native history adapter test and ReqLLM tests |
 

lib/imp/predict/react_v2.ex

@@ -28,8 +28,19 @@ defmodule Imp.Predict.ReActV2 do
       execute and are recorded, and a `submit` in the same step still wins.
       Outputs that fail validation are recorded as that call's result, the same
       error a bad `submit` records, and the loop continues.
-    * `max_iters`, or a prediction error. The loop forces one more request with
-      `tool_choice` naming `submit` (`termination_reason: :forced_submit`).
+    * `max_iters`. What the step limit does is `on_max_iters`. The default,
+      `:forced_submit`, makes one more request with `tool_choice` naming
+      `submit` (`termination_reason: :forced_submit`). `:last_prose` makes one
+      more request with no tools in it at all, so the only thing the model can
+      do is speak; that prose is the single text output and
+      `termination_reason: :last_prose`, and a completion that says nothing is
+      an empty answer rather than an error. `:last_prose` needs a signature
+      with exactly one output of type `:string`, and is refused at
+      construction otherwise. `:last_prose_note` puts one line of host text in
+      front of that request as a user message; Imp writes no sentence of its
+      own.
+    * A prediction error. The loop forces one more request with `tool_choice`
+      naming `submit` (`termination_reason: :forced_submit`).
 
   A step's outputs are `next_thought` and `tool_calls`. The provider holds the
   tool roster natively, so a step normally comes back as native tool calls. A
@@ -45,7 +56,8 @@ defmodule Imp.Predict.ReActV2 do
   forced request says nothing
   about why by default; `:forced_submit_notice`, a string or a 1-arity function
   of the termination reason, adds one user-visible turn saying so, which is kept
-  in the returned history like any other turn. If a provider cannot
+  in the returned history like any other turn, and `:last_prose_note` does the
+  same for the `:last_prose` request. If a provider cannot
   honor that tool contract, a tools-disabled typed extractor derives the task
   outputs from the original inputs and accumulated history.
 
@@ -69,10 +81,12 @@ defmodule Imp.Predict.ReActV2 do
     :signature,
     :react,
     :forced_submit_notice,
+    :last_prose_note,
     tools: %{},
     max_iters: 20,
     tool_policy: :allow,
     prose: :answer,
+    on_max_iters: :forced_submit,
     finish_on: %{}
   ]
 
@@ -98,6 +112,15 @@ defmodule Imp.Predict.ReActV2 do
     # mainstream tool loop does. `:forced_submit` keeps the older behaviour of
     # one more request with `tool_choice` naming submit.
     prose: [type: {:in, [:answer, :forced_submit]}, default: :answer],
+    # What the step limit does. `:forced_submit` makes one more request with
+    # `tool_choice` naming submit. `:last_prose` makes one more request with no
+    # tools in it, so the only thing the model can do is speak, and what it
+    # says is the answer; it needs a signature with one text output for that
+    # prose to be.
+    on_max_iters: [type: {:in, [:forced_submit, :last_prose]}, default: :forced_submit],
+    # One line of text put in front of the `:last_prose` request as a user
+    # message. nil says nothing, and Imp never writes a sentence of its own.
+    last_prose_note: [type: {:or, [:string, nil]}, default: nil],
     # Tools that end the turn with the outputs they carry, the shape Pydantic
     # AI calls an output tool. Name to
     # `fn arguments, result, inputs -> {:finish, outputs} | :continue end`.
@@ -113,6 +136,13 @@ defmodule Imp.Predict.ReActV2 do
       raise ArgumentError, "submit is reserved by Imp.Predict.ReActV2"
     end
 
+    if opts[:on_max_iters] == :last_prose and not single_text_output?(signature) do
+      raise ArgumentError,
+            "Imp.Predict.ReActV2.new/3: on_max_iters: :last_prose needs a signature with " <>
+              "exactly one output of type :string, got: " <>
+              inspect(Imp.Signature.output_names(signature))
+    end
+
     submit = Imp.Tool.new(:submit, "Submit the final outputs for the task.", & &1)
     tools = Map.put(tools, :submit, submit)
 
@@ -161,7 +191,9 @@ defmodule Imp.Predict.ReActV2 do
       max_iters: opts[:max_iters],
       tool_policy: opts[:tool_policy],
       forced_submit_notice: opts[:forced_submit_notice],
+      last_prose_note: opts[:last_prose_note],
       prose: opts[:prose],
+      on_max_iters: opts[:on_max_iters],
       finish_on: resolve_finish_on!(opts[:finish_on], tools)
     }
   end
@@ -268,8 +300,15 @@ defmodule Imp.Predict.ReActV2 do
     end
   end
 
-  defp run(react, history, inputs, pending, turn, max_iters, execution) when turn >= max_iters,
-    do: forced_submit(react, history, inputs, pending, :max_iters, turn, nil, execution)
+  defp run(react, history, inputs, pending, turn, max_iters, execution) when turn >= max_iters do
+    case react.on_max_iters do
+      :forced_submit ->
+        forced_submit(react, history, inputs, pending, :max_iters, turn, nil, execution)
+
+      :last_prose ->
+        last_prose(react, history, pending, turn)
+    end
+  end
 
   defp run(react, history, inputs, pending, turn, max_iters, execution) do
     case predict(react.react, react, history, pending) do
@@ -393,22 +432,61 @@ defmodule Imp.Predict.ReActV2 do
     end
   end
 
+  # The step limit under `on_max_iters: :last_prose`. The last request carries
+  # no tools, so the only thing the model can do is speak, and what it says is
+  # the single text output. A completion that says nothing is an empty answer:
+  # the run is over either way, and there is nothing to force.
+  defp last_prose(react, history, pending, turn) do
+    history = append_note(history, react.signature, react.last_prose_note)
+
+    case predict(last_prose_program(react), react, history, pending) do
+      {:ok, prediction, history} ->
+        calls = prediction |> Imp.get(:tool_calls, []) |> normalize_calls(turn)
+        emit_reasoning(prediction, turn)
+        history = append_last_step(history, pending, prediction, calls)
+        final_prediction(last_prose_outputs(react.signature, prediction), history, :last_prose)
+
+      {:error, reason, history} ->
+        termination =
+          if context_window_exceeded?(reason), do: :context_window_exceeded, else: :max_iters
+
+        incomplete_prediction(history, termination, reason)
+    end
+  end
+
+  # A request with no tools. Both provider keys go: the chat completions body
+  # carries `tools` and `tool_choice` together or not at all, and a request
+  # that names a tool choice without a roster is invalid.
+  defp last_prose_program(react) do
+    %{react.react | config: Keyword.drop(react.react.config, [:tools, :tool_choice])}
+  end
+
+  defp last_prose_outputs(signature, prediction) do
+    case parse_prose(signature, prediction) do
+      {:ok, outputs} ->
+        outputs
+
+      :none ->
+        [%Imp.Signature.Field{name: name}] = signature.outputs
+        %{name => nil}
+    end
+  end
+
   # The notice is what the model is told, so it goes into the durable history
   # rather than into one request: the record of the run carries it, and the
   # prompt renders it as the last user message before the forced request.
-  defp append_forced_submit_notice(react, history, reason) do
-    case notice_text(react.forced_submit_notice, reason) do
-      text when is_binary(text) and text != "" ->
-        case Imp.Signature.input_names(react.signature) do
-          [first | _rest] -> append_history(history, %{first => text})
-          [] -> history
-        end
+  defp append_forced_submit_notice(react, history, reason),
+    do: append_note(history, react.signature, notice_text(react.forced_submit_notice, reason))
 
-      _none ->
-        history
+  defp append_note(history, signature, text) when is_binary(text) and text != "" do
+    case Imp.Signature.input_names(signature) do
+      [first | _rest] -> append_history(history, %{first => text})
+      [] -> history
     end
   end
 
+  defp append_note(history, _signature, _none), do: history
+
   defp notice_text(nil, _reason), do: nil
   defp notice_text(text, _reason) when is_binary(text), do: text
   defp notice_text(fun, reason) when is_function(fun, 1), do: fun.(reason)
@@ -460,7 +538,7 @@ defmodule Imp.Predict.ReActV2 do
     submit_calls = %ToolCalls{tool_calls: Enum.filter(calls.tool_calls, &submit?/1)}
 
     if submit_calls.tool_calls == [] do
-      history = maybe_append_forced_observation(history, pending, prediction, calls)
+      history = append_last_step(history, pending, prediction, calls)
       extract_final(react, inputs, history, reason, initial_error)
     else
       case execute_calls(react, submit_calls, execution, inputs) do
@@ -488,7 +566,10 @@ defmodule Imp.Predict.ReActV2 do
     end
   end
 
-  defp maybe_append_forced_observation(history, pending, prediction, calls) do
+  # The completion of the run's last request, thought and any calls, as this
+  # turn's history event. A completion that said nothing and called nothing
+  # adds no turn.
+  defp append_last_step(history, pending, prediction, calls) do
     thought = Imp.get(prediction, :next_thought)
 
     if thought in [nil, ""] and calls.tool_calls == [] do
@@ -763,18 +844,24 @@ defmodule Imp.Predict.ReActV2 do
   # validated through the same parse a `submit`'s arguments go through, so a
   # constrained output is not quietly filled with something it excludes.
   defp prose_answer(%__MODULE__{prose: :forced_submit}, _prediction), do: :none
+  defp prose_answer(react, prediction), do: parse_prose(react.signature, prediction)
 
-  defp prose_answer(react, prediction) do
-    with [%Imp.Signature.Field{type: type, name: name}] <- react.signature.outputs,
-         true <- type in [:string, "string"],
+  defp parse_prose(signature, prediction) do
+    with true <- single_text_output?(signature),
+         [%Imp.Signature.Field{name: name}] <- signature.outputs,
          prose when is_binary(prose) and prose != "" <- Imp.get(prediction, :next_thought),
-         {:ok, parsed} <- Imp.Adapter.Chat.parse(react.signature, %{name => prose}, []) do
+         {:ok, parsed} <- Imp.Adapter.Chat.parse(signature, %{name => prose}, []) do
       {:ok, Imp.Prediction.to_map(parsed)}
     else
       _not_an_answer -> :none
     end
   end
 
+  defp single_text_output?(%Imp.Signature{outputs: [%Imp.Signature.Field{type: type}]}),
+    do: type in [:string, "string"]
+
+  defp single_text_output?(_signature), do: false
+
   defp execute_call(
          _react,
          %ToolCall{name: @malformed_tool_call, arguments: %{received: received}},

lib/imp/saving.ex

@@ -240,6 +240,8 @@ defmodule Imp.Saving do
       "tools" => dump_tools(Map.delete(react.tools, :submit), "ReActV2"),
       "max_iters" => react.max_iters,
       "prose" => Atom.to_string(react.prose),
+      "on_max_iters" => Atom.to_string(react.on_max_iters),
+      "last_prose_note" => react.last_prose_note,
       "finish_on" => dump_finish_on(react.finish_on),
       "tool_policy" => dump_tool_policy(react.tool_policy, "ReActV2 tool policy")
     }
@@ -574,6 +576,8 @@ defmodule Imp.Saving do
       tools: Map.put(tools, :submit, submit),
       max_iters: require_non_negative_integer!(state["max_iters"], "ReActV2 max_iters"),
       prose: load_react_v2_prose!(state["prose"]),
+      on_max_iters: load_react_v2_on_max_iters!(state["on_max_iters"]),
+      last_prose_note: load_react_v2_last_prose_note!(state["last_prose_note"]),
       finish_on: load_finish_on!(state["finish_on"]),
       tool_policy: load_tool_policy!(state["tool_policy"], "ReActV2 tool policy")
     }
@@ -1163,6 +1167,21 @@ defmodule Imp.Saving do
   defp load_react_v2_prose!(other),
     do: raise(ArgumentError, "invalid saved ReActV2 prose: #{inspect(other)}")
 
+  # A dump written before ReActV2 had the option carries no "on_max_iters" key,
+  # and the step limit forced a submit then.
+  defp load_react_v2_on_max_iters!(nil), do: :forced_submit
+  defp load_react_v2_on_max_iters!("forced_submit"), do: :forced_submit
+  defp load_react_v2_on_max_iters!("last_prose"), do: :last_prose
+
+  defp load_react_v2_on_max_iters!(other),
+    do: raise(ArgumentError, "invalid saved ReActV2 on_max_iters: #{inspect(other)}")
+
+  defp load_react_v2_last_prose_note!(nil), do: nil
+  defp load_react_v2_last_prose_note!(note) when is_binary(note), do: note
+
+  defp load_react_v2_last_prose_note!(other),
+    do: raise(ArgumentError, "invalid saved ReActV2 last_prose_note: #{inspect(other)}")
+
   defp dump_react_mode!(:provider_native), do: "provider_native"
   defp dump_react_mode!(:dspy_3_2_1), do: "dspy_3_2_1"
 

priv/public_api.json

@@ -7372,7 +7372,9 @@
       "struct_fields": [
         "finish_on",
         "forced_submit_notice",
+        "last_prose_note",
         "max_iters",
+        "on_max_iters",
         "prose",
         "react",
         "signature",