docs: document ModelOption.THINKING behaviour for Granite

docs/docs/how-to/configure-model-options.md

@@ -120,7 +120,7 @@ non-deterministic session.
 | `ModelOption.STREAM` | `bool` | `False` | Enable streaming output. |
 | `ModelOption.STREAM_TIMEOUT` | `float \| None` | `120.0` | Timeout in seconds applied to every chunk, including time-to-first-token. Only applies to streaming responses; non-streaming calls are unaffected. If no chunk arrives within this window the stream aborts with a `TimeoutError`. Set to `None` to disable. Increase for slow local inference. |
 | `ModelOption.STOP_SEQUENCES` | `list[str]` | `None` | Strings that halt generation when produced by the model. |
-| `ModelOption.THINKING` | varies | `None` | Enable or configure reasoning/thinking mode (model-dependent). |
+| `ModelOption.THINKING` | `bool \| str` | `None` | Enable or configure reasoning/thinking mode. See [Reasoning and thinking mode](#reasoning-and-thinking-mode) below. |
 | `ModelOption.CONTEXT_WINDOW` | `int` | backend default | Context window size override. |
 | `ModelOption.TOOLS` | `list[MelleaTool]` | `None` | Tools exposed to the model for tool calling. |
 | `ModelOption.TOOL_CHOICE` | `str` | `"auto"` | Tool selection strategy (`"none"`, `"auto"`, or a specific tool name). |
@@ -176,6 +176,67 @@ mot = await m.ainstruct(
 )
 ```
 
+## Reasoning and thinking mode
+
+`ModelOption.THINKING` enables or configures a model's reasoning/thinking mode.
+Accepted values and their effect are backend-dependent:
+
+| Backend | `True` | `False` | `"low"` / `"medium"` / `"high"` |
+| ------- | ------ | ------- | -------------------------------- |
+| Native `OllamaModelBackend` | Enables thinking (Ollama `think=True`) | Disables thinking (`think=False`) | Passed through to Ollama's `think=` param, which handles string effort levels itself |
+| `OpenAIBackend` / LiteLLM (OpenAI-compatible) | Enables thinking (`reasoning_effort="medium"`, plus `chat_template_kwargs.enable_thinking=True` for vLLM-served templates) | Disables thinking on vLLM-served/OpenAI-compatible servers that honour `chat_template_kwargs`. **Real OpenAI reasoning models, and LiteLLM targets that aren't Ollama, deliberately never receive `reasoning_effort="none"`** (real OpenAI rejects that value) — there is no supported way to fully disable reasoning on them via `ModelOption.THINKING` | Sent as `reasoning_effort` verbatim — a top-level request parameter, independent of any chat template |
+| `LocalHFBackend` | Forwards to whichever chat-template variable is declared (`think`, `thinking`, or `enable_thinking`) | Same, `False` value | Forwarded verbatim as the chat template's own `reasoning_effort` variable when the template declares one. This is a different transport than the OpenAI backend's top-level parameter — it only takes effect if the served model's template exposes that variable |
+
+For Granite 4.2 specifically: the chat template only distinguishes `"low"`
+effort from everything else — `reasoning_effort == "low"` triggers genuine
+low-effort (short) reasoning, while `"medium"`/`"high"` are accepted but
+behave the same as `True`/omitted (full-length reasoning). This holds across
+all three backends, provided the serving runtime forwards the effort level
+into the chat template (Ollama and vLLM do). Granite defaults to thinking
+**on** when `ModelOption.THINKING` is not set at all.
+
+`LocalHFBackend` also parses Granite's `<think>...</think>` block out of the
+response, so `result.thinking` and `result.value` are populated separately —
+matching the other backends — rather than leaving the reasoning trace
+embedded raw in `result.value`. This split is skipped for streaming (`m serve`)
+calls, where the reasoning trace still arrives inline in `result.value`;
+incremental splitting for streaming is tracked separately in
+[#1604](https://github.com/generative-computing/mellea/issues/1604).
+
+```python
+import mellea
+from mellea.backends import ModelOption, model_ids
+from mellea.backends.ollama import OllamaModelBackend
+
+m = mellea.MelleaSession(
+    backend=OllamaModelBackend(model_id=model_ids.IBM_GRANITE_4_2_3B)
+)
+
+# Full reasoning (the default for Granite 4.2)
+answer = m.instruct("What is 17 * 24?", model_options={ModelOption.THINKING: True})
+print(answer.thinking)  # reasoning trace
+print(answer.value)     # final answer
+# Output will vary — reasoning traces are non-deterministic.
+
+# Short, low-effort reasoning — use when you need an answer within a small
+# token budget rather than an exhaustive trace
+answer = m.instruct("What is 17 * 24?", model_options={ModelOption.THINKING: "low"})
+
+# No reasoning at all
+answer = m.instruct("What is 17 * 24?", model_options={ModelOption.THINKING: False})
+```
+
+> **Full example:** [`docs/examples/thinking_mode.py`](https://github.com/generative-computing/mellea/blob/main/docs/examples/thinking_mode.py)
+
+If you're serving Granite 4.2 via vLLM, make sure your vLLM install picks up
+the model's latest reasoning-parser update (shipped in the model's Hugging
+Face files) — an older cached parser produces stale thinking behavior.
+
+For non-Granite thinking models served through an OpenAI-compatible endpoint
+(e.g. Qwen3 on vLLM), see
+[Empty `value` from a thinking-mode model](../integrations/openai.md#empty-value-from-a-thinking-mode-model)
+in the OpenAI integration guide.
+
 ## System prompts
 
 Set a system prompt with `ModelOption.SYSTEM_PROMPT`. At session level it applies to all

docs/docs/integrations/openai.md

@@ -395,6 +395,13 @@ final answer. The OpenAI backend reports the response faithfully — the model
 genuinely returned `content=None` — but the reasoning content is preserved
 separately on the underlying `ModelOutputThunk`.
 
+> **Preferred entry point:** for backends that honour it (this one included),
+> `ModelOption.THINKING` is the portable way to enable/disable/level thinking —
+> see [Reasoning and thinking mode](../how-to/configure-model-options.md#reasoning-and-thinking-mode).
+> The `extra_body`/`enable_thinking` pattern below remains useful as a fallback
+> for runtime-specific params `ModelOption.THINKING` doesn't cover, and for the
+> `default_extra_body` merge semantics described below.
+
 Diagnose with:
 
 ```python

docs/examples/thinking_mode.py

@@ -0,0 +1,39 @@
+# pytest: ollama, e2e
+
+"""Demonstrates ModelOption.THINKING against a local Ollama Granite model.
+
+Requires `ollama serve` running with `granite4.2:3b` pulled
+(`ollama pull granite4.2:3b`).
+"""
+
+import mellea
+from mellea.backends import ModelOption, model_ids
+from mellea.backends.ollama import OllamaModelBackend
+
+m = mellea.MelleaSession(
+    backend=OllamaModelBackend(model_id=model_ids.IBM_GRANITE_4_2_3B)
+)
+
+question = "What is 17 * 24?"
+
+# Full reasoning — Granite 4.2 thinks by default even without THINKING set,
+# but setting it explicitly makes the intent visible in the code.
+full = m.instruct(question, model_options={ModelOption.THINKING: True})
+print("=== THINKING=True ===")
+print("thinking:", full.thinking)
+print("answer:", full.value)
+assert full.thinking
+
+# Low-effort reasoning — a short trace, useful when the token budget is tight.
+low = m.instruct(question, model_options={ModelOption.THINKING: "low"})
+print("=== THINKING='low' ===")
+print("thinking:", low.thinking)
+print("answer:", low.value)
+assert low.thinking
+
+# No reasoning at all.
+off = m.instruct(question, model_options={ModelOption.THINKING: False})
+print("=== THINKING=False ===")
+print("thinking:", off.thinking)
+print("answer:", off.value)
+assert not off.thinking