Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions docs/docs/advanced/intrinsics.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Adapter functions are adapter-accelerated operations for RAG quality checks. The
LoRA/aLoRA adapters loaded directly into the Hugging Face backend — faster and more
reliable than prompting a general-purpose model for these specialized micro-tasks.

> **Backend note:** Adapter functions work with two backends:
> **Backend note:** Adapter functions work with three backends:
>
> - **LocalHFBackend** — loads LoRA/aLoRA adapters from the catalog at runtime.
> A local Granite Switch checkpoint can instead use
Expand All @@ -25,8 +25,13 @@ reliable than prompting a general-purpose model for these specialized micro-task
> `load_embedded_adapters=True`. Only adapter functions embedded in the model are
> available — check the model's `adapter_index.json` for the list.
> See `docs/docs/examples/granite-switch/README.md`
> - **OllamaModelBackend**: uses an Ollama model that bundles the adapter,
> for example `gabegoodhart/granite4.1-uncertainty:3b`, which is `granite4.1:3b`
> plus the uncertainty aLoRA. Ollama bundles one adapter per model, so pass
> `adapter_models={"uncertainty": "<tag>", ...}` to route each adapter function
> to its model. Install `mellea[switch]` to download the adapter's `io.yaml`.
>
> Adapter functions do not work with Ollama or other remote backends.
> Adapter functions do not work with other remote backends.

Set up the backend once and reuse it across adapter function calls:

Expand Down
4 changes: 3 additions & 1 deletion docs/docs/advanced/lora-and-alora-adapters.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,9 @@ Apple Silicon Mac with sufficient VRAM for the chosen base model. Uploading requ
Hugging Face account.

> **Backend note:** Custom-trained adapters can only be loaded into `LocalHFBackend`.
> They do not work with Ollama, OpenAI, or other remote backends.
> They do not work with OpenAI or other remote backends. To use one with Ollama,
> convert it to GGUF and bundle it into an Ollama model with a Modelfile `ADAPTER`
> line, then pass that model tag via `OllamaModelBackend(adapter_models=...)`.
>
> Granite Switch models ship with pre-trained adapter functions embedded in the
> model weights. Use them through `OpenAIBackend` with a served checkpoint, or
Expand Down
35 changes: 35 additions & 0 deletions docs/examples/intrinsics/uncertainty_ollama.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# pytest: e2e, ollama, qualitative

"""Example usage of the uncertainty/certainty intrinsic with Ollama.

Evaluates how certain the model is about its response to a user question.
The context should contain a user question followed by an assistant answer.

Ollama bundles one adapter per model, so the uncertainty adapter is served by
its own model tag (`granite4.1:3b` plus the uncertainty aLoRA). Pass that tag
via `adapter_models`; normal chat still uses the base model.

Requires `mellea[switch]` to download the adapter's `io.yaml`.

To run this script from the root of the Mellea source tree, use the command:
```
uv run python docs/examples/intrinsics/uncertainty_ollama.py
```
"""

from mellea import model_ids, start_backend
from mellea.stdlib import functional as mfuncs
from mellea.stdlib.components.intrinsic import core

ctx, backend = start_backend(
"ollama",
model_id=model_ids.IBM_GRANITE_4_1_3B,
context_type="chat",
adapter_models={"uncertainty": "gabegoodhart/granite4.1-uncertainty:3b"},
)

response, ctx = mfuncs.chat("What is 2 + 2?", ctx, backend) # type: ignore
print(f"Response: {response.content}")

result = core.check_certainty(ctx, backend) # type: ignore
print(f"Certainty score: {result}")
Loading
Loading