Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
[![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](LICENSE)
[![PyPI](https://img.shields.io/pypi/v/google-adk)](https://pypi.org/project/google-adk/)
[![NPM Version](https://img.shields.io/npm/v/@google/adk)](https://www.npmjs.com/package/@google/adk)
[![Go Doc](https://img.shields.io/badge/Go%20Package-Doc-blue.svg)](https://pkg.go.dev/google.golang.org/adk)
[![Go Doc](https://img.shields.io/badge/Go%20Package-Doc-blue.svg)](https://pkg.go.dev/google.golang.org/adk/v2)
[![Maven Central Java](https://img.shields.io/maven-central/v/com.google.adk/google-adk)](https://search.maven.org/artifact/com.google.adk/google-adk)
[![Maven Central Kotlin](https://img.shields.io/maven-central/v/com.google.adk/google-adk-kotlin-core)](https://search.maven.org/artifact/com.google.adk/google-adk-kotlin-core)

Expand Down
229 changes: 196 additions & 33 deletions docs/agents/models/litert-lm.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,73 @@
# LiteRT-LM model host for ADK agents

<div class="language-support-tag">
<span class="lst-supported">Supported in ADK</span><span class="lst-python">Python v0.1.0</span>
<span class="lst-supported">Supported in ADK</span><span class="lst-python">Python v0.1.0</span><span class="lst-kotlin">Kotlin v0.4.0</span>
</div>

[LiteRT-LM](https://github.com/google-ai-edge/LiteRT-LM) is a C++ library to
efficiently run language models across edge platforms.
On desktop (Linux, macOS, and Windows), ADK integrates with LiteRT-LM-hosted
models through the LiteRT-LM server launched by the LiteRT-LM CLI `lit`.
You can use the
[LiteRT-LM](https://github.com/google-ai-edge/LiteRT-LM)
library to efficiently run language models locally on various compute devices
without requiring specialized processors such as graphics processing units
(GPU) or tensor processing units (TPU). LiteRT-LM supports many models,
including Google Gemma models as well as third-party models. This guide
provides instructions for setting up LiteRT-LM with ADK for the following
languages:

## Get started
- [Python](#python)
- [Kotlin](#kotlin)

LiteRT-LM works with the `Gemini` class. You only have to set the `base_url` and
`model` parameters.
## Python

These instructions describe how to use LiteRT-LM server with ADK in Python
with a Gemma open weights model, including using LiteRT-LM 's local hosting
model server `lit`.

### Install resources

You need to download a model to use with LiteRT-LM, and the `lit` CLI tool
to help you find a model download it.

#### Install `lit` CLI tool

Download and install the `lit` CLI tool by following these
[instructions](https://github.com/google-ai-edge/LiteRT-LM?tab=readme-ov-file#desktop-cli-lit)
in the LiteRT-LM GitHub repository.

#### Download a model

Before you start the server, you need to download a model. You'll need a
_Hugging Face_ user access token to download a LiteRT-LM model using `lit`. You
can get a token for your _Hugging Face_ account
[here](https://huggingface.co/settings/tokens).

To see a list of models available for download, use the `lit list` command:

```bash
lit list --show_all
```

Download a model using the `lit pull` command:

```bash
export HUGGING_FACE_HUB_TOKEN="**your Hugging Face token**"
lit pull gemma3n-e2b
```

### Configure your agent

Configure your agent to connect to LiteRT-LM and a hosted model.
When running Gemma models with LiteRT-LM, you configure a `Gemini`
model class with the model identifier and local network address.

To use LiteRT-LM with ADK and a Gemma model:

1. Set `base_url` to the LiteRT-LM server URL, for example: `localhost:8001`.
2. Set `model` to the LiteRT-LM model name, for example: `gemma3n-e2b`.

The following example code shows how to configure an agent to
connect to the locally hosted LiteRT-LM instance serving the Gemma
model configuration described above:

```py
from google.adk.agents import Agent
from google.adk.models import Gemini
Expand Down Expand Up @@ -47,55 +98,167 @@ Then run the agent as usual:
adk web
```

## Running the LiteRT-LM Server
### Running the LiteRT-LM server

The LiteRT-LM server is a separate process that serves LiteRT-LM models. It is
started by the LiteRT-LM CLI tool `lit`.

### Download the `lit` CLI tool
#### Run the server

Download the `lit` CLI tool by following these
[instructions](https://github.com/google-ai-edge/LiteRT-LM?tab=readme-ov-file#desktop-cli-lit)
in the LiteRT-LM GitHub repository.
After downloading a model, start the LiteRT-LM server locally by running the
following command:

```bash
lit serve --port 8001
```

### Download a model
!!! tip "Local Server Port Number"

Before you start the server, you need to download a model. You'll need a
*Hugging Face* user access token to download a LiteRT-LM model using `lit`. You
can get a token for your *Hugging Face* account
[here](https://huggingface.co/settings/tokens).
You may choose any port number for the LiteRT-LM server as long as it matches the `base_url` you set in the `Gemini` class in your agent code.

To see a list of models available for download, use the `lit list` command:
#### Debugging

To see incoming requests to the LiteRT-LM server and the exact input sent to the
model, use the `--verbose` flag:

```bash
lit list --show_all
lit serve --port 8001 --verbose
```

Download a model using the `lit pull` command:
## Kotlin

These instructions describe how to use LiteRT-LM with ADK in Kotlin using
the `com.google.adk.kt.litertlm` package.

### Install resources

You need to download a model to use with LiteRT-LM, and the `litert-lm` CLI tool
to help you find a model download it.

#### Install LiteRT-LM CLI

Prerequisites: Python 3.10 or higher

To install the CLI, run:

```bash
export HUGGING_FACE_HUB_TOKEN="**your Hugging Face token**"
lit pull gemma3n-e2b
pip install --upgrade litert-lm
```

### Run the server
For additional installation methods, such as using uv, see
[LiteRT-LM CLI Installation Guide](https://developers.google.com/edge/litert-lm/cli/installation).

After downloading a model, start the LiteRT-LM server locally by running the
following command:
#### Download a model

Download a model compatible with LiteRT-LM to use the `litert-lm` CLI tool.
Use `litert-lm` to download models directly from Hugging Face:

```bash
lit serve --port 8001
litert-lm import \
--from-huggingface-repo litert-community/gemma-4-E2B-it-litert-lm \
gemma-4-E2B-it.litertlm
```

!!! tip "Local Server Port Number"
Once downloaded, the model is stored locally at:

You may choose any port number for the LiteRT-LM server as long as it matches the `base_url` you set in the `Gemini` class in your agent code.
```
~/.litert-lm/models/gemma-4-E2B-it.litertlm/model.litertlm
```

### Debugging
For more details about `litert-lm`, refer to the
[LiteRT-LM CLI Usage Guide](https://developers.google.com/edge/litert-lm/cli/usage).

To see incoming requests to the LiteRT-LM server and the exact input sent to the
model, use the `--verbose` flag:
### Add dependencies

ADK Kotlin works with LiteRT-LM through an adapter package,
`com.google.adk:google-adk-kotlin-litertlm`.

In your `build.gradle.kts`, add `com.google.adk:google-adk-kotlin-litertlm` and
`com.google.ai.edge.litertlm:litertlm-jvm` to your dependencies:

```kt
repositories {
mavenCentral()
google()
}

dependencies {
implementation("com.google.adk:google-adk-kotlin-core:0.4.0")
implementation("com.google.adk:google-adk-kotlin-litertlm:0.4.0")
implementation("com.google.ai.edge.litertlm:litertlm-jvm:0.13.1")
// other dependencies...
}
```

### Configure agent model

Run a local model for your agent with LiteRT-LM by configuring a
`LiteRtLmModel` object as part of your `LlmAgent` object. If you do not
already have a ADK Kotlin project, follow the
[Kotlin Quickstart for ADK](/get-started/kotlin/)
getting started guide. The following code example shows you how to
configure an `LlmAgent`, and set the `model` parameter to a `LiteRtLmModel`:

```kt
object HelloTimeAgent {

// Get model path from environment variable.
private val modelPath: String by lazy {
System.getenv("LITERT_LM_MODEL_PATH")
?: throw IllegalStateException(
"LITERT_LM_MODEL_PATH environment variable must be set pointing to a .litertlm file."
)
}

@JvmField
val rootAgent =
LlmAgent(
name = "hello_time_agent",
description = "Tells the current time in a specified city.",
model =
LiteRtLmModel.create(
EngineConfig(modelPath = modelPath, backend = Backend.CPU())
),
instruction =
Instruction(
"You are a helpful assistant that tells the current time in a city. " +
"Use the 'getCurrentTime' tool for this purpose."
),
tools = TimeService().generatedTools(),
)
}
```

In this example, the path to the LiteRT-LM model file is read from the
environment variable `LITERT_LM_MODEL_PATH`. The model will be run on the CPU.
You can run the model on a GPU by setting `backend = Backend.GPU()`.

When you run the agent, set `LITERT_LM_MODEL_PATH` to the location of the model
file, for example: `~/.litert-lm/models/gemma-4-E2B-it.litertlm/model.litertlm`.

### Run your agent

If you followed the [Kotlin Quickstart for ADK](/get-started/kotlin/)
with the above modifications, you can run your ADK agent using the command-line
REPL with the environment variable `LITERT_LM_MODEL_PATH` set to the path of
the model file:

```bash
lit serve --port 8001 --verbose
LITERT_LM_MODEL_PATH=~/.litert-lm/models/gemma-4-E2B-it.litertlm/model.litertlm ./gradlew run
```

Example interaction:

```
Agent hello_time_agent is ready. Type 'exit' to quit.

You > what's your name?

hello_time_agent > I am Gemma 4, a Large Language Model developed by Google DeepMind.

You > what time is it in paris?

hello_time_agent > calls tool: getCurrentTime

hello_time_agent > The time in Paris is 10:30 am.
```
21 changes: 12 additions & 9 deletions docs/api-reference/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,16 @@

The Agent Development Kit (ADK) provides comprehensive API references across supported languages, allowing you to dive deep into all available classes, methods, and functionalities.

{{#
CONTRIBUTORS: Keep all API reference links below as full `https://` absolute URLs.
These API docs (python/, typescript/, java/, kotlin/, cli/, agentconfig/, rest/)
are externally generated HTML, not MkDocs Markdown pages. If written as relative
or root-relative paths, such as `python/index.html` or `/api-reference/python/`,
the mkdocs-llmstxt plugin rewrites them to non-existent `.md` URLs, such as
`https://adk.dev/api-reference/python/index.md`, producing broken 404 links in
llms.txt / llms-full.txt. See #1716 and #1717 for history.
#}}

<div class="grid cards" markdown>

- :fontawesome-brands-python:{ .lg .middle } **Python API Reference**
Expand All @@ -10,9 +20,6 @@ The Agent Development Kit (ADK) provides comprehensive API references across sup
Explore the complete API documentation for the Python Agent Development Kit. Discover detailed information on all modules, classes, functions, and examples to build sophisticated AI agents with Python.

[:octicons-arrow-right-24: View Python API Docs](https://adk.dev/api-reference/python/) <br>
<!-- Assuming your Python API docs are in a 'python' subdirectory -->
<!-- Or link to an external ReadTheDocs, etc. -->
<!-- [:octicons-arrow-right-24: View Python API Docs](python/index.html) -->

<!-- This comment forces a block separation -->

Expand All @@ -22,8 +29,6 @@ The Agent Development Kit (ADK) provides comprehensive API references across sup
Access the complete API documentation for the TypeScript Agent Development Kit. Find detailed information on all packages, classes, and methods to build powerful and flexible AI agents with TypeScript.

[:octicons-arrow-right-24: View Typescript API Docs](https://adk.dev/api-reference/typescript/) <br>
<!-- Assuming your Typescript API docs are in a 'typescript' subdirectory -->
<!-- [:octicons-arrow-right-24: View Typescript API Docs](typescript/index.html) -->

<!-- This comment forces a block separation -->

Expand All @@ -32,7 +37,8 @@ The Agent Development Kit (ADK) provides comprehensive API references across sup
---
Explore the complete API documentation for the Go Agent Development Kit. Discover detailed information on all modules, classes, and functions to build sophisticated AI agents with Go.

[:octicons-arrow-right-24: View Go API Docs](https://pkg.go.dev/google.golang.org/adk) <br>
[:octicons-arrow-right-24: View Go v2.x API Docs](https://pkg.go.dev/google.golang.org/adk/v2) <br>
[:octicons-arrow-right-24: View Go v1.x API Docs](https://pkg.go.dev/google.golang.org/adk) <br>

<!-- This comment forces a block separation -->

Expand All @@ -42,9 +48,6 @@ The Agent Development Kit (ADK) provides comprehensive API references across sup
Access the comprehensive Javadoc for the Java Agent Development Kit. This reference provides detailed specifications for all packages, classes, interfaces, and methods, enabling you to develop robust AI agents using Java.

[:octicons-arrow-right-24: View Java API Docs](https://adk.dev/api-reference/java/) <br>
<!-- Assuming your Java API docs (Javadocs) are in a 'java' subdirectory -->
<!-- Or link to an external Javadoc hosting site -->
<!-- [:octicons-arrow-right-24: View Java API Docs](java/index.html) -->

<!-- This comment forces a block separation -->

Expand Down
2 changes: 1 addition & 1 deletion docs/integrations/mimir.md
Original file line number Diff line number Diff line change
Expand Up @@ -145,4 +145,4 @@ session = await runner.session_service.create_session(
- [adk-mimir-memory on GitHub](https://github.com/Perseus-Computing-LLC/adk-mimir-memory)
- [adk-mimir-memory on PyPI](https://pypi.org/project/adk-mimir-memory/)
- [Mimir (backing service)](https://github.com/Perseus-Computing-LLC/mimir)
- [Perseus (context engine)](https://github.com/Perseus-Computing-LLC/perseus)
- [Perseus Context integration](/integrations/perseus/)
3 changes: 2 additions & 1 deletion docs/runtime/runconfig.md
Original file line number Diff line number Diff line change
Expand Up @@ -324,5 +324,6 @@ your language:

- [Python API reference](../api-reference/python/google-adk.html#google.adk.agents.RunConfig)
- [TypeScript API reference](../api-reference/typescript/interfaces/RunConfig.html)
- [Go API reference](https://pkg.go.dev/google.golang.org/adk/agent#RunConfig)
- [Go API reference](https://pkg.go.dev/google.golang.org/adk/v2/agent#RunConfig)
- [Java API reference](../api-reference/java/com/google/adk/agents/RunConfig.html)
- [Kotlin API reference](../api-reference/kotlin/google-adk-kotlin-core/com.google.adk.kt.agents/-run-config/index.html)
6 changes: 3 additions & 3 deletions docs/skills/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -287,11 +287,11 @@ You can define Skills within the code of your agent, as shown below.
```go
import (
"os"

"google.golang.org/adk/tool/skilltoolset/skill"
"google.golang.org/adk/tool/skilltoolset"
)

// ...

source := skill.NewFileSystemSource(os.DirFS("./skills"))
Expand All @@ -301,7 +301,7 @@ You can define Skills within the code of your agent, as shown below.
// source, _, err = skill.WithFrontmatterPreloadSource(ctx, source)
// source, _, err = skill.WithCompletePreloadSource(ctx, source)
// For more information about these and other wrappers, see
// https://pkg.go.dev/google.golang.org/adk/tool/skilltoolset/skill#Source.
// https://pkg.go.dev/google.golang.org/adk/v2/tool/skilltoolset/skill#Source.

skillToolset, err := skilltoolset.New(ctx, skilltoolset.Config{
Source: source,
Expand Down
Loading
Loading