Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion agentic/docs/project-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,7 @@ Example: `plots/scatter-basic/` contains everything for the basic scatter plot.
- **`core/database/repositories.py`**: Data access layer
- **`api/`**: FastAPI backend (routers, schemas, dependencies, cache, analytics, MCP server)
- **`app/`**: React frontend (Vite 8 + TypeScript 6 + MUI 9)
- **`agents/`**: Agent network on Google ADK 2.11 and Vertex AI ("Use with my data"; Claude Haiku 5.5 by default, Gemini 3.8 Flash behind `AGENT_PROVIDER`); the runtime core (agents, plot pipeline, plugins, render layer, private `/v1` service) runs locally, see `agents/README.md`; design in `docs/concepts/agent-network.md`
- **`agents/`**: Agent network on Google ADK 2.13 and Vertex AI ("Use with my data"; Claude Haiku 5.5 by default, Gemini 3.8 Flash behind `AGENT_PROVIDER`); the runtime core (agents, plot pipeline, plugins, render layer, private `/v1` service) runs locally, see `agents/README.md`; design in `docs/concepts/agent-network.md`
- **`agentic/`**: AI workflow layer (composable phases, prompt templates, runtime state)
- **`agentic/workflows/`**: Click CLI scripts (plan, build, test, review + orchestrators)
- **`agentic/commands/`**: Markdown prompt templates
Expand Down
2 changes: 1 addition & 1 deletion agents/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Agent network

This directory holds the anyplot agent network: the service that lets an admin paste their own data on a plot page and get that plot adapted, rendered and reviewed ("Use with my data"). It is built with [Google ADK](https://adk.dev) 2.11. The design, including every contract and guardrail, is in [Agent network design](../docs/concepts/agent-network.md).
This directory holds the anyplot agent network: the service that lets an admin paste their own data on a plot page and get that plot adapted, rendered and reviewed ("Use with my data"). It is built with [Google ADK](https://adk.dev) 2.13. The design, including every contract and guardrail, is in [Agent network design](../docs/concepts/agent-network.md).

## What is built

Expand Down
2 changes: 1 addition & 1 deletion agents/anyplot/agent.py
Original file line number Diff line number Diff line change
Expand Up @@ -194,7 +194,7 @@ def with_session_part(system: types.ContentUnion | None, block: str) -> types.Co
async def add_session_context(callback_context: CallbackContext, llm_request: LlmRequest) -> LlmResponse | None:
"""Append the session block to the root's system instruction, as a part of its own.

ADK 2.11 sends an agent's `instruction` next to a `static_instruction` as user content:
ADK 2.13 sends an agent's `instruction` next to a `static_instruction` as user content:
a `<<<BEGIN_SYSTEM_INSTRUCTION>>>` block with a preamble, inserted into the user's own
turn (`flows/llm_flows/prompt/_instructions.py`). Claude Haiku 5.5 read that as a user
who forges system text: in a prep-eval sample of 2026-10-15 its thinking called the block
Expand Down
23 changes: 4 additions & 19 deletions agents/anyplot/code/loader.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@
ISO dates), so no `sep`, `decimal` or date format is needed. The column dtypes from
the dataset profile map to pandas dtypes that keep missing values (`integer` to
`Int64`, `number` to `float64`, `boolean` to `boolean`, `text` to `string`); datetime
columns go to `parse_dates`. A dataset with a number column is read with
`float_precision="round_trip"`, so every float is the one `data.csv` writes in its
shortest round-trip form (`ROUND_TRIP`). `import pandas as pd` is added after the last
module-level import when the file does not have it.
columns go to `parse_dates`. A number column needs no float option: since pandas 3.1
`read_csv` has one converter, and every float is the one `data.csv` writes in its
shortest round-trip form. `import pandas as pd` is added after the last module-level
import when the file does not have it.

`data.csv` writes a missing cell as an empty cell and nothing else, so a text cell such as
`NA` (Namibia, North America), `None` or `null` is a value there. pandas reads those tokens
Expand Down Expand Up @@ -115,21 +115,11 @@ def na_values(dtype_map: dict[str, str], dates: list[str]) -> dict[str, list[str
return values


ROUND_TRIP: Literal["round_trip"] = "round_trip"
"""pandas' exact float reader: its default C reader misreads about one in four 17-digit numbers by one unit in
the last place (measured on pandas 3.0, 2026-10-15), so the plot would not draw the values `data.csv` holds."""


ISO_DATES: Literal["ISO8601"] = "ISO8601"
"""The `date_format` of every date column: `data.csv` writes ISO dates only, and without it pandas guesses a
format per column and warns "Could not infer format" for years such as 0001 (review of #12196)."""


def round_trip(dtype_map: dict[str, str]) -> bool:
"""Whether the read needs `float_precision="round_trip"`: some column is read as float64."""
return PANDAS_DTYPES["number"] in dtype_map.values()


def _literal(text: str) -> str:
"""A double-quoted Python string literal (JSON escapes are valid Python escapes)."""
return json.dumps(text, ensure_ascii=False)
Expand All @@ -149,15 +139,12 @@ def _statement(indent: str, dtype_map: dict[str, str], dates: list[str]) -> str:
f"{_literal(name)}: " + ('[""]' if dtype_map.get(name) == TEXT_DTYPE else NA_NAME)
for name in [*dtype_map, *(name for name in dates if name not in dtype_map)]
]
exact = round_trip(dtype_map)
arguments = [_literal(DATA_FILE)]
if dtype_items:
arguments.append("dtype={" + ", ".join(dtype_items) + "}")
if date_items:
arguments.append("parse_dates=[" + ", ".join(date_items) + "]")
arguments.append(f'date_format="{ISO_DATES}"')
if exact:
arguments.append(f'float_precision="{ROUND_TRIP}"')
if per_column:
arguments += ["keep_default_na=False", "na_values={" + ", ".join(na_items) + "}"]
prefix = _tokens_statement(indent) + "\n" + indent if per_column else ""
Expand All @@ -172,8 +159,6 @@ def _statement(indent: str, dtype_map: dict[str, str], dates: list[str]) -> str:
if date_items:
lines += [f"{inner}parse_dates=[", *(f"{entry}{item}," for item in date_items), f"{inner}],"]
lines.append(f'{inner}date_format="{ISO_DATES}",')
if exact:
lines.append(f'{inner}float_precision="{ROUND_TRIP}",')
if per_column:
lines += [f"{inner}keep_default_na=False,", f"{inner}na_values={{", *(f"{entry}{item}," for item in na_items)]
lines.append(f"{inner}}},")
Expand Down
47 changes: 38 additions & 9 deletions agents/anyplot/data/prepare.py
Original file line number Diff line number Diff line change
Expand Up @@ -631,7 +631,7 @@ def _apply(frame: pd.DataFrame, operation: Operation, carried: _Carried) -> tupl


def _read(parsed: ParsedDataset) -> pd.DataFrame:
"""The canonical CSV as the loader reads it; `round_trip` keeps every number's shortest form stable.
"""The canonical CSV as the loader reads it; pandas' reader keeps every number's shortest form stable.

Only an empty cell is missing (`keep_default_na=False`, as the run form's loader reads
it): a text category `NA` (Namibia, North America) that the parse kept is a category here too.
Expand All @@ -641,7 +641,6 @@ def _read(parsed: ParsedDataset) -> pd.DataFrame:
dtype=pandas_dtypes(parsed.column_dtypes),
parse_dates=list(parsed.parse_dates),
date_format="ISO8601", # `data.csv` writes ISO dates only: no per-column guess, no "Could not infer" warning
float_precision="round_trip",
keep_default_na=False,
na_values=[""],
)
Expand Down Expand Up @@ -793,6 +792,35 @@ def _order_open(dates: pd.Series) -> bool:
return bool(present) and all(value.day <= 12 for value in present) and any(v.day != v.month for v in present)


def _decimal_total(values: pd.Series) -> ExactDecimal:
"""The total of a float column's cells as `data.csv` writes them, added in decimals.

A float's `repr` is its shortest form, the text the user reads, so this is the sum they
would get by hand. Adding the floats themselves leaves the noise of their binary form in
a total of decimal numbers, and how much depends on the method: pandas 3.1 dropped its
compensated group sum for sorted keys (`4024.2` came back as `4024.2000000000003`), and
even an exactly rounded `math.fsum` gives `377.78000000000003` for cells that add up to
`377.78`. Such a total would reach `data.csv` and the plot as a value no one wrote.
`decimal` keeps 28 significant digits, more than a float holds. Infinities of both signs
have no total: the result is then not a number.
"""
try:
return sum((ExactDecimal(repr(value)) for value in values.dropna().tolist()), ExactDecimal(0))
except InvalidOperation:
return ExactDecimal("NaN")


def _group_sum(values: pd.Series) -> float:
"""A float group's total (`_decimal_total`); a group with no value totals 0, as in pandas."""
return float(_decimal_total(values))


def _group_mean(values: pd.Series) -> float:
"""A float group's mean from its decimal total; a group with no value has none."""
count = values.count()
return float(_decimal_total(values) / count) if count else math.nan


def _aggregate(frame: pd.DataFrame, op: Aggregate, carried: _Carried) -> tuple[pd.DataFrame, str]:
keys = _distinct(op.by)
for name in [*keys, op.column]:
Expand Down Expand Up @@ -827,11 +855,12 @@ def _aggregate(frame: pd.DataFrame, op: Aggregate, carried: _Carried) -> tuple[p
totals = [sum(int(value) for value in values.dropna()) for _, values in grouped]
aggregated, rounded = _whole_numbers(totals, grouped.size().index)
case "sum":
aggregated = grouped.sum()
case "mean":
aggregated = grouped.agg(_group_sum)
case "mean" if is_integer_dtype(work[op.column].dtype):
aggregated = grouped.mean()
if is_integer_dtype(work[op.column].dtype): # a mean is a float: inputs past 2**53 are counted
cast = _past_exact(work[op.column])
cast = _past_exact(work[op.column]) # a mean is a float: inputs past 2**53 are counted
case "mean":
aggregated = grouped.agg(_group_mean)
case "count":
aggregated = grouped.count()
case "min":
Expand Down Expand Up @@ -1074,9 +1103,9 @@ def _derive(frame: pd.DataFrame, op: Derive, carried: _Carried) -> tuple[pd.Data
what = f"sum of {', '.join(columns)}"
case "share":
if keys:
total = values[0].groupby([frame[key] for key in keys], dropna=False).transform("sum")
total = values[0].groupby([frame[key] for key in keys], dropna=False).transform(_group_sum)
else:
total = pd.Series(values[0].sum(), index=frame.index)
total = pd.Series(_group_sum(values[0]), index=frame.index)
# A total that overflowed to infinity would give finite shares of 0: no share for its group instead.
result = values[0] / total.where((total != 0) & np.isfinite(total))
what = f"share of {columns[0]} in the total" + (f" of each {' and '.join(keys)}" if keys else "")
Expand Down Expand Up @@ -1787,7 +1816,7 @@ def _period_starts(series: pd.Series, period: str) -> pd.Series:
days = pd.Series(series.dt.normalize(), index=series.index)
if period == "day":
return days
return pd.Series(days - pd.to_timedelta(days.dt.weekday, unit="D"), index=series.index)
return pd.Series(days - pd.to_timedelta(days.dt.day_of_week, unit="D"), index=series.index)


def _period_labels(series: pd.Series, period: str) -> pd.Series:
Expand Down
48 changes: 30 additions & 18 deletions agents/anyplot/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
`google.adk.models.anthropic_llm.Claude` with an `AsyncAnthropicVertex` client built
from the settings (ADK's own builds one from `GOOGLE_CLOUD_PROJECT` and
`GOOGLE_CLOUD_LOCATION`, which would let the environment pick the project).
Reasoning depth is the Claude `effort` of `AnthropicGenerateContentConfig`; ADK 2.11
Reasoning depth is the Claude `effort` of `AnthropicGenerateContentConfig`; ADK 2.13
ignores `thinking_config.thinking_level` for Claude and raises when both are set.

Claude thinks adaptively by default (since 2026-10-11): the root with
Expand All @@ -27,7 +27,7 @@
parts, which the answer guard, ADK's output parsing and the stream translator all skip.

Two details keep the replayed thinking blocks harmless. Left unset, Haiku 5.5 would
think with the display omitted, and ADK 2.11 sends such a signature-only block back as
think with the display omitted, and ADK 2.13 sends such a signature-only block back as
`redacted_thinking`, which breaks the root's call after a tool result; the summarized
display avoids that. And the root replays its thinking block after a tool result while
the session block between the two calls changed (until 2026-10-15 ADK also moved it
Expand All @@ -43,13 +43,16 @@
block) goes to Claude as one system block per part, with ADK's prompt-cache breakpoint on
the first only (`system_blocks`), so a session block that changes leaves the policy cached.

ADK 2.11 sends no response schema to Claude: `basic.py` sets
`config.response_schema` for a tool-less agent with an `output_schema`, and
`anthropic_llm.py` never reads it, so the adapter and reviewer would get free text.
`VertexClaude` closes that gap with Claude's structured outputs: when a request
carries a response schema and no tools, it sets `output_config.format` to that
schema (`structured_output_format`), so constrained decoding makes the answer text
valid JSON of exactly that shape, and ADK's `validate_schema` parses it as it is.
ADK 2.13 delivers a response schema to Claude as a `structured_output` tool
(`anthropic_llm._OutputSchemaTool`; ADK 2.11 did not send the schema at all): `basic.py`
sets `config.response_schema` for a tool-less agent with an `output_schema`, and the
tool is forced with thinking off and only offered with thinking on, so a thinking
adapter or reviewer could still answer in free text. `VertexClaude` uses Claude's
structured outputs instead: when a request carries a response schema and no tools,
it drops ADK's tool and sets `output_config.format` to that schema
(`structured_output_format`), so constrained decoding makes the answer text valid
JSON of exactly that shape, and ADK's `validate_schema` parses it as it is. ADK's
unwrapping of the tool call finds no call and leaves the answer alone.
The judge answers the same way. The grammar enforces types, enums, required fields
and `additionalProperties: false`; what it cannot enforce (string lengths, item
counts, a verdict's `ok` agreeing with its defects) the SDK's `transform_schema`
Expand Down Expand Up @@ -91,7 +94,7 @@
from pathlib import Path
from typing import Any, Literal, Protocol

from anthropic import transform_schema
from anthropic import NOT_GIVEN, transform_schema
from google.adk.models.anthropic_llm import AnthropicGenerateContentConfig, Claude
from google.adk.models.base_llm import BaseLlm
from google.adk.models.google_llm import Gemini
Expand Down Expand Up @@ -291,8 +294,13 @@ def structured_output_format(schema: Any) -> dict[str, Any]:
signature no longer matches the conversation prefix (`thinking.block_binding`)."""


def _has_tools(llm_request: LlmRequest) -> bool:
return bool(llm_request.config and llm_request.config.tools)
def _structured_schema(llm_request: LlmRequest) -> Any:
"""The schema a request answers under as a structured output, or None.

A request with tools of its own (the root) has none: its answer is chat.
"""
config = llm_request.config
return config.response_schema if config and not config.tools else None


def system_parts(llm_request: LlmRequest) -> list[str]:
Expand Down Expand Up @@ -349,17 +357,21 @@ def _anthropic_client(self) -> Any:
def _build_anthropic_kwargs(
self, llm_request: LlmRequest, messages: Any, tools: Any, tool_choice: Any, thinking: Any
) -> dict[str, Any]:
"""ADK's request arguments, plus the structured-output format when a response schema is set.
"""ADK's request arguments, with a response schema sent as the structured-output format.

The format joins ADK's `output_config` (which holds the effort) rather than
replacing it. A request with tools (the root) gets no format: its answer is chat.
The format replaces the `structured_output` tool ADK delivers a schema as (a tool
and a format on one call would leave two ways to answer), so it goes before ADK
builds the arguments, whose cache breakpoints also look at the tools. The format
joins ADK's `output_config` (which holds the effort) rather than replacing it.
"""
schema = _structured_schema(llm_request)
if schema:
tools, tool_choice = NOT_GIVEN, NOT_GIVEN
kwargs: dict[str, Any] = super()._build_anthropic_kwargs(llm_request, messages, tools, tool_choice, thinking)
parts = system_parts(llm_request)
if len(parts) > 1:
kwargs["system"] = system_blocks(parts, kwargs.get("system"))
schema = llm_request.config.response_schema if llm_request.config else None
if schema is not None and not _has_tools(llm_request):
if schema:
kwargs["output_config"] = {
**(kwargs.get("output_config") or {}),
"format": structured_output_format(schema),
Expand All @@ -381,8 +393,8 @@ def _build_anthropic_kwargs(
async def generate_content_async(
self, llm_request: LlmRequest, stream: bool = False
) -> AsyncGenerator[LlmResponse, None]:
structured = bool(llm_request.config and llm_request.config.response_schema) and not _has_tools(llm_request)
# A structured answer is read whole, so it never streams.
structured = bool(_structured_schema(llm_request))
async for response in super().generate_content_async(llm_request, stream=stream and not structured):
yield response

Expand Down
2 changes: 1 addition & 1 deletion agents/anyplot/render/runtimes/python.py
Original file line number Diff line number Diff line change
Expand Up @@ -236,7 +236,7 @@ def loader_block(
return to_run_form(working, columns=columns, dtypes=dtypes, parse_dates=parse_dates)

def loader_notes(self, kinds: ColumnKinds) -> list[str]:
"""None: `Int64`, `float64` with `float_precision="round_trip"` and the parsed datetimes keep every cell."""
"""None: `Int64`, `float64` and the parsed datetimes keep every cell."""
return []

def data_json(self, csv_text: str, kinds: ColumnKinds | None) -> str | None:
Expand Down
2 changes: 1 addition & 1 deletion agents/anyplot/sub_agents/adapter.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
adapter prompt plus the verbatim catalogue sources for its library), so its prompt
prefix stays cacheable, and it never sees the conversation: `include_contents='none'`
and the `before_model_callback` keeps only the last content, the fenced
`AdaptRequest` the pipeline passes as node input. (ADK 2.11 also prepends the
`AdaptRequest` the pipeline passes as node input. (ADK 2.13 also prepends the
invocation's user message to a single-turn agent that runs under its own isolation
scope; the callback removes it, so the adapter reads nothing but the request.)

Expand Down
3 changes: 1 addition & 2 deletions agents/evals/stress.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@

from agents.anyplot.briefs import adapter_profile, profile_json, profile_summary, spec_brief
from agents.anyplot.code.edits import apply_plan, new_literal_chars
from agents.anyplot.code.loader import ISO_DATES, ROUND_TRIP, loader_arguments, na_values, round_trip
from agents.anyplot.code.loader import ISO_DATES, loader_arguments, na_values
from agents.anyplot.code.regions import find_regions
from agents.anyplot.data.bindings import check_bindings, default_bindings
from agents.anyplot.data.kinds import read_kinds
Expand Down Expand Up @@ -177,7 +177,6 @@ def loader_outcome(parsed: ParsedDataset) -> tuple[str, float]:
parse_dates=dates,
keep_default_na=missing is None,
na_values=missing,
float_precision=ROUND_TRIP if round_trip(dtypes) else None,
date_format=ISO_DATES if dates else None,
)
except Exception as exc: # the class is the finding; the message could quote the data
Expand Down
2 changes: 1 addition & 1 deletion agents/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -1325,7 +1325,7 @@ async def _ingest_paste(
delta = ingested.state_delta()
# Read again: the chip's choice may have landed while the judges ran, and a choice is never overwritten.
# From this read to the append below nothing suspends: `InMemorySessionService.get_session` and
# `append_event` (ADK 2.11) contain no `await` that yields, and the code between them is synchronous,
# `append_event` (ADK 2.13) contain no `await` that yields, and the code between them is synchronous,
# so on the one event loop no `PUT /language` can land in between; the read and the conditional
# write are atomic. A persistent session service with I/O in either call would need a
# compare-and-set instead. Keep any new `await` out of this block.
Expand Down
Loading
Loading