Feat/web search agent

CHANGELOG.md

@@ -8,7 +8,7 @@ All notable changes to this package are recorded here. The format follows
 
 ### Added
 
-- **The agent's saved logins need no picking.** `agent()`, `AgentRun.follow_up()` and `agent_batches` take `logins`: left out, the API reads each site the workspace has a saved login for signed in with that site's login (`"auto"`), and every other site signed out, so one run can read several sites signed in; `"none"`, or a list of login ids, changes that. `login_id` is the older form of a list of one. **This needs an API with `logins`**, on the API's feature branch.
+- **The agent's saved logins need no picking.** `agent()`, `AgentRun.follow_up()` and `agent_batches` take `logins`, the saved logins the agent may sign in with: left out, every one (`"auto"`); `"none"`, or a list of login ids, changes that. The agent reads pages signed out, as the public sees them, and signs in page by page where a page needs the account, with the login for that page's site, so one run can read a site's public pages signed out and its account pages signed in. `login_id` is the older form of a list of one. **This needs an API with `logins`**, on the API's feature branch.
 - **Follow-ups and conversations.** `AgentRun.follow_up(prompt, ...)` asks the next question in a run's conversation -- it takes what `agent()` takes, answers after everything asked and found before it, and may read again, free, any page the conversation read -- and returns its `AgentRun`. `conversations(q=None, limit=25)` walks the workspace's conversations, the latest message first; `conversation(id)` reads one with every run in it. A message that is not the conversation's latest is a 409 `conflict`. **These need an API with follow-ups**, on the API's feature branch, not yet on mesharc.dev.
 - **`agent()` takes `login_id=`, `data_connections=` and `proposals=`**: a saved login its site is read with; up to two of the workspace's data-provider connections (People Data Labs, Financial Modeling Prep), which the agent looks company and financial data up with on your own key, at no MeshArc credits, each record one of its `sources` as `{url, title, provider}`; and leave to propose things to watch, for a member to approve. Each is sent only when given. `trace()` knows the `lookup` step. **These need an API with saved logins, data providers and proposals**, on the API's feature branch.
 - **`arc.agent_batches`: the agent over every row of a table.** `preview`, `start`, `get`, `rows`, `wait`, `list`, `cancel` and `export` -- one agent run a row, its prompt and starting URLs filled from the row's columns as `{{column}}`, its answer the row's new columns; `max_credits_per_row` caps each row and the batch never spends more than rows x that cap. **This needs an API with `/agent/batch`**, on the API's feature branch.
@@ -44,6 +44,13 @@ All notable changes to this package are recorded here. The format follows
 - **`run_agent`, a sixteenth MCP tool, so an assistant can hand a research question to the agent.** It needs write access; a read-only connection is answered `{"code": "read_only"}`. It returns at once with a job, hosted or local, and a retry that lands on a finished run gets its result; `get_job` and `cancel_job` take `kind="agent"` to follow and stop it, and a run past its 7 days answers `status: "expired"`. A finished answer carries `fieldSources`, where each value of `data` came from, and keeps within 60,000 characters: `fieldSources` is left out first, then its sources are cut to the first 50, then `data` to its JSON text, with a note saying where the full answer is. The model's tokens are charged at the model provider's price plus 20%. It sends its own idempotency key, made from its inputs and the ten-minute window, so a retry in the same ten minutes lands on the same run; a 409 saying that key is still being accepted (code `in_flight`, or an older API's message naming the Idempotency-Key) is answered as running, and any other 409 as an error. An output schema describes its answer. **This needs an API that has the `/agent` route**, which is not live on mesharc.dev yet.
 - **`continue_agent`, a seventeenth MCP tool, carries on an agent run that stopped at its credit limit.** `continue_agent(id, max_credits=None, max_steps=None)` starts a new run on the same thread, which resumes where the stopped one was and keeps the pages it read (not paid for again); left out, the budget and steps are the stopped run's. It needs write access; a read-only connection is answered `{"code": "read_only"}`. It returns at once with the new run's job, as `run_agent` does, for `get_job` and `cancel_job` with `kind="agent"`, and sends its own idempotency key, made from its inputs and the ten-minute window. A run already carried on (a retry past that window included) is answered with the run that carried it on, and nothing new is started. A run that did not stop at its limit is answered with code `conflict`, an unknown one `not_found`, one past its 7 days `expired`. A `credit_limit` answer's note now points at `continue_agent` with the run's id, or at the run that already carried it on. **This needs an API that has the `/agent/{id}/continue` route**, which is not live on mesharc.dev yet.
 
+### Fixed
+
+- `agent_batches.wait()` waits while a table is `cancelling` -- its rows still stopping -- and returns once it has ended. It returned at `cancelling`, before the rows had stopped.
+- `AgentRun.follows_run_id` and `followed_by` name the message a follow-up follows and the one asked after it, as `continues_run_id` and `continued_by` do for a continuation.
+- `monitors.create` and `update` type `sentiment` as `False` or `{"connectionId", "only"}`; `True` was allowed by the type but always refused by the API.
+- `rows(q=...)` and the MCP tool `get_monitor_rows` find a record however far down the list it is, and `total` counts the matches. The API now searches every record before cutting a page; before, a search looked at one page only. **This needs the API's fix**, on its feature branch.
+
 ## 0.6.0 - 2026-10-07
 
 Two MCP tool names change, so **this breaks anything that calls `web_search` or `get_page` by name**; both still do what they did, under the new names below. The Python client's API is unchanged apart from two additions: `web_search()` keeps its name there.

README.md

@@ -283,12 +283,12 @@ A follow-up answers after everything asked and found in the conversation before
 run = arc.agent(
     "How many employees does Stripe have, and what was Apple's revenue last year?",
     data_connections=["conn_pdl", "conn_fmp"],   # the workspace's data providers, one per provider
-    login_id=None,                               # a saved login: its site read signed in
+    logins="auto",                               # the saved logins it may sign in with (the default)
     proposals=True,                              # it may propose things to watch
 )
 ```
 
-`data_connections=` names up to two of the workspace's data-provider connections (`arc.connections`, below): People Data Labs for company data, Financial Modeling Prep for financial data. The agent may then look things up on your own key — the provider bills it, MeshArc charges no credits — and each record found is one of `run.sources`, as `{url, title, provider}`, `url` being the request it came from (your key is never in it). `login_id=` is one of the workspace's saved logins (`arc.logins`): pages on its site are read signed in, every other site signed out; logins are switched on per deployment. `proposals=True` lets the agent propose crawling a site or keeping a search or question on a schedule; nothing is made until a member approves it (`arc.proposals`).
+`data_connections=` names up to two of the workspace's data-provider connections (`arc.connections`, below): People Data Labs for company data, Financial Modeling Prep for financial data. The agent may then look things up on your own key — the provider bills it, MeshArc charges no credits — and each record found is one of `run.sources`, as `{url, title, provider}`, `url` being the request it came from (your key is never in it). The agent reads pages signed out and signs in, page by page, where a page needs the account, with the workspace's saved login for that page's site (`arc.logins`); `logins=` says which it may use (every one by default, `"none"`, or a list of ids), and `login_id=` is the older form of a list of one; logins are switched on per deployment. `proposals=True` lets the agent propose crawling a site or keeping a search or question on a schedule; nothing is made until a member approves it (`arc.proposals`).
 
 ## Monitors: a search or a question, asked again on a schedule
 

mesharc/__init__.py

@@ -27,7 +27,7 @@
 import random
 import time
 from datetime import date
-from typing import Any, Dict, Iterable, Iterator, List, Optional, Union
+from typing import Any, Dict, Iterable, Iterator, List, Literal, Optional, Union
 from urllib.parse import parse_qs, urlparse
 
 import httpx
@@ -74,7 +74,8 @@ def __init__(self, message: str, job_id: str = "") -> None:
 
 
 def _running(status: Any) -> bool:
-    return status in ("queued", "running")
+    # "cancelling" is a table whose rows are still stopping: not finished yet.
+    return status in ("queued", "running", "cancelling")
 
 
 def _cursor_of(next_url: str) -> str:
@@ -329,7 +330,7 @@ def create(self, kind: str, request: Json, schedule: str, name: Optional[str] =
                webhook: Union[str, Json, None] = None, baseline_id: Optional[str] = None,
                idempotency_key: Optional[str] = None, budget: Optional[Json] = None, alerts: Optional[Json] = None,
                add_to_project: Optional[str] = None, rows: Union[bool, Json, None] = None,
-               destination: Optional[str] = None, sentiment: Union[bool, Json, None] = None) -> Json:
+               destination: Optional[str] = None, sentiment: Union[Literal[False], Json, None] = None) -> Json:
         """Keep a search or an agent request and run it on a schedule.
 
         ``kind`` is ``search`` or ``agent``; ``request`` is the body of a
@@ -387,7 +388,7 @@ def update(self, monitor_id: str, name: Optional[str] = None, schedule: Optional
                status: Optional[str] = None, webhook: Union[str, Json, None] = None,
                budget: Optional[Json] = None, alerts: Optional[Json] = None, add_to_project: Optional[str] = None,
                rows: Union[bool, Json, None] = None, destination: Optional[str] = None,
-               sentiment: Union[bool, Json, None] = None) -> Json:
+               sentiment: Union[Literal[False], Json, None] = None) -> Json:
         """Change a monitor's ``name``, ``schedule``, ``status`` (``active`` or
         ``paused``), ``webhook`` or any setting ``create`` takes. Only the
         ones that are not None are sent; ``webhook=""`` removes the webhook,
@@ -944,6 +945,16 @@ def continued_by(self) -> Optional[str]:
         """The run that carried this one on: its answer replaces this one's partial."""
         return self.envelope.get("continuedBy") or None
 
+    @property
+    def follows_run_id(self) -> Optional[str]:
+        """The message this one follows in its conversation (``follow_up()``); None for its first."""
+        return self.envelope.get("followsRunId") or None
+
+    @property
+    def followed_by(self) -> Optional[str]:
+        """The follow-up asked after this message; None until one is."""
+        return self.envelope.get("followedBy") or None
+
     def refresh(self) -> AgentRun:
         """Read the run as it stands now. Returns the handle."""
         self.envelope = self._h("GET", f"/agent/{self.id}")
@@ -1362,11 +1373,13 @@ def agent(self, prompt: str, urls: Optional[Iterable[str]] = None, schema: Optio
         secret its messages are signed with comes back once, as
         ``AgentRun.webhook_secret``.
 
-        The workspace's saved logins need no picking: each page on a site
-        with one is read signed in with that site's login, every other page
-        signed out, so one run can read several sites signed in. ``logins``
-        changes that -- ``"none"``, or a list of login ids to keep it to;
-        left out, the API's ``"auto"`` -- and ``login_id`` is the older form
+        The workspace's saved logins need no picking: the agent reads pages
+        signed out, as the public sees them, and signs in page by page
+        where a page needs the account, with the login for that page's
+        site -- so one run can read a site's public pages signed out and
+        its account pages signed in. ``logins`` says which it may use --
+        ``"none"``, or a list of login ids to keep it to; left out, the
+        API's ``"auto"``, every one -- and ``login_id`` is the older form
         of a list of one.
         ``data_connections`` names up to two of the workspace's data
         providers (one per provider): the agent may then look up company

tests/test_client_workspace.py

@@ -48,6 +48,7 @@ def test_a_follow_up_posts_its_own_request_to_the_runs_conversation():
     assert (req.method, req.url.path, req.headers["Idempotency-Key"]) == ("POST", "/api/v1/agent/r1/follow-up", "f-1")
     assert body(req) == {"prompt": "and their prices?", "maxCredits": 300, "dataConnections": ["c1"], "timeout": 120}
     assert (nxt.id, nxt.envelope["followsRunId"]) == ("r2", "r1")
+    assert (nxt.follows_run_id, nxt.followed_by, first.follows_run_id) == ("r1", None, None)
 
 
 def test_a_follow_up_of_a_message_that_is_not_the_latest_is_a_409():
@@ -94,6 +95,10 @@ def test_waiting_for_a_table_polls_until_it_ends_and_times_out_plainly(monkeypat
     monkeypatch.setattr("time.sleep", lambda s: None)
     arc = scripted({"id": "b1", "status": "running"}, {"id": "b1", "status": "done"})
     assert arc.agent_batches.wait("b1", poll=0)["status"] == "done"
+    # A table being cancelled still has rows stopping: it is waited for until it has ended.
+    arc = scripted({"id": "b1", "status": "cancelling"}, {"id": "b1", "status": "cancelled"})
+    assert arc.agent_batches.wait("b1", poll=0)["status"] == "cancelled"
+    assert len(arc.calls) == 2
     arc = scripted({"id": "b1", "status": "running"})
     with pytest.raises(MeshArcTimeoutError) as exc:
         arc.agent_batches.wait("b1", poll=0, timeout=0)