feat: carry a form from the DSS to the client

CHANGELOG.md

@@ -7,6 +7,7 @@ All notable changes to this project are recorded here, in the format of
 ## [Unreleased]
 
 ### Added
+- A `form` item in the answer: a form the DSS offers reaches the client instead of being dropped.
 - CI publishes the image to `ghcr.io/openagrinet/experience-api` from main and release tags.
 - `EXPERIENCE_API_DSS_MODE=http` calls the real DSS: the §7 request, its event stream read leniently.
 - A Dockerfile and a compose file that run the API with a healthcheck.

docs/api-contracts/api-contract.md

@@ -315,15 +315,42 @@ Example in the parked fixtures: `response/answered.json`
 | `outcome.status` | Yes | `answered`, `partially_answered`, `rejected`, `no_match`, `requires_input`, `unavailable`. For logs; the client does not branch on it (§5.3) |
 | `outcome.cause` | Yes, may be `null` | Open set, passed through. Treated as an opaque string |
 | `content` | Yes, may be `[]` | In order. Empty only when `error` is present |
-| `content[].type` | Yes | `text` or `refusal` |
-| `content[].text` | Yes | Markdown, as today |
+| `content[].type` | Yes | `text`, `refusal` or `form` |
+| `content[].text` | `text` and `refusal` | Markdown, as today |
 | `content[].citations` | `text` only; may be `[]` | `start`/`end` are Unicode code-point offsets into `text` |
+| `content[].formId`, `context`, `fields` | `form` only | An empty form to fill in. Shape below |
 | `sources` | Yes, may be `[]` | `{ id, name, url? }`. What citations point at |
 | `error` | No | Present only when the turn failed. `Error` shape (§6) |
 
 `confidence` is dropped on purpose: nothing in the client should act on it yet.
 It is easy to add later.
 
+A `form` item asks the farmer for details, such as for a grievance:
+
+```json
+{
+  "type": "form",
+  "formId": "f-1",
+  "context": { "scheme": "PMFBY", "category": "Grievance", "action": "support" },
+  "fields": [
+    { "name": "applicantPhone", "label": "Phone", "kind": "text", "required": true, "pattern": "^[0-9]{10}$" },
+    { "name": "season", "label": "Season", "kind": "select", "required": false,
+      "options": [{ "value": "Kharif", "label": "Kharif" }] }
+  ]
+}
+```
+
+| Field | Required | Notes |
+|---|:--:|---|
+| `formId` | Yes | Names this one form |
+| `context` | Yes | Opaque. The client keeps it as received |
+| `fields[].name`, `label`, `kind`, `required` | Yes | `kind` is `text`, `textarea`, `select` or `secret` today; an open set |
+| `fields[].pattern`, `minLength`, `maxLength` | No | Present only when the field has the rule |
+| `fields[].options` | No | `{ value, label }`. Present only for a `select` |
+
+Sending a filled form back is not in this contract yet. The answer still
+carries the form, so the client can draw it.
+
 ### 5.3 How the client renders each outcome
 
 **The client draws `content` and checks `error`. It never picks words by
@@ -337,12 +364,12 @@ through untouched.
 | `partially_answered` | A mix — answer, a refusal for part of the ask, maybe a question | Draws each item in order | `response/partially-answered.json` |
 | `rejected` | A `refusal` item | Draws it, styled as a refusal | `response/rejected.json` |
 | `no_match` | Text ("I could not find a source for that") | Draws it | `response/no-match.json` |
-| `requires_input` | A question back ("Which district?") | Draws it. The user's next message is the reply | `response/requires-input.json` |
+| `requires_input` | A question back ("Which district?"), or text and a `form` | Draws it. The user's next message is the reply | `response/requires-input.json` |
 | `unavailable` | Text saying what is down, **or nothing**, plus `error` | Draws `content`; if empty, shows its own message for `error.code`. Retry if `error.retryable`. Kept out of history | `response/unavailable.json`, `response/unavailable-internal.json` |
 
 The client branches on exactly two things:
 
-1. **`content[].type`** — `text` or `refusal`, for styling.
+1. **`content[].type`** — `text` or `refusal` for styling; `form` to draw a form.
 2. **`error`** — present means the turn failed: show Retry when
    `error.retryable` is true, and keep the turn out of history.
 
@@ -532,6 +559,7 @@ Reshaping a terminal frame into `FinalAnswer`:
 | `content[]` `text` | `{ type: "text", text, citations }` |
 | `content[].annotations[]` (`url_citation`) | `citations[]`: `sourceId`, `startIndex` → `start`, `endIndex` → `end`. `url`/`sourceName` dropped — they are on `sources` |
 | `content[]` `refusal` | `{ type: "refusal", text }` |
+| `content[]` `form` | `{ type: "form", formId, context, fields }`, unchanged. A form without `formId` or `context` fails the frame |
 | `sources[]` | `{ id, name, url }`. `kind` dropped |
 | `error` (`code`, `message`, `retryable`, `retryAfterSeconds`) | `error`, same fields |
 

src/experience_api/chat/adapters/dss/mapping.py

@@ -18,6 +18,9 @@
     DssEvent,
     DssFinished,
     DssStarted,
+    FormBlock,
+    FormField,
+    FormOption,
     Message,
     Outcome,
     RefusalBlock,
@@ -176,6 +179,8 @@ def _answer(event: str, message: schemas.ResponseMessage) -> Answer:
 
 
 def _block(item: schemas.ContentItem) -> Block | None:
+    if item.type == "form":
+        return _form(item)
     if item.text is None:
         return None
     if item.type == "refusal":
@@ -194,3 +199,28 @@ def _block(item: schemas.ContentItem) -> Block | None:
         )
     logger.warning("dss_content_type=%s skipped=unknown", item.type)
     return None
+
+
+def _form(item: schemas.ContentItem) -> FormBlock:
+    # Without its id and context the client cannot send the form back.
+    if item.form_id is None or item.context is None:
+        raise DssProtocolError("malformed form item: formId or context missing")
+    return FormBlock(
+        form_id=item.form_id,
+        context=item.context,
+        fields=tuple(
+            FormField(
+                name=f.name,
+                label=f.label,
+                kind=f.kind,
+                required=f.required,
+                pattern=f.pattern,
+                min_length=f.min_length,
+                max_length=f.max_length,
+                options=tuple(
+                    FormOption(value=o.value, label=o.label) for o in f.options
+                ),
+            )
+            for f in item.fields
+        ),
+    )

src/experience_api/chat/adapters/dss/schemas.py

@@ -106,10 +106,31 @@ class Annotation(_Response):
     end_index: int | None = None
 
 
+class FormOption(_Response):
+    value: str
+    label: str
+
+
+class FormField(_Response):
+    name: str
+    label: str
+    kind: str
+    required: bool = False
+    pattern: str | None = None
+    min_length: int | None = None
+    max_length: int | None = None
+    options: list[FormOption] = []
+
+
 class ContentItem(_Response):
+    """One item of any type. `form_id`, `context` and `fields` are a `form`'s."""
+
     type: str
     text: str | None = None
     annotations: list[Annotation] = []
+    form_id: str | None = None
+    context: dict[str, str] | None = None
+    fields: list[FormField] = []
 
 
 class Outcome(_Response):

src/experience_api/chat/adapters/http/mapping.py

@@ -12,6 +12,7 @@
     ChatTurn,
     Completed,
     Delta,
+    FormBlock,
     Location,
     Message,
     RefusalBlock,
@@ -64,9 +65,13 @@ def to_final_answer(ids: TurnIds, answer: Answer) -> schemas.FinalAnswer:
     )
 
 
-def _block(block: Block) -> schemas.TextContent | schemas.RefusalContent:
+def _block(
+    block: Block,
+) -> schemas.TextContent | schemas.RefusalContent | schemas.FormContent:
     if isinstance(block, RefusalBlock):
         return schemas.RefusalContent(text=block.text)
+    if isinstance(block, FormBlock):
+        return _form(block)
     return schemas.TextContent(
         text=block.text,
         citations=[
@@ -76,6 +81,31 @@ def _block(block: Block) -> schemas.TextContent | schemas.RefusalContent:
     )
 
 
+def _form(block: FormBlock) -> schemas.FormContent:
+    return schemas.FormContent(
+        form_id=block.form_id,
+        context=dict(block.context),
+        fields=[
+            schemas.FormField(
+                name=f.name,
+                label=f.label,
+                kind=f.kind,
+                required=f.required,
+                pattern=f.pattern,
+                min_length=f.min_length,
+                max_length=f.max_length,
+                # Left out, not `[]`, when a field has none: the client's type
+                # marks it optional.
+                options=[
+                    schemas.FormOption(value=o.value, label=o.label) for o in f.options
+                ]
+                or None,
+            )
+            for f in block.fields
+        ],
+    )
+
+
 def to_wire_event(event: ChatEvent) -> tuple[str, BaseModel]:
     """The event's name on the stream, and its data."""
 

src/experience_api/chat/adapters/http/schemas.py

@@ -116,6 +116,29 @@ class RefusalContent(_WireOut):
     text: str
 
 
+class FormOption(_WireOut):
+    value: str
+    label: str
+
+
+class FormField(_WireOut):
+    name: str
+    label: str
+    kind: str
+    required: bool
+    pattern: str | None = None
+    min_length: int | None = None
+    max_length: int | None = None
+    options: list[FormOption] | None = None
+
+
+class FormContent(_WireOut):
+    type: Literal["form"] = "form"
+    form_id: str
+    context: dict[str, str]
+    fields: list[FormField]
+
+
 class Source(_WireOut):
     id: str
     name: str
@@ -147,7 +170,7 @@ class FinalAnswer(_WireOut):
     assistant_message_id: str
     trace_id: str
     outcome: Outcome
-    content: list[TextContent | RefusalContent]
+    content: list[TextContent | RefusalContent | FormContent]
     sources: list[Source]
     error: Error | None = None
 

src/experience_api/chat/domain.py

@@ -2,6 +2,7 @@
 
 from __future__ import annotations
 
+from collections.abc import Mapping
 from dataclasses import dataclass
 from typing import Literal
 
@@ -58,7 +59,37 @@ class RefusalBlock:
     text: str
 
 
-Block = TextBlock | RefusalBlock
+@dataclass(frozen=True)
+class FormOption:
+    value: str
+    label: str
+
+
+@dataclass(frozen=True)
+class FormField:
+    """One field the farmer fills in, with the only rules the DSS sends."""
+
+    name: str
+    label: str
+    kind: str
+    required: bool
+    pattern: str | None = None
+    min_length: int | None = None
+    max_length: int | None = None
+    options: tuple[FormOption, ...] = ()
+
+
+@dataclass(frozen=True)
+class FormBlock:
+    """An empty form the DSS asks the farmer to fill in. `context` is opaque:
+    the client sends it back as received, so the DSS can find the form again."""
+
+    form_id: str
+    context: Mapping[str, str]
+    fields: tuple[FormField, ...]
+
+
+Block = TextBlock | RefusalBlock | FormBlock
 
 
 @dataclass(frozen=True)

tests/unit/chat/test_dss_frames.py

@@ -17,6 +17,9 @@
     DssEvent,
     DssFinished,
     DssStarted,
+    FormBlock,
+    FormField,
+    FormOption,
     Outcome,
     RefusalBlock,
     Source,
@@ -130,6 +133,91 @@ def test_an_unknown_content_type_is_skipped() -> None:
     assert finished.answer.content == (TextBlock("Hi."),)
 
 
+def test_a_form_comes_through_whole() -> None:
+    raw = frame(
+        "turn.completed",
+        2,
+        {
+            "outcome": {"status": "requires_input", "cause": None},
+            "content": [
+                {"type": "text", "text": "Please fill in this form."},
+                {
+                    "type": "form",
+                    "formId": "f-1",
+                    "context": {
+                        "scheme": "PMFBY",
+                        "category": "Grievance",
+                        "action": "support",
+                    },
+                    "fields": [
+                        {
+                            "name": "applicantPhone",
+                            "label": "Phone",
+                            "kind": "text",
+                            "required": True,
+                            "pattern": "^[0-9]{10}$",
+                            "minLength": 10,
+                            "maxLength": 10,
+                        },
+                        {
+                            "name": "season",
+                            "label": "Season",
+                            "kind": "select",
+                            "required": False,
+                            "options": [{"value": "Kharif", "label": "Kharif"}],
+                        },
+                    ],
+                },
+            ],
+            "sources": [],
+        },
+    )
+    (finished,) = _events(raw)
+
+    assert isinstance(finished, DssFinished)
+    assert finished.answer.content == (
+        TextBlock("Please fill in this form."),
+        FormBlock(
+            form_id="f-1",
+            context={"scheme": "PMFBY", "category": "Grievance", "action": "support"},
+            fields=(
+                FormField(
+                    name="applicantPhone",
+                    label="Phone",
+                    kind="text",
+                    required=True,
+                    pattern="^[0-9]{10}$",
+                    min_length=10,
+                    max_length=10,
+                ),
+                FormField(
+                    name="season",
+                    label="Season",
+                    kind="select",
+                    required=False,
+                    options=(FormOption(value="Kharif", label="Kharif"),),
+                ),
+            ),
+        ),
+    )
+
+
+def test_a_form_without_its_id_is_a_protocol_error() -> None:
+    raw = frame(
+        "turn.completed",
+        2,
+        {
+            "outcome": {"status": "requires_input", "cause": None},
+            "content": [{"type": "form", "context": {}, "fields": []}],
+            "sources": [],
+        },
+    )
+    data = raw.decode().split("data: ", 1)[1].strip()
+
+    with pytest.raises(DssProtocolError, match="form"):
+        to_dss_events("turn.completed", data)
+
+
 def test_a_malformed_frame_names_the_problem_but_not_the_text() -> None:
     # The DSS's text can echo the user's words; it must not reach a log line
     # through an exception message.