fix(#1687): substitute, don't send, an embedding input with no text

[DevNexsler]

Ticket #1687 — doc-organizer: an embedding input with no text is rejected by the
route and takes the whole batch with it.

What this fixes

providers/embed/limits.py already truncated an input past the model's context window
(#0569), because an embeddings request is all-or-nothing and the input never gets
shorter. The mirror case was unguarded. Measured against
openrouter.ai/api/v1/embeddings (qwen/qwen3-embedding-8b) on 2026-08-27:

Input	Result
["first", "", "third"]	400 zod too_small, "path": ["input", 1] — the gateway's own validation, before any upstream sees the batch
[" "]	400 {"detail":"Prompt must not be empty"} — the upstream, and intermittently: three runs answered 400 / 400 / 200
[" "]	400, same upstream shape (3/3)
["first", " ", "third"]	200 OK (3/3)
["\n"], ["\t \n"], ["."]	200 OK

Two layers reject, and the second one is a coin flip depending on which upstream the
gateway routed to — so the guard keys on content, not length. Any input whose
.strip() is empty is replaced with EMPTY_INPUT_PLACEHOLDER (".", one printable
character, the smallest input no layer treats as absent).

Substituting rather than dropping is forced by EmbedProvider's one-vector-per-input
contract: _call_embeddings indexes the response back onto the request, so dropping a
slot would silently misalign every later vector in the batch. The check runs after
truncation, because a text whose only content sits past the token limit is bounded down
to whitespace and is then the same problem.

The bound lives at the provider boundary, so all four embed providers inherit it from
the seam that already owns "what will this route accept" — no call site re-derives it.
max_input_tokens: 0 still disables the context-window bound; the route's refusal of a
text-free input is not a tunable.

staging/provider_sim learns both rejections first, so the hermetic tier can reproduce
this at all (#1658) — the empty-string case with the gateway's exact zod body, the
whitespace-only case with the upstream's. The sim answers the second deterministically
even though the real route is intermittent: a simulator stricter than the real route
cannot produce a false green, a more permissive one can.

Where the empty input comes from. attachment_context_refresh.build_contextualized_text
returns "" when the anchor chunk is context-only and the refreshed context is empty;
TaxonomyStore._build_row / .search embed a raw label. The indexing chunk path
cannot produce one — every chunk is prefixed with a non-empty _build_chunk_context
header whose title falls back to the doc id.

Correction to the ticket: the production 400s are not this defect

The ticket was filed on the theory that the 103 Client error '400 ... /v1/embeddings'
skips in indexer.log were empty inputs masking #1655's character-cap 422s. They are
not. qwen/qwen3-embedding-8b is served through more than one upstream, and they report
an oversized input differently — the same 195000-character body returned both:

422 {"error":{"message":"Value error, The input sequence should have less than 131072 characters..."}}
400 {"code":20015,"message":"The parameter is invalid. Please check again.","data":null}

That code: 20015 body is logged verbatim in indexer.log for documents::000dU on
2026-07-24 (the #0569 attachment). So a document that reports 422 four times and 400 on
the fifth has not changed and has not hit a second defect — it hit a different upstream.

Consequence for reviewers: the ticket's acceptance box "Client error '400 .*v1/embeddings
stops appearing" closes with #1655 / PR #145 (the character bound), not with this PR.
Do not read #1655's acceptance as failed because 400s remain, and do not tick that box as
evidence for this guard. If 400s on this route survive a landed character bound, that is
the point at which a separate text-free input actually exists — that is the discriminating
check. The bodies are indistinguishable in the log only because raise_for_status()
discards them (#1657).

Tests

Red → green, proven both ways. With the guard reverted (constant kept so collection
still works): 6 failed, 34 passed, the provider-through-sim test failing with the exact
production string Client error '400 Bad Request' for url '.../api/v1/embeddings'. With
the guard: 45 passed.

Full make gate (artifacts .evals/gate-runs/20260827-062919/):

tier	result
static	pass — ruff clean, 2033 collected
unit	1731 passed
integration	90 passed
staging-e2e	39 passed, 2 deselected; tool coverage 21/21 traced; attachment-path audit pass
live	167 passed, 4 skipped (pytest -m live -q --ignore=tests/e2e after live preflight: PASS)

The gate's own live tier aborts at session start on any branch at this base — the
known #1654 defect: tests/conftest.py derives the tier marker with "_live" in fname,
which also matches tests/e2e/test_health_liveness.py, whose tests/e2e/conftest.py
pytest.exit()s because gate.py already tore the staging stack down. tests/conftest.py
is not in this diff, and pytest -m live --collect-only shows those two e2e files being
pulled in. The runbook's prescribed workaround (--ignore=tests/e2e) is what the 167
above ran.

Candidate-container qualification (AGENTS.md 3c)

Source SHA 7a8254969326cc2a53a2e2fb1c61945e81c3550a, isolated manager-owned Compose
project, images
sha256:db96b8232b8719a1009e12f6767618e77b0d50a732ef002f40bb19913359669b (app) and
sha256:18194eda58ac4613810d20d6d4ff20273e7fd4618670d97a1964b257b6c98c9c (provider-sim).

• Failing path, inside the shipped image, over the network to the shipped provider-sim:
  ["first", "", "third"] → 3 vectors, dim=768, order preserved across the empty slot,
  with the guard's warning in the container's own log.
• Neighbouring happy path at the public boundary: deposit → POST /api/index/document
  → documents::00001 → retrievable via MCP file_search (presence, not rank).
• All three containers health=healthy restarts=0 oom=false exit=0; /health status: ok.
• docker compose down -v removed containers, network and all three volumes.

---

Maint ticket: #1687 — Maint-Manager tickets/review/1687-doc-organizer-an-empty-embed-input-400s-the-whole-b.md
On merge: verify this ticket's acceptance boxes against production (not tests),
tick them, then move the ticket to tickets/done/ and set status: done.
A merged PR is not a closed ticket — an unmerged PR is not a shipped fix.

Note for the closer: the box "Client error '400 .*v1/embeddings stops appearing in
doc-organizer:/data/index/indexer.log`" is not closeable by this PR — see the
correction above. Close it with #1655. The check that this PR is what it claims:

docker exec doc-organizer sh -c \
  "grep -a 'input carries no text' /data/index/indexer.log | tail"

A hit means the guard fired on a real production input and the batch survived; no hit
after a full sweep means no text-free input reached the provider in that window, which
is also a pass.

🤖 Generated with Claude Code