Skip to content

Stop indexing the AG-UI documentation mirror that upstream deleted - #169

Merged
jpr5 merged 2 commits into
mainfrom
fix/unindex-retired-agui-mirror
Sep 13, 2026
Merged

Stop indexing the AG-UI documentation mirror that upstream deleted#169
jpr5 merged 2 commits into
mainfrom
fix/unindex-retired-agui-mirror

Conversation

@jpr5

@jpr5 jpr5 commented Sep 13, 2026

Copy link
Copy Markdown
Contributor

What is wrong

showcase/shell-docs/src/content/ag-ui/ no longer exists. CopilotKit#7092,
"fix(docs): remove retired AG-UI mirror", merged 2026-09-11T22:44:24Z
(tylerslaton, merge 3e86b8d55) — 112 files, 20 additions, 25,059
deletions — deleted the whole tree plus src/app/sitemap.ts and
src/app/llms-mdx/[[...slug]]/route.ts. Confirmed against the live API:

$ gh api repos/CopilotKit/CopilotKit/contents/showcase/shell-docs/src/content --jq '.[].name'
docs
doctest.json
framework-overviews
reference
snippets

#164 (298b7fa) had added showcase/shell-docs/src/content/ag-ui/**/*.mdx to
the docs source's file_patterns. Its full walk completed
2026-09-11T08:11:58Z — about 14 hours before the deletion landed. We
indexed a tree that was retired that night.

Today the production sitemap still advertises 96 /ag-ui/ URLs from a stale
build; 88 serve 200 and 8 already return 500. As the site rebuilds, those URLs
die, and our index keeps handing them out.

No content becomes unsearchable. search-ag-ui-docs already indexes the
upstream ag-ui-protocol/ag-ui repo (~1,317 chunks) and answers with
docs.ag-ui.com links, which resolve. Only the link target changes — to the
one that will still exist.

What changed

  1. Removed the ag-ui file_patterns entry and the comment block above it
    that argued for indexing the tree.

  2. Restored both tool descriptions to their pre-Index the 96 AG-UI documentation pages hosted on docs.copilotkit.ai #164 text, verbatim. This
    is the most important part of the change. search-docs claimed to cover
    "the AG-UI protocol pages hosted on the CopilotKit docs site
    (docs.copilotkit.ai/ag-ui/...)" — a false claim, and it is what an LLM
    reads to decide which tool to call. It now reads:

    Search the CopilotKit product documentation (https://docs.copilotkit.ai) —
    guides, concepts, quickstarts, API reference, and how-tos for building with
    CopilotKit. Use this for CopilotKit usage and configuration questions. NOT
    for AG-UI protocol docs (use search-ag-ui-docs) and NOT for source code
    (use search-code). This is a semantic search, so prefer performing multiple
    queries with different phrases instead of a single long query, until you
    find all the context you need.

    search-ag-ui-docs — which Index the 96 AG-UI documentation pages hosted on docs.copilotkit.ai #164 softened into "one of two near-identical
    copies" — is again described as the AG-UI documentation, with no mention
    of a second copy.

  3. unclaimed_exempt_paths: stale comment replaced, no exemption added.
    Index the 96 AG-UI documentation pages hosted on docs.copilotkit.ai #164's comment said content/ag-ui/ "is deliberately absent from this list
    and always will be: it is not exempt, it is CLAIMED". That reasoning is
    obsolete. The directory is gone, so the unclaimed_content audit has nothing
    to walk there and cannot fire either way; an exemption would only blind the
    audit to whatever lands at that path next.

Per-file decision on the rest of #164

git show 298b7fa --stat lists three files.

File Decision Reasoning
deploy/copilotkit-docs.yaml revert the ag-ui parts The subject of this PR. Nothing else #164 touched in it.
scripts/test-path-filter.ts KEEP, comment corrected #164 changed only comments here. One named "three page subtrees (docs/, reference/, ag-ui/)"; now two. Reverting verbatim would restore a comment that was already wrong before #164 — it claimed the source path is content/docs/, which #159 had widened to content/.
src/__tests__/copilotkit-docs-config.test.ts KEEP, assertions updated Deploy-config coverage barely existed before this file, and that absence is exactly why both of this config's silent failures — the unindexed reference tree, and now this — stayed invisible. Deleting it would restore the blind spot. Its ag-ui cases are inverted (the tree must not be claimed) and a third describe block is new: what search-docs advertises. Its machine-relay block postdates #164 and is untouched.

RED → GREEN

Harness calls the shipped walkSourceFiles() and deriveUrl() over the
shipped deploy/copilotkit-docs.yaml. No database, no network, no
production.

The checkout is CopilotKit 4b7c0d35fd505e981a03cd7d3cba400234b0eb4b — the
first parent of the deletion merge, i.e. main immediately before #7092.
That commit is required to demonstrate the before-state at all: at any commit
after it the files are simply gone, so a red run there would prove nothing
about our config.

RED (config as on main):

TOTAL enumerated: 962
  content/docs/:      682
  content/reference/: 184
  content/ag-ui/:     96
  [PRESENT] .../content/ag-ui/concepts/agents.mdx        -> https://docs.copilotkit.ai/ag-ui/concepts/agents
  [PRESENT] .../content/ag-ui/concepts/architecture.mdx  -> https://docs.copilotkit.ai/ag-ui/concepts/architecture
  [PRESENT] .../content/ag-ui/agentic-protocols.mdx      -> https://docs.copilotkit.ai/ag-ui/agentic-protocols
  [PRESENT] .../content/ag-ui/sdk/js/core/events.mdx     -> https://docs.copilotkit.ai/ag-ui/sdk/js/core/events

GREEN (this branch, same command, same checkout):

TOTAL enumerated: 866
  content/docs/:      682
  content/reference/: 184
  content/ag-ui/:     0
  [ ABSENT] .../content/ag-ui/concepts/agents.mdx
  [ ABSENT] .../content/ag-ui/concepts/architecture.mdx
  [ ABSENT] .../content/ag-ui/agentic-protocols.mdx
  [ ABSENT] .../content/ag-ui/sdk/js/core/events.mdx

File-count delta: 962 → 866, exactly −96. The prose and reference counts do
not move.

Negative check: the existing 866 URLs are byte-identical

#159 and #164 both came close to breaking the ordered, first-match-wins
url_derivation.strip_prefix. Removing a pattern must not disturb it. Full
path → URL manifests for the docs/ and reference/ subtrees, before and
after:

red rows: 866    green rows: 866
diff exit=0, 0 lines             cmp: BYTE-IDENTICAL
md5  c88778a79e5b9d4da7e5bf34a0a27cbb  (both)
docs/quickstart.mdx            -> https://docs.copilotkit.ai/quickstart
reference/hooks/useAgent.mdx   -> https://docs.copilotkit.ai/reference/hooks/useAgent

Mutation-tested, so the empty diff is not vacuous. Swapping the two
strip_prefix entries so the shorter content/ prefix wins first — the exact
defect this check exists to catch — produces 1,364 differing rows, every
prose page gaining a spurious /docs segment
(/quickstart → /docs/quickstart). The check fires. Restoring the config
returns the manifest to byte-identical.

The new assertions are mutation-tested the same way: against #164's config,
7 of them fail (4 ag-ui coverage cases, 3 tool-description cases).

Local gate

npm run build ✅ · npx tsc --noEmit ✅ · npm test 198 files / 3,949
tests passed, 0 failed, 1 skipped ✅ · node scripts/check-test-shapes.mjs
✅ (no new violations) · npx prettier --check on all three touched files ✅

No pre-existing failure to control for — the suite is fully green.

Deploying this: a reindex IS required

Merging this config change does not remove already-indexed chunks. The ~866
ag-ui chunks sitting in the docs source stay there until a full walk of that
source rewrites it. The chunks will keep being returned, with dying
docs.copilotkit.ai/ag-ui/... links, until then.

#160's config_fingerprint should force that walk automatically, since
file_patterns is folded into the fingerprint. Measured on the two configs:

8274a4799a6d331b97b48a4a74a74cc5   docs source @ origin/main
2bf63ca62d74ba10ad4698363f3c5f7b   docs source @ this branch

The branch value is exactly the pre-#164 fingerprint, so the stored value no
longer matches and the orchestrator takes the full-walk path.

How an operator confirms it, after deploy and the next reindex:

  1. /health — the docs source's document count falls from ~962 back toward
    ~866, i.e. down by roughly the ag-ui contribution (~866 chunks across
    96 files    ).
  2. search-docs for something only those pages said (e.g. "AG-UI
    RunAgentInput", "AG-UI Rust SDK subscriber") returns no
    docs.copilotkit.ai/ag-ui/... URLs; search-ag-ui-docs answers the same
    query with docs.ag-ui.com links.
  3. The reindex audit reports no new unclaimed cluster for
    CopilotKit/CopilotKit.

If the count does not move, the fingerprint path did not fire and a manual full
reindex of the docs source is the fallback.

jpr5 added 2 commits September 13, 2026 09:05
@jpr5
Pin that the docs source no longer claims the retired AG-UI mirror (RED)
a5286d6 
CopilotKit#7092 deleted showcase/shell-docs/src/content/ag-ui/ upstream on
2026-09-11, about 14 hours after #164 added that tree to the docs source's
file_patterns. The config still claims 96 files whose source is gone, and
search-docs still advertises docs.copilotkit.ai/ag-ui/... links that die as
the site rebuilds.

These assertions fail against the shipped YAML as it stands: 4 ag-ui coverage
cases plus both tool-description cases. A third describe block is new — a
search tool's description is what an LLM reads to route a query, so a stale
coverage claim there was invisible to every test we had.
@jpr5
Stop indexing the AG-UI mirror upstream deleted, and unsay that we co…
22d03d3 
…ver it

CopilotKit#7092 (merged 2026-09-11T22:44:24Z, 112 files, -25,059 lines)
deleted showcase/shell-docs/src/content/ag-ui/ from CopilotKit/CopilotKit.
#164 had added that tree to the docs source's file_patterns and its full walk
finished 2026-09-11T08:11:58Z — about 14 hours earlier. We indexed a tree that
was retired that night.

Removed the ag-ui file_patterns entry and the comment block arguing for it.
Restored both tool descriptions to their pre-#164 text, verbatim. The
search-docs one is the point of this change: it claimed to cover "the AG-UI
protocol pages hosted on the CopilotKit docs site (docs.copilotkit.ai/ag-ui/
...)", which is what an LLM reads to pick a tool, and those pages no longer
exist. search-ag-ui-docs already indexes the canonical upstream copy and hands
back docs.ag-ui.com links that resolve, so nothing becomes unsearchable — only
the link target changes to the one that survives.

The unclaimed_exempt_paths comment claiming content/ag-ui/ is "CLAIMED, and
always will be" is replaced rather than inverted: the directory is gone, so
the audit cannot fire on it either way and an exemption would only blind it to
whatever lands at that path next. No exemption added.

scripts/test-path-filter.ts kept, comment corrected — #164 only edited comments
there, and one now names a third subtree that does not exist.
@jpr5
jpr5 merged commit c567825 into main Sep 13, 2026
7 checks passed
@jpr5
jpr5 deleted the fix/unindex-retired-agui-mirror branch September 13, 2026 16:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@jpr5