Skip to content

docs: generate version-scoped llms.txt artifacts#2188

Merged
chickenlj merged 1 commit into
agentscope-ai:mainfrom
jujn:docs/update-llms-full
Jul 15, 2026
Merged

docs: generate version-scoped llms.txt artifacts#2188
chickenlj merged 1 commit into
agentscope-ai:mainfrom
jujn:docs/update-llms-full

Conversation

@jujn

@jujn jujn commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator

Summary

Close #2185
This PR replaces the root-level LLM documentation artifacts with version-scoped outputs:

  • /v1/llms.txt
  • /v1/llms-full.txt
  • /v2/llms.txt
  • /v2/llms-full.txt

The goal is to avoid mixing AgentScope Java 1.x and 2.x documentation in a single llms-full.txt, so AI coding tools can ingest the correct documentation set for the target version.

Changes

  • Add a local Sphinx build-finished extension to generate version-scoped llms.txt and llms-full.txt files from docs/_toc.yml.
  • Remove the default sphinx_llms_txt integration to stop generating root-level /llms.txt and /llms-full.txt.
  • Filter generated LLM docs to English v1/en and v2/en pages, excluding blogs and community pages.
  • Update AI coding docs and the AI context menu to use versioned LLM documentation URLs.

@jujn jujn requested review from a team and Copilot July 14, 2026 03:53

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@codecov

codecov Bot commented Jul 14, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@oss-maintainer oss-maintainer left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Documentation update — generates version-scoped llms.txt artifacts for better LLM context management.

Findings

  • [Info] Adds 287 lines of documentation across 6 files. Structure looks good.

Verdict

LGTM.


Automated review by github-manager

@AgentScopeJavaBot AgentScopeJavaBot added enhancement New feature or request area/build Build, CI, BOM, distribution labels Jul 14, 2026

@AgentScopeJavaBot AgentScopeJavaBot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Review

This PR replaces root-level llms.txt/llms-full.txt with version-scoped outputs (/v1/llms.txt, /v2/llms.txt, etc.) to prevent mixing v1 and v2 documentation in a single LLM index file. The implementation is a new Sphinx build-finished extension (versioned_llms.py) that parses _toc.yml and generates per-version artifacts, removing the third-party sphinx-llms-txt dependency.

The overall architecture is clean and consistent with existing extensions like root_index_notice.py. However, there's one notable gap: the v2 root: page (v2/en/intro) defined in _toc.yml is not parsed by _load_ordered_docs(), which only handles - file: lines. This means the v2 intro page won't appear in v2/llms.txt or v2/llms-full.txt, unlike v1 where the intro is listed as a chapter entry.

Additionally, the text.find("\\n---", 4) in _extract_frontmatter() could match --- within frontmatter values, causing premature truncation. Using python-frontmatter library or a regex-based parser would be more robust.

Comment thread docs/_sphinx_extensions/versioned_llms.py
Comment thread docs/_sphinx_extensions/versioned_llms.py

@AgentScopeJavaBot AgentScopeJavaBot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Review

This PR replaces root-level llms.txt/llms-full.txt with version-scoped outputs (/v1/llms.txt, /v2/llms.txt, etc.) to prevent mixing v1 and v2 documentation in a single LLM index file. The implementation is a new Sphinx build-finished extension (versioned_llms.py) that parses _toc.yml and generates per-version artifacts, removing the third-party sphinx-llms-txt dependency.

The overall architecture is clean and consistent with existing extensions like root_index_notice.py. However, there's one notable gap: the v2 root: page (v2/en/intro) defined in _toc.yml is not parsed by _load_ordered_docs(), which only handles - file: lines. This means the v2 intro page won't appear in v2/llms.txt or v2/llms-full.txt, unlike v1 where the intro is listed as a chapter entry.

Additionally, the text.find("\\n---", 4) in _extract_frontmatter() could match --- within frontmatter values, causing premature truncation. Using python-frontmatter library or a regex-based parser would be more robust.

docs: list[tuple[str, str]] = []
seen: set[str] = set()
active_index: int | None = None

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[major] The _toc.yml v2 entry root: v2/en/intro is not parsed by _load_ordered_docs(), which only matches - file: lines. This means the v2 intro page will be missing from v2/llms.txt and v2/llms-full.txt. Consider adding logic to also parse root: lines at the start of the method, or confirm this is intentional and add a comment explaining why.



def _extract_frontmatter(text: str) -> tuple[dict[str, str], str]:
if not text.startswith("---\n"):

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[minor] text.find("\\n---", 4) can match --- substrings within frontmatter values (e.g., a multi-line description containing --- separators), causing premature frontmatter truncation. Consider using a regex like re.search(r'\\n---\\s*\\n', text[4:]) or the python-frontmatter library for more robust parsing.

@chickenlj chickenlj merged commit a95abab into agentscope-ai:main Jul 15, 2026
6 checks passed
@jujn jujn deleted the docs/update-llms-full branch July 15, 2026 06:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/build Build, CI, BOM, distribution enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update llms-full.txt

5 participants