Skip to content

[docs] Self-healing documentation fixes from issue analysis - 2026-05-26#34801

Closed
github-actions[bot] wants to merge 1 commit into
mainfrom
doc-healer/forecast-cli-docs-2026-05-26-dca1091116387711
Closed

[docs] Self-healing documentation fixes from issue analysis - 2026-05-26#34801
github-actions[bot] wants to merge 1 commit into
mainfrom
doc-healer/forecast-cli-docs-2026-05-26-dca1091116387711

Conversation

@github-actions
Copy link
Copy Markdown
Contributor

Self-Healing Documentation Fixes

This PR was automatically created by the Daily Documentation Healer workflow.

Gaps Fixed

  • gh aw forecast command is completely undocumented in docs/src/content/docs/setup/cli.md — Added a #### \forecast` section under **Monitoring** describing the command's purpose, flags (--days, --period, --sample, --eval, --repo, --json), Monte Carlo projection method, skipped-run exclusion (introduced in #34740), refocused effective-token output (#34750), A/B variant split, and --eval` backtesting semantics. The new section links to the existing Forecast Specification and Effective Tokens pages.

Root Cause

The gh aw forecast command was introduced on 2026-05-11 in #31377 and has received two feature changes within the last 7 days (#34740 on 2026-05-25 making it interruption-aware and excluding skipped runs, and #34750 on 2026-05-25 refocusing output on effective-token predictions). The cli.md reference page — which lists every other gh aw subcommand — never received a corresponding entry.

The gap surfaced in cli-consistency issue #33322 (closed not_planned on 2026-05-19). Per doc-healer policy for not_planned closures, the issue was cross-referenced against commits in the same 7-day window; the recent forecast-related commits confirm an independent code-driven documentation gap separate from the original aggregate report.

DDUw's pre-flight (Step 1) scans only PRs merged in the last 24 hours, so feature additions older than that window are not re-checked against cli.md. There is no DDUw step that enumerates the live CLI surface (cobra.Command set) and compares it to the #### subcommand sections in cli.md, so commands added on older days drift silently.

💡 DDUw Improvement Suggestions

DDUw Improvement Suggestions

Add a new heuristic step to .github/workflows/daily-doc-updater.md that detects CLI commands missing from docs/src/content/docs/setup/cli.md, independent of recent merges. Concretely:

  1. Step 4a — CLI surface vs. reference parity check (new step before "Identify Documentation Gaps"):

    ### 4a. CLI Surface Parity Check
    
    Enumerate every top-level `gh aw` subcommand registered in `pkg/cli/*_command.go` (one `NewXxxCommand` function per file). For each command, confirm a matching `#### \`<command>\`` heading exists in `docs/src/content/docs/setup/cli.md`. If a command has no matching heading, treat it as a confirmed documentation gap regardless of merge recency.
    
    Suggested grep:
    

    grep -hoP 'Use:\s+"\K[a-z][a-z0-9-]' pkg/cli/_command.go | sort -u

    
    Compare the resulting set against `grep -oP '^#### \\`\K[a-z][a-z0-9 -]*' docs/src/content/docs/setup/cli.md`.
    
  2. Step 1c clarification — recent commits trigger drift checks for the affected feature even when the original triggering PR is older than 24h. Currently Step 1c only re-checks closed-issue body items. Adding "if commits in the 7-day window touch a feature file (e.g., pkg/cli/forecast*.go), confirm that feature has a section in cli.md" would have caught this gap on the day of Make forecast interruption-aware, ignore skipped runs, and clean up empty CI output #34740/Refocus forecast output on effective-token predictions by removing yield and episode metrics #34750.

These two additions together close the class of gaps where DDUw misses an undocumented feature because no merge in the last 24 hours explicitly says "add docs for X."

Related Issues

References

Generated by 📝 Daily Documentation Healer · opus47 27.9M ·

  • expires on May 29, 2026, 12:13 AM UTC

Add a `forecast` section to docs/src/content/docs/setup/cli.md covering
the command's purpose, flags, Monte Carlo projection method, skipped-run
exclusion, A/B variant split, and `--eval` backtesting mode. Links to
the existing forecast and effective-tokens specifications.

Closes #aw_dh1
@github-actions github-actions Bot added automation documentation Improvements or additions to documentation labels May 26, 2026
@pelikhan pelikhan closed this May 26, 2026
@github-actions github-actions Bot added the closed:unknown PR was closed without merging: unknown label May 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automation closed:unknown PR was closed without merging: unknown documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant