[prompt-analysis] Copilot PR Prompt Analysis - 2026-05-25

[github-actions[bot]]

Summary

Analysis Period: Last 30 days (May 2026)
Total PRs: 1,000 | Merged: 804 (80.6%) | Closed: 193 (19.4%) | Open: 3

Overall merge success rate is strong at 80.6%, consistent with the prior week (81.3%). The most significant differentiator between successful and closed PRs is the WIP label: PRs without [WIP] in the title merge at 82.4%, while WIP-labeled PRs merge at only 46.9%.

---

Prompt Categories and Success Rates

Category	Total	Merged	Closed	Success Rate
Bug Fix	815	665	148	81.8%
Feature Addition	161	121	39	75.6%
Refactoring	13	10	3	76.9%
Documentation	2	1	1	50.0%
Testing	4	4	0	100.0%
Other	5	3	2	60.0%

Note: Categories are not mutually exclusive — PRs are assigned to the first matching category by priority.

---

Prompt Analysis

✅ Successful Prompt Patterns

Common characteristics in merged PRs:

• Average prompt length: 703 words
• Optimal range: 150–500 words (82.8% merge rate — highest of all buckets)
• Most common title words: fix, add, workflow, agent, model, safe
• Prompts describe the problem clearly and include what changed and why

Example merged prompts:

1. PR fix: skip unlock job when activation was skipped #34124 (101 words): fix: skip unlock job when activation was skipped — describes the incorrect behavior, root cause, and specific files changed in pkg/workflow/compiler.
2. PR fix: pass X-GitHub-Api-Version as top-level header in dispatchWorkflow #33419 (100 words): fix: pass X-GitHub-Api-Version as top-level header — explains the API misconfiguration with technical precision and the exact parameter path that was wrong.
3. PR Refocus forecast output on effective-token predictions by removing yield and episode metrics #34750: Refocus forecast output on effective-token predictions — narrows scope clearly, explains what was removed and why.

❌ Unsuccessful Prompt Patterns

Common characteristics in closed PRs:

• Average prompt length: 820 words (longer ≠ better)
• WIP-labeled PRs: 53.1% closure rate vs 17.6% for non-WIP
• Very long (500–1000 words): only 41.7% success — worst performing bucket
• Prompts with vague, exploratory, or incomplete intentions

Example closed prompts:

1. PR [WIP] Fix cross-package inconsistencies in specifications #34662: [WIP] Fix cross-package inconsistencies in specifications — WIP label signals incomplete intent; 500–1000 word body with broad scope.
2. PR Planning to add OTLP to workflows #31065 (6 words): Planning to add OTLP to workflows — placeholder description with no actionable detail.
3. PR <empty> #31203: (empty body) — No prompt provided; essentially a blank submission.

---

Key Insights

• WIP label is the strongest failure signal: [WIP] PRs have a 53.1% closure rate vs 17.6% for non-WIP. Agents appear to submit exploratory or incomplete work under WIP, which then gets rejected. If a task isn't ready, hold the PR rather than submitting WIP.
• Sweet spot is 150–500 words: This range achieves the highest merge rate (82.8%). Both very short (<50 words, 73.7%) and long-but-not-comprehensive (500–1000 words, 41.7%) perform worse. Concise, well-scoped prompts outperform verbose ones.
• Testing prompts have a 100% success rate: All 4 PRs categorized as test-related were merged. Prompts focused on adding test coverage or fixing tests are highly actionable and well-received.

---

Recommendations

Based on today's analysis:

1. DO keep prompts in the 150–500 word range — precise descriptions of the problem, change, and rationale without padding.
2. DO use specific action verbs (fix, add) paired with a technical component name — mirrors the pattern of 82%+ merged PR titles.
3. DO describe both what changed and why it was wrong — the highest-confidence merged PRs include root cause context.
4. AVOID submitting PRs with [WIP] in the title — they close at 53% vs 18% for non-WIP. Use draft PRs instead if you need a staging area.
5. AVOID prompts in the 500–1000 word range without clear structure — this is the worst-performing bucket (41.7%). If a prompt is long, use headers to structure it clearly.

---

Historical Trends

Date	Total PRs	Merged	Closed	Success Rate
2026-05-15	1,000	797	200	79.9%
2026-05-17	1,000	809	191	80.9%
2026-05-18	1,000	811	185	81.4%
2026-05-19	993	811	182	81.7%
2026-05-24	1,000	810	186	81.3%
2026-05-25	1,000	804	193	80.6%

Trend: Success rate has been stable in the 80–82% range for the past 10 days. Today shows a slight dip (80.6%) compared to the 81.3% peak on May 24, largely due to a small uptick in WIP closures. No concerning downward trend.

---

References: §26419502431

Generated by 🔍 Copilot PR Prompt Pattern Analysis · sonnet46 1.2M · ◷

• expires on May 26, 2026, 9:03 PM UTC