Overview
- Date: 2026-09-09
- Students simulated: 46 × 1000 Monte Carlo runs
- Workshop steps available: 30/30
- Overall success rate: 23.4% (95% Monte Carlo interval: 23.0%–23.8%)
- Highest-dropout step:
04-actions-intro (18.0% conditional dropout among 44,166 at-risk runs; 95% interval: 17.6%–18.3%)
- Lowest curriculum quality step:
04-github-actions-intro.md (overall score 5.39/10)
- Learning KPI index: 2.88/10 (active_learning 4.18 · checkpoint_quality 0.00 · scaffolding 5.00)
- Model:
2026-07-survival-model-v2 / 2026-07-assumption-model-v2 (parameter hash 2024391902)
- Limitation: synthetic results reflect explicit model assumptions; intervals exclude model and population-assumption uncertainty
Part Summary
| Part |
Files |
Mean Score |
Std Dev |
| Part 1 — core path (lessons 00–14) |
15 |
6.69 / 10.0 |
±1.74 |
| Part 2 — advanced (lessons 15+) |
15 |
6.07 / 10.0 |
±0.36 |
| Overall corpus |
30 |
6.38 / 10.0 |
±– |
No steps are classified as other in this run.
Critical Findings
04-actions-intro is now the single highest-dropout step (18.0%, ~7,944 failures cohort-wide) driven entirely by concept-overload — the page has zero troubleshooting cues, zero callouts, and no fallback explanation if the labeling exercise or Actions/agentic comparison doesn't land the first time (active_learning 3.9/10, checkpoint_quality 0.0/10 in the rubric).- Agent-derived content evidence shows Step 8 (
08-run-your-workflow) actually has strong state readiness: the source .md and .lock.yml are both explicitly committed and pushed in Step 7 (line 105-121), and the billing side quest walks both centralized and personal billing paths to completion. After incorporating this evidence, Step 8's conditional dropout fell from 100% (lexical false-positive) to ~1.4%, showing the content itself is not the blocker — the earlier lexical-proximity heuristic under-detected commit/push instructions split across adjacent code fences.
- Learning quality is weak even where students persist: the cohort-wide Learning KPI index is 2.88/10, dragged down almost entirely by
checkpoint_quality scoring 0.0/10 on every main step. Investigation shows this is a rubric-detection artifact — the shared rubric's CHECKPOINT_RE only matches a literal ✅ character in the heading, while all 30 main steps use the :white_check_mark: shortcode, so genuine checkpoints are systematically undercounted, not absent.
- The most important repair belongs to Part 1 (
00–14, lessons 04 and 05 specifically) — that is where the highest-dropout, lowest-score steps concentrate, while Part 2 has a much tighter, more consistent score distribution (±0.36 vs ±1.74).
Top Repairs to Prioritize
Note: some student dropout is expected and acceptable. Repairs must maintain or improve the learning KPI index — do not lower the cognitive bar or remove practice to chase headline completion numbers.
- Add a troubleshooting/recovery callout and a second worked example to
04-github-actions-intro.md to address the concept-overload dropout without cutting content (completion impact: ↑ · learning KPI impact: ↑, targets active_learning + checkpoint recognition)
- Fix the
CHECKPOINT_RE detection gap in curriculum_assessment.py to also match the :white_check_mark: shortcode heading, so checkpoint_quality scores reflect the real (well-formed) checklists already present across the corpus (completion impact: ↔ · learning KPI impact: ↑, corrects a measurement bug rather than content)
- Strengthen
05-agentic-workflows-intro.md with a tighter before/after concept summary and one additional worked contrast example to reduce the agentic-concept-gap failure mode while preserving its existing multi-activity structure (completion impact: ↑ · learning KPI impact: ↑, targets active_learning + scaffolding)
Dropout by step
| Step |
At-risk runs |
Dropouts |
Conditional dropout rate |
95% MC interval |
Failure mode |
Top reason |
| 04-actions-intro |
44,166 |
7,944 |
18.0% |
17.6%–18.3% |
learning barrier |
Concept overload — learners skim the Actions explanation and reach later steps without a stable mental model |
| 05-agentic-intro |
36,222 |
6,533 |
18.0% |
17.6%–18.4% |
learning barrier (mostly) / access barrier (GHES subset) |
Agentic concept gap — learners don't fully internalize the shift from deterministic jobs to goal-oriented workflows; smaller share is a GHES enablement gap |
| 07-first-workflow |
22,109 |
4,731 |
21.4% |
20.9%–21.9% |
access barrier |
Copilot access missing — the account lacks usable Copilot access before the required model-access check |
| 05c-agentic-practice |
29,689 |
3,598 |
12.1% |
11.8%–12.5% |
learning barrier |
Agentic classification gap — learners struggle to reliably decide agentic vs. standard workflow in the practice exercises |
| 05b-agentic-security |
26,091 |
2,457 |
9.4% |
9.1%–9.8% |
learning barrier |
Agentic security gap — learners don't fully absorb the safe-outputs/sandbox guardrail model |
| 06-install-gh-aw |
23,634 |
1,525 |
6.5% |
6.1%–6.8% |
access barrier |
Extension install friction — auth/token-scope issues around gh extension install |
| 02-setup |
46,000 |
1,834 |
4.0% |
– |
access barrier |
Setup friction — Codespace launch and readiness checks |
| 08-run-your-workflow |
17,378 |
~240 |
1.4% |
1.3%–1.7% |
access barrier (residual) |
Actions:write / model-access-not-configured edge cases; step content itself is well-supported |
Curriculum quality and learning KPIs
| Step file |
Overall score |
active_learning |
checkpoint_quality |
scaffolding |
Learning KPI index |
Lowest rubric dimension |
Repair priority |
| 04-github-actions-intro.md |
5.39 |
3.9 |
0.0 |
5.0 |
2.78 |
checkpoint_quality (detection gap) |
High |
| 05-agentic-workflows-intro.md |
5.43 |
2.4 |
0.0 |
5.0 |
2.24 |
checkpoint_quality (detection gap) |
High |
| 05b-agentic-workflows-security.md |
5.75 |
2.5 |
0.0 |
5.0 |
2.27 |
checkpoint_quality (detection gap) |
Medium |
| 08-run-your-workflow.md |
5.67 |
3.0 |
0.0 |
5.0 |
2.36 |
checkpoint_quality (detection gap) |
Low (content already well-supported) |
| 02a-setup-codespace.md |
6.09 |
5.0 |
0.0 |
5.0 |
3.18 |
checkpoint_quality (detection gap) |
Low |
| 05c-agentic-workflows-practice.md |
6.22 |
6.2 |
0.0 |
5.0 |
3.62 |
checkpoint_quality (detection gap) |
Medium |
| 07-your-first-workflow.md |
6.25 |
6.3 |
0.0 |
5.0 |
3.65 |
checkpoint_quality (detection gap) |
Medium |
| 06-install-gh-aw.md |
10.0 |
3.5 |
0.0 |
5.0 |
2.64 |
checkpoint_quality (detection gap) |
Low |
| Cohort mean (30 steps) |
6.38 |
4.18 |
0.0 |
5.0 |
2.88 |
checkpoint_quality (detection gap, corpus-wide) |
— |
Segment breakdowns
By technical level
| Level |
Mean success rate |
n |
| advanced |
49.4% |
5 |
| actions-user |
48.3% |
11 |
| github-basic |
15.4% |
19 |
| beginner |
0.5% |
11 |
By personality
| Personality |
Mean success rate |
n |
| impatient |
26.7% |
6 |
| methodical |
25.9% |
12 |
| skeptical |
23.2% |
7 |
| confused |
22.0% |
6 |
| curious |
20.6% |
15 |
By UI preference
| UI preferred |
Mean success rate |
| false (CLI-first) |
33.9% |
| true (browser-first) |
11.9% |
Notable student journeys (3)
- Surprising success: Learner 011 (
actions-user, methodical, backend-dev, CLI, browser-not-preferred) posted the cohort's highest success rate (62.4%) — methodical CLI-comfortable learners with prior Actions experience clear the terminal-heavy Step 6/7/8 sequence with the least friction.
- Unexpected dropout: Learner 016 (
advanced, impatient, data-science) had a surprisingly low 38.4% success rate for an advanced-level profile, most often stalling at 07-first-workflow — impatience combined with the required Copilot access pre-check produces friction even for technically strong learners who skip verification steps.
- Content-gap case: All
beginner-level, no-coding/program-manager background students (e.g., Learners 017, 020, 022, 029, 038) recorded 0% success rate, uniformly dropping at 04-actions-intro — this reflects a genuine learning barrier (concept overload with no recovery path) compounding with a population assumption that beginners have very low baseline concept-absorption capacity; it is a model assumption, not an observed human result.
Generated by 🔬 Workshop Student Simulator · copilot · auto · 231.3 AIC · ⌖ 7.23 AIC · ⊞ 15.2K · ◷
Overview
04-actions-intro(18.0% conditional dropout among 44,166 at-risk runs; 95% interval: 17.6%–18.3%)04-github-actions-intro.md(overall score 5.39/10)2026-07-survival-model-v2/2026-07-assumption-model-v2(parameter hash2024391902)Part Summary
No steps are classified as
otherin this run.Critical Findings
04-actions-introis now the single highest-dropout step (18.0%, ~7,944 failures cohort-wide) driven entirely byconcept-overload— the page has zero troubleshooting cues, zero callouts, and no fallback explanation if the labeling exercise or Actions/agentic comparison doesn't land the first time (active_learning 3.9/10, checkpoint_quality 0.0/10 in the rubric).08-run-your-workflow) actually has strong state readiness: the source.mdand.lock.ymlare both explicitly committed and pushed in Step 7 (line 105-121), and the billing side quest walks both centralized and personal billing paths to completion. After incorporating this evidence, Step 8's conditional dropout fell from 100% (lexical false-positive) to ~1.4%, showing the content itself is not the blocker — the earlier lexical-proximity heuristic under-detected commit/push instructions split across adjacent code fences.checkpoint_qualityscoring 0.0/10 on every main step. Investigation shows this is a rubric-detection artifact — the shared rubric'sCHECKPOINT_REonly matches a literal✅character in the heading, while all 30 main steps use the:white_check_mark:shortcode, so genuine checkpoints are systematically undercounted, not absent.00–14, lessons 04 and 05 specifically) — that is where the highest-dropout, lowest-score steps concentrate, while Part 2 has a much tighter, more consistent score distribution (±0.36 vs ±1.74).Top Repairs to Prioritize
Note: some student dropout is expected and acceptable. Repairs must maintain or improve the learning KPI index — do not lower the cognitive bar or remove practice to chase headline completion numbers.
04-github-actions-intro.mdto address the concept-overload dropout without cutting content (completion impact: ↑ · learning KPI impact: ↑, targets active_learning + checkpoint recognition)CHECKPOINT_REdetection gap incurriculum_assessment.pyto also match the:white_check_mark:shortcode heading, so checkpoint_quality scores reflect the real (well-formed) checklists already present across the corpus (completion impact: ↔ · learning KPI impact: ↑, corrects a measurement bug rather than content)05-agentic-workflows-intro.mdwith a tighter before/after concept summary and one additional worked contrast example to reduce theagentic-concept-gapfailure mode while preserving its existing multi-activity structure (completion impact: ↑ · learning KPI impact: ↑, targets active_learning + scaffolding)Dropout by step
gh extension installCurriculum quality and learning KPIs
Segment breakdowns
By technical level
By personality
By UI preference
Notable student journeys (3)
actions-user, methodical, backend-dev, CLI, browser-not-preferred) posted the cohort's highest success rate (62.4%) — methodical CLI-comfortable learners with prior Actions experience clear the terminal-heavy Step 6/7/8 sequence with the least friction.advanced, impatient, data-science) had a surprisingly low 38.4% success rate for an advanced-level profile, most often stalling at07-first-workflow— impatience combined with the required Copilot access pre-check produces friction even for technically strong learners who skip verification steps.beginner-level,no-coding/program-managerbackground students (e.g., Learners 017, 020, 022, 029, 038) recorded 0% success rate, uniformly dropping at04-actions-intro— this reflects a genuine learning barrier (concept overload with no recovery path) compounding with a population assumption that beginners have very low baseline concept-absorption capacity; it is a model assumption, not an observed human result.