diff --git a/.cursor/rules/video-playwright.mdc b/.cursor/rules/video-playwright.mdc new file mode 100644 index 0000000..5f8eadc --- /dev/null +++ b/.cursor/rules/video-playwright.mdc @@ -0,0 +1,55 @@ +--- +description: Video recording uses the isolated video/ Playwright package — never ad-hoc installs +alwaysApply: true +--- + +# Short-form videos & Playwright + +Exploration videos (9:16 vertical) are recorded by the **isolated** `website/video/` package. +Playwright is intentionally **not** in the root `package.json` (CI stays lean). + +Chromium is **shared with the OG pipeline** — if `npm run og:check` passes, video recording +can use the same browser cache. + +## One-time human setup (Holger's machine only) + +From `feelyourprotocol/website/`: + +```bash +npm run og:setup # if not done yet — downloads Chromium (~once per machine) +npm run video:setup # npm install in video/ only +npm run video:check # should print: OK: Playwright Chromium is installed... +``` + +**Holger has already run og:setup.** Do not run setup again unless he explicitly asks. + +`video/` CLIs reuse `og/` bootstrap — redirect Cursor's sandbox `PLAYWRIGHT_BROWSERS_PATH` +to the developer's real browser cache. Use `required_permissions: ["all"]` and **never** +run `video:setup` / `playwright install` as the first fix. + +## Agent rules (mandatory) + +### NEVER do these + +- `npm run video:setup`, `npm run og:setup`, or any `playwright install` +- Ad-hoc `npx playwright install` or sandbox browser downloads +- Retry setup when record fails — that is almost never the fix + +### ALWAYS do these + +1. Run video commands with **`required_permissions: ["all"]`** +2. Use npm scripts from `website/`: + - `npm run video:preflight` + - `npm run video:record -- ` +3. Optional fast check: `npm run video:preflight` with `required_permissions: ["all"]` + - `status: ready` → proceed + - `status: needs_agent_permissions` → re-run with `required_permissions: ["all"]` + - `status: needs_human_setup` → ask user to run `npm run og:check` locally + +### When record fails + +1. Permission errors → re-run with `required_permissions: ["all"]` +2. Still fails → ask user for `npm run og:check` output; do **not** run setup +3. Selector timeout / 404 → fix app or playbook, not Playwright setup + +See `video/README.md` and `docs/contributing/video-pipeline.md`. diff --git a/.gitignore b/.gitignore index 813a59b..2749606 100644 --- a/.gitignore +++ b/.gitignore @@ -35,4 +35,10 @@ roadmap/.vitepress/cache SERVER_NOTES.md # Dist -dist/ \ No newline at end of file +dist/ + +# Isolated Playwright packages (local node_modules + artifacts) +og/node_modules +og/.chromium-ready.json +video/node_modules +video/projects/*/output/ \ No newline at end of file diff --git a/docs/.vitepress/config.ts b/docs/.vitepress/config.ts index ca484fd..99abc55 100644 --- a/docs/.vitepress/config.ts +++ b/docs/.vitepress/config.ts @@ -101,6 +101,8 @@ export default defineConfig({ { text: 'Styling & Design', link: '/contributing/styling' }, { text: 'Code Conventions', link: '/contributing/code-conventions' }, { text: 'Third-Party Libraries', link: '/contributing/third-party-libraries' }, + { text: 'Video Pipeline', link: '/contributing/video-pipeline' }, + { text: 'Authoring a Video Short', link: '/contributing/video-authoring' }, ], }, { diff --git a/docs/contributing/video-authoring.md b/docs/contributing/video-authoring.md new file mode 100644 index 0000000..b7e6ccc --- /dev/null +++ b/docs/contributing/video-authoring.md @@ -0,0 +1,386 @@ +# Authoring a video short + +How to plan, write, and record a Feel Your Protocol vertical short (~45–60 s) from a +live exploration. + +Operational commands and file layout: [Video pipeline](/contributing/video-pipeline). +Reference project: `video/projects/eip-8024/`. + +--- + +## Goals + +Each short should: + +1. Show **real** exploration UI (not mockups). +2. Use the **video overlay language** (black bars, Bebas Neue / Barlow Condensed) — not site chrome. +3. Guide the viewer's eye with layered overlays without overcrowding the frame. +4. Start with a **title-card thumbnail** (frame 0 after trim). + +Rendering lives in `src/video/`; copy and timing live in `video/projects//`. +Run `npm run website:build` before recording whenever overlay components change. + +--- + +## Workflow overview + +``` +Plan story arc → content.json (overlays, annotations, highlights) + → zones.json (focus areas) + → playbook.json (beats + structure) + → npm run video:generate ← final *-final.mp4 (video + voice) + → npm run video:record --no-voice ← silent .webm only + → verify frame 0 thumbnail + audio sync +``` + +Silent-only path (no voice): omit `narration.json` / `voice/` and set manual `cue`/`wait` in `playbook.json`. +See [Voice-over pipeline](/contributing/video-voice). + +--- + +## 1. Plan the story arc + +Break the video into **beats** — one playbook step each. Each beat can include: + +| Phase | Playbook | Purpose | +|-------|----------|---------| +| Guide | `annotate` | Pointer on live UI (parallel through the beat) | +| Cue | `overlay` + `cue` | Story text the viewer reads before action | +| Reveal | `selectExample`, `step`, `scroll`, `expandCompanion` | Exploration action | +| Hold | `wait` | Pause after action so the result sinks in | + +Typical arc for an opcode demo: + +1. **Title** — hook and topic +2. **Context** — hardfork / problem statement +3. **Setup** — select example, grow state (steps) +4. **Climax** — the one opcode that matters (slow step + highlights) +5. **Extra** — companion panel, calculator, etc. (optional) +6. **Recap** — tie back to the takeaway (text-only on current state) +7. **Outro** — closing line + CTAs + +Use `npm run video:storyboard -- ` to preview timing before recording. + +--- + +## 2. Project files + +Create `video/projects//`: + +| File | Contents | +|------|----------| +| `content.json` | `overlays`, `annotations`, `highlightSets` | +| `playbook.json` | Ordered beats with timing and actions | +| `zones.json` | Focus-area selectors for `focus` and annotation targets | +| `output/` | Recorded `.webm` (gitignored) | + +Add the exploration to `video/src/explorationRegistry.ts` if not already listed. + +### `zones.json` — focus areas + YouTube Shorts safe zone + +`focusAreas` / `placements` drive playbook `focus` scrolling. Add **`safeZone`** (540×960 px) so readable content clears Shorts platform chrome: + +| Edge | Default | Notes | +|------|---------|-------| +| top | 56px | Username / top controls (text sits lower inside black bars) | +| bottom | 120px | Like / comment / caption row | +| left | 32px | Edge crop — e.g. keep “3 NEW OPCODES” fully visible | +| right | 48px | Right-side action column | + +Black overlay bars stay **edge-to-edge**; only the **text inside** is inset (same horizontal safe zone as exploration UI). CSS applies `--video-safe-left` / `--video-safe-right` to `.video-banner__line`, punch lines, and title/outro band content — not as outer margin on the black box. + +Vertical rhythm inside top/bottom ribbons uses `--video-ribbon-clear-top` (Shorts chrome clearance, ~60% of `--video-safe-top`), `--video-ribbon-pad-top`, and `--video-ribbon-pad-bottom` (top still larger than bottom). Tune in `src/video/video.css` if a project’s banner feels cramped. + +--- + +## 3. Story overlays (`content.json` → `overlays`) + +Story overlays carry the narrator text. + +| Type | Use | +|------|-----| +| `title-card` | Opening — full band layout (see [Title and outro cards](#title-and-outro-cards)) | +| `outro-card` | Closing + CTAs | +| `punch` | Single headline + optional subline | +| `split` | Two staggered lines | +| `list-flash` | Headline + bullet list | + +In the playbook, pair overlays with **`placement: "top-banner"`** and a **`focus`** key +so the exploration panel stays visible below the banner. + +**Pattern:** + +```json +{ + "beat": "stack-grow", + "overlay": "stack-cue", + "placement": "top-banner", + "focus": "stack", + "cue": 3000, + "step": { "count": 9, "interval": 360 }, + "wait": 1600 +} +``` + +Show overlay → `cue` ms → hide → run actions → `wait` ms. + +--- + +## 4. UI annotations (`content.json` → `annotations`) + +Annotations are black callouts with a ring on a UI target. They answer: *where should I look?* + +```json +"annotations": { + "guide-stack": { + "target": "stack", + "label": "Live stack", + "hint": "Watch depth grow", + "side": "top" + } +} +``` + +In the playbook: `"annotate": "guide-stack"`. The annotation **stays visible** while the +beat runs — including after the story overlay hides and steps begin — whenever the banner +alone does not point at the action area. If a **top-banner story overlay** covers the +annotation target, the callout is suppressed automatically (avoids collisions like +“Example presets” over “3 NEW OPCODES”). + +Targets are keys from `zones.json` `focusAreas`, or built-in aliases: + +| Target key | Resolves to | +|------------|-------------| +| `disassembly-active` | `[data-disassembly-active="true"]` | +| `stack-top` | `[data-stack-depth="1"]` | +| `stack-depth-{n}` | `[data-stack-depth="{n}"]` | + +Explorations used in videos need stable `data-testid` / `data-stack-depth` attributes on +the elements you annotate (see `BytecodeStepperEC` for the EIP-8024 pattern). + +--- + +## 5. Climax highlights (`content.json` → `highlightSets`) + +Use a third layer for the **peak moment** — translucent yellow **area boxes** over the +exact text cells (opcode mnemonic, stack value). No labels, no full-row width, no site chrome. + +Target the **text cell**, not the full row: + +| Target key | Highlights | +|------------|------------| +| `disassembly-active-opcode` | Opcode text on the active PC row (windup) | +| `disassembly-dupn` | `DUPN …` mnemonic row (payoff — PC may have moved on) | +| `stack-top-value` | `[0x…]` on stack depth 1 | +| `stack-depth-{n}-value` | Value cell at depth *n* | + +Requires `data-disassembly-opcode`, `data-stack-value` on exploration widgets (see +`BytecodeStepperEC`). + +Define sets in `content.json`: + +```json +"highlightSets": { + "dupn-windup": { + "marks": [ + { "target": "disassembly-active-opcode", "padX": 12, "padY": 10 }, + { "target": "stack-depth-17-value", "padX": 12, "padY": 10 } + ] + }, + "dupn-payoff": { + "marks": [ + { "target": "disassembly-dupn", "padX": 14, "padY": 12 }, + { "target": "stack-top-value", "padX": 14, "padY": 12 } + ] + } +} +``` + +Wire timing in the playbook `step` action: + +```json +"step": { + "count": 9, + "interval": 380, + "climaxFrom": 9, + "climaxInterval": 1050, + "climaxPauseMs": 550, + "highlightSet": "dupn-windup", + "highlightAfterStep": 9, + "highlightSetAfter": "dupn-payoff", + "holdAfterClimaxMs": 1500 +} +``` + +| Field | Purpose | +|-------|---------| +| `climaxFrom` | 1-based step index to enter slow motion | +| `climaxInterval` | Step delay during climax (ms) | +| `climaxPauseMs` | Pause before the first climax step | +| `highlightSet` | Highlights shown when climax starts | +| `highlightAfterStep` | Step number after which to switch sets | +| `highlightSetAfter` | Payoff highlight set | +| `holdAfterClimaxMs` | Extra hold after payoff highlights | + +--- + +## 6. Pacing and cue times + +Allocate enough read time for overlay density: + +| Overlay density | Recommended `cue` | +|-----------------|-------------------| +| Headline + subline (`punch`, `split`) | 3000–3400 ms | +| Headline + list (`list-flash`, 3+ items) | 4000–4200 ms | +| Title / outro band cards | 4500–5000 ms | +| Text-only recap (no actions) | 4000+ ms (`wait` only) | + +Step intervals: ~360–400 ms for routine steps; ~1000 ms+ for the climax step. + +Example timings from `eip-8024`: + +| Beat | Overlay | Timing | +|------|---------|--------| +| hook | title-card | `wait` 4500 ms | +| context | glamsterdam | `cue` 3400 ms | +| opcodes | three-opcodes | `cue` 4200 ms, `wait` 1200 ms | +| stack-grow | stack-cue | `cue` 3000 ms, `wait` 1600 ms | +| dupn-hit | dupn-cue | `cue` 3000 ms + climax step | +| calculator | calculator-tease | `cue` 2800 ms, `wait` 2400 ms | +| recap | old-limit-cue | `wait` 4200 ms (no actions) | +| outro | outro-card | `wait` 5000 ms | + +Storyboard validation warns when `cue` is missing on overlay+action beats or when ids +do not resolve in `content.json` / `zones.json`. + +--- + +## 7. Title and outro cards + +Full-screen cards use **`VideoBandShell`**: + +- Middle **~3/5** — black band with hero type (title, subtitle, hook / closing, CTAs). +- Top and bottom **~1/5** — live exploration peeking through. + +| Card | Peek role | +|------|-----------| +| Title | Show that a live demo runs underneath | +| Outro | Keep exploration visible behind CTAs | + +Outro CTAs support `variant: "secondary"` (smaller, e.g. Forkcast) and `"primary"` +(larger, e.g. FYP). Use `"READ ABOUT THE EIP"` for Forkcast, not generic "READ MORE". + +CSS: `html.fyp-video-capture` locks hero font sizes; `html.fyp-video-band-active` during band cards. + +--- + +## 8. Recap and example switches + +**Recap beats** should stay on the current exploration state — overlay text only, no +`selectExample` or `step`. Point `focus` at the result the viewer just watched (e.g. stack). + +```json +{ + "beat": "recap", + "overlay": "old-limit-cue", + "annotate": "guide-stack", + "focus": "stack", + "wait": 4200 +} +``` + +Write recap copy that references what already happened ("You just copied depth 17"), not +a new demo. + +Switch examples mid-video only when the overlay **introduces** a new scenario and the +reset is intentional. Pick an example that matches the overlay message (e.g. do not use +`invalid-dupn` to explain the historical DUP depth-16 limit — that example shows a +shallow-stack revert, a different concept). + +--- + +## 9. Record and verify + +```bash +npm run website:build +npm run video:record -- --preview # 540×960 — fast iteration +npm run video:generate -- # 1080×1920 *-final.mp4 (2× upscale on mux) +``` + +All captures use a **540×960 Playwright viewport** — the layout reference. Preview and +full-res differ only at mux: `video:generate` upscales 2× to 1080×1920 (lanczos). Composition +is identical; do not hand-upscale a preview file. + +After recording: + +1. Check the trim log — frame 0 should be the title band (valid stats: dark pixel ratio + ~0.8, mean luminance ~40 in the center crop). +2. Scrub the `.webm` at **0:00** (thumbnail), the climax timestamp, and the outro. +3. Confirm overlays, annotations, and highlights appear as storyboarded. + +The recorder hides the exploration until the title card is ready, preloads display fonts, +and post-trims lead-in with ffmpeg. Requires `ffmpeg` on PATH. + +--- + +## Troubleshooting + +| If you see… | Check… | +|-------------|--------| +| Missing overlays / annotations in output | Run `npm run website:build` before record | +| Gray letterboxing | Playwright `recordVideo.size` matches viewport dimensions | +| Frame 0 shows exploration UI | Trim log; title-band detection needs black center band | +| Small fallback font on thumbnail | Font preload (`display=block`) and capture CSS | +| Top banner covers example selector | Banner max-height CSS; `focus` uses `block: 'start'` scroll | +| Climax highlights / slow motion absent | All climax fields present on `step` in playbook JSON | +| UI resets without explanation | Remove mid-video `selectExample`; use text-only recap | +| Banner overlaps during focus | `zones.json` scroll margins; `--video-top-banner-max-h` | + +--- + +## Guidelines + +- Keep **three layers separate**: story overlays, annotations, climax highlights. +- Use **area highlights** on text cells (`*-opcode`, `*-value`) — not full rows. +- Give **dense overlays ≥ 3 s** cue time; run storyboard before recording. +- Make **frame 0** the title card; verify after every record. +- Put **`data-testid`** on controls the playbook clicks. + +--- + +## Reference example: EIP-8024 + +| ~Time | Beat | Content | +|-------|------|---------| +| 0:00 | title-card | EIP-8024, Glamsterdam, opcodes, stack-too-deep hook | +| 0:08 | glamsterdam → opcodes | Hardfork; three opcodes; select DUPN | +| 0:14 | stack-grow | 17 pushes — stack grows | +| 0:22 | dupn-hit | DUPN copies depth 17 — climax with markers + slow step | +| 0:34 | calculator | Companion / magic-byte calculator | +| 0:39 | recap | Classic DUP depth 16 vs depth 17 just demonstrated | +| 0:44 | outro | Closing + Forkcast + FYP CTAs | + +Files: `video/projects/eip-8024/{content,playbook,zones}.json`. + +--- + +## Checklist + +- [ ] Story arc defined in beats +- [ ] `zones.json` covers every `focus` and annotation target +- [ ] Dense overlays have adequate `cue` (3000–4200 ms); storyboard reviewed +- [ ] Annotations on beats with UI action after overlay hides +- [ ] Climax beat has `highlightSets` + climax timing (if applicable) +- [ ] Recap beats are text-only on current state; no unexplained example switches +- [ ] First step is `title-card` +- [ ] `npm run website:build` before record +- [ ] Frame 0 thumbnail verified after record + +--- + +## Related + +- [Video pipeline](/contributing/video-pipeline) — architecture, overlay types, selectors +- [Voice-over pipeline](/contributing/video-voice) — ElevenLabs narration, timing, mux +- [Architecture — Video pipeline](/guide/architecture#video-pipeline-short-form) +- `video/README.md` — CLI and Playwright selectors diff --git a/docs/contributing/video-pipeline.md b/docs/contributing/video-pipeline.md new file mode 100644 index 0000000..efff84e --- /dev/null +++ b/docs/contributing/video-pipeline.md @@ -0,0 +1,256 @@ +# Video pipeline + +Short-form vertical videos (YouTube Shorts, Instagram Reels, X) that educate about +EIPs using live Feel Your Protocol explorations. + +## Architecture + +Two layers stay separate: + +1. **Website (`src/video/`)** — overlay rendering and video-mode chrome stripping. + Activated with `?fyp-video=1`. Does not contain video copy or playbooks. +2. **Video package (`video/`)** — `content.json`, `playbook.json`, Playwright recording CLI. + Mirrors the isolated `og/` package pattern. + +``` +website:build → static server → Playwright recordVideo + ↑ + ?fyp-video=1 + injected __FYP_VIDEO_CONFIG__ +``` + +## URL parameters + +| Param | Scope | Description | +|-------|-------|-------------| +| `fyp-video=1` | Any route | Video capture mode — hides header, footer, roadmap banner | +| `example=` | Explorations | Pre-selects an example preset on load | + +Example deep link: + +``` +/eip-8024-stack-opcodes-dupn-swapn-exchange?fyp-video=1&example=swapn +``` + +The `example` param works for **all** explorations: + +- E-Component explorations (`BytecodeStepperEC`, `PrecompileInterfaceEC`) via shared init logic +- Custom explorations (`eip-7594`, `eip-7928`) via `resolveInitialExample()` and `useExplorationExampleQuery()` + +Invalid keys fall back to each exploration's default example. + +For precompile explorations that also support field-level share URLs (`?b=`, `?hash=`, …), +`?example=` takes precedence when present. + +## Overlay system + +Overlays use a **separate visual language** from the main site: black bars, Bebas Neue / +Barlow Condensed, no rounded corners, topic color as accent only. + +Components in `src/video/`: + +| File | Role | +|------|------| +| `VideoShell.vue` | Overlay layer + `window.__FYP_VIDEO__` bridge | +| `VideoOverlay.vue` | Routes to type-specific overlay components | +| `overlays/VideoTitleCard.vue` | Full-screen movie title | +| `overlays/VideoSplitOverlay.vue` | Staggered split read flow | +| `video.css` | Capture-only typography and animations | + +Overlay types in `content.json`: + +| Type | Use | +|------|-----| +| `title-card` | Full-screen opening (black, movie poster) | +| `outro-card` | Full-screen closing | +| `punch` | Massive single-line highlight bar | +| `split` | Staggered black bars (upper-left → lower-right) | +| `list-flash` | Bold bullet list | + +Copy is injected via `window.__FYP_VIDEO_CONFIG__` before navigation. + +Full-screen **title** and **outro** cards use `VideoBandShell`: middle ~3/5 black hero band, +top/bottom ~1/5 peek zones show live exploration. See +[Authoring a video short](/contributing/video-authoring#title-and-outro-cards). + +### UI annotations (`content.json` → `annotations`) + +Parallel **guide callouts** point at exploration regions while story text may hide: + +```json +"annotations": { + "guide-stack": { + "target": "stack", + "label": "Live stack", + "hint": "Watch depth grow", + "side": "top" + } +} +``` + +Reference playbook step: `"annotate": "guide-stack"`. Targets use `zones.json` focus area keys +or built-in aliases (`disassembly-active-opcode`, `disassembly-dupn`, `stack-top-value`, …). +Callouts are **auto-suppressed** when a top-banner overlay covers the target — see +[Authoring § annotations](/contributing/video-authoring#4-ui-annotations-contentjson--annotations). + +### Climax highlights (`content.json` → `highlightSets`) + +Third overlay layer — translucent yellow **area boxes** over text cells during peak +moments (opcode mnemonic, stack value — not full rows). Playbook climax fields on `step`: + +| Field | Purpose | +|-------|---------| +| `climaxFrom` | 1-based step index to enter slow motion | +| `climaxInterval` | Step delay during climax (ms) | +| `climaxPauseMs` | Pause before first climax step | +| `highlightSet` | Highlight set id before climax step | +| `highlightAfterStep` | Switch highlights after this step completes | +| `highlightSetAfter` | Payoff highlight set id | +| `holdAfterClimaxMs` | Extra hold after payoff | + +See [Authoring a video short](/contributing/video-authoring#climax-highlights-content-json-highlightsets). + +## Story beats (cue → reveal) + +Each playbook step can tell a **story beat** synchronized with the exploration: + +| Field | Purpose | +|-------|---------| +| `beat` | Label for storyboard / validation | +| `overlay` | Text the viewer reads **before** the action | +| `annotate` | UI guide callout id — **stays visible** while story overlay hides and actions run | +| `placement` | `top-banner` or `bottom-banner` — keeps the focus area clear | +| `focus` | UI region to keep visible (see `zones.json`) | +| `cue` | Read time (ms) — overlay stays up, then hides before actions | +| `wait` | Hold time (ms) after actions — let the viewer absorb | + +**Pattern:** show overlay → `cue` → hide → step / select example → `wait` + +Preview the narrative timeline: + +```bash +npm run video:storyboard -- eip-8024 +npm run video:record -- eip-8024 --dry-run # storyboard + validation +``` + +### Safe zones (`zones.json`) + +Per-project map of exploration focus areas and overlay placements (540×960 reference). +Black ribbon backgrounds stay edge-to-edge; text is inset horizontally and vertically via +`--video-safe-*` and `--video-ribbon-*` CSS vars — details in +[Authoring § safe zone](/contributing/video-authoring#zonesjson--focus-areas--youtube-shorts-safe-zone). +Author overlays in `top-banner` so stack, disassembly, or companion stay visible below. + +## Playbook format + +Each project lives in `video/projects//`: + +| File | Purpose | +|------|---------| +| `content.json` | Overlay copy, annotations, highlight sets | +| `playbook.json` | Story beats: `overlay`, `cue`, `focus`, `selectExample`, `step`, … | +| `zones.json` | Focus-area selectors + banner placement metadata | +| `output/` | Recorded `.webm` (gitignored) | + +Example beat — read, then step with stack visible: + +```json +{ + "beat": "stack-grow", + "overlay": "stack-cue", + "placement": "top-banner", + "focus": "stack", + "cue": 2400, + "step": { "count": 9, "interval": 340 }, + "wait": 1400 +} +``` + +## Automation hooks + +Stable selectors for Playwright (see `video/README.md` for the full list). + +When adding new interactive controls to explorations, add a `data-testid` if the +video playbook needs to click them. + +`BytecodeStepperEC` auto-scrolls the disassembly and stack panels during stepping +(active PC row + deep stack entries stay in view). Playwright also syncs panel +scroll after each `step` action in the playbook. + +## Local development + +Preview video mode in the dev server (overlays only appear when config is injected — use record dry-run to validate copy): + +```bash +npm run dev +# /eip-8024-stack-opcodes-dupn-swapn-exchange?fyp-video=1&example=dupn +``` + +## Recording + +From `website/`: + +```bash +# ── Final (video + voice) — upload / listen with audio ── +npm run video:generate:preview -- eip-8024 # 540×960 → *-final.mp4 +npm run video:generate -- eip-8024 # 1080×1920 → *-final.mp4 (same layout, 2× upscale) + +# ── Silent intermediate only (no audio) — always 540×960 capture ── +npm run video:record -- eip-8024 --preview --no-voice +npm run video:record -- eip-8024 --dry-run +``` + +Playwright always records at **540×960** (layout reference). Full-res deliverables are +upscaled to 1080×1920 during mux — never a separate 1080px CSS viewport. + +Final: `video/projects/eip-8024/output/--final.mp4` +Intermediate: same stem with `.webm` (Playwright capture, no audio track). + +**Always run `npm run website:build` before recording** if `src/video/` changed (`video:generate` does this automatically). + +Recordings auto-trim lead-in (Playwright captures black/hidden frames before the title +card renders) and verify frame 0 shows the title band for preview thumbnails. Detection +uses center-crop dark-pixel ratio — not brightness alone. Requires `ffmpeg` on PATH. + +Details: [Record and verify](/contributing/video-authoring#record-and-verify). + +## Voice-over (ElevenLabs) + +Narration is synthesized first; character timestamps drive playbook timing; silent video is +muxed with audio at the end. Full workflow: [Voice-over pipeline](/contributing/video-voice). + +```bash +npm run video:voice:synth -- # ElevenLabs → voice/ +npm run video:voice:plan -- # voice-aligned storyboard + +# Final (video + audio): +npm run video:generate:preview -- +npm run video:generate -- + +# Silent webm only: +npm run video:record -- --preview --no-voice +``` + +## Tests + +Unit tests cover: + +- `src/libs/__tests__/exampleFromQuery.spec.ts` — query parsing and resolution +- `src/libs/__tests__/exampleFromQuery.explorations.spec.ts` — all live explorations +- `src/video/__tests__/` — video mode detection and bridge logic +- `video/src/__tests__/` — project loading, playbook duration, lead-in trim + +Playwright recording is **not** run in CI (same policy as OG images). + +## Authoring guide + +[Authoring a video short](/contributing/video-authoring) — how to plan beats, write +overlays/annotations/highlights, set pacing, record, and verify thumbnails. Reference: +`video/projects/eip-8024/`. + +## Adding a new video + +1. Create `video/projects//content.json` and `playbook.json` +2. Add exploration to `video/src/explorationRegistry.ts` if not already listed +3. Run `npm run video:record -- ` + +No website changes required if the exploration type is already supported. diff --git a/docs/contributing/video-voice.md b/docs/contributing/video-voice.md new file mode 100644 index 0000000..f1f7a59 --- /dev/null +++ b/docs/contributing/video-voice.md @@ -0,0 +1,333 @@ +# Voice-over pipeline + +How to add ElevenLabs narration to Feel Your Protocol short-form videos. + +Operational baseline: [Video pipeline](/contributing/video-pipeline), +[Authoring a video short](/contributing/video-authoring). + +--- + +## Core idea + +**Today:** you hand-pick `cue` / `wait` / step `interval` ms in `playbook.json`, record a +silent `.webm`, publish. + +**With voice:** ElevenLabs drives the **clock**. You write spoken copy, the API returns audio +plus **character-level timestamps**. The playbook timing is **derived** (then refined) to +match that audio. Recording stays silent; **ffmpeg muxes** video + voice at the end. + +ElevenLabs does **not** accept our time markers — it gives markers **back**. The workflow +flips: + +``` +OLD: story plan → manual ms timing → record +NEW: story plan → narration script → synthesize → derive ms timing → record → mux +``` + +You still need a **rough visual plan before synthesis** (which beats, which actions, climax). +Voice generation is a **refinement step**, not a substitute for storyboarding. + +--- + +## What we keep + +| Layer | Change | +|-------|--------| +| Title / outro cards | Keep — can be spoken over (band still shows peek + type) | +| Story overlays (`punch`, `split`, …) | Keep copy — may differ from spoken script | +| Annotations | Keep — still guide the eye during action | +| Climax highlights | Keep — timing derived from voice + action budget | +| Playwright recording | Keep — still drives the browser | +| `zones.json`, exploration actions | Keep | + +On-screen text and spoken narration **may diverge** (shorter banners, fuller voice). Link +them by **beat id**, not by identical strings. + +--- + +## Project layout (planned) + +Per project `video/projects//`: + +| File / dir | Role | +|------------|------| +| `playbook.json` | **Structure** — beats, overlay ids, actions, focus, climax fields (intervals optional once voice timing exists) | +| `narration.json` | **Spoken copy** — segments tied to beat ids | +| `content.json` | Unchanged — visual overlay copy, annotations, highlights | +| `voice/` | Generated artifacts (gitignored): `full.mp3`, `alignment.json`, `manifest.json` | +| `output/` | Silent `.webm` + final muxed `.mp4` | + +Credentials: `video/.env` (`ELEVENLABS_API_KEY`, `ELEVENLABS_VOICE_ID`, optional `ELEVENLABS_MODEL_ID=eleven_v3`). + +--- + +## `narration.json` format (proposed) + +One segment per beat that has speech. Beats without an entry stay silent (e.g. pure action +holds) or use `"text": ""`. + +```json +{ + "model": "eleven_v3", + "segments": [ + { + "beat": "hook", + "text": "EIP-8024. Three new stack opcodes. Glamsterdam is coming." + }, + { + "beat": "context", + "text": "Solidity developers know this error — stack too deep." + }, + { + "beat": "opcodes", + "text": "DUPN, SWAPN, and EXCHANGE let the EVM reach deeper into the stack." + }, + { + "beat": "stack-grow", + "text": "Watch the stack grow — seventeen values pushed." + }, + { + "beat": "dupn-hit", + "text": "Now DUPN copies from depth seventeen. No compiler gymnastics." + }, + { + "beat": "recap", + "text": "Classic DUP stops at depth sixteen. You just copied depth seventeen." + }, + { + "beat": "outro", + "text": "Read the EIP on Forkcast. Explore it on Feel Your Protocol." + } + ] +} +``` + +**Segment strategy:** synthesize **one segment per beat** (multiple API calls), then +concatenate with a short crossfade or silence gap (~200–400 ms). This gives clean +`startMs` / `endMs` per beat without parsing a monologue. Alternative: one full script + +substring alignment lookup (see below) — better prosody, harder mapping. + +--- + +## ElevenLabs API artifacts + +Endpoint (already validated): + +`POST /v1/text-to-speech/{voice_id}/with-timestamps` + +Response (per call): + +```json +{ + "audio_base64": "…", + "alignment": { + "characters": ["E", "I", "P", …], + "character_start_times_seconds": [0.0, 0.04, …], + "character_end_times_seconds": [0.04, 0.09, …] + } +} +``` + +From alignment we can: + +1. **Duration** — last `character_end_times_seconds` × 1000. +2. **Substring timing** — find beat text in the character stream → `startMs`, `endMs`. +3. **Word boundaries** — group characters between spaces for caption-style overlays (v2+). + +Store per segment in `voice/manifest.json`: + +```json +{ + "generatedAt": "2026-07-04T…", + "audioFile": "voice/full.mp3", + "totalDurationMs": 47200, + "beats": { + "context": { "startMs": 5200, "endMs": 9100, "segmentFile": "voice/seg-context.mp3" }, + "opcodes": { "startMs": 9300, "endMs": 14200 } + } +} +``` + +--- + +## Deriving playbook timing from voice + +For each playbook step with a narration segment: + +| Playbook field | Derivation | +|----------------|------------| +| `cue` | `beat.endMs - beat.startMs` while overlay is visible (narrator reads banner copy or parallel narration) | +| Overlay hide | When beat speech ends (or slightly before last word if action should start on final syllable) | +| `step.interval` | Fit N steps into `(nextBeat.startMs - actionStartMs - holdBudget)` | +| Climax `climaxInterval` | Stretch to cover spoken emphasis on dupn-hit beat | +| `wait` | `nextBeat.startMs - currentBeatEndMs - transitionMs`, floored at ~400 ms | + +**Action beats** (stack-grow, dupn-hit): speech often plays **during** overlay (`cue`), then +overlay hides and steps run **while the next segment hasn't started yet**. Budget step +intervals from the **gap** between segment end and next segment start, minus highlight holds. + +**Text-only beats** (recap): `wait` = segment duration (no `cue` split). + +**Title / outro:** treat like overlay-only beats — speech over the band; `wait` = voice +duration (+ optional 300 ms tail). + +### Merge strategy + +Do **not** hand-edit derived ms in `playbook.json`. Use a generated overlay: + +- `playbook.json` — structure only (or legacy manual timing as fallback). +- `voice/manifest.json` — authoritative timestamps from ElevenLabs. +- `playbookRunner` loads **merged** step timing: `structure ⊕ voice manifest`. + +If voice files are missing, fall back to manual `cue`/`wait` (silent video mode). + +--- + +## Pipeline commands (planned) + +From `website/`: + +```bash +# One-shot final (video + voice) — open the *-final.mp4, not the .webm +npm run video:generate:preview -- eip-8024 +npm run video:generate -- eip-8024 +npm run video:generate -- eip-8024 --skip-synth --preview # reuse voice/, re-record + mux + +# Silent webm only (visual iteration, no audio): +npm run video:record -- eip-8024 --preview --no-voice + +# Or step-by-step: +npm run video:voice:synth -- eip-8024 +npm run video:voice:plan -- eip-8024 +npm run website:build +npm run video:record -- eip-8024 --preview +npm run video:voice:mux -- eip-8024 +``` + +Skip synth/plan to record with manual timing (no `voice/manifest.json`). + +Implementation modules (under `video/src/voice/`): + +| Module | Responsibility | +|--------|----------------| +| `loadEnv.ts` | Load `video/.env` | +| `elevenlabs.ts` | API client (`with-timestamps`) | +| `synthesize.ts` | Segment loop, write mp3 + raw alignments | +| `concatAudio.ts` | ffmpeg concat segments → `full.mp3` | +| `alignBeats.ts` | Build `manifest.json` beat → start/end ms | +| `mergeTiming.ts` | Apply manifest to playbook steps | +| `mux.ts` | ffmpeg: copy video, encode AAC, `-shortest` | + +--- + +## Lead-in trim and audio sync + +Recording trims black/hidden lead-in from the `.webm` (see `trimLeadIn.ts`). Audio must +start at the **same** timeline origin as trimmed video. + +Options (pick one in implementation): + +1. **Anchor on title beat** — narration segments exclude pre-title silence; manifest + `startMs` is relative to first title frame (after trim). Simplest. +2. **Prepend silence** — mux step adds `trimOffsetMs` silence to audio to match ffmpeg trim. +3. **Record trim offset** — `recordVideo` writes `output/trim-meta.json`; mux reads it. + +Recommend **(1)** for v1: first segment is `hook` title speech; video timeline zero = title +card visible; no voice before that. + +--- + +## Procedural workflow (agent + human) + +Use this checklist for each new exploration short. + +### Phase A — Story (mostly unchanged) + +1. Define beats in `playbook.json` — overlay ids, actions, focus, climax fields. +2. Write visual copy in `content.json` (overlays, annotations, highlightSets). +3. Define `zones.json` focus areas. +4. Run `npm run video:storyboard -- ` — validate structure **without** voice. + +Rough target length: ~45–60 s total voice (YouTube Shorts). + +### Phase B — Narration script + +1. Add `narration.json` — one segment per spoken beat. +2. Write for **ear**, not screen: conversational, present tense, short sentences. +3. Eleven v3: optional inline tags (`[excited]`, `[slowly]`) sparingly on climax beat. +4. Read aloud yourself once — aim ~130–150 wpm equivalent. + +### Phase C — Synthesize + +1. Ensure `video/.env` is set. +2. `npm run video:voice:synth -- ` +3. Listen to `voice/full.mp3` — re-run after script edits (cache segments by text hash to save credits). + +### Phase D — Timing plan + +1. `npm run video:voice:plan -- ` — prints voice-aligned storyboard: + ``` + 00:00 hook voice 4.2s (overlay) + 00:04 context voice 3.8s → hide → wait 0.6s + 00:08 opcodes voice 5.1s → selectExample + … + ``` +2. Refine: if stack-grow speech is longer than 9 steps can fill, either shorten script or + slow step interval / reduce step count. +3. Climax beat: ensure `climaxInterval` + highlight holds fit inside segment + gap. + +### Phase E — Record + verify + +1. `npm run website:build && npm run video:record -- --preview` +2. Scrub silent video against `voice/full.mp3` in a player (manual sync check). +3. Fix timing via narration edits or `voice/plan overrides` (optional small manual offsets in manifest). + +### Phase F — Mux + publish + +1. `npm run video:voice:mux -- ` → `output/--final.mp4` +2. Verify frame 0 thumbnail (title band) and audio start. +3. Full-res record + mux when satisfied. + +--- + +## Implementation order (suggested) + +| Step | Deliverable | +|------|-------------| +| 1 | `narration.json` schema + EIP-8024 sample script | +| 2 | `voice/synthesize` CLI + segment mp3 + alignment JSON | +| 3 | `concatAudio` + `manifest.json` beat timestamps | +| 4 | `mergeTiming` integrated into `playbookRunner` / storyboard | +| 5 | Record eip-8024 preview with voice-derived timing | +| 6 | `mux` CLI + docs update in video-pipeline.md | +| 7 | Optional: word-level captions from alignment (v2) | + +--- + +## Testing + +- Unit tests: substring → ms lookup, merge timing math, manifest validation against playbook beat ids. +- No ElevenLabs calls in CI — fixture alignment JSON from real API response. +- Manual: preview record + mux for one project before merging pipeline defaults. + +--- + +## Agent instructions + +When the user asks to add voice to a video project: + +1. Read this doc + existing `playbook.json` / `content.json`. +2. Draft `narration.json` segments for beats that should be spoken. +3. Run synth → plan → storyboard; report total duration and per-beat ms. +4. Adjust narration length if derived step intervals fall below ~280 ms or total exceeds ~60 s. +5. Record preview, mux, ask user to listen on phone. +6. Do **not** commit `video/.env` or `voice/` artifacts. + +--- + +## Related + +- [Video pipeline](/contributing/video-pipeline) — recording, overlays, trim +- [Authoring a video short](/contributing/video-authoring) — beats, pacing, climax +- `video/src/loadEnv.ts` — env loading for ElevenLabs +- `video/.env.example` — credential template diff --git a/docs/guide/architecture.md b/docs/guide/architecture.md index 1071bc9..59111df 100644 --- a/docs/guide/architecture.md +++ b/docs/guide/architecture.md @@ -145,3 +145,15 @@ The project uses a hybrid testing approach: - **E2E tests** (Cypress) as lean smoke tests for critical navigation flows and page-level integration Unit tests live alongside their components in `__tests__/` folders. E2E tests are consolidated in `cypress/e2e/`. + +## Video pipeline (short-form) + +Vertical exploration videos for YouTube Shorts, Reels, and X are recorded via an +isolated Playwright package in `video/` (same pattern as `og/` for social preview images). + +- **Website overlay shell** — `src/video/`, activated with `?fyp-video=1` +- **Content & playbooks** — `video/projects//` (JSON, separate from the site) +- **Deep-linking** — `?example=` pre-selects example presets on all explorations + +See [Video pipeline](/contributing/video-pipeline) for the workflow and +[Authoring a video short](/contributing/video-authoring) for how to write a project. diff --git a/docs/public/llms.txt b/docs/public/llms.txt index 1dc1b67..6624c37 100644 --- a/docs/public/llms.txt +++ b/docs/public/llms.txt @@ -28,6 +28,8 @@ at contributors and developers who want to understand or extend the project. - [Styling & Design](https://docs.feelyourprotocol.org/contributing/styling.html): The exploration design system and class conventions - [Code Conventions](https://docs.feelyourprotocol.org/contributing/code-conventions.html): Coding standards and test commands - [Third-Party Libraries](https://docs.feelyourprotocol.org/contributing/third-party-libraries.html): How new library dependencies are evaluated and added +- [Video Pipeline](https://docs.feelyourprotocol.org/contributing/video-pipeline.html): Short-form exploration videos — architecture, overlays, recording +- [Authoring a Video Short](https://docs.feelyourprotocol.org/contributing/video-authoring.html): How to plan, write, and record exploration shorts ## Related diff --git a/docs/special-actions/ice-cream-week.md b/docs/special-actions/ice-cream-week.md index a259537..f54ef98 100644 --- a/docs/special-actions/ice-cream-week.md +++ b/docs/special-actions/ice-cream-week.md @@ -2,8 +2,8 @@ Temporary home-page gag (see [Special Actions overview](/special-actions/)): a meme ice cream stand where visitors buy a scoop with **$FYP on Base** and receive a **soulbound NFT** receipt. -::: warning Summer gag, real on-chain flow -Playful sandbox — not financial advice, not a product roadmap. **IceCreamStand** is deployed on Base; wallet integration on the home page is the remaining step. Code lives in `src/ice-cream/` and stays in the repo after the stand comes down; only the home integration is removed. +::: info Archived on home (July 2026) +The stand is **off the home page**. **IceCreamStand** remains on Base; `public/ice-cream/` metadata URLs stay live for soulbound receipts. Code lives in `src/ice-cream/` for reference and possible reuse. ::: ## Scope @@ -46,10 +46,10 @@ src/ice-cream/ ├── purchasePort.ts # IceCreamPurchasePort — viem later ├── memes.ts # vendor + NFT image pairs ├── assets/ # vendor WebP + nft/ PNG masters -└── __tests__/ # remove folder with feature if desired +└── __tests__/ # module + IceCreamHomeSection.spec.ts (keep when archived) ``` -**Home integration (single point):** +**Home integration (single point — currently removed):** ```vue import { IceCreamHomeSection } from '@/ice-cream' @@ -57,7 +57,9 @@ import { IceCreamHomeSection } from '@/ice-cream' ``` -To end Ice Cream Week: delete the import and `` line in `HomeView.vue`. Optionally remove `describe('Ice Cream Week')` from `HomeView.spec.ts`. +To **activate:** add the import and `` in `HomeView.vue` (above Latest). + +To **deactivate:** remove those two lines only. Keep `src/ice-cream/` and its tests. Do **not** add special-action assertions to `HomeView.spec.ts` — home placement is covered by `IceCreamHomeSection.spec.ts`. See [Special Actions lifecycle](/special-actions/#lifecycle). ## Purchase flow (phases) diff --git a/docs/special-actions/index.md b/docs/special-actions/index.md index acf0726..47bfcbb 100644 --- a/docs/special-actions/index.md +++ b/docs/special-actions/index.md @@ -6,8 +6,17 @@ Short-lived or playful features on the main site — summer gags, one-off events These pages document how we built and operate specific actions. Add a new page under `docs/special-actions/` and link it here when another action ships. ::: +## Lifecycle + +| Phase | Home page | Tests | +| ----- | --------- | ----- | +| **Active** | Import the action’s `*HomeSection` in `HomeView.vue` | Module tests in `src//__tests__/` (always) | +| **Archived** | Remove import + component from `HomeView.vue` | Same module tests — **do not** delete; **do not** add special-action blocks to `HomeView.spec.ts` | + +**Test convention:** `HomeView.spec.ts` covers permanent home content only. Each special action keeps its own tests under `src//__tests__/`, including a `*HomeSection.spec.ts` for the home section wrapper. When you deactivate, remove home wiring only — the module test suite stays green and documents how to re-enable. + ## Active & archived | Action | Status | Doc | | ------ | ------ | --- | -| Ice Cream Week | Active (home page) | [Ice Cream Week](./ice-cream-week) | +| Ice Cream Week | Archived (home removed July 2026; contract + metadata live) | [Ice Cream Week](./ice-cream-week) | diff --git a/package.json b/package.json index a72cfec..263099f 100644 --- a/package.json +++ b/package.json @@ -33,6 +33,17 @@ "social:preview": "vite preview --config vite.social.config.ts", "capture:social": "run-s social:build capture:social:run", "capture:social:run": "npm run capture:social:run --prefix og --", + "video:setup": "npm run setup --prefix video", + "video:check": "npm run check --prefix video", + "video:preflight": "npm run preflight --prefix video", + "video:record": "npm run record --prefix video --", + "video:storyboard": "npm run storyboard --prefix video --", + "video:voice:synth": "npm run voice:synth --prefix video --", + "video:voice:plan": "npm run voice:plan --prefix video --", + "video:voice:mux": "npm run voice:mux --prefix video --", + "video:generate": "npm run website:build && npm run generate --prefix video --", + "video:generate:preview": "npm run website:build && npm run generate --prefix video -- --preview", + "video:render": "npm run video:generate --", "website:preview": "vite preview --outDir dist/website", "docs:dev": "vitepress dev docs", "docs:build": "vitepress build docs", diff --git a/src/App.vue b/src/App.vue index d858151..8c95732 100644 --- a/src/App.vue +++ b/src/App.vue @@ -1,5 +1,5 @@ diff --git a/src/components/CompanionSheet.vue b/src/components/CompanionSheet.vue index 0510547..ed8593f 100644 --- a/src/components/CompanionSheet.vue +++ b/src/components/CompanionSheet.vue @@ -1,10 +1,22 @@