Skip to content

HANDOFF.md

Latest commit

 

History

History
143 lines (120 loc) · 9.19 KB

HANDOFF.md

File metadata and controls

143 lines (120 loc) · 9.19 KB
Raw

Interactive Reader — Authoritative Project Handoff & Save-State

Status: Active Project Save-State (Supersedes all previous handoff documents)
Current Authoritative Milestone:
Hardware Acceleration, Local Server & Browser Viewer = FROZEN / VERIFIED = 401 passed, 31 warnings repo-wide

1. Authoritative Milestones Summary

Milestone Status Verified Test Count Notes
Incremental Streaming & Unit-Level Resume v2 FROZEN / VERIFIED 100% green Dynamic unit scheduling, prefix hashing, static file:/// playback
Character & Pronunciation Studio v1 FROZEN / VERIFIED 100% green Pywebview desktop bridge, ephemeral audio preview, sidecar config persistence
Dialogue Attribution Hardening v2.1 FROZEN / VERIFIED 100% green High-precision attribution (direct speech verbs, conversational context, unquoted speaker tags, 1-token boundary tolerance)
Dialogue Review Workflow v1 FROZEN / VERIFIED 100% green Review card queue, filter buckets, explicit speaker overrides, live reader span focus
Explicit "Not Dialogue" v1 FROZEN / VERIFIED 143 passed in focused suites Dedicated not_dialogue token ranges, suppression in speech planning, instant review card toggle
Global Pronunciation Lexicon v1 FROZEN / VERIFIED 351 passed repo-wide Cross-book user lexicon at ~/.interactive_reader/lexicon.json, atomic writes, pure speech pipeline boundary
Reusable Character Roster v1 FROZEN / VERIFIED 374 passed repo-wide Reusable character voice library at ~/.interactive_reader/characters.json, detached snapshot materialization
Hardware Acceleration & Reader Server v1 FROZEN / VERIFIED 401 passed repo-wide GPU auto-detection (Arc/CUDA/MPS), CPU intra-op thread tuning, --serve HTTP host & --browser launcher

2. Reusable Character Roster Architecture (v1)

  • Storage Location & Envelope:
    • Persisted at ~/.interactive_reader/characters.json (customizable programmatically via get_global_characters_path(custom_path)).
    • Envelope schema: {"version": 1, "scope": "global", "profiles": [...]}.
  • Strict Validation & Corruption Protection:
    • Strict JSON validation rejecting missing/incorrect versions, wrong scopes, and duplicate character_ids without silent normalization or corruption.
    • If the global file is malformed on disk, global edits are cleanly disabled in the Studio UI with a warning notice while book-local editing continues uninterrupted.
  • Atomic Writes & Concurrency Protection:
    • Atomic writes via temporary files (NamedTemporaryFile + replace).
    • SHA-256 lost-update protection with safe non-conflicting multi-window merge: distinct character IDs merged cleanly, concurrent conflicting edits on the same character_id rejected.
  • Canonical Model Reuse:
    • Fully reuses canonical CharacterVoiceProfile fields: character_id, display_name, voice (named single voice or weighted blend tuple), aliases, pause_before, pause_after.
    • No synthetic speed or non-canonical fields added.
  • Dormant by Default (GLOBAL ROSTER != ACTIVE CHARACTER SET):
    • Characters existing in the global roster do NOT automatically participate in dialogue attribution for open documents.
    • Unrelated characters in different books with the same display name (e.g., "John") never accidentally receive global voices unless explicitly assigned.
  • Explicit Materialization ("Use in This Book"):
    • Clicking "Use in This Book" copies an explicit detached snapshot into NarrationConfig.dialogue.profiles and saves the local sidecar (chapter.ireader.json).
    • Subsequent modifications to the global character in the roster do NOT mutate existing book sidecars or cause spurious audio staleness.
    • Local materialized profiles remain fully editable within each book.
  • ID Collision UX:
    • Explicit confirmation prompts prevent accidental overwrites when adding to global roster or materializing to a book with an existing matching character_id.
  • Dialogue Review Integration:
    • Review speaker select dropdown groups choices into This Book (Active) and Global Roster (Click to Activate).
    • Selecting a global character automatically materializes it into the active book before setting the SpeakerOverride.

3. Global Pronunciation Lexicon Architecture (v1)

  • Storage Location & Envelope:
    • Persisted at ~/.interactive_reader/lexicon.json ({"version": 1, "scope": "global", "rules": [...]}).
  • Strict Precedence Hierarchy: 
    Book Rule (sidecar) > Legacy Embedded Global Rule (sidecar) > User Global Rule (~/.interactive_reader/lexicon.json) > Baseline G2P (Misaki)
  • Pure Speech Planning Boundary:
    • build_speech_pipeline(config, effective_lexicon=None) remains pure and deterministic.
    • Speech planners do not read environment variables or perform filesystem lookups.
    • Global lexicon is loaded at the application boundary (cli.py, studio.py, window.py).
  • Contextual Masking UI:
    • Book overrides display [Book Override] with (Masking Global: /.../).
    • Removing a book override restores and unmasks the underlying [Global] rule cleanly.

4. Frozen Architectural Invariants (Do Not Reopen)

  1. Canonical Document & Token Model:
    • Word and punctuation tokens are immutable semantic tokens with stable 0-indexed integer IDs.
    • Text rendering in reader HTML maps 1-to-1 to tokens via span[data-token="<i>"].
  2. Pure SpeechPlan & SpeechSpan:
    • Text transformations, pause insertion, and voice assignments produce deterministic SpeechSpans consumed by the generation controller.
  3. Dialogue Attribution Hardening (v2.1):
    • High-precision name/alias matching using direct speech verbs, conversational turn continuity, unquoted attribution tags, and 1-token boundary tolerance.
    • No network calls or non-deterministic LLMs in core attribution.
  4. Explicit "Not Dialogue":
    • Negative constraint ranges (not_dialogue) are stored in DialogueConfig and suppress dialogue attribution during planning.
  5. Dialogue Review Workflow:
    • Interactive review card with queue filter buckets (UNKNOWN, ALL, CONTEXTUAL, DIRECT, OVERRIDDEN, NOT_DIALOGUE).
    • Clean excerpt display without doubled quotation marks.
  6. Save vs Apply/Regenerate:
    • Configuration edits persist immediately to disk sidecars.
    • Speech synthesis regeneration is on-demand and non-blocking via GenerationController.
  7. Prefix-Preserving Semantic Audio Invalidation:
    • Audio unit hashes determine output-based staleness.
    • Changes invalidate only from first_mismatch onward (first_mismatch .. total_units-1), preserving valid leading audio units.
  8. Adaptive Priority Scheduling:
    • Foreground user playback requests take priority over background sequential synthesis.
  9. Desktop App Architecture:
    • Local-first native pywebview app (file:/// URI compatible).
    • No localhost HTTP servers, no background daemons, no asyncio loop conflicts (threaded SynthesisScheduler).

5. Current Product Direction & Progress

Vision:
Interactive Reader is evolving into a local-first, deterministic personal audiobook studio where:

  • Source markdown documents remain canonical.
  • Narration knowledge (pronunciations and character voice identities) is globally reusable across chapters and books.
  • Corrections and customizations remain explicit, transparent, and inspectable.

Recent Progress:

  • We can now reuse pronunciation rules globally across all books without duplicating sidecar entries.
  • We can now maintain a reusable library of character voice profiles and materialize them into any book with a single click.
  • Repetitive per-chapter setup friction is substantially reduced without introducing risky heuristic cross-book identity matching.

6. Known Limitations & Future Evidence to Watch

  1. Technical / Non-Character Quotation False Positives:
    • Non-dialogue quotes (e.g. code terms, scare quotes, technical excerpts) require manual marking via "Not Dialogue".
  2. Book / Multi-Chapter Scope Needs:
    • Observe real multi-chapter reading sessions to evaluate whether explicit per-chapter materialization becomes repetitive enough to justify a formal multi-chapter Project/Book scope.
  3. Cold-Start & Model Loading:
    • Kokoro ONNX model initial load time on CPU/GPU.
  4. Expressive Narration & Style Controls:
    • Character speed / pitch modulation and expressive tone controls.
  5. Studio UX Ergonomics:
    • Real-world ergonomic feedback from reading continuous long-form literature.

7. Resume Instructions for Future Sessions

============================================================
RESUME INSTRUCTIONS FOR NEXT SESSION
============================================================
1. Read this HANDOFF.md document.
2. Check git status to verify clean working tree.
3. Do NOT reopen or refactor frozen architecture.
4. Use the reader on real multi-chapter / book material.
5. Identify the single highest-friction product problem from actual use evidence.
6. Formulate and propose one narrow next feature plan before implementing it.
============================================================