Skip to content

ARCHITECTURE.md

Latest commit

 

History

History
82 lines (66 loc) · 4.49 KB

ARCHITECTURE.md

File metadata and controls

82 lines (66 loc) · 4.49 KB
Raw

Architecture Overview

This document describes the architectural principles, execution model, and core invariants of Interactive Reader.

For active project state, milestone history, and test verification baselines, see HANDOFF.md.

1. System Overview

Interactive Reader turns plain text and Markdown into progressively generated, interactive audiobooks with word-level synchronization and character voice assignments.

Markdown / Text Document
           │
           ▼
   [ Document Parser ] ────► Canonical Semantic Tokens (0-indexed)
           │
           ▼
[ SpeechPlanningPipeline ] ─► SpeechPlan (Spans, Voices, Pauses)
           │
           ▼
  [ SynthesisScheduler ] ───► Kokoro TTS Engine (Inference Lock)
           │
           ▼
    [ ReaderBundle ] ──────► chapter.html + chapter.wav + chapter.ireader/chunks

2. Core Architectural Principles

1. Canonical Document & Token Identity

  • Markdown and text are parsed into an immutable Document consisting of semantic Token instances.
  • Every word and punctuation mark has a stable 0-indexed integer ID.
  • The rendered reader HTML maps 1-to-1 to these tokens via <span data-token="<i>">, guaranteeing that audio timestamps, highlighting, and user click-to-seek always reference unambiguous token boundaries.

2. Pure & Deterministic Speech Planning

  • The SpeechPlanningPipeline transforms the canonical token stream into a sequence of SpeechSpan objects with assigned voices, rate adjustments, and boundary pauses.
  • Canonical Transform Order: 
    DefaultSpeechPlanner
  → DialogueTransform
  → TechnicalReadingTransform
  → FootnoteTransform
  → PronunciationTransform
  • Pure Function Boundary: Speech planners and transforms are pure and deterministic. They never perform filesystem I/O or read environment variables. All global and sidecar configuration is resolved at the application boundary (cli.py, studio.py, window.py) and passed explicitly.

3. Deterministic Dialogue Attribution

  • Core dialogue attribution is performed using rule-based and conversational context matching:
    • Direct speech verbs (said, asked, replied).
    • Conversational turn continuity and alternating speaker exchanges.
    • Character proximity and unquoted speaker tags.
    • 1-token boundary tolerance for dialogue punctuation.
  • Zero Network / Non-Deterministic Dependencies: Core dialogue attribution requires no LLM calls and no internet access.

4. ReaderBundle & Zero-Server Playback

  • Output is packaged as a ReaderBundle:
    • chapter.html — Interactive reader shell.
    • chapter.wav — Assembled full audio file (when synthesis finishes).
    • chapter.ireader/chunks/ — Chunk audio (.wav) and alignment metadata (.js).
    • chapter.ireader/state.json — Generation progress and unit hashes.
  • Static file:/// Compatible: The reader runs completely offline directly from the local filesystem without requiring a background HTTP daemon or localhost server.

5. Adaptive Priority Scheduling

  • The background SynthesisScheduler generates audio units sequentially.
  • When a user clicks ahead to read an ungenerated section, the scheduler dynamically reprioritizes that unit to the front of the queue, minimizing playback latency.

6. Prefix-Preserving Semantic Invalidation

  • Each synthesis unit carries a deterministic SHA-256 hash of its input text, voice parameters, and pronunciation rules.
  • When narration settings change (e.g., in the Desktop Studio), staleness inspection identifies the first_mismatch unit index.
  • Only units from first_mismatch onward are invalidated and queued for regeneration; preceding valid units are preserved without re-synthesis.

7. Save vs. Apply / Regenerate Lifecycle

  • Save: Edits in the Desktop Studio persist immediately to disk sidecars (chapter.ireader.json or global files).
  • Regenerate: Audio synthesis is decoupled from saving and runs on demand via the GenerationController. Users can continue reading while background generation proceeds.

8. Synchronous Architecture & Concurrency

  • The core synthesis pipeline is synchronous and thread-safe.
  • A centralized INFERENCE_LOCK coordinates model access, allowing ephemeral audio previews in the Studio to safely share the neural TTS model with active background synthesis threads.
  • No asyncio event loops are used in core processing, avoiding thread-loop conflicts with desktop window wrappers (pywebview).