feat(epub): Japanese CJK line-breaking PR F — Utf8ClusterAssembler + viewport-width CJK wrapping

[mtskf]

Summary

Final PR (PR F) of the Japanese CJK line-breaking effort. This is where behaviour changes: Japanese (CJK) paragraphs now wrap naturally at the viewport edge instead of breaking every ~66 characters. English / Latin / BiDi / hyphenation layout is unchanged.

Builds on PR D (ITextMetrics scaffolding + host characterization tests) and PR E (WordJoin enum + Utf8 predicates + parser bridge), both already on master.

Root cause

The parser flushed accumulated text into a "word" every MAX_WORD_SIZE (200 bytes ≈ 66 Japanese chars). CJK has no spaces, so each 200-byte chunk became a layout word and the line broke after each one — leaving short ragged lines mid-paragraph. The real defect: the layout's join flag could only express "space + breakable" and "no-space + non-breakable", but never the "no-space + breakable" state CJK needs (you may break between any two ideographs, with no space). PR E generalised the flag to the three-state WordJoin; this PR makes CJK characters first-class breakable units.

Commits

• feat(parser): add Utf8ClusterAssembler with grapheme cluster glue — new host-safe state machine (lib/Epub/Epub/parsers/Utf8ClusterAssembler.{h,cpp}) owning UTF-8 decode + grapheme-cluster assembly + WordJoin policy. static methods (external linkage so host gtest links directly), fixed buffers (no heap), no Arduino/HAL/expat. Handles cross-callback carry-over (pendingUtf8), base+extender clustering (pendingCjkBase), stage-time style snapshot, NUL-safe decode. + 12-case host suite (test/parser_cluster/).
• feat(parser): emit CJK as breakable words via Utf8ClusterAssembler — integrates the assembler into ChapterHtmlSlimParser: clusterState member, flushPendingText()/FlushedKind, currentFontStyle(), the 13 boundary flush sites + nested-table OPEN/DROP symmetric pair + startNewTextBlock drainer, and the 750-token CJK ceiling. + 2 layout-proxy tests pinning the Latin→Glue / CJK→CjkBreak inline-close policy.
• chore: bump SECTION_FILE_VERSION + full verification — 26 → 27 so stale .crosspoint/sections/*.bin regenerate (layout output changed; binary structure did not). docs/file-formats.md synced to v27.

Design notes

• English layout is byte-identical. currentFontStyle() is a verbatim extraction of the old flushPartWordBuffer style logic (including underline); the Latin emit path is unchanged. Only CJK produces the new CjkBreak state.
• Inline-style boundaries respect token kind. 日<em>本</em>語 stays wrappable (CJK keeps CjkBreak); quick<em>ly</em> stays glued (Latin keeps Glue). Driven by flushPendingText() returning FlushedKind.
• Stage-time style snapshot. A CJK base staged inside <u> flushes with underline even after </u> has closed.
• Memory (ESP32-C3, ~380KB RAM, no PSRAM). CJK becomes one std::string token per character; the 750-token ceiling (run after each CJK emit) bounds the transient layout vector, freed per page.

Out of scope (deliberately deferred)

Kinsoku (禁則処理), vertical text, BiDi+CJK reorder fallback, cluster-aware rendering of extenders in GfxRenderer (extenders rasterise as normal-advance glyphs; layout keeps them on the same line), and parser-shell host-side test coverage (the parser TU pulls expat/HAL — not host-compilable). See the plan for rationale.

Verification

• Host gtest suite green: ParserClusterTest 12/12, ParsedTextLayoutTest 14/14 (incl. 2 new policy-proxy tests), full tree 132/132, 0 warnings under -Wall -Wextra -pedantic.
• Firmware build is CI-gated (no PlatformIO on the dev machine).
• 🔲 Device QA (human): confirm the 訳者序 paragraph wraps to the page edge with no mid-paragraph short lines; delete .crosspoint/ first (also forced by the v27 bump).

🤖 Generated with Claude Code