diff --git a/CHANGELOG.md b/CHANGELOG.md index 7b7bdb86..31a2ac22 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,78 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [0.14.1] - 2026-08-18 + +> Patch: everyday-stable 0.14.x. Same slim public SKU as 0.14.0, plus the two +> Settings/auth probes that were still lying on a daily machine, and one +> command that installs the notarized .app instead of re-signing it. + +### Fixed + +- **STT Test is a file probe.** Settings Test no longer POSTs to a live + Voice Lab WebSocket (`STT_ENDPOINT`). Known live sockets map to + `/v1/audio/transcriptions`; loopback `:8446` → `:8444`. +- **ChatGPT sign-in no longer requires Responses write.** OAuth persists + identity after exchange. `api.responses.write` stays a lane Test, so + Codex public tokens can sign in. +- **STT remapper test names loopback explicitly** so `make check` Semgrep + does not treat a templated live-socket URL as an open WebSocket. +- **Overlay copy stays evidence-only in the quality-chain test.** Isolated + `make verify` no longer greened that path by reading a leftover host + lexicon. Voice Lab finalize is still the teach gesture. + +### Changed + +- **`make release-stable`** is the everyday cut: slim sign + notarize + + `verify-dmg`, then install that stapled Developer ID `.app` to + `/Applications` without re-signing. `make install-app` remains the + local-release / Lab path. +- **`make release-full` is fail-closed.** Whisper embed no longer falls + back to a slim dylib when the HF snapshot is weights-only. It uses the + composed `~/.codescribe/models` tree from `make download-model`. +- `SITE_VERSION` stays `0.13.3` until a published GitHub release. + +## [0.14.0] - 2026-08-17 + +> Minor: developer Lab surface, Dictionary helper file-pass, bus word pins, +> and a 30-minute Whisper idle. Production DMG still has no Lab menu. + +### Added + +- **Developer Lab on keyed `make install-app`.** `CSDeveloperSurface` bakes + only when Sparkle + production license public keys resolve on the machine. + Public `git clone && make` stays Lab-off. Production DMG refuses the bit. +- **Lab mode overlay-off.** Developer veto hides the daily HUD without + flipping the tray "Transcription Overlay" toggle. Leftover UserDefaults + cannot hide overlay on a production bundle. +- **Dictionary helper Retranscribe.** Follows Settings `asr_mode`: + `local_power` → `hq:` candle file pass, `cloud` → `cloud:` file pass, + `apple_only` disabled. Missing archive refuses — never `last_session.wav`. +- **Voice Lab on the website** (`/voice/lab`). Teacher + Seal Atlas as a + Codescribe module, not a sidecar: same `teach()` triangle as + `codescribe-teacher`, idle until Run; Atlas HTML loads only on demand. +- **Overlay stays the canvas.** Assistive hold/toggle stays on the live + overlay (composer mic is the only Agent-owned capture). Action row whispers + at rest. Retranscribe runs Full HQ / Cloud on `last_session.wav`. Forest + glass drinks the desktop; the panel stays non-key until you click FINAL. + +### Changed + +- **Whisper idle is 30 minutes after the last finished decode**, not 60 + seconds from load. The running process only picks this up after + `install-app` + relaunch. +- **Settings matches the live STT contract.** Dictation owns the ASR mode + picker (`apple_only` / `local_power` / `cloud`) and writes + `CODESCRIBE_ASR_MODE` plus explicit Cloud consent. Final pass is no longer + an engine control. Active STT is the last serving take (`local_apple` → + Apple). `STT_ENDPOINT` is the live WSS socket on Dictation. Retranscribe + toasts the real error, including a missing `last_session.wav`. +- **Quality HTML is Seal Atlas.** `codescribe-corpus` writes + `quality/seal-atlas.{profile}.html` as the report (handshake in + `docs/quality-reports/CONTRACT.md`). Qube scores move to + `quality/qube.{profile}.html` and stay a footnote. Gold take 01 remains + `docs/quality-reports/seal-atlas.take01.html`. + ## [0.13.3] - 2026-08-13 > The agent-stability and STT-truth-layer wave: one dictation pipeline with an @@ -51,7 +123,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed - **Whisper residency is bounded and observable** — the normal idle-weight TTL - is now 300 seconds (five minutes), while `CODESCRIBE_WHISPER_IDLE_UNLOAD_SECS=0` + is now 60 seconds (one minute), while `CODESCRIBE_WHISPER_IDLE_UNLOAD_SECS=0` remains the explicit power-user keep-warm override. INFO lifecycle events now expose the effective TTL plus load/unload/reclaim counts and durations without logging audio or transcript content. Host `vmmap` reclaim remains a release diff --git a/Cargo.lock b/Cargo.lock index a0a60ab4..80406866 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -729,7 +729,7 @@ dependencies = [ [[package]] name = "codescribe" -version = "0.13.3" +version = "0.14.1" dependencies = [ "anyhow", "arboard", @@ -772,7 +772,7 @@ dependencies = [ [[package]] name = "codescribe-core" -version = "0.13.3" +version = "0.14.1" dependencies = [ "anyhow", "async-trait", @@ -822,7 +822,7 @@ dependencies = [ [[package]] name = "codescribe-ffi" -version = "0.13.3" +version = "0.14.1" dependencies = [ "anyhow", "async-trait", diff --git a/Cargo.toml b/Cargo.toml index 83630eda..4c6f6991 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -3,7 +3,7 @@ resolver = "2" members = [".", "core", "bridge"] [workspace.package] -version = "0.13.3" +version = "0.14.1" edition = "2024" authors = ["Vetcoders "] @@ -18,7 +18,7 @@ tracing = "0.1" [package] name = "codescribe" -version = "0.13.3" +version = "0.14.1" edition = "2024" description = "Speech-to-text for macOS — SwiftUI front-end over a Rust engine (UniFFI bridge)" authors = ["Vetcoders "] diff --git a/Makefile b/Makefile index 65fb97a7..86ce34e8 100644 --- a/Makefile +++ b/Makefile @@ -11,7 +11,7 @@ test-engine test-engine-apple test-engine-candle test-teacher \ demo demo-raw demo-assistive check verify semgrep fix clean help corpus-census test-corpus-parity \ dist-preflight dist-preflight-signed verify-canaries smoke-canaries \ - dmg dmg-signed release-standard release-full release-dmgs notarize verify-dmg download-model download-e5 download-embedder ensure-models \ + dmg dmg-signed release-standard release-full release-dmgs release-stable install-app-release notarize verify-dmg download-model download-e5 download-embedder ensure-models \ hooks SHELL := /bin/bash @@ -182,7 +182,11 @@ config: install-app: @echo "Building $(CODESCRIBE_APP_NAME).app (SwiftUI, optimized local profile) via scripts/build-app.sh ..." @echo "Local install uses the development license verifier; CODESCRIBE_LICENSE_PUBLIC_KEY_HEX is reserved for distribution builds." - @env -u CODESCRIBE_LICENSE_PUBLIC_KEY_HEX $(MAKE) --no-print-directory app PROFILE=local-release + @BIT=$$(./scripts/developer-surface-gate.sh); \ + echo "Developer surface: $$BIT (1 needs legit Sparkle + license public keys on this machine)."; \ + env -u CODESCRIBE_LICENSE_PUBLIC_KEY_HEX \ + CODESCRIBE_DEVELOPER_SURFACE=$$BIT \ + $(MAKE) --no-print-directory app PROFILE=local-release @APP_SRC="macos/build/Build/Products/Release/Codescribe.app"; \ if [ ! -d "$$APP_SRC" ]; then \ echo "Build product missing: $$APP_SRC — 'make app PROFILE=local-release' did not produce the app."; \ @@ -256,7 +260,11 @@ bump: esac; \ new="$$major.$$minor.$$patch"; \ sed -i '' "s/^version = \"$$current\"/version = \"$$new\"/" $(VERSION_FILE); \ - echo "Bumped: v$$current -> v$$new" + if [ -f README.md ]; then \ + sed -i '' "s/badge\/version-$$current-/badge\/version-$$new-/" README.md; \ + sed -i '' "s/current source version is \`$$current\`/current source version is \`$$new\`/" README.md; \ + fi; \ + echo "Bumped: v$$current -> v$$new (Cargo.toml + README). Update CHANGELOG by hand. Do not bump site/src/lib/release.ts here." bump-patch: @$(MAKE) bump TYPE=patch @@ -1179,7 +1187,8 @@ help: @printf '%s\n' ' make install-no-embed DEV/RECOVERY: no optional embeds (runtime paths only)' @printf '%s\n' ' make release-codescribe-embedded Fat dylib with Whisper baked in (not daily)' @printf ' $(HELP_C_GREEN)%-18s$(HELP_C_RESET) %s\n' 'config' 'Edit ~/.codescribe/.env' - @printf ' $(HELP_C_GREEN)%-18s$(HELP_C_RESET) %s\n' 'install-app' 'Install to /Applications' + @printf ' $(HELP_C_GREEN)%-18s$(HELP_C_RESET) %s\n' 'install-app' 'Local-release install to /Applications (may re-sign; Lab if keys resolve)' + @printf ' $(HELP_C_GREEN)%-18s$(HELP_C_RESET) %s\n' 'release-stable' 'Everyday: notarize slim DMG + install that stapled .app' @printf '\n' @printf ' $(HELP_C_YELLOW)%s$(HELP_C_RESET)\n' 'RELEASE & DISTRIBUTION' @printf ' $(HELP_C_GREEN)%-18s$(HELP_C_RESET) %s\n' 'dmg' 'Build DMG (ad-hoc signed)' @@ -1308,6 +1317,40 @@ release-standard: dist-preflight-signed fi; \ ./scripts/verify-dmg-payload.sh "$$DMG" --variant slim --version "$$VERSION" +# Install the already-built Release .app without re-signing. Re-signing with +# Apple Development (install-app) drops the notarization ticket and gives the +# everyday machine a different identity than the public DMG. +install-app-release: + @APP_SRC="macos/build/Build/Products/Release/$(CODESCRIBE_APP_NAME).app"; \ + DEST="/Applications/$(CODESCRIBE_APP_NAME).app"; \ + if [ ! -d "$$APP_SRC" ]; then \ + echo "ERROR: no Release app at $$APP_SRC — run make release-standard first."; \ + exit 1; \ + fi; \ + if ! codesign --verify --deep --strict "$$APP_SRC" >/dev/null 2>&1; then \ + echo "ERROR: Release app is not strictly signed. Refusing to install it."; \ + exit 1; \ + fi; \ + echo "Stopping running $(CODESCRIBE_APP_NAME) (if any) ..."; \ + pkill -x "$(CODESCRIBE_APP_NAME)" 2>/dev/null || true; \ + sleep 1; \ + echo "Installing stapled Release app to $$DEST (no re-sign) ..."; \ + mkdir -p /Applications; \ + rsync -a --delete "$$APP_SRC/" "$$DEST/"; \ + echo "Installed: $$DEST"; \ + defaults read "$$DEST/Contents/Info" CFBundleShortVersionString; \ + defaults read "$$DEST/Contents/Info" CSBuildCommit; \ + if stapler validate "$$DEST" >/dev/null 2>&1; then \ + echo "staple: valid"; \ + else \ + echo "WARNING: staple not valid yet — run make release-standard (it notarizes) before treating this as everyday-stable."; \ + fi + +# Everyday stable cut: the slim public DMG and the same stapled .app in +# /Applications. Does not bake Lab. Does not touch SITE_VERSION / tags / GitHub. +release-stable: release-standard install-app-release + @echo "Everyday stable is local (slim DMG + /Applications). Tag, push, gh release, and SITE_VERSION stay the operator button." + # Optional fat SKU: bake Whisper (~1GB+) into the app. Not the daily path. # Ends with the fail-closed payload gate (full = Silero + Whisper embedded, # MiniLM runtime resource). diff --git a/README.md b/README.md index 5d54053c..e7dda202 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # ⌜ Codescribe ⌟ -[![Version](https://img.shields.io/badge/version-0.13.3-6a9bcc)](Cargo.toml) +[![Version](https://img.shields.io/badge/version-0.14.1-6a9bcc)](Cargo.toml) [![License: FSL-1.1-ALv2](https://img.shields.io/badge/license-FSL--1.1--ALv2-d97757)](LICENSE) [![CI](https://github.com/vetcoders/codescribe/actions/workflows/rust.yml/badge.svg)](https://github.com/vetcoders/codescribe/actions/workflows/rust.yml) [![Landing](https://img.shields.io/badge/site-vetcoders.github.io%2Fcodescribe-788c5d)](https://vetcoders.github.io/codescribe/) @@ -59,7 +59,7 @@ flowchart TB > **Current runtime truth:** live overlay preview is local Whisper. Cloud STT is configurable in Settings, but in the current build it is still a **post-capture** path rather than live cloud preview. -> **Status:** current source version is `0.13.3` (see `Cargo.toml`) and ships as a native macOS tray/settings/overlay app with local live preview, tiered settings (`settings.json` + Keychain + optional `.env`), and quality-loop tooling. +> **Status:** current source version is `0.14.1` (see `Cargo.toml`) and ships as a native macOS tray/settings/overlay app with local live preview, tiered settings (`settings.json` + Keychain + optional `.env`), and quality-loop tooling. See: [`docs/WHISPER_LIVE.md`](docs/WHISPER_LIVE.md) | [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) @@ -405,7 +405,8 @@ make format # cargo fmt ``` make app # Debug SwiftUI app build make app PROFILE=local-release # Optimized local SwiftUI app build -make install-app # Build + install /Applications/Codescribe.app +make install-app # Local-release install (may re-sign; Lab only if keys resolve) +make release-stable # Everyday: notarize slim DMG + install that stapled .app make release-qube # Build qube CLI tools make install # Install qube CLI tools + repo-local hooks make release-dmgs # Build both signed + notarized release DMGs diff --git a/app/controller/mod.rs b/app/controller/mod.rs index ad5910e1..6d9dca3e 100644 --- a/app/controller/mod.rs +++ b/app/controller/mod.rs @@ -78,7 +78,7 @@ use crate::os::selection::{ wait_for_frontmost_app, }; use crate::os::shortcut_registry; -use codescribe_core::asr_session::GatewaySessionAvailability; +use codescribe_core::asr_session::gateway_session_availability; use context_bucket::ContextBucket; #[cfg(test)] pub(crate) use context_bucket::ContextMarker; @@ -103,8 +103,10 @@ use delivery_route::{ DeliveryFacts, DeliveryIntent, DeliveryRoute, delivery_intent_from_session, format_delivery_route_line, overlay_insert_facts, resolve_delivery_route, target_is_self_app, }; +#[cfg(test)] +pub(crate) use final_pass::FinalPassRoutingMode; pub(crate) use final_pass::{ - FinalPassAction, FinalPassRoutingMode, FinalPassStages, SmartTailGapSource, StopPathBudget, + FinalPassAction, FinalPassStages, SmartTailGapSource, StopPathBudget, StreamingCompletenessEvidence, append_tail_gap, apply_committed_density_floor, apply_residual_required_demotion, assess_streaming_completeness, completeness_label, compose_stop_path_residual_from_partials, final_pass_action, final_pass_routing_mode, @@ -285,6 +287,17 @@ impl Drop for AtomicFlagGuard { } } +/// Keep the last session WAV at a stable path so overlay Retranscribe +/// (Full HQ / Cloud) can re-run without depending on a temp file still +/// being there after stop. +fn retain_last_session_audio(path: &std::path::Path) { + let dest = crate::config::Config::config_dir().join("last_session.wav"); + match std::fs::copy(path, &dest) { + Ok(_) => info!("last_session.wav retained at {}", dest.display()), + Err(err) => warn!("last_session.wav retain failed: {err:#}"), + } +} + /// What one stop-and-process pass produced: the delivery decision plus the /// per-phase wall clock that the stop-path budget line reports. #[derive(Debug, Clone, Default)] @@ -537,7 +550,8 @@ impl RecordingController { // // Product invariant: recording readiness is NOT engine readiness. // Audio capture must start the moment the user presses record; the - // live pipeline and the final pass lazy-load the engine on first use. + // live local refinement and explicit Retranscribe lazy-load the + // engine on first use. // A failed prewarm is a warning, not an app or recording failure. // The idle-unload reaper (commit 2b8bb1f) may legitimately drop the // engine later and the next call reloads it — pinning it here would @@ -643,8 +657,8 @@ impl RecordingController { } /// Cross the product truth boundary exactly once. Engine finals and engine - /// session close are still mutable draft stages: Smart/Always final pass, - /// adjudication, dictionary cleanup, and formatting all happen later. The + /// session close are still mutable draft stages: adjudication, dictionary + /// cleanup, and formatting all happen later. Normal stop has no file pass. The /// text handed here is the same text used for history and delivery. async fn seal_active_transcript(&self, text: String) { let bus = self.active_transcript_bus.read().await.clone(); @@ -2300,10 +2314,8 @@ impl RecordingController { Arc::clone(&session_telemetry), transcript_bus.clone(), ); - rec.configure_layer1( - &UserSettings::load(), - GatewaySessionAvailability::Unavailable, - ); + let settings = UserSettings::load(); + rec.configure_layer1(&settings, gateway_session_availability(&config)); if !cfg!(test) { let language_hint = language.whisper_hint().map(str::to_string); // Audio-first cold start: do not preflight Whisper here. The @@ -2515,10 +2527,8 @@ impl RecordingController { Arc::clone(&self.session_telemetry), transcript_bus.clone(), ); - recorder.configure_layer1( - &UserSettings::load(), - GatewaySessionAvailability::Unavailable, - ); + let settings = UserSettings::load(); + recorder.configure_layer1(&settings, gateway_session_availability(&config)); // Skip actual audio stream in tests (no CoreAudio device needed) let language_hint = language.whisper_hint().map(str::to_string); @@ -2580,9 +2590,9 @@ impl RecordingController { /// Stop a toggle session under a watchdog. /// - /// The full stop — recorder drain, final-pass STT, post-process, delivery — - /// must finish within `STOP_TIMEOUT`. When it does not (final-pass deadlock - /// on the Metal device, lock contention, a `stop` blocked in a CoreAudio + /// The full stop — recorder drain, live-session adjudication, post-process, + /// delivery — must finish within `STOP_TIMEOUT`. When it does not (lock + /// contention, a `stop` blocked in a CoreAudio /// callback), recovery forces `Idle` so the next toggle press registers, the /// badge clears, and the tray stops claiming idle over a hung recording. async fn stop_toggle_and_adjudicate(&self) -> Result<()> { @@ -2590,9 +2600,9 @@ impl RecordingController { return Ok(()); } - // Watchdog: full stop+adjudicate (recorder.stop + final-pass STT + post-process - // + paste) must complete within STOP_TIMEOUT. If it stalls — final-pass deadlock - // on Metal device, RwLock contention, recorder.stop blocked on cpal callback — + // Watchdog: full stop+adjudicate (recorder.stop + live truth + post-process + // + paste) must complete within STOP_TIMEOUT. If it stalls — RwLock + // contention or recorder.stop blocked on a cpal callback — // force recovery to Idle so subsequent toggle presses register, badge clears, // and tray reflects truth instead of showing Idle while recording is hung. match tokio::time::timeout(STOP_TIMEOUT, self.stop_toggle_and_adjudicate_inner()).await { @@ -2857,17 +2867,14 @@ impl RecordingController { result.map(|_| ()) } - /// Turn a stopped recording into a delivered transcript: decide the final - /// pass, adjudicate which transcript is the truth, then run the text - /// pipeline. + /// Turn a stopped recording into a delivered transcript: drain the live + /// transcript, adjudicate its truth, then run the text pipeline. /// - /// Final-pass routing is typed rather than heuristic. `Always` re-passes the - /// whole file; `Smart` on a complete streaming transcript skips; `Smart` on - /// an incomplete one transcribes only the uncommitted audio tail and appends - /// it, because committed streaming text is immutable under the append-only - /// overlay doctrine and a full re-pass there would rewrite what the user - /// already watched being typed. `Off` still emits a skipped LocalFinalPass - /// so provenance stays honest — it gates the WAV re-pass, never the lexicon. + /// Normal stop never uploads or decodes the completed WAV. Both local and + /// cloud refinement happen inside the live session; whole-file inference is + /// reserved for explicit Retranscribe actions (Overlay, Dictionary, + /// Teacher). A typed skipped LocalFinalPass is still emitted so old + /// provenance readers remain honest while the legacy routing UI is retired. /// /// A missing tail boundary is carried by the type, not by a sentinel: the /// `Skip` variant produces no `from_secs` at all, so handing `0.0` to the @@ -2898,6 +2905,9 @@ impl RecordingController { } else { None }; + if let Some(path) = &audio_path { + retain_last_session_audio(path.as_path()); + } let recording_timestamp = chrono::Local::now(); @@ -2907,47 +2917,17 @@ impl RecordingController { let use_local_stt = config.use_local_stt; let raw_save_enabled = raw_save_enabled(assistive); - let cloud_config = if use_local_stt { - None - } else { - match (config.stt_endpoint.clone(), config.stt_api_key.clone()) { - (Some(endpoint), Some(api_key)) - if !endpoint.trim().is_empty() && !api_key.trim().is_empty() => - { - Some((endpoint, api_key)) - } - _ => None, - } - }; - let assistive_loop = assistive && self.assistive_loop_active.load(Ordering::SeqCst); let mut local_final_pass_verdict = None; - let mut cloud_verdict_opt = None; - let mut cloud_handle: Option>> = + // Normal recording never uploads the completed WAV. Cloud audio is + // already carried by the bounded live WSS Layer 1 session; multipart + // file upload belongs only to explicit retranscribe surfaces. + let cloud_verdict_opt = None; + let cloud_handle: Option>> = None; let mut local_final_pass_attempted = false; - if let Some((cloud_endpoint, cloud_api_key)) = cloud_config { - if let Some(path) = &audio_path { - let cloud_path = path.as_path().to_path_buf(); - let cloud_language = language_opt.map(str::to_string); - cloud_handle = Some(tokio::spawn(async move { - crate::client::transcribe_cloud( - &cloud_path, - cloud_language.as_deref(), - &cloud_endpoint, - &cloud_api_key, - ) - .await - })); - } else { - warn!("Cloud STT disabled: no audio file available"); - } - } else if !use_local_stt { - warn!("Cloud STT disabled: STT_ENDPOINT/STT_API_KEY missing"); - } - let routing_mode = final_pass_routing_mode(); let prefer_apple = codescribe_core::stt::active_engine_is_apple(); let streaming_engine_label = if prefer_apple { @@ -2955,10 +2935,10 @@ impl RecordingController { } else { "streaming_whisper" }; - // Honest mode: Off never runs a full file re-pass (Apple or not). - // Smart/Always decide via final_pass_action — no silent rewrite. - let run_local_final_pass = - use_local_stt && !matches!(routing_mode, FinalPassRoutingMode::Off); + // Product contract: a normal stop is never a file-pass trigger. Keep + // the legacy typed routing below inert until it is removed with the + // obsolete settings surface; explicit retranscribe owns file decoding. + let run_local_final_pass = false; let effective_routing = routing_mode; let mut final_pass_secs = 0.0; let mut final_pass_stages = FinalPassStages::default(); @@ -3554,19 +3534,6 @@ impl RecordingController { ); } - if !use_local_stt { - if let Some(handle) = cloud_handle.take() { - info!("Awaiting cloud STT as selected transcript backend"); - match handle.await { - Ok(Ok(verdict)) => cloud_verdict_opt = Some(verdict), - Ok(Err(e)) => error!("Cloud transcription failed: {}", e), - Err(e) => error!("Cloud transcription task failed: {}", e), - } - } else { - warn!("Cloud backend unavailable (cloud disabled or missing credentials)"); - } - } - let session_telemetry = snapshot_session_telemetry(&self.session_telemetry); let mut truth_verdict = adjudicate_recording_truth( use_local_stt, @@ -3718,8 +3685,8 @@ impl RecordingController { write_truth_sidecar_logged(&audio_saved_path, &truth_metadata); } - // No-speech stops still paid the final pass — keep the stage - // receipt so latency truth covers every real stop. + // Keep the all-zero legacy stage receipt so latency truth + // explicitly proves that no file pass ran on this stop. info!("{}", format_final_pass_stages_line(final_pass_stages)); self.seal_active_transcript(String::new()).await; return Ok(ProcessRecordingOutcome::no_speech(reason)); @@ -3833,8 +3800,8 @@ impl RecordingController { /// post-process, format per mode, persist, then paste or hand off. /// /// The dictionary step (lexicon, cleanup, semantic gate) always runs — it is - /// independent of the final-pass mode, which routes only the optional WAV - /// re-pass. Formatting then follows the mode the hotkey chose, and the + /// independent of the legacy final-pass setting. Formatting then follows + /// the mode the hotkey chose, and the /// Light+ sentence-shape floor is applied to every lane that *promised* /// formatting. RAW lanes are excluded on purpose: `force_raw` and Toggle-OFF /// promise the user their literal words. diff --git a/app/presentation/transcript_bus.rs b/app/presentation/transcript_bus.rs index 66dfa00a..a9acf215 100644 --- a/app/presentation/transcript_bus.rs +++ b/app/presentation/transcript_bus.rs @@ -5,12 +5,14 @@ //! It never opens audio, re-transcribes a file, or reconstructs text from UI //! deltas. One append-only JSON object is flushed per state transition. +use std::collections::BTreeMap; use std::fs::{File, OpenOptions}; use std::io::{self, Write}; use std::path::{Path, PathBuf}; use std::sync::Mutex; use chrono::{SecondsFormat, Utc}; +use codescribe_core::audio::capture_receipt::session_energy_db; use codescribe_core::pipeline::contracts::TranscriptSegment; use serde::{Deserialize, Serialize}; @@ -65,6 +67,32 @@ impl TranscriptDraftStatus { } } +/// Grain of one published span. Word pins are engine evidence; utterance +/// grain is the honest fallback when Apple committed a window, not words. +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "snake_case")] +pub enum TranscriptWordGrain { + #[default] + Word, + Utterance, +} + +fn is_word_grain(grain: &TranscriptWordGrain) -> bool { + matches!(grain, TranscriptWordGrain::Word) +} + +/// One span on the capture PCM clock: text + samples + intensity. +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +pub struct TranscriptWordSpan { + pub text: String, + pub sample_start: u64, + pub sample_end: u64, + #[serde(skip_serializing_if = "Option::is_none")] + pub energy_db: Option, + #[serde(default, skip_serializing_if = "is_word_grain")] + pub grain: TranscriptWordGrain, +} + /// Append-only public event contract. `text` is always clean reducer truth; /// unfiltered engine `raw_text` never crosses this boundary. #[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] @@ -84,6 +112,8 @@ pub struct CleanTranscriptEvent { pub text: String, #[serde(default, skip_serializing_if = "Vec::is_empty")] pub segments: Vec, + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub words: Vec, #[serde(skip_serializing_if = "Option::is_none")] pub pipeline_session_id: Option, } @@ -106,6 +136,7 @@ struct TranscriptBusWriter { sequence: u64, started: bool, sealed: bool, + drafts: BTreeMap, } impl TranscriptBus { @@ -155,6 +186,7 @@ impl TranscriptBus { sequence: 0, started: false, sealed: false, + drafts: BTreeMap::new(), }), sample_rate_override, }; @@ -182,6 +214,7 @@ impl TranscriptBus { pub fn publish_draft(&self, status: TranscriptDraftStatus, utterance: TranscriptDraft) { let status = status.as_str(); let sample_rate = self.sample_rate(); + let words = word_spans_from_draft(&utterance, sample_rate); let event = CleanTranscriptEvent { schema: "codescribe.transcript.v1".to_string(), sequence: 0, @@ -195,8 +228,9 @@ impl TranscriptBus { sample_end: sample_rate.map(|rate| seconds_to_sample(utterance.end_seconds, rate)), audio_start_seconds: Some(utterance.start_seconds), audio_end_seconds: Some(utterance.end_seconds), - text: utterance.text, - segments: utterance.segments, + text: utterance.text.clone(), + segments: utterance.segments.clone(), + words, pipeline_session_id: None, }; @@ -208,6 +242,7 @@ impl TranscriptBus { tracing::warn!(session_id = %self.session.session_id, %status, "transcript draft ignored after product seal"); return; } + writer.drafts.insert(utterance.utterance_id, utterance); if let Err(error) = self .ensure_started_locked(&mut writer) .and_then(|_| self.write_event_locked(&mut writer, event)) @@ -227,6 +262,8 @@ impl TranscriptBus { if writer.sealed { return; } + let sample_rate = self.sample_rate(); + let clock = aggregate_seal_clock(&writer.drafts, sample_rate); let event = CleanTranscriptEvent { schema: "codescribe.transcript.v1".to_string(), sequence: 0, @@ -235,13 +272,14 @@ impl TranscriptBus { utterance_id: None, emitted_at: String::new(), status: "transcript_sealed".to_string(), - sample_rate_hz: self.sample_rate(), - sample_start: None, - sample_end: None, - audio_start_seconds: None, - audio_end_seconds: None, + sample_rate_hz: sample_rate, + sample_start: clock.sample_start, + sample_end: clock.sample_end, + audio_start_seconds: clock.audio_start_seconds, + audio_end_seconds: clock.audio_end_seconds, text, - segments: Vec::new(), + segments: clock.segments, + words: clock.words, pipeline_session_id, }; match self @@ -287,6 +325,7 @@ impl TranscriptBus { audio_end_seconds: None, text: String::new(), segments: Vec::new(), + words: Vec::new(), pipeline_session_id: None, }, )?; @@ -345,6 +384,108 @@ fn seconds_to_sample(seconds: f32, sample_rate: u32) -> u64 { (f64::from(seconds) * f64::from(sample_rate)).round() as u64 } +fn finite_audio_window(start: f32, end: f32) -> Option<(f32, f32)> { + if start.is_finite() && end.is_finite() && end > start { + Some((start, end)) + } else { + None + } +} + +fn word_span_from_seconds( + text: String, + start: f32, + end: f32, + sample_rate: Option, + grain: TranscriptWordGrain, +) -> Option { + let (start, end) = finite_audio_window(start, end)?; + if text.trim().is_empty() { + return None; + } + let rate = sample_rate.filter(|rate| *rate > 0)?; + let sample_start = seconds_to_sample(start, rate); + let sample_end = seconds_to_sample(end, rate).max(sample_start.saturating_add(1)); + Some(TranscriptWordSpan { + text, + sample_start, + sample_end, + energy_db: session_energy_db(sample_start, sample_end), + grain, + }) +} + +fn word_spans_from_draft( + utterance: &TranscriptDraft, + sample_rate: Option, +) -> Vec { + if !utterance.segments.is_empty() { + return utterance + .segments + .iter() + .filter_map(|segment| { + word_span_from_seconds( + segment.text.clone(), + segment.start_ts, + segment.end_ts, + sample_rate, + TranscriptWordGrain::Word, + ) + }) + .collect(); + } + word_span_from_seconds( + utterance.text.clone(), + utterance.start_seconds, + utterance.end_seconds, + sample_rate, + TranscriptWordGrain::Utterance, + ) + .into_iter() + .collect() +} + +struct SealClock { + sample_start: Option, + sample_end: Option, + audio_start_seconds: Option, + audio_end_seconds: Option, + segments: Vec, + words: Vec, +} + +fn aggregate_seal_clock( + drafts: &BTreeMap, + sample_rate: Option, +) -> SealClock { + let mut audio_start = None; + let mut audio_end = None; + let mut segments = Vec::new(); + let mut words = Vec::new(); + for draft in drafts.values() { + if let Some((start, end)) = finite_audio_window(draft.start_seconds, draft.end_seconds) { + audio_start = Some(audio_start.map_or(start, |seen: f32| seen.min(start))); + audio_end = Some(audio_end.map_or(end, |seen: f32| seen.max(end))); + } + segments.extend(draft.segments.iter().cloned()); + words.extend(word_spans_from_draft(draft, sample_rate)); + } + SealClock { + sample_start: match (sample_rate, audio_start) { + (Some(rate), Some(start)) => Some(seconds_to_sample(start, rate)), + _ => None, + }, + sample_end: match (sample_rate, audio_end) { + (Some(rate), Some(end)) => Some(seconds_to_sample(end, rate)), + _ => None, + }, + audio_start_seconds: audio_start, + audio_end_seconds: audio_end, + segments, + words, + } +} + fn expand_tilde(path: &str) -> PathBuf { if path == "~" { return directories::BaseDirs::new() @@ -414,6 +555,14 @@ mod tests { assert_eq!(lines[1].sample_end, Some(72_000)); assert_eq!(lines[2].status, "transcript_sealed"); assert_eq!(lines[2].text, "clean final"); + assert_eq!(lines[2].sample_start, Some(12_000)); + assert_eq!(lines[2].sample_end, Some(72_000)); + assert_eq!(lines[2].audio_start_seconds, Some(0.25)); + assert_eq!(lines[2].audio_end_seconds, Some(1.5)); + assert_eq!(lines[2].words.len(), 1); + assert_eq!(lines[2].words[0].sample_start, 12_000); + assert_eq!(lines[2].words[0].sample_end, 72_000); + assert_eq!(lines[2].words[0].grain, TranscriptWordGrain::Utterance); assert_eq!( lines.iter().map(|event| event.sequence).collect::>(), vec![1, 2, 3] @@ -428,4 +577,62 @@ mod tests { ); } } + + #[test] + fn seal_publishes_word_spans_on_the_pcm_clock() { + let temp = tempfile::tempdir().unwrap(); + let path = temp.path().join("words.jsonl"); + let bus = TranscriptBus::open_at( + TranscriptSession { + session_id: "session-words".to_string(), + mode: TranscriptMode::Dictation, + }, + path.clone(), + Some(16_000), + ) + .unwrap(); + bus.publish_draft( + TranscriptDraftStatus::Created, + TranscriptDraft { + utterance_id: 3, + text: "dwa slowa".to_string(), + start_seconds: 1.0, + end_seconds: 2.0, + segments: vec![ + TranscriptSegment { + text: "dwa".to_string(), + start_ts: 1.0, + end_ts: 1.4, + }, + TranscriptSegment { + text: "slowa".to_string(), + start_ts: 1.4, + end_ts: 2.0, + }, + ], + }, + ); + bus.publish_sealed("dwa slowa".to_string(), None); + + let lines: Vec = std::fs::read_to_string(&path) + .unwrap() + .lines() + .map(|line| serde_json::from_str(line).unwrap()) + .collect(); + let seal = lines + .iter() + .find(|event| event.status == "transcript_sealed") + .expect("seal"); + assert_eq!(seal.sample_start, Some(16_000)); + assert_eq!(seal.sample_end, Some(32_000)); + assert_eq!(seal.segments.len(), 2); + assert_eq!(seal.words.len(), 2); + assert_eq!(seal.words[0].text, "dwa"); + assert_eq!(seal.words[0].sample_start, 16_000); + assert_eq!(seal.words[0].sample_end, 22_400); + assert_eq!(seal.words[0].grain, TranscriptWordGrain::Word); + assert_eq!(seal.words[1].text, "slowa"); + assert_eq!(seal.words[1].sample_start, 22_400); + assert_eq!(seal.words[1].sample_end, 32_000); + } } diff --git a/bin/codescribe-corpus.rs b/bin/codescribe-corpus.rs index e9a81674..c210aa65 100644 --- a/bin/codescribe-corpus.rs +++ b/bin/codescribe-corpus.rs @@ -33,6 +33,9 @@ use codescribe_core::asr_session::GatewaySessionAvailability; use codescribe_core::config::UserSettings; use codescribe_core::pipeline::contracts::{EngineEvent, LayerSource}; use codescribe_core::quality::engine_contract::{CORPUS_REPORT_SCHEMA, ENGINE_CONTRACT_ID}; +use codescribe_core::quality::seal_atlas_html::{ + SealAtlasPage, SealAtlasStats, render_seal_atlas_html, +}; use codescribe_core::util::safe_path::{safe_open, safe_symlink_or_copy_bounded}; use serde::{Deserialize, Serialize}; use sha2::{Digest, Sha256}; @@ -1133,7 +1136,8 @@ async fn run_worker(args: WorkerArgs) -> Result<()> { let input_hashes_unchanged = rows .iter() .all(|row| row.audio_hash_unchanged && row.reference_hash_unchanged); - let quality_html = format!("quality/{}.html", args.profile.token()); + let quality_html = format!("quality/seal-atlas.{}.html", args.profile.token()); + let qube_html = format!("quality/qube.{}.html", args.profile.token()); let quality_report = build_quality_report( args.profile, &args.language, @@ -1154,8 +1158,28 @@ async fn run_worker(args: WorkerArgs) -> Result<()> { metrics_reference: MetricsReference::Corpus, local_transcription: LocalTranscriptionMode::LocalWhisper, }; + let atlas = SealAtlasPage { + title: format!("Seal Atlas — corpus {}", args.profile.token()), + lede: format!( + "Corpus profile {}. One take, one PCM clock. Words from SealedSpan.words when a dump is attached — not from the final string.", + args.profile.token() + ), + stats: SealAtlasStats { + sealed_spans: quality_report.entries.len().to_string(), + ..SealAtlasStats::default() + }, + findings: vec![format!( + "Qube scores (footnote) live at {}. Avg WER is not the live engine.", + qube_html + )], + dump_present: false, + }; atomic_write_private( &output_root.join(&quality_html), + render_seal_atlas_html(&atlas).as_bytes(), + )?; + atomic_write_private( + &output_root.join(&qube_html), render_qube_html(&quality_report, &quality_config).as_bytes(), )?; diff --git a/bridge/src/config.rs b/bridge/src/config.rs index eba43bbb..9feef611 100644 --- a/bridge/src/config.rs +++ b/bridge/src/config.rs @@ -125,8 +125,9 @@ pub struct CsSettings { /// `"whisper"`. `None` means the built-in auto policy. Written back via /// `update_config` with the same key (promoted → settings.json). pub stt_engine: Option, - /// Final-pass routing (`FINAL_PASS_MODE`): `"always"` | `"smart"` | `"off"`. - /// `None` means Smart default. Written back via `update_config`. + /// Legacy stop-file-pass token (`FINAL_PASS_MODE`). Runtime ignores it + /// on stop; Settings no longer exposes Always/Smart/Off. Persist `off` + /// if a value must still be written. pub final_pass_mode: Option, // ── LLM backend (base) ── pub llm_endpoint: Option, @@ -165,6 +166,15 @@ pub struct CsSettings { pub emit_words_max: Option, pub buffered_interim_sec: Option, pub backend_max_upload_mb: Option, + /// Product ASR lane (`CODESCRIBE_ASR_MODE`): `"apple_only"` | + /// `"local_power"` | `"cloud"`. Written back via `update_config`. + pub asr_mode: Option, + /// Audio-egress consent (`CODESCRIBE_CLOUD_CONSENT`): `"granted"` | + /// `"denied"`. Cloud without `granted` resolves to Apple-only. + pub cloud_consent: Option, + /// Libraxis gateway session-mint URL (`CODESCRIBE_ASR_GATEWAY_URL`). + /// Session mint, not the live WSS socket (`STT_ENDPOINT`). + pub asr_gateway_url: Option, } /// Live, non-secret impact summary shown before a full local-data reset. @@ -645,6 +655,21 @@ impl CodescribeConfig { settings.backend_max_upload_mb, &env_file, ), + asr_mode: effective_settings_string( + "CODESCRIBE_ASR_MODE", + settings.asr_mode.clone(), + &env_file, + ), + cloud_consent: effective_settings_string( + "CODESCRIBE_CLOUD_CONSENT", + settings.cloud_consent.clone(), + &env_file, + ), + asr_gateway_url: effective_settings_string( + "CODESCRIBE_ASR_GATEWAY_URL", + settings.asr_gateway_url.clone(), + &env_file, + ), } } diff --git a/bridge/src/hotkeys.rs b/bridge/src/hotkeys.rs index 56e37bb8..826edeae 100644 --- a/bridge/src/hotkeys.rs +++ b/bridge/src/hotkeys.rs @@ -25,7 +25,9 @@ use tokio::sync::broadcast::error::RecvError; use crate::agent_delivery::{ CsAgentDeliveryListener, set_delivery_listener, spawn_delivery_forwarder, }; -use crate::recording::{CsAnnotationKind, CsLayerSummary, CsTranscriptionListener}; +use crate::recording::{ + CsAnnotationKind, CsLayerSummary, CsTranscription, CsTranscriptionListener, +}; use crate::{CsError, CsLanguage}; /// Shared process-wide slot for the lazily-created `RecordingController`. @@ -633,11 +635,23 @@ impl CodescribeHotkeys { start_recording_with_event(HotkeyEvent::ToggleNormal).await } - /// Start the same toggle flow in the assistive lane for UI-initiated recording. + /// Start the same toggle flow in the assistive lane. Overlay owns this + /// route — the Agent composer mic is a separate, UI-initiated capture. pub async fn start_assistive_recording(&self) -> Result<(), CsError> { start_recording_with_event(HotkeyEvent::ToggleAssistive).await } + /// Overlay Retranscribe: `hq:` / `cloud:` prefixes pick the pass. + /// Bare paths are a Full HQ file pass. + pub async fn transcribe_file(&self, path: String) -> Result { + crate::recording::transcribe_session_file(path).await + } + + /// Stable path of the last retained session WAV, if it exists. + pub fn last_session_audio_path(&self) -> Option { + crate::recording::last_session_audio_path() + } + /// Stop the active legacy-controller recording flow, if one is live. pub async fn stop_recording(&self) -> Result<(), CsError> { let Some(controller) = current_controller(&shared_controller()) else { diff --git a/bridge/src/recording.rs b/bridge/src/recording.rs index 949e0775..81f4b848 100644 --- a/bridge/src/recording.rs +++ b/bridge/src/recording.rs @@ -277,6 +277,125 @@ impl From<&LayerSummary> for CsLayerSummary { } } +/// Path prefixes pick the Retranscribe pass: +/// - `hq:` or no prefix — Full HQ file pass (`transcribe_file_verdict`) +/// - `cloud:` — Cloud pass (`transcribe_cloud` with Settings STT credentials) +pub(crate) async fn transcribe_session_file(path: String) -> Result { + let (pass, file_path) = split_retranscribe_path(&path); + match pass { + RetranscribePass::Hq => tokio::task::spawn_blocking(move || transcribe_file_hq(file_path)) + .await + .map_err(|e| CsError::Recording { + msg: format!("transcribe_file task join error: {e}"), + })?, + RetranscribePass::Cloud => transcribe_file_cloud(file_path).await, + } +} + +/// `~/.codescribe/last_session.wav` when the last stop retained audio. +pub(crate) fn last_session_audio_path() -> Option { + let dest = codescribe_core::config::Config::config_dir().join("last_session.wav"); + dest.exists().then(|| dest.to_string_lossy().into_owned()) +} + +enum RetranscribePass { + Hq, + Cloud, +} + +fn split_retranscribe_path(path: &str) -> (RetranscribePass, String) { + if let Some(rest) = path.strip_prefix("cloud:") { + (RetranscribePass::Cloud, rest.to_string()) + } else if let Some(rest) = path.strip_prefix("hq:") { + (RetranscribePass::Hq, rest.to_string()) + } else { + (RetranscribePass::Hq, path.to_string()) + } +} + +fn transcribe_file_hq(path: String) -> Result { + let verdict = codescribe_core::stt::transcribe_file_verdict(std::path::Path::new(&path), None) + .map_err(|e| CsError::Recording { msg: e.to_string() })?; + Ok(CsTranscription { + text: verdict.text, + language: "und".to_string(), + }) +} + +async fn transcribe_file_cloud(path: String) -> Result { + let config = codescribe_core::config::Config::load(); + let endpoint = config + .stt_endpoint + .clone() + .filter(|value| !value.trim().is_empty()); + let key = config + .stt_api_key + .clone() + .filter(|value| !value.trim().is_empty()) + .unwrap_or_default(); + let Some(endpoint) = endpoint else { + return Err(CsError::Recording { + msg: "Cloud pass needs STT_ENDPOINT".to_string(), + }); + }; + // Same invert as Settings → Test: a stored Voice Lab socket is not a + // multipart URL. Public HTTPS file URLs stay file. + let endpoint = codescribe_core::stt::tail_provider::file_probe_endpoint(&endpoint); + if codescribe_core::stt::tail_provider::stt_auth_mode(&endpoint) + != codescribe_core::stt::tail_provider::SttAuthMode::Unauthenticated + && key.is_empty() + { + return Err(CsError::Recording { + msg: "Cloud pass needs STT_API_KEY for this endpoint".to_string(), + }); + } + let verdict = + codescribe::client::transcribe_cloud(std::path::Path::new(&path), None, &endpoint, &key) + .await + .map_err(|e| CsError::Recording { msg: e.to_string() })?; + Ok(CsTranscription { + text: verdict.text, + language: "und".to_string(), + }) +} + +#[cfg(test)] +mod retranscribe_tests { + use super::*; + + #[test] + fn retranscribe_path_prefixes_select_hq_or_cloud() { + assert!(matches!( + split_retranscribe_path("/tmp/last_session.wav"), + (RetranscribePass::Hq, path) if path == "/tmp/last_session.wav" + )); + assert!(matches!( + split_retranscribe_path("hq:/tmp/last_session.wav"), + (RetranscribePass::Hq, path) if path == "/tmp/last_session.wav" + )); + assert!(matches!( + split_retranscribe_path("cloud:/tmp/last_session.wav"), + (RetranscribePass::Cloud, path) if path == "/tmp/last_session.wav" + )); + } + + #[test] + fn cloud_pass_inverts_voice_lab_socket_to_file() { + assert_eq!( + codescribe_core::stt::tail_provider::file_probe_endpoint( + "ws://127.0.0.1:8446/v1/audio/transcribe" + ), + "http://127.0.0.1:8444/v1/audio/transcriptions" + ); + assert_eq!( + codescribe_core::stt::tail_provider::file_probe_endpoint( + "https://api.libraxis.cloud/v1/audio/transcriptions" + ), + "https://api.libraxis.cloud/v1/audio/transcriptions" + ); + } +} + /// Foreign callback trait — dictation events forwarded to Swift. /// /// Distilled from the engine's richer `EngineEvent` stream: diff --git a/core/asr_session/bootstrap.rs b/core/asr_session/bootstrap.rs index 84b4434a..3aef9bff 100644 --- a/core/asr_session/bootstrap.rs +++ b/core/asr_session/bootstrap.rs @@ -1,9 +1,10 @@ -//! Recording-start policy that joins persisted mode truth, consent, and a -//! minted gateway connection into the provider decision consumed by C1. +//! Recording-start policy that joins persisted mode truth, consent, and the +//! canonical STT endpoint into the provider decision consumed by C1. //! -//! The gateway mint itself is deliberately outside this repository boundary. -//! Until a caller supplies one validated, short-lived connection, cloud mode -//! is treated as unavailable and recording continues with Apple + lexicon. +//! An explicit `ws`/`wss` URL is the live socket. A public HTTPS file URL +//! (`/v1/audio/transcriptions`) stays file — OpenAI and Libraxis the same — +//! so recording continues with Apple + lexicon unless the socket is stored. +//! Loopback file URLs still map onto the local Voice Lab worker. use std::fmt; @@ -14,21 +15,70 @@ use super::cloud::{ }; use super::consent::authorize_cloud_egress; use super::recorder::Layer1Decision; -use crate::config::{AsrProductMode, UserSettings}; +use crate::config::{AsrProductMode, Config, UserSettings}; -/// Availability of one short-lived gateway session at recording start. +/// Availability of one validated live session at recording start. /// /// `Invalid` is distinct from `Unavailable` for content-free diagnostics. The -/// raw endpoint and bearer never cross this enum and are never formatted. +/// raw endpoint and credential never cross this enum and are never formatted. pub enum GatewaySessionAvailability { - /// No mint response is available (offline, timeout, or gateway absent). + /// No known live endpoint is available. Unavailable, - /// A mint response or normalized connection failed validation. + /// The resolved live connection failed validation. Invalid, - /// A validated, single-use WebSocket endpoint and bearer. + /// A validated WebSocket endpoint and endpoint-owned credential. Ready(GatewayConnection), } +/// Resolve the live WebSocket session directly from canonical STT config. +/// +/// Stored `ws`/`wss` is the live socket (Voice Lab on this host). A public +/// OpenAI-compatible file URL is never rewritten into a socket. Retranscribe +/// and Settings → Test keep the file path. +pub fn gateway_session_availability(config: &Config) -> GatewaySessionAvailability { + let Some(endpoint) = config + .stt_endpoint + .as_deref() + .filter(|value| !value.trim().is_empty()) + else { + return GatewaySessionAvailability::Unavailable; + }; + let Some(endpoint) = live_websocket_endpoint(endpoint) else { + return GatewaySessionAvailability::Unavailable; + }; + let credential = config.stt_api_key.as_deref().unwrap_or_default(); + match GatewayConnection::new(endpoint, credential) { + Ok(connection) => GatewaySessionAvailability::Ready(connection), + Err(_) => GatewaySessionAvailability::Invalid, + } +} + +fn live_websocket_endpoint(endpoint: &str) -> Option { + let mut url = reqwest::Url::parse(endpoint).ok()?; + let host = url.host_str()?.trim_matches(['[', ']']); + match url.scheme() { + "ws" | "wss" => return Some(url.to_string()), + "http" | "https" => {} + _ => return None, + } + + let loopback = host.eq_ignore_ascii_case("localhost") + || host + .parse::() + .is_ok_and(|address| address.is_loopback()); + if !loopback { + return None; + } + + let websocket_scheme = if url.scheme() == "https" { "wss" } else { "ws" }; + url.set_scheme(websocket_scheme).ok()?; + if url.path().ends_with("/transcriptions") { + let path = url.path().trim_end_matches("transcriptions").to_string() + "transcribe"; + url.set_path(&path); + } + Some(url.to_string()) +} + impl fmt::Debug for GatewaySessionAvailability { fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { formatter.write_str(match self { @@ -43,7 +93,7 @@ impl fmt::Debug for GatewaySessionAvailability { /// /// Cloud can arm only when the settings resolver still says `cloud`, explicit /// audio-egress authorization succeeds, and the caller supplies a validated -/// minted gateway connection. Every other state is normal Apple + lexicon. +/// live gateway connection. Every other state is normal Apple + lexicon. /// In particular, an unavailable cloud session never falls through to local /// power or Whisper. pub fn layer1_decision_for_recording( @@ -149,4 +199,45 @@ mod tests { ); assert_eq!(probe().saturating_sub(before), 0); } + + #[test] + fn public_file_url_is_not_a_live_socket() { + assert_eq!( + live_websocket_endpoint("https://api.libraxis.cloud/v1/audio/transcriptions"), + None + ); + assert_eq!( + live_websocket_endpoint("https://api.openai.com/v1/audio/transcriptions"), + None + ); + assert_eq!( + live_websocket_endpoint("https://custom.example/v1/audio/transcriptions"), + None + ); + assert_eq!( + live_websocket_endpoint("wss://api.libraxis.cloud/v1/audio/transcribe").as_deref(), + Some("wss://api.libraxis.cloud/v1/audio/transcribe") + ); + assert_eq!( + live_websocket_endpoint("http://127.0.0.1:8000/v1/audio/transcriptions").as_deref(), + Some("ws://127.0.0.1:8000/v1/audio/transcribe") + ); + assert_eq!( + live_websocket_endpoint("ws://127.0.0.1:8446/v1/audio/transcribe").as_deref(), + Some("ws://127.0.0.1:8446/v1/audio/transcribe") + ); + } + + #[test] + fn loopback_live_socket_needs_no_credential() { + let config = Config { + stt_endpoint: Some("http://localhost:8000/v1/audio/transcriptions".into()), + stt_api_key: None, + ..Config::default() + }; + assert!(matches!( + gateway_session_availability(&config), + GatewaySessionAvailability::Ready(_) + )); + } } diff --git a/core/asr_session/cloud.rs b/core/asr_session/cloud.rs index 1a3e22b7..db30bf88 100644 --- a/core/asr_session/cloud.rs +++ b/core/asr_session/cloud.rs @@ -1,9 +1,11 @@ -//! Dedicated live cloud transport for the normalized Libraxis gateway contract. +//! Dedicated live cloud transport for the Libraxis Voice Lab WebSocket. //! -//! This module is intentionally a second path beside the legacy whole-file -//! WebSocket uploader. It owns a live session: one start message, bounded PCM -//! frames, normalized receive events, and a bounded end/drain. It does not own -//! recorder wiring, consent, provider selection, or a vendor protocol. +//! This module owns normal live capture: a `config` message, bounded base64 PCM +//! `chunk` messages, periodic `flush`, and a bounded `end`/drain. The receive +//! adapter converts Voice Lab events into Codescribe's normalized vocabulary. +//! Whole-file multipart upload lives outside this session and is reserved for +//! explicit retranscribe actions. This module does not own recorder wiring, +//! consent, or provider selection. //! //! Provider ordering is evidence, not authority. Transcript revisions are //! compared only inside their utterance, duplicates and stale revisions are @@ -15,12 +17,13 @@ use std::fmt; use std::thread::JoinHandle; use std::time::{Duration, Instant}; +use base64::{Engine as _, engine::general_purpose::STANDARD as BASE64}; use futures_util::{SinkExt, StreamExt}; use serde::{Deserialize, Serialize}; use tokio::sync::mpsc; use tokio::time::timeout; use tokio_tungstenite::tungstenite::client::IntoClientRequest; -use tokio_tungstenite::tungstenite::http::header::{AUTHORIZATION, HeaderValue}; +use tokio_tungstenite::tungstenite::http::header::{AUTHORIZATION, HeaderName, HeaderValue}; use tokio_tungstenite::tungstenite::{Error as WebSocketError, Message}; use tokio_tungstenite::{MaybeTlsStream, WebSocketStream, connect_async}; @@ -89,14 +92,15 @@ impl CloudSessionLimits { } } -/// Gateway endpoint and short-lived bearer minted outside the desktop transport. +/// Live endpoint and its endpoint-owned authentication credential. /// /// Its `Debug` representation is deliberately content-free. Endpoints can carry /// signed query parameters and bearer values are credentials; neither belongs /// in logs, panic output, or telemetry. pub struct GatewayConnection { endpoint: String, - bearer: String, + credential: String, + auth_mode: crate::stt::tail_provider::SttAuthMode, } impl GatewayConnection { @@ -106,10 +110,10 @@ impl GatewayConnection { /// allowed but remains redacted by the type's `Debug` implementation. pub fn new( endpoint: impl Into, - bearer: impl Into, + credential: impl Into, ) -> Result { let endpoint = endpoint.into(); - let bearer = bearer.into(); + let credential = credential.into(); let parsed = reqwest::Url::parse(&endpoint).map_err(|_| AsrErrorKind::Protocol)?; let host = parsed .host_str() @@ -117,17 +121,24 @@ impl GatewayConnection { .ok_or(AsrErrorKind::Protocol)?; let encrypted = parsed.scheme() == "wss"; let loopback = matches!(host, "localhost" | "127.0.0.1" | "::1"); + let auth_mode = crate::stt::tail_provider::stt_auth_mode(&endpoint); if (!encrypted && !(parsed.scheme() == "ws" && loopback)) || !parsed.username().is_empty() || parsed.password().is_some() - || bearer.trim().is_empty() + || (auth_mode != crate::stt::tail_provider::SttAuthMode::Unauthenticated + && credential.trim().is_empty()) { return Err(AsrErrorKind::Protocol); } - let authorization = format!("Bearer {bearer}"); - HeaderValue::from_str(&authorization).map_err(|_| AsrErrorKind::Protocol)?; - Ok(Self { endpoint, bearer }) + if auth_mode != crate::stt::tail_provider::SttAuthMode::Unauthenticated { + HeaderValue::from_str(credential.trim()).map_err(|_| AsrErrorKind::Protocol)?; + } + Ok(Self { + endpoint, + credential, + auth_mode, + }) } } @@ -136,7 +147,7 @@ impl fmt::Debug for GatewayConnection { formatter .debug_struct("GatewayConnection") .field("endpoint", &"[REDACTED]") - .field("bearer", &"[REDACTED]") + .field("credential", &"[REDACTED]") .finish() } } @@ -205,6 +216,7 @@ impl GatewayPcmFrame { self.pcm_s16le.len() } + #[cfg(test)] fn into_wire_bytes(self) -> Vec { let mut wire = Vec::with_capacity(8 + self.pcm_s16le.len()); wire.extend_from_slice(&self.sequence_id.to_be_bytes()); @@ -213,6 +225,115 @@ impl GatewayPcmFrame { } } +/// Stateful adapter from the proven Voice Lab wire into Codescribe's strict +/// normalized event vocabulary. +struct VoiceLabReceiveState { + session_id: String, + next_event_id: u64, + utterance_id: u64, + revision: u64, +} + +impl VoiceLabReceiveState { + fn new(session_id: String) -> Self { + Self { + session_id, + next_event_id: 1, + utterance_id: 1, + revision: 0, + } + } + + fn event_id(&mut self) -> Result { + let id = self.next_event_id; + self.next_event_id = self + .next_event_id + .checked_add(1) + .ok_or(AsrErrorKind::Protocol)?; + Ok(format!("voice-lab-{id}")) + } + + fn adapt(&mut self, text: &str) -> Result, AsrErrorKind> { + if let Ok(event) = serde_json::from_str::(text) { + return Ok(Some(event)); + } + let value: serde_json::Value = + serde_json::from_str(text).map_err(|_| AsrErrorKind::Protocol)?; + let message_type = value + .get("type") + .and_then(serde_json::Value::as_str) + .ok_or(AsrErrorKind::Protocol)?; + match message_type { + "ack" | "ready" => Ok(None), + "transcript.partial" | "transcript" => { + let text = voice_lab_text(&value)?; + self.revision = self.revision.checked_add(1).ok_or(AsrErrorKind::Protocol)?; + let event_id = self.event_id()?; + Ok(Some(GatewayEvent::Partial { + event_id, + session_id: self.session_id.clone(), + utterance_id: self.utterance_id, + revision: self.revision, + text, + start_ms: None, + end_ms: None, + })) + } + "transcript.final" => { + let text = voice_lab_text(&value)?; + self.revision = self.revision.checked_add(1).ok_or(AsrErrorKind::Protocol)?; + let event_id = self.event_id()?; + let event = GatewayEvent::Final { + event_id, + session_id: self.session_id.clone(), + utterance_id: self.utterance_id, + revision: self.revision, + text, + start_ms: None, + end_ms: None, + }; + self.utterance_id = self + .utterance_id + .checked_add(1) + .ok_or(AsrErrorKind::Protocol)?; + self.revision = 0; + Ok(Some(event)) + } + "error" => { + let code = match value.get("code").and_then(serde_json::Value::as_str) { + Some("auth" | "unauthorized" | "forbidden") => GatewayErrorCode::Auth, + Some("quota" | "payment_required") => GatewayErrorCode::Quota, + Some("rate_limited") => GatewayErrorCode::RateLimited, + Some("timeout") => GatewayErrorCode::Timeout, + Some("backpressure") => GatewayErrorCode::Backpressure, + _ => GatewayErrorCode::Protocol, + }; + let event_id = self.event_id()?; + Ok(Some(GatewayEvent::Error { + event_id, + session_id: self.session_id.clone(), + utterance_id: self.utterance_id, + code, + })) + } + "end" | "session.ended" => Ok(Some(GatewayEvent::SessionEnded { + session_id: self.session_id.clone(), + })), + _ => Err(AsrErrorKind::Protocol), + } + } +} + +fn voice_lab_text(value: &serde_json::Value) -> Result { + value + .get("text") + .and_then(serde_json::Value::as_str) + .map(str::trim) + .filter(|text| !text.is_empty()) + .map(str::to_string) + .ok_or(AsrErrorKind::Protocol) +} + impl fmt::Debug for GatewayPcmFrame { fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { formatter @@ -379,9 +500,9 @@ enum WorkerSignal { Closed, } -/// Real bounded WebSocket actor for the normalized gateway contract. +/// Real bounded WebSocket actor for the Voice Lab wire contract. /// -/// The socket and bearer live on a dedicated current-thread Tokio runtime. +/// The socket and credential live on a dedicated current-thread Tokio runtime. /// The synchronous provider side only performs bounded `try_send`/`try_recv` /// channel operations; it never performs network I/O on the audio callback. pub struct GatewayWebSocketTransport { @@ -432,7 +553,7 @@ impl CloudGatewayTransport for GatewayWebSocketTransport { let (command_tx, command_rx) = mpsc::channel(self.limits.outbound_queue_capacity); let (event_tx, event_rx) = mpsc::channel(self.limits.inbound_queue_capacity); - // Move the short-lived bearer into the socket worker. The synchronous + // Move the credential into the socket worker. The synchronous // provider retains no spare credential copy after session start. let connection = self.connection.take().ok_or(AsrErrorKind::Protocol)?; let limits = self.limits; @@ -546,9 +667,22 @@ async fn run_gateway_socket( .as_str() .into_client_request() .map_err(|_| AsrErrorKind::Protocol)?; - let authorization = HeaderValue::from_str(&format!("Bearer {}", connection.bearer)) - .map_err(|_| AsrErrorKind::Protocol)?; - request.headers_mut().insert(AUTHORIZATION, authorization); + match connection.auth_mode { + crate::stt::tail_provider::SttAuthMode::Unauthenticated => {} + crate::stt::tail_provider::SttAuthMode::Bearer => { + let authorization = + HeaderValue::from_str(&format!("Bearer {}", connection.credential.trim())) + .map_err(|_| AsrErrorKind::Protocol)?; + request.headers_mut().insert(AUTHORIZATION, authorization); + } + crate::stt::tail_provider::SttAuthMode::ApiKey => { + let value = HeaderValue::from_str(connection.credential.trim()) + .map_err(|_| AsrErrorKind::Protocol)?; + request + .headers_mut() + .insert(HeaderName::from_static("x-api-key"), value); + } + } let connected = timeout(limits.connect_timeout, connect_async(request)) .await @@ -556,7 +690,13 @@ async fn run_gateway_socket( .map_err(|error| classify_socket_error(&error))?; let (mut socket, _) = connected; - let start = serde_json::to_string(&config).map_err(|_| AsrErrorKind::Protocol)?; + // Proven Voice Lab wire: credentials stay in the WebSocket handshake, + // never in the JSON body. + let start = serde_json::json!({ + "type": "config", + "language": config.locale.as_deref().unwrap_or("pl"), + }) + .to_string(); send_socket_message( &mut socket, Message::Text(start.into()), @@ -564,25 +704,47 @@ async fn run_gateway_socket( ) .await?; + let mut receive_state = VoiceLabReceiveState::new(config.session_id.clone()); + let mut flush = tokio::time::interval_at( + tokio::time::Instant::now() + Duration::from_millis(2_500), + Duration::from_millis(2_500), + ); loop { tokio::select! { command = command_rx.recv() => { match command { Some(GatewayCommand::Pcm(frame)) => { + let chunk = serde_json::json!({ + "type": "chunk", + "audio_base64": BASE64.encode(&frame.pcm_s16le), + "sample_rate": config.audio.sample_rate_hz, + "encoding": "pcm16", + }).to_string(); send_socket_message( &mut socket, - Message::Binary(frame.into_wire_bytes().into()), + Message::Text(chunk.into()), limits.send_timeout, ).await?; } Some(GatewayCommand::End) => { - let end = serde_json::json!({"type": "session.end"}).to_string(); + let flush = serde_json::json!({"type": "flush"}).to_string(); + send_socket_message( + &mut socket, + Message::Text(flush.into()), + limits.send_timeout, + ).await?; + let end = serde_json::json!({"type": "end"}).to_string(); send_socket_message( &mut socket, Message::Text(end.into()), limits.send_timeout, ).await?; - return drain_gateway_tail(&mut socket, limits, event_tx).await; + return drain_gateway_tail( + &mut socket, + limits, + event_tx, + &mut receive_state, + ).await; } Some(GatewayCommand::Abort) | None => { let _ = socket.close(None).await; @@ -590,8 +752,22 @@ async fn run_gateway_socket( } } } + _ = flush.tick() => { + let message = serde_json::json!({"type": "flush"}).to_string(); + send_socket_message( + &mut socket, + Message::Text(message.into()), + limits.send_timeout, + ).await?; + } incoming = socket.next() => { - if forward_gateway_message(incoming, &mut socket, limits.send_timeout, event_tx).await? { + if forward_gateway_message( + incoming, + &mut socket, + limits.send_timeout, + event_tx, + &mut receive_state, + ).await? { return Err(AsrErrorKind::Transport); } } @@ -603,13 +779,22 @@ async fn drain_gateway_tail( socket: &mut WebSocketStream>, limits: CloudSessionLimits, event_tx: &mpsc::Sender, + receive_state: &mut VoiceLabReceiveState, ) -> Result<(), AsrErrorKind> { let deadline = tokio::time::Instant::now() + limits.close_timeout; loop { let incoming = tokio::time::timeout_at(deadline, socket.next()) .await .map_err(|_| AsrErrorKind::Transport)?; - if forward_gateway_message(incoming, socket, limits.send_timeout, event_tx).await? { + if forward_gateway_message( + incoming, + socket, + limits.send_timeout, + event_tx, + receive_state, + ) + .await? + { return Ok(()); } } @@ -620,11 +805,13 @@ async fn forward_gateway_message( socket: &mut WebSocketStream>, send_timeout: Duration, event_tx: &mpsc::Sender, + receive_state: &mut VoiceLabReceiveState, ) -> Result { match incoming { Some(Ok(Message::Text(text))) => { - let event: GatewayEvent = - serde_json::from_str(text.as_ref()).map_err(|_| AsrErrorKind::Protocol)?; + let Some(event) = receive_state.adapt(text.as_ref())? else { + return Ok(false); + }; let ended = matches!(event, GatewayEvent::SessionEnded { .. }); event_tx .send(WorkerSignal::Event(event)) @@ -1285,6 +1472,54 @@ mod tests { assert!(serde_json::from_str::(vendor_specific).is_err()); } + #[test] + fn voice_lab_wire_is_adapted_without_credential_fields() { + let mut state = VoiceLabReceiveState::new(session_id().to_string()); + assert_eq!( + state.adapt(r#"{"type":"ack","received_bytes":320}"#), + Ok(None) + ); + + let partial = state + .adapt(r#"{"type":"transcript.partial","text":"pierwszy"}"#) + .expect("valid partial") + .expect("partial event"); + assert!(matches!( + partial, + GatewayEvent::Partial { + utterance_id: 1, + revision: 1, + ref text, + .. + } if text == "pierwszy" + )); + + let final_event = state + .adapt(r#"{"type":"transcript.final","text":"pierwszy final"}"#) + .expect("valid final") + .expect("final event"); + assert!(matches!( + final_event, + GatewayEvent::Final { + utterance_id: 1, + revision: 2, + ref text, + .. + } if text == "pierwszy final" + )); + assert!(matches!( + state + .adapt(r#"{"type":"transcript.final","text":"drugi"}"#) + .expect("second final") + .expect("second event"), + GatewayEvent::Final { + utterance_id: 2, + revision: 1, + .. + } + )); + } + #[test] fn local_sequence_is_global_across_reordered_utterances_and_duplicates() { let duplicate = partial("u2-r1", 2, 1, "drugi"); @@ -1511,5 +1746,9 @@ mod tests { GatewayConnection::new(format!("{plain_websocket}127.0.0.1:9000/live"), "token") .is_ok() ); + assert!( + GatewayConnection::new(format!("{plain_websocket}127.0.0.1:9000/live"), "").is_ok(), + "loopback live STT must not require a key" + ); } } diff --git a/core/asr_session/mod.rs b/core/asr_session/mod.rs index 7947c83c..84b451d1 100644 --- a/core/asr_session/mod.rs +++ b/core/asr_session/mod.rs @@ -17,11 +17,12 @@ //! Layer 0 canvas choice and Layer 1 refiner mode on two separate axes. //! - [`fake`] — a deterministic in-memory provider for tests and later cuts. //! - [`cloud`] — the dedicated live gateway session, bounded PCM transport, -//! and Codescribe-owned stream-global event sequencing. +//! Voice Lab wire adapter, and Codescribe-owned stream-global event +//! sequencing. //! - [`local_helper`] — the provider-compatible, injected child-process //! boundary whose confirmed exit is the local-weight reclaim authority. //! - [`bootstrap`] — the recording-start join between persisted mode/consent -//! truth and a validated, short-lived gateway session. +//! truth and a validated live endpoint. //! - [`recorder`] — the per-recording lane the live session drives: injected //! [`recorder::Layer1Decision`], bounded non-blocking PCM fan-out, volatile //! partial draft, and typed degrade paths that always land on canvas + @@ -32,15 +33,14 @@ //! //! ## What deliberately does NOT live here //! -//! No settings UI, gateway session-mint client, vendor protocol, or local -//! model. The live socket consumes only a normalized gateway contract; the -//! mode/consent *records* live in `crate::config::cloud_asr` (the settings -//! brain), while [`bootstrap`] joins that truth to the recorder. +//! No settings UI or local model. The Voice Lab wire is isolated behind the +//! normalized provider contract; the mode/consent *records* live in +//! `crate::config::cloud_asr` (the settings brain), while [`bootstrap`] joins +//! that truth to the recorder. //! -//! The existing whole-file `client::transcribe_cloud` / `transcribe_websocket` -//! API is **outside** this contract. It uploads one completed recording and is -//! a stop/recovery path, not a live session — routing it through this interface -//! and calling the result "live" is precisely the confusion this seam prevents. +//! The existing whole-file `client::transcribe_cloud` API is **outside** this +//! contract. It uploads one completed recording only for explicit retranscribe +//! surfaces — normal recording never routes that file pass through this seam. //! //! ## Doctrine encoded in the types //! @@ -74,7 +74,9 @@ pub mod recorder; #[cfg(test)] mod tests; -pub use bootstrap::{GatewaySessionAvailability, layer1_decision_for_recording}; +pub use bootstrap::{ + GatewaySessionAvailability, gateway_session_availability, layer1_decision_for_recording, +}; pub use cloud::{ CloudGatewayTransport, CloudSessionLimits, CloudSessionTelemetry, GatewayConnection, GatewayErrorCode, GatewayEvent, GatewayPcmFrame, GatewaySessionConfig, GatewayTransportPoll, diff --git a/core/audio/capture_receipt.rs b/core/audio/capture_receipt.rs index fe02ac8b..46bdc50d 100644 --- a/core/audio/capture_receipt.rs +++ b/core/audio/capture_receipt.rs @@ -32,6 +32,20 @@ pub const CLIP_ABS: f32 = 0.99; static LAST_RECEIPT: OnceLock>> = OnceLock::new(); static LAST_OPEN_PATH: OnceLock>> = OnceLock::new(); +static SESSION_ENERGY: OnceLock> = OnceLock::new(); + +/// One capture hop on the session PCM axis. Intensity lives here, not on tokens. +#[derive(Debug, Clone, Copy)] +struct EnergyHop { + sample_start: u64, + sample_end: u64, + rms: f32, +} + +#[derive(Debug, Default)] +struct SessionEnergyClock { + hops: Vec, +} fn last_receipt_slot() -> &'static Mutex> { LAST_RECEIPT.get_or_init(|| Mutex::new(None)) @@ -41,6 +55,62 @@ fn last_open_path_slot() -> &'static Mutex> { LAST_OPEN_PATH.get_or_init(|| Mutex::new(None)) } +fn session_energy_slot() -> &'static Mutex { + SESSION_ENERGY.get_or_init(|| Mutex::new(SessionEnergyClock::default())) +} + +/// Open a new capture epoch's energy ladder. Call at live-session start only. +pub fn begin_session_energy_clock() { + *session_energy_slot() + .lock() + .unwrap_or_else(|e| e.into_inner()) = SessionEnergyClock::default(); +} + +fn record_session_energy_hop(sample_start: u64, sample_end: u64, rms: f32) { + if sample_end <= sample_start || !rms.is_finite() || rms < 0.0 { + return; + } + session_energy_slot() + .lock() + .unwrap_or_else(|e| e.into_inner()) + .hops + .push(EnergyHop { + sample_start, + sample_end, + rms, + }); +} + +/// Mean RMS of hops overlapping `[sample_start, sample_end)`, as dBFS. +/// +/// Missing hops, inverted ranges, or a silent window return `None`. This is +/// intensity on the PCM clock — not a confidence score. +pub fn session_energy_db(sample_start: u64, sample_end: u64) -> Option { + if sample_end <= sample_start { + return None; + } + let hops = session_energy_slot() + .lock() + .unwrap_or_else(|e| e.into_inner()); + let mut weighted = 0.0_f64; + let mut covered = 0.0_f64; + for hop in &hops.hops { + let lo = hop.sample_start.max(sample_start); + let hi = hop.sample_end.min(sample_end); + if hi <= lo { + continue; + } + let width = (hi - lo) as f64; + weighted += f64::from(hop.rms) * width; + covered += width; + } + if covered <= 0.0 { + return None; + } + let db = linear_to_db((weighted / covered) as f32); + db.is_finite().then_some(db) +} + /// Remember the live capture path (device / rate / channels) without a new TCC prompt. pub fn publish_open_capture_path(meta: CapturePathMeta) { *last_open_path_slot() @@ -195,7 +265,9 @@ impl CaptureLevelAccumulator { sum_sq += f64::from(x) * f64::from(x); } let rms = (sum_sq / samples.len() as f64).sqrt() as f32; + let sample_start = self.sample_count; self.sample_count += samples.len() as u64; + record_session_energy_hop(sample_start, self.sample_count, rms); self.digital_zero_samples += zeros; self.clipping_samples += clips; if peak > self.peak_linear { @@ -524,4 +596,22 @@ mod tests { "emitting the WARN must not change the terminal class" ); } + + #[test] + fn session_energy_db_is_pcm_range_intensity() { + begin_session_energy_clock(); + let mut acc = CaptureLevelAccumulator::new(); + acc.push_samples(&vec![0.0; 160]); + acc.push_samples(&vec![0.1; 160]); + acc.push_samples(&vec![0.0; 160]); + assert!( + session_energy_db(0, 160).is_none(), + "digital-zero hops have no finite dBFS" + ); + let speech = session_energy_db(160, 320).expect("speech hop"); + assert!(speech.is_finite()); + assert!(session_energy_db(480, 640).is_none()); + begin_session_energy_clock(); + assert!(session_energy_db(160, 320).is_none()); + } } diff --git a/core/build.rs b/core/build.rs index ec2efba9..ec730687 100644 --- a/core/build.rs +++ b/core/build.rs @@ -61,9 +61,9 @@ const LOCAL_INSTALL_ENV: &str = "CODESCRIBE_LOCAL_INSTALL"; /// /// Each asset has its own policy, and they are not symmetrical: Silero VAD is /// non-negotiable and its absence panics; MiniLM, Whisper and TTS are opt-in. -/// A requested-but-missing model degrades to a `cargo:warning` -/// and a runtime lookup rather than failing the build — only the committed -/// Silero file is treated as a repo invariant. +/// MiniLM and TTS degrade to a `cargo:warning` when requested but missing. +/// Whisper embed is fail-closed: `CODESCRIBE_EMBED_WHISPER=1` without a +/// complete snapshot must not produce a `_full` artifact that is actually slim. fn main() { println!("cargo:rerun-if-changed=Cargo.toml"); println!("cargo:rerun-if-env-changed=CODESCRIBE_EMBED_MODEL"); @@ -149,15 +149,15 @@ fn main() { .expect("Failed to write embedded_model_data.rs"); println!("cargo:rustc-cfg=embed_model"); } else if embed_whisper_requested && !no_embed && !model_exists { - println!( - "cargo:warning=Whisper model not found for embedding: {}", - model_path.display() - ); - println!( - "cargo:warning=Download with: hf download {}", - DEFAULT_WHISPER_REPO + panic!( + "CODESCRIBE_EMBED_WHISPER=1 but no complete Whisper snapshot at {}. \ +Need config.json + tokenizer.json + mel_filters.npz + weights/model.safetensors. \ +The HF repo {} is weights-only; compose it with `make download-model` into \ +~/.codescribe/models/{}, or set CODESCRIBE_MODEL_PATH to that directory.", + model_path.display(), + DEFAULT_WHISPER_REPO, + DEFAULT_MODEL_NAME ); - println!("cargo:warning=Falling back to runtime Whisper lookup for this build"); } // TTS model embedding (optional, via CODESCRIBE_EMBED_TTS=1) @@ -426,32 +426,66 @@ fn resolve_embed_model_path(manifest_dir: &str, embed_model: &str) -> PathBuf { Path::new(manifest_dir).join("models").join(embed_model) } +/// True when a directory can be baked into the fat SKU. +/// +/// The default HF Whisper repo is weights-only. `make download-model` composes +/// tokenizer + mel into `~/.codescribe/models/`. Incomplete snapshots +/// must not win over that composed tree. +fn whisper_dir_complete(path: &Path) -> bool { + let weights = if path.join("weights.safetensors").exists() { + path.join("weights.safetensors") + } else { + path.join("model.safetensors") + }; + path.join("config.json").exists() + && path.join("tokenizer.json").exists() + && path.join("mel_filters.npz").exists() + && weights.exists() +} + /// Locate the Whisper snapshot to embed. /// -/// `CODESCRIBE_MODEL_PATH` wins when it points at a real snapshot — it is the -/// explicit operator override. Otherwise a repo-id-shaped name is looked up in -/// the HF cache, the default name falls back to the default repo, and anything -/// left resolves as a plain path. +/// `CODESCRIBE_MODEL_PATH` wins when it is a complete snapshot. The composed +/// `~/.codescribe/models` tree is next — that is what `make download-model` +/// writes. An HF cache hit is used only when it already has tokenizer + mel. fn resolve_whisper_embed_model_path( manifest_dir: &str, embed_model: &str, default_repo: &str, ) -> PathBuf { - // CODESCRIBE_MODEL_PATH takes priority — explicit user override if let Ok(model_path) = env::var("CODESCRIBE_MODEL_PATH") { let p = PathBuf::from(model_path.trim()); - if p.join("config.json").exists() { + if whisper_dir_complete(&p) { return p; } } + if let Some(home) = dirs::home_dir() { + let composed = home.join(".codescribe").join("models").join(embed_model); + if whisper_dir_complete(&composed) { + return composed; + } + if embed_model == DEFAULT_MODEL_NAME { + let default_composed = home + .join(".codescribe") + .join("models") + .join(DEFAULT_MODEL_NAME); + if whisper_dir_complete(&default_composed) { + return default_composed; + } + } + } if embed_model.contains('/') { if let Some(snapshot) = find_hf_snapshot(embed_model) { - return snapshot; + if whisper_dir_complete(&snapshot) { + return snapshot; + } } } else if embed_model == DEFAULT_MODEL_NAME && let Some(snapshot) = find_hf_snapshot(default_repo) { - return snapshot; + if whisper_dir_complete(&snapshot) { + return snapshot; + } } resolve_embed_model_path(manifest_dir, embed_model) } diff --git a/core/config/models.rs b/core/config/models.rs index 22696858..68d56fb6 100644 --- a/core/config/models.rs +++ b/core/config/models.rs @@ -65,6 +65,20 @@ fn is_complete_whisper_model_dir(path: &Path) -> bool { .any(|name| path.join(name).exists()) } +/// Whether a candidate models root owns at least one complete Whisper model. +/// +/// A bundled `Resources/models` directory may contain only another model +/// family (for example the semantic embedder). Treating mere directory +/// existence as Whisper ownership shadows the user-installed fp16 model and +/// incorrectly drops runtime resolution to the legacy q8 cache. +fn models_root_contains_complete_whisper_model(path: &Path) -> bool { + fs::read_dir(path).is_ok_and(|entries| { + entries + .filter_map(std::result::Result::ok) + .any(|entry| is_complete_whisper_model_dir(&entry.path())) + }) +} + /// Find a complete Hugging Face cache snapshot for a model reference. /// /// A reference containing `/` is treated as a repo id and looked up directly. @@ -144,7 +158,7 @@ impl ModelManager { // 1. Bundled .app: Contents/MacOS/binary -> Contents/Resources/models/ let bundled_path = exe_dir.join("../Resources/models"); - if bundled_path.exists() { + if models_root_contains_complete_whisper_model(&bundled_path) { return bundled_path .canonicalize() .context("Failed to canonicalize bundled models path"); @@ -156,7 +170,7 @@ impl ModelManager { // means "directory with ALL models" — hijacking it from tests sends // runtime Whisper resolution to the wrong place. let dev_path = exe_dir.join("../../models"); - if dev_path.exists() { + if models_root_contains_complete_whisper_model(&dev_path) { return dev_path .canonicalize() .context("Failed to canonicalize dev models path"); @@ -164,7 +178,7 @@ impl ModelManager { // 3. Direct ./models/ (running from repo root) let local_path = PathBuf::from("../../models"); - if local_path.exists() { + if models_root_contains_complete_whisper_model(&local_path) { return local_path .canonicalize() .context("Failed to canonicalize local models path"); @@ -615,6 +629,24 @@ mod tests { fs::write(path.join("model.safetensors"), "weights").unwrap(); } + /// A bundle containing only the semantic embedder must not claim ownership + /// of Whisper resolution and hide the user's complete fp16 install. + #[test] + fn embedder_only_models_root_does_not_qualify_as_whisper_root() { + let temp_dir = TempDir::new().unwrap(); + let models_dir = temp_dir.path().join("models"); + let embedder = models_dir.join("embedder"); + fs::create_dir_all(&embedder).unwrap(); + fs::write(embedder.join("config.json"), "{}").unwrap(); + fs::write(embedder.join("tokenizer.json"), "{}").unwrap(); + fs::write(embedder.join("model.safetensors"), "weights").unwrap(); + + assert!(!models_root_contains_complete_whisper_model(&models_dir)); + + create_complete_whisper_model(&models_dir.join(DEFAULT_MODEL)); + assert!(models_root_contains_complete_whisper_model(&models_dir)); + } + /// Smoke: `list_models` succeeds against the live models dir. #[test] #[serial] diff --git a/core/llm/account_auth/device_code.rs b/core/llm/account_auth/device_code.rs index 13dd6038..579ff94d 100644 --- a/core/llm/account_auth/device_code.rs +++ b/core/llm/account_auth/device_code.rs @@ -27,7 +27,6 @@ use crate::llm::account_auth::pkce::PkceCodes; use crate::llm::account_auth::server::exchange_code_for_tokens; use crate::llm::account_auth::{ AccountAuthError, AccountTokens, issuer_for, provider_oauth_config, store_account_tokens, - verify_responses_write_access, }; use crate::llm::provider::ProviderKind; @@ -317,7 +316,6 @@ pub async fn complete_device_code_login( match config.provider { ProviderKind::XaiResponses => { let tokens = poll_xai_device_token(&client, config, device_code).await?; - verify_responses_write_access(config.provider, &tokens.access_token).await?; store_account_tokens(config.provider, &tokens) } _ => { @@ -347,7 +345,6 @@ pub async fn complete_device_code_login( &code_resp.authorization_code, ) .await?; - verify_responses_write_access(config.provider, &tokens.access_token).await?; store_account_tokens(config.provider, &tokens) } } diff --git a/core/llm/account_auth/mod.rs b/core/llm/account_auth/mod.rs index 5cd3688d..d4a3d7a7 100644 --- a/core/llm/account_auth/mod.rs +++ b/core/llm/account_auth/mod.rs @@ -508,12 +508,10 @@ pub fn store_account_tokens( save_key(account, &payload).map_err(|error| AccountAuthError::Storage(error.to_string())) } -/// Official Responses endpoint a freshly minted account token must be able to -/// write to. `None` ⇒ no probe exists for this provider and sign-in verifies -/// nothing extra. Scoped to OpenAI: its login above requests identity scopes -/// only, and a token without `api.responses.write` still exchanges cleanly — -/// the field failure this probe exists for (five raw 401s in one morning, -/// 2026-08-14). +/// Optional official Responses URL for a *capability* probe. `None` ⇒ this +/// provider has no such probe. Sign-in must not call this: ChatGPT OAuth is +/// an identity row; Responses write is a separate lane/credential. Coupling +/// them (2026-08-14) made Codex public tokens unable to save a session. fn responses_probe_endpoint(provider: ProviderKind) -> Option { match provider { // The env override exists for hermetic tests (and emergency ops): @@ -534,14 +532,11 @@ fn responses_probe_endpoint(provider: ProviderKind) -> Option { /// it at a mock; production leaves it unset (official endpoint). pub const RESPONSES_PROBE_URL_ENV: &str = "CODESCRIBE_RESPONSES_PROBE_URL"; -/// Verify a just-exchanged account token can actually use the Responses API -/// BEFORE it is persisted as "connected". +/// Classify whether a token can write the official Responses API. /// -/// The probe spends no tokens: an empty JSON body is authorized before it is -/// validated, so a healthy token answers 400 (validation) while a -/// scope-starved one answers 401. Only that definitive 401 fails the login; -/// transport errors stay fail-open — the exchange itself just proved the -/// network, and a flaky probe must not lock out an otherwise valid sign-in. +/// Not a sign-in gate. Persist is provider-local identity; this probe is a +/// later row/lane Test. Empty body: 400 = authorized, 401 = no write scope. +/// Transport errors stay fail-open. pub async fn verify_responses_write_access( provider: ProviderKind, access_token: &str, @@ -801,9 +796,8 @@ mod tests { (EnvGuard::set_path("CODESCRIBE_DATA_DIR", dir.path()), dir) } - /// The 401 field failure (2026-08-14): a scope-starved token must fail the - /// login BEFORE persisting, while a healthy token's 400 validation answer - /// passes — the probe authorizes before it validates and spends nothing. + /// Capability probe only: 401 means no Responses write; 400 means the + /// token authorized. Login persist does not call this. #[tokio::test] async fn responses_probe_rejects_401_and_passes_validation_400() { let mut starved_server = mockito::Server::new_async().await; @@ -816,7 +810,7 @@ mod tests { let starved_url = format!("{}/v1/responses", starved_server.url()); let error = verify_responses_write_access_at(&starved_url, "starved-token") .await - .expect_err("401 must reject the sign-in"); + .expect_err("401 classifies as no Responses write"); assert!(error.to_string().contains("api.responses.write")); starved.assert_async().await; diff --git a/core/llm/account_auth/server.rs b/core/llm/account_auth/server.rs index a84aaa90..1a6b57fa 100644 --- a/core/llm/account_auth/server.rs +++ b/core/llm/account_auth/server.rs @@ -30,7 +30,7 @@ use tiny_http::{Header, Request, Response, Server, StatusCode}; use crate::llm::account_auth::pkce::{PkceCodes, generate_pkce}; use crate::llm::account_auth::{ AccountAuthError, AccountTokens, LoginFlow, ProviderOAuthConfig, TokenRequestEncoding, - issuer_for, provider_oauth_config, store_account_tokens, verify_responses_write_access, + issuer_for, provider_oauth_config, store_account_tokens, }; use crate::llm::provider::ProviderKind; @@ -333,18 +333,10 @@ async fn process_request( .await { Ok(tokens) => { - // A token that cannot write to the Responses API must never be - // shown as "connected" — the first real prompt would answer a - // raw 401 (field failure, 2026-08-14). Reject BEFORE persisting. - if let Err(error) = - verify_responses_write_access(opts.provider, &tokens.access_token).await - { - return HandledRequest::ResponseAndExit { - headers: Vec::new(), - body: format!("Sign-in rejected: {error}").into_bytes(), - result: Err(error), - }; - } + // Persist identity for this vendor row only. A Responses-write + // probe against api.openai.com is a different provider/lane + // capability and must not veto ChatGPT sign-in (develop/main + // store here; Codex public tokens never carry api.responses.write). if let Err(error) = store_account_tokens(opts.provider, &tokens) { return HandledRequest::Response( Response::from_string(format!("Unable to persist account tokens: {error}")) @@ -620,20 +612,6 @@ mod tests { let _tokens = EnvGuard::unset(OPENAI_ACCOUNT_TOKENS_ACCOUNT); let mut issuer = mockito::Server::new_async().await; - // Healthy scope: the probe's empty body earns a 400 validation answer. - // Pinned to the mock — an unpinned probe would leak to api.openai.com. - let probe = issuer - .mock("POST", "/v1/responses") - .match_header("authorization", "Bearer account-access") - .with_status(400) - .with_body(r#"{"error":"invalid input"}"#) - .expect(1) - .create_async() - .await; - let _probe_url = EnvGuard::set( - crate::llm::account_auth::RESPONSES_PROBE_URL_ENV, - &format!("{}/v1/responses", issuer.url()), - ); let _mock = issuer .mock("POST", "/oauth/token") .match_body(mockito::Matcher::AllOf(vec![ @@ -673,15 +651,14 @@ mod tests { load_account_tokens(ProviderKind::OpenAiResponses).expect("tokens were stored"); assert_eq!(stored.access_token, "account-access"); assert_eq!(stored.refresh_token.as_deref(), Some("account-refresh")); - probe.assert_async().await; } - /// The field failure (five raw 401s, 2026-08-14): a token that exchanges - /// cleanly but cannot write to the Responses API must FAIL the login and - /// must never be persisted as "connected". + /// Provider independence: ChatGPT identity stores after a clean exchange. + /// Official Responses write is a different row/lane; login must not probe it + /// (Codex public tokens never carry `api.responses.write`). #[tokio::test] #[serial] - async fn scope_starved_token_fails_login_and_is_not_stored() { + async fn chatgpt_identity_token_stores_without_responses_write() { let _disable = EnvGuard::set("CODESCRIBE_DISABLE_KEYCHAIN", "1"); let _tokens = EnvGuard::unset(OPENAI_ACCOUNT_TOKENS_ACCOUNT); @@ -695,16 +672,9 @@ mod tests { .await; let probe = issuer .mock("POST", "/v1/responses") - .match_header("authorization", "Bearer scope-starved") - .with_status(401) - .with_body(r#"{"error":"Missing scopes: api.responses.write"}"#) - .expect(1) + .expect(0) .create_async() .await; - let _probe_url = EnvGuard::set( - crate::llm::account_auth::RESPONSES_PROBE_URL_ENV, - &format!("{}/v1/responses", issuer.url()), - ); let mut opts = openai_opts("client"); opts.issuer = issuer.url(); @@ -716,16 +686,15 @@ mod tests { "http://127.0.0.1:{}/auth/callback?code=auth-code&state=starved-state", login.actual_port ); - let _response = reqwest::get(&callback).await.expect("callback request"); - let error = login + let response = reqwest::get(&callback).await.expect("callback request"); + assert!(response.status().is_success()); + login .block_until_done() .await - .expect_err("scope-starved login must fail"); - assert!(error.to_string().contains("api.responses.write")); - assert!( - load_account_tokens(ProviderKind::OpenAiResponses).is_err(), - "a rejected token must not be stored" - ); + .expect("identity login stores"); + let stored = + load_account_tokens(ProviderKind::OpenAiResponses).expect("identity token stored"); + assert_eq!(stored.access_token, "scope-starved"); probe.assert_async().await; } diff --git a/core/llm/client.rs b/core/llm/client.rs index 104606e6..e03733f1 100644 --- a/core/llm/client.rs +++ b/core/llm/client.rs @@ -812,9 +812,13 @@ async fn transcribe_multipart( /// Send a single multipart STT transcription request (used by retry loop) async fn transcribe_multipart_request(url: &str, api_key: &str, form: Form) -> Result { - let response = get_client() - .post(url) - .header("x-api-key", api_key) + let request = get_client().post(url); + let request = match crate::stt::tail_provider::stt_auth_mode(url) { + crate::stt::tail_provider::SttAuthMode::Unauthenticated => request, + crate::stt::tail_provider::SttAuthMode::Bearer => request.bearer_auth(api_key), + crate::stt::tail_provider::SttAuthMode::ApiKey => request.header("x-api-key", api_key), + }; + let response = request .multipart(form) .send() .await diff --git a/core/llm/key_liveness.rs b/core/llm/key_liveness.rs index 3e6e3164..4c12f39b 100644 --- a/core/llm/key_liveness.rs +++ b/core/llm/key_liveness.rs @@ -91,7 +91,16 @@ pub fn probe_api_key_liveness(account: &str) -> ApiKeyLivenessResult { } let config = Config::load(); - let Some(api_key) = lane_truth::secret(account) else { + let api_key = lane_truth::secret(account); + let stt_is_unauthenticated = account == "STT_API_KEY" + && config + .stt_endpoint + .as_deref() + .filter(|value| !value.trim().is_empty()) + .map(crate::stt::tail_provider::stt_auth_mode) + .unwrap_or(crate::stt::tail_provider::SttAuthMode::Unauthenticated) + == crate::stt::tail_provider::SttAuthMode::Unauthenticated; + let Some(api_key) = api_key.or_else(|| stt_is_unauthenticated.then(String::new)) else { return ApiKeyLivenessResult::new( account, ApiKeyLivenessStatus::Missing, @@ -153,17 +162,22 @@ pub fn probe_api_key_liveness(account: &str) -> ApiKeyLivenessResult { /// Probe the configured multipart STT slot with 100 ms of synthetic silence. /// The response body is never surfaced; only auth/quota/transport status is. +/// A live WebSocket URL is remapped to the file worker first — Test is not a +/// handshake against the Voice Lab socket. fn probe_stt_key( client: &Client, config: &Config, account: &str, api_key: &str, ) -> ApiKeyLivenessResult { - let endpoint = config - .stt_endpoint - .clone() - .filter(|value| !value.trim().is_empty()) - .unwrap_or_else(|| "http://127.0.0.1:8000/v1/audio/transcriptions".to_string()); + let endpoint = crate::stt::tail_provider::file_probe_endpoint( + config + .stt_endpoint + .as_deref() + .map(str::trim) + .filter(|value| !value.is_empty()) + .unwrap_or("http://127.0.0.1:8000/v1/audio/transcriptions"), + ); if crate::stt::tail_provider::validate_remote_endpoint(&endpoint).is_err() { return ApiKeyLivenessResult::new( account, @@ -203,12 +217,21 @@ fn probe_stt_key( .text("model", "whisper-1") .text("language", "pl") .text("response_format", "json"); - let response = client - .post(&endpoint) - .header("x-api-key", api_key) - .multipart(form) - .send(); - response_result(account, endpoint, response) + let request = client.post(&endpoint); + let auth_mode = crate::stt::tail_provider::stt_auth_mode(&endpoint); + let request = match auth_mode { + crate::stt::tail_provider::SttAuthMode::Unauthenticated => request, + crate::stt::tail_provider::SttAuthMode::Bearer => request.bearer_auth(api_key), + crate::stt::tail_provider::SttAuthMode::ApiKey => request.header("x-api-key", api_key), + }; + let response = request.multipart(form).send(); + let mut result = response_result(account, endpoint, response); + if auth_mode == crate::stt::tail_provider::SttAuthMode::Unauthenticated + && result.status == ApiKeyLivenessStatus::Ok + { + result.message = "local STT endpoint accepts unauthenticated requests".to_string(); + } + result } /// Classify one provider HTTP response. This is the tested contract; network @@ -490,8 +513,53 @@ mod tests { ); let request = server.join().expect("STT probe server"); assert!(request.starts_with("POST /v1/audio/transcriptions HTTP/1.1")); - assert!(request.to_ascii_lowercase().contains("x-api-key: test-key")); + let request_lower = request.to_ascii_lowercase(); + assert!(!request_lower.contains("x-api-key:")); + assert!(!request_lower.contains("authorization:")); assert!(request.contains("codescribe-key-probe.wav")); + assert_eq!( + result.message, + "local STT endpoint accepts unauthenticated requests" + ); + } + + /// A stored Voice Lab socket is remapped onto the file worker before POST. + #[test] + fn stt_probe_maps_live_websocket_to_multipart() { + let listener = TcpListener::bind("127.0.0.1:0").expect("bind STT live-socket probe server"); + let address = listener + .local_addr() + .expect("STT live-socket probe address"); + let live = format!("ws://127.0.0.1:{}/v1/audio/transcribe", address.port()); + let expected = format!( + "http://127.0.0.1:{}/v1/audio/transcriptions", + address.port() + ); + let server = thread::spawn(move || { + let (mut stream, _) = listener.accept().expect("accept remapped STT probe"); + let mut buffer = [0_u8; 8192]; + let bytes_read = stream.read(&mut buffer).expect("read remapped STT probe"); + stream + .write_all( + b"HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nContent-Length: 11\r\nConnection: close\r\n\r\n{\"text\":\"\"}", + ) + .expect("write remapped STT probe response"); + String::from_utf8_lossy(&buffer[..bytes_read]).to_string() + }); + let client = Client::builder() + .timeout(PROBE_TIMEOUT) + .connect_timeout(PROBE_TIMEOUT) + .build() + .expect("build remapped STT probe client"); + let config = Config { + stt_endpoint: Some(live), + ..Config::default() + }; + let result = probe_stt_key(&client, &config, "STT_API_KEY", "test-key"); + assert_eq!(result.status, ApiKeyLivenessStatus::Ok); + assert_eq!(result.probed_endpoint.as_deref(), Some(expected.as_str())); + let request = server.join().expect("remapped STT probe server"); + assert!(request.starts_with("POST /v1/audio/transcriptions HTTP/1.1")); } /// 2xx means the provider accepted the key and returned a usable response. diff --git a/core/pipeline/stream_postprocess.rs b/core/pipeline/stream_postprocess.rs index 8d0207bc..9eededd8 100644 --- a/core/pipeline/stream_postprocess.rs +++ b/core/pipeline/stream_postprocess.rs @@ -1602,7 +1602,7 @@ mod tests { assert_eq!(lexicon.custom_canonicals, vec!["FooBar".to_string()]); } - /// Overlay correction → custom lexicon → hot-reload teaches the next transcript. + /// Overlay copy is evidence-only. Voice Lab finalize teaches the custom lexicon. #[test] #[serial] fn overlay_correction_chain_teaches_custom_lexicon_for_next_transcript() { @@ -1620,7 +1620,7 @@ mod tests { crate::quality::overlay_quality::extract_lexicon_candidates("uni agentka", "Junie"); assert_eq!(candidates, vec![("uni agentka".into(), "Junie".into())]); - let quality_path = crate::quality::overlay_quality::commit_overlay_correction( + let commit = crate::quality::overlay_quality::commit_overlay_correction( "uni agentka", "uni agentka", "Junie", @@ -1628,8 +1628,10 @@ mod tests { Some("whisper-test".into()), Some("copy"), ) - .expect("commit overlay correction") - .quality_path; + .expect("commit overlay correction"); + assert!(commit.evidence_only); + assert_eq!(commit.pairs_learned, 0); + let quality_path = commit.quality_path; assert!(quality_path.starts_with(&temp_root)); assert!(quality_path.ends_with("corrections.jsonl")); @@ -1647,8 +1649,24 @@ mod tests { ); let custom_path = crate::config::Config::config_dir().join("lexicon.custom.jsonl"); - let custom = std::fs::read_to_string(&custom_path).expect("read custom lexicon"); - assert!(custom.contains(r#""term":"Junie""#)); + assert!( + !custom_path.exists(), + "overlay copy must not write the custom lexicon" + ); + + std::thread::sleep(std::time::Duration::from_millis(50)); + let outcome = crate::quality::overlay_quality::finalize_voice_lab_correction( + &correction_id, + "Junie Prime", + ) + .expect("finalize Voice Lab correction"); + assert_eq!(outcome.pairs_learned, 1); + assert_eq!(outcome.record.revision, record.revision + 1); + assert_eq!(outcome.record.edited_text, "Junie Prime"); + + let custom = + std::fs::read_to_string(&custom_path).expect("read custom lexicon after teach"); + assert!(custom.contains(r#""term":"Junie Prime""#)); assert!(custom.contains(r#""uni agentka""#)); let mut custom_rules = Vec::new(); @@ -1660,9 +1678,9 @@ mod tests { Some(&mut custom_canonicals), ); assert_eq!(count, 1); - assert_eq!(custom_canonicals, vec!["Junie".to_string()]); + assert_eq!(custom_canonicals, vec!["Junie Prime".to_string()]); - let mut lexicon = Lexicon { + let lexicon = Lexicon { builtin_rules: Vec::new(), custom_rules, custom_path: custom_path.clone(), @@ -1672,23 +1690,6 @@ mod tests { protected_canonicals: Vec::new(), custom_canonicals, }; - assert_eq!(lexicon.apply("uni agentka"), "Junie"); - assert_eq!( - lexicon.apply("Następny transcript: uni agentka."), - "Następny transcript: Junie." - ); - - std::thread::sleep(std::time::Duration::from_millis(50)); - let outcome = crate::quality::overlay_quality::finalize_voice_lab_correction( - &correction_id, - "Junie Prime", - ) - .expect("finalize Voice Lab correction"); - assert_eq!(outcome.pairs_learned, 1); - assert_eq!(outcome.record.revision, record.revision + 1); - assert_eq!(outcome.record.edited_text, "Junie Prime"); - - lexicon.maybe_reload(); assert_eq!(lexicon.custom_rules.len(), 1); assert_eq!(lexicon.apply("uni agentka"), "Junie Prime"); assert_eq!( diff --git a/core/pipeline/streaming/apple_live_session.rs b/core/pipeline/streaming/apple_live_session.rs index 6060e60c..f49c9dee 100644 --- a/core/pipeline/streaming/apple_live_session.rs +++ b/core/pipeline/streaming/apple_live_session.rs @@ -45,9 +45,12 @@ use crate::asr_session::recorder::{ }; use crate::asr_session::{SessionId as Layer1SessionId, SessionInput as Layer1SessionInput}; use crate::audio::capture_receipt::{ - CaptureLevelAccumulator, CapturePathMeta, emit_capture_level_receipt, + CaptureLevelAccumulator, CapturePathMeta, begin_session_energy_clock, + emit_capture_level_receipt, +}; +use crate::pipeline::contracts::{ + DropKind, EngineEvent, EventSink, LayerSource, TranscriptSegment, }; -use crate::pipeline::contracts::{DropKind, EngineEvent, EventSink, TranscriptSegment}; use crate::pipeline::stream_postprocess::StreamPostProcessor; use crate::stt::apple_stt::{LiveStreamEvent, LiveStreamSession}; use crate::stt::tail_patcher::{SkipReasonCode, TailPatchConfig, TailPatchOutcome}; @@ -57,7 +60,9 @@ use crate::stt::tail_provider::{ }; use super::live_audio_buffer::{DEFAULT_RETENTION_SECS, LiveAudioBuffer, ResolvedAudioWindow}; -use super::progressive_seal::{AppleCommit, ProgressiveSealMachine, SealTick, seal_span_text}; +use super::progressive_seal::{ + AppleCommit, ProgressiveSealMachine, SealTick, SealedSpan, seal_span_text, +}; #[cfg(test)] use super::session::emit_tail_patch_result; use super::session::{ @@ -319,6 +324,7 @@ pub(crate) async fn apple_stream_transcription_session( mut lifecycle_events, } = config; let mut capture_level = CaptureLevelAccumulator::new(); + begin_session_energy_clock(); // Hands-free silence is the ENGINE LIFECYCLE on this lane, not a chunker // knob: SFSpeech still owns phrase boundaries inside an utterance, but the // threshold decides when the engine rests (mic + Silero keep watching) and @@ -537,6 +543,7 @@ pub(crate) async fn apple_stream_transcription_session( // (the T0 `merge_live_layer1` seam), owned by the stop-path truth // adjudicator once the settings cut arms real providers. let layer1_outcome = layer1_lane.stop(); + let layer1_candidate = layer1_outcome.refined_transcript(); if let Some(reason) = layer1_lane.take_degrade_notice() { emit_layer1_degrade_warning(event_sink.as_ref(), reason); } @@ -557,6 +564,7 @@ pub(crate) async fn apple_stream_transcription_session( ); } + let mut sealed_spans = Vec::new(); match worker.join() { Ok(Ok(outcome)) => { info!( @@ -566,6 +574,7 @@ pub(crate) async fn apple_stream_transcription_session( under_commit_escalations = outcome.under_commit_escalations, "Apple progressive live session finished" ); + sealed_spans = outcome.sealed_spans; } Ok(Err(e)) => { warn!("Apple live stream worker failed: {e:#}"); @@ -582,6 +591,17 @@ pub(crate) async fn apple_stream_transcription_session( } log_tail_patch_session_receipt(tail_patch_lane.replacements(), tail_patch_lane.skipped()); + let mut live_cloud_patches = 0u64; + if let Some(candidate) = layer1_candidate { + for event in plan_live_layer1_gap_patches(&sealed_spans, &candidate) { + event_sink.on_event(&event); + live_cloud_patches = live_cloud_patches.saturating_add(1); + } + info!( + provider_chars = candidate.chars().count(), + live_cloud_patches, "Live cloud Layer 1 reconciled against committed Apple floor" + ); + } emit_capture_level_receipt( event_sink.as_ref(), &capture_level.finalize(CapturePathMeta::resolve(sample_rate, 1, None)), @@ -589,7 +609,9 @@ pub(crate) async fn apple_stream_transcription_session( emit_session_finalised( event_sink.as_ref(), session_id, - tail_patch_lane.replacements(), + tail_patch_lane + .replacements() + .saturating_add(live_cloud_patches), ); } @@ -817,13 +839,18 @@ impl AppleSealState { // so the inline-format buffer may chunk-format it while dictation // continues. Sync + non-blocking; no-op unless the flag is armed. crate::llm::inline_format::on_chunk_sealed(sealed.id, &sealed.text); + let segments = if sealed.words.is_empty() { + pending.segments + } else { + timed_words_to_segments(&sealed.words, self.sample_rate) + }; let _ = ev_tx.send(EngineEvent::UtteranceFinal { utterance_id: sealed.id, text: sealed.text, raw_text: pending.raw_text, start_ts: pending.start_ts, end_ts: pending.end_ts, - segments: pending.segments, + segments, vad_speech_pct: None, avg_logprob: None, compression_ratio: None, @@ -869,6 +896,165 @@ struct AppleStreamOutcome { unresolved_windows: u64, /// How many seals escalated an unplaceable Layer 1 under-commit (W-C). under_commit_escalations: u64, + sealed_spans: Vec, +} + +#[derive(Debug)] +struct LivePatchToken { + utterance_id: u64, + start: usize, + end: usize, +} + +/// Convert provider-neutral Layer 1 gap-fill into existing bounded utterance +/// patches. The merge first preserves every Apple token; only tokens present +/// in the merged result but absent from that floor become zero-width inserts. +fn plan_live_layer1_gap_patches(spans: &[SealedSpan], candidate: &str) -> Vec { + if spans.is_empty() || candidate.trim().is_empty() { + return Vec::new(); + } + let live = spans + .iter() + .map(|span| span.text.trim()) + .filter(|text| !text.is_empty()) + .collect::>() + .join(" "); + let merged = crate::quality::merge_live_layer1(&live, candidate); + if merged.provider_fill_tokens == 0 { + return Vec::new(); + } + + let live_tokens = crate::quality::teacher::tokenize(&live); + let merged_tokens = crate::quality::teacher::tokenize(&merged.text); + let mapped = mapped_live_tokens(spans); + if mapped.len() != live_tokens.len() { + warn!( + mapped_tokens = mapped.len(), + live_tokens = live_tokens.len(), + "Live cloud gap planner refused inconsistent span token map" + ); + return Vec::new(); + } + let ops = crate::quality::teacher::align_words(&live_tokens, &merged_tokens); + let mut patches = Vec::new(); + let mut previous_live: Option = None; + let mut index = 0usize; + while index < ops.len() { + match &ops[index] { + crate::quality::teacher::AlignOp::InsertB { .. } => { + let start = index; + while matches!( + ops.get(index), + Some(crate::quality::teacher::AlignOp::InsertB { .. }) + ) { + index += 1; + } + let words = ops[start..index] + .iter() + .filter_map(|op| match op { + crate::quality::teacher::AlignOp::InsertB { b } => { + Some(merged_tokens[*b].surface.as_str()) + } + _ => None, + }) + .collect::>() + .join(" "); + let next_live = ops[index..].iter().find_map(live_op_index); + if let Some(previous) = previous_live.and_then(|idx| mapped.get(idx)) { + patches.push(EngineEvent::ReplaceRange { + utterance_id: previous.utterance_id, + start: previous.end, + end: previous.end, + text: format!(" {words}"), + source: LayerSource::TailPatch, + }); + } else if let Some(next) = next_live.and_then(|idx| mapped.get(idx)) { + patches.push(EngineEvent::ReplaceRange { + utterance_id: next.utterance_id, + start: next.start, + end: next.start, + text: format!("{words} "), + source: LayerSource::TailPatch, + }); + } + } + crate::quality::teacher::AlignOp::Substitute { a, b } => { + if let Some(live_token) = mapped.get(*a) { + patches.push(EngineEvent::ReplaceRange { + utterance_id: live_token.utterance_id, + start: live_token.start, + end: live_token.end, + text: merged_tokens[*b].surface.clone(), + source: LayerSource::TailPatch, + }); + } + previous_live = Some(*a); + index += 1; + } + op => { + previous_live = live_op_index(op).or(previous_live); + index += 1; + } + } + } + + // Multiple inserts into one utterance use offsets from the same immutable + // Apple text. Apply right-to-left so an earlier insertion cannot shift a + // later one's char boundary. + patches.sort_by(|left, right| { + patch_position(right) + .cmp(&patch_position(left)) + .then_with(|| patch_utterance(right).cmp(&patch_utterance(left))) + }); + patches +} + +fn live_op_index(op: &crate::quality::teacher::AlignOp) -> Option { + match op { + crate::quality::teacher::AlignOp::Equal { a, .. } + | crate::quality::teacher::AlignOp::DeleteA { a } + | crate::quality::teacher::AlignOp::Substitute { a, .. } => Some(*a), + crate::quality::teacher::AlignOp::InsertB { .. } => None, + } +} + +fn mapped_live_tokens(spans: &[SealedSpan]) -> Vec { + let mut mapped = Vec::new(); + for span in spans { + let chars = span.text.chars().collect::>(); + let mut cursor = 0usize; + while cursor < chars.len() { + while cursor < chars.len() && chars[cursor].is_whitespace() { + cursor += 1; + } + let start = cursor; + while cursor < chars.len() && !chars[cursor].is_whitespace() { + cursor += 1; + } + if start < cursor { + mapped.push(LivePatchToken { + utterance_id: span.id, + start, + end: cursor, + }); + } + } + } + mapped +} + +fn patch_position(event: &EngineEvent) -> usize { + match event { + EngineEvent::ReplaceRange { start, .. } => *start, + _ => 0, + } +} + +fn patch_utterance(event: &EngineEvent) -> u64 { + match event { + EngineEvent::ReplaceRange { utterance_id, .. } => *utterance_id, + _ => 0, + } } /// Resolve a sealed utterance back to its audio span, then release what can @@ -958,6 +1144,21 @@ fn seconds_to_captured_sample(seconds: f32, sample_rate: u32, captured_end: u64) ((seconds as f64 * sample_rate.max(1) as f64).round() as u64).min(captured_end) } +fn timed_words_to_segments(words: &[TimedTailSegment], sample_rate: u32) -> Vec { + let rate = sample_rate.max(1) as f32; + words + .iter() + .filter(|word| { + word.range.sample_end > word.range.sample_start && !word.text.trim().is_empty() + }) + .map(|word| TranscriptSegment { + text: word.text.clone(), + start_ts: word.range.sample_start as f32 / rate, + end_ts: word.range.sample_end as f32 / rate, + }) + .collect() +} + fn apple_segments_on_pcm_clock( state: &AppleSealState, segments: &[TranscriptSegment], @@ -1125,6 +1326,22 @@ fn apply_conservative_fusion( return fallback; } let decision = conservative_fuse(&apple_words, &whisper); + if !decision.unresolved.is_empty() { + // An unresolved fusion verdict means the rewrite text intentionally + // kept Apple's shorter alternative. Consuming the fallback here used + // to erase already-computed, safely anchored gap appends — exactly the + // first-utterance loss visible in the operator's local take. Keep the + // pending Apple span immutable and let the bounded patch lane land. + let receipt = fusion_receipt(utterance_id, &decision); + let _ = ev_tx.send(EngineEvent::Warning { + code: receipt.code.as_str().to_string(), + message: format!( + "fusion unresolved={} agreements={} gap_fills={}; bounded fallback retained", + receipt.unresolved, receipt.agreements, receipt.gap_fills + ), + }); + return fallback; + } if !state.progressive.try_rewrite(utterance_id, &decision.text) { // The span sealed before fusion could rewrite it. That is a refusal of // THIS route, not a verdict on the recovery: Layer 1 already computed @@ -1143,16 +1360,6 @@ fn apply_conservative_fusion( }); return fallback; } - if !decision.unresolved.is_empty() { - let receipt = fusion_receipt(utterance_id, &decision); - let _ = ev_tx.send(EngineEvent::Warning { - code: receipt.code.as_str().to_string(), - message: format!( - "fusion unresolved={} agreements={} gap_fills={}", - receipt.unresolved, receipt.agreements, receipt.gap_fills - ), - }); - } TailPatchOutcome::NoChange } @@ -2204,6 +2411,7 @@ fn apple_stream_worker( filtered_empty_drops: state.filtered_empty_drops, unresolved_windows: state.unresolved_windows, under_commit_escalations: state.under_commit_escalations, + sealed_spans: state.progressive.sealed_spans().to_vec(), }) } @@ -2414,6 +2622,68 @@ mod tests { } } + fn sealed_span(id: u64, text: &str) -> SealedSpan { + SealedSpan { + id, + text: text.to_string(), + end_secs_millis: id as u32 * 1_000, + range: TailSampleRange { + session: "live-cloud-gap-test".to_string(), + capture_epoch: 0, + sample_start: (id - 1) * 16_000, + sample_end: id * 16_000, + }, + words: Vec::new(), + apple_evidence: TailProviderEvidence { + source: TailEvidenceSource::AppleSpeech, + revision: None, + stability: TailEvidenceStability::Final, + timing_quality: TailTimingQuality::Synthetic, + avg_logprob: None, + }, + whisper_evidence: None, + whisper_words: Vec::new(), + silero_utterance_id: None, + } + } + + #[test] + fn live_cloud_gap_plan_preserves_apple_and_inserts_missing_words() { + let spans = vec![ + sealed_span(1, "I będziesz miał po prostu lokalnej teraz sobie."), + sealed_span(2, "Możesz odczytać i też pow."), + ]; + let candidate = "I będziesz miał po prostu z lokalnej sesji teraz sobie. Możesz odczytać i też powkurwiać się razem."; + let patches = plan_live_layer1_gap_patches(&spans, candidate); + assert!( + !patches.is_empty(), + "provider-only gaps must become patches" + ); + + let mut rendered = spans + .iter() + .map(|span| (span.id, span.text.clone())) + .collect::>(); + for patch in &patches { + let utterance_id = patch_utterance(patch); + patch + .apply_to_committed_text(rendered.get_mut(&utterance_id).expect("known span")) + .expect("bounded patch"); + } + let patched = rendered.into_values().collect::>().join(" "); + let live = spans + .iter() + .map(|span| span.text.as_str()) + .collect::>() + .join(" "); + assert_eq!( + patched, + crate::quality::merge_live_layer1(&live, candidate).text + ); + assert!(patched.contains("z lokalnej sesji")); + assert!(patched.contains("powkurwiać się razem")); + } + /// Integration boundary: the real Apple state owns the progressive /// machine. Apple commit + live partial alone stay pending; an elapsed /// Whisper window seals before session end. diff --git a/core/pipeline/streaming/session.rs b/core/pipeline/streaming/session.rs index 16dc6b90..03a6ba27 100644 --- a/core/pipeline/streaming/session.rs +++ b/core/pipeline/streaming/session.rs @@ -14,7 +14,8 @@ use tracing::{debug, error, info, warn}; use crate::asr_session::recorder::{Layer1Decision, RecorderLifecycleEvents}; use crate::audio::capture_receipt::{ - CaptureLevelAccumulator, CapturePathMeta, emit_capture_level_receipt, + CaptureLevelAccumulator, CapturePathMeta, begin_session_energy_clock, + emit_capture_level_receipt, }; use crate::audio::chunker::{SpeechEvent, SpeechSession}; use crate::pipeline::contracts::{ @@ -558,6 +559,7 @@ pub(crate) async fn vad_transcription_session( info!("Transcription session started (event-based pipeline)"); let session_id = uuid::Uuid::new_v4().to_string(); let mut capture_level = CaptureLevelAccumulator::new(); + begin_session_energy_clock(); let mut session = if let Some(sec) = utterance_silence_sec { SpeechSession::new_utterance_with_silence(sample_rate, sec) diff --git a/core/quality/engine_contract.rs b/core/quality/engine_contract.rs index 72452f17..80901318 100644 --- a/core/quality/engine_contract.rs +++ b/core/quality/engine_contract.rs @@ -17,6 +17,58 @@ pub const ENGINE_CONTRACT_ID: &str = "the-engine/v1"; /// Path of the agent-facing prose lock, relative to the repo root. pub const ENGINE_CONTRACT_DOC: &str = "docs/THE_ENGINE_CONTRACT.md"; +/// Path of the HTML-surface contract, relative to the repo root. +pub const QUALITY_HTML_CONTRACT_DOC: &str = "docs/quality-reports/CONTRACT.md"; + +/// Failures if this string is not a Seal Atlas quality report. +pub fn validate_quality_html(html: &str) -> Vec { + let lowered = html.to_ascii_lowercase(); + let mut failures = Vec::new(); + if !html.contains(r#"name="engine-contract""#) || !html.contains(ENGINE_CONTRACT_ID) { + failures.push("missing meta engine-contract=the-engine/v1".into()); + } + if !html.contains(r#"name="quality-report-surface""#) + || !lowered.contains("seal-atlas") && !lowered.contains("seal atlas") + { + failures.push("missing meta quality-report-surface=seal-atlas".into()); + } + if !lowered.contains("seal atlas") && !lowered.contains("seal-atlas") { + failures.push("title/body must name Seal Atlas".into()); + } + if !html.contains(r#"class="stat""#) { + failures.push("Voice Lab handshake needs div.stat cards".into()); + } + if !lowered.contains("word-grain") { + failures.push("must label word-grain".into()); + } + if !lowered.contains("utterance-grain") { + failures.push("must label utterance-grain".into()); + } + if !lowered.contains("clock-lie") && !html.contains("kłamstwo zegarowe") { + failures.push("clock-lie must be a first-class finding".into()); + } + if !html.contains("SealedSpan.words") + && !lowered.contains("sealedspan.words") + && !lowered.contains("sealed spans") + { + failures.push("must name SealedSpan.words or sealed spans".into()); + } + if !lowered.contains("whisper") { + failures.push("must mention whisper_words / Whisper on the same clock".into()); + } + if lowered.contains("

codescribe quality report

") { + failures.push("retired Qube H1 is not a Seal Atlas".into()); + } + let wer_at = lowered.find("avg wer"); + let footnote_at = lowered.find("footnote"); + if let Some(wer) = wer_at + && !footnote_at.map(|footnote| footnote < wer).unwrap_or(false) + { + failures.push("Avg WER may only appear after a footnote marker".into()); + } + failures +} + /// What a quality report is allowed to treat as the document vs a proposal. #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] #[serde(rename_all = "snake_case")] @@ -421,6 +473,43 @@ mod tests { assert_eq!(QUALITY_REPORT_SURFACE, "seal-atlas"); } + #[test] + fn gold_atlas_passes_html_handshake() { + let root = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join(".."); + let path = root.join(SEAL_ATLAS_GOLD_HTML); + let body = std::fs::read_to_string(&path) + .unwrap_or_else(|err| panic!("{} must exist: {err}", path.display())); + let failures = validate_quality_html(&body); + assert!( + failures.is_empty(), + "gold take 01 failed handshake: {failures:?}" + ); + } + + #[test] + fn html_contract_doc_exists() { + let root = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join(".."); + let body = std::fs::read_to_string(root.join(QUALITY_HTML_CONTRACT_DOC)) + .unwrap_or_else(|err| panic!("{QUALITY_HTML_CONTRACT_DOC} must exist: {err}")); + for needle in [ + "quality-report-surface", + "div class=\"stat\"", + "word-grain", + "clock-lie", + "quality/seal-atlas.", + ] { + assert!(body.contains(needle), "CONTRACT.md missing {needle:?}"); + } + } + + #[test] + fn retired_qube_title_fails_handshake() { + let fake = r#"Codescribe Quality Report +

Codescribe Quality Report

Avg WER 12%

"#; + let failures = validate_quality_html(fake); + assert!(failures.len() >= 3, "{failures:?}"); + } + #[test] fn gold_atlas_html_is_a_pcm_instrument_not_a_wer_table() { let root = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join(".."); diff --git a/core/quality/mod.rs b/core/quality/mod.rs index 6bcf2c0d..3cf6c7ca 100644 --- a/core/quality/mod.rs +++ b/core/quality/mod.rs @@ -22,13 +22,16 @@ pub mod overlay_quality; pub mod qube_daemon; /// Qube report types and serialization for quality/donor telemetry surfaces. pub mod qube_report; +/// Seal Atlas HTML renderer — the quality-report surface corpus writes. +pub mod seal_atlas_html; /// Teacher loop: attention flags, lexicon feedback, polygon token helpers. pub mod teacher; pub use engine_contract::{ CORPUS_REPORT_SCHEMA, ENGINE_CONTRACT, ENGINE_CONTRACT_ID, EngineContract, - render_engine_contract_html, + render_engine_contract_html, validate_quality_html, }; +pub use seal_atlas_html::{SealAtlasPage, SealAtlasStats, render_seal_atlas_html}; pub use teacher::{ Layer1MergeMode, Layer1MergedDelivery, MergeMode, MergedDelivery, TeacherInput, TeacherReport, merge_live_layer1, merge_live_whisper, merge_live_whisper_with_terms, report_to_html, teach, diff --git a/core/quality/overlay_quality.rs b/core/quality/overlay_quality.rs index 70967d4a..81f4152d 100644 --- a/core/quality/overlay_quality.rs +++ b/core/quality/overlay_quality.rs @@ -947,14 +947,15 @@ where Ok(()) } -/// Result of a successful overlay-quality commit (evidence always; learn when Correction). +/// Result of a successful overlay-quality commit (evidence always; learn only +/// on an explicit teach gesture, never on overlay copy/close). #[derive(Debug, Clone, PartialEq, Eq)] pub struct OverlayCorrectionCommit { /// The `corrections.jsonl` the evidence line was appended to. pub quality_path: PathBuf, /// Lexicon pairs actually upserted from this commit (0 when evidence-only or filtered). pub pairs_learned: u32, - /// True when the formatting level is not Correction — record saved, nothing taught. + /// True when this commit did not teach the custom lexicon. pub evidence_only: bool, } @@ -971,6 +972,13 @@ impl OverlayCorrectionCommit { } } +/// Overlay copy/close/send never writes `lexicon.custom.jsonl`. +/// Highlighted-span teach (`action=teach-span`) still does. Speech-gap +/// teach (`teach-span-gap`) stays evidence-only. +fn overlay_commit_teaches_lexicon(_mode: &str, action: Option<&str>) -> bool { + matches!(action, Some("teach-span") | Some("teach-dictionary")) +} + /// High-level: save the quality record for the overlay edit AND feed lexicon candidates. /// Called from bridge (and tests). Returns path + honest pairs-learned count. /// `action` (e.g. "copy", "send", "close") is carried into meta for future analytics (P2-03 triage over-correct). @@ -1038,7 +1046,11 @@ pub fn commit_overlay_correction_with_confidence( .map(FormattingPolicy::parse) .transpose()? .map(|level| level.as_str().to_string()); - let teaches = formatting_level.as_deref() == Some(FormattingPolicy::Correction.as_str()); + // Overlay copy/close/send is evidence. Teaching from that diff is how + // 2026-08-17 learned "pisanie Żyda" → "mi się nie wydaje" and "w 3 4" → + // "Dwa Trzy Cztery Pięć". Lexicon grows only on an explicit teach gesture + // (highlighted span / Voice Lab), never from a formatting-level flag. + let teaches = overlay_commit_teaches_lexicon(mode, action); let record = QualityRecord::new_with_confidence( raw_text.to_string(), delivered_text.to_string(), @@ -1693,7 +1705,7 @@ mod tests { let delivered = format!("{body}zaznaczenie koniec"); let edited = format!("{body}selection koniec"); - let commit = commit_overlay_correction( + let evidence = commit_overlay_correction( &delivered, &delivered, &edited, @@ -1701,7 +1713,10 @@ mod tests { Some("whisper".into()), Some("copy"), ) - .expect("commit long dictation fix"); + .expect("commit long dictation evidence"); + assert_eq!(evidence.pairs_learned, 0); + let commit = teach_span("zaznaczenie", "selection", "lexicon_corrected") + .expect("explicit teach of the one-word fix"); assert_eq!(commit.pairs_learned, 1); assert_eq!(commit.acknowledgement_message(), "Saved — 1 pair learned"); @@ -1936,12 +1951,12 @@ mod tests { .expect("commit should succeed"); let p = commit.quality_path.clone(); assert!(p.ends_with("corrections.jsonl")); - assert!( - commit.pairs_learned >= 1, - "correction should teach at least one pair" + assert_eq!( + commit.pairs_learned, 0, + "overlay copy is evidence, not a lexicon teacher" ); - assert!(!commit.evidence_only); - assert!(commit.acknowledgement_message().contains("learned")); + assert!(commit.evidence_only); + assert_eq!(commit.acknowledgement_message(), "Saved as evidence"); // Proof of isolation: the quality file landed under the overridden DATA_DIR // (config_dir + quality_dir respect it; real ~/.codescribe untouched). assert!( @@ -2140,16 +2155,6 @@ mod tests { Some("send"), ) .expect("second correction"); - let lexicon_path = Config::config_dir().join("lexicon.custom.jsonl"); - let mut lexicon_file = OpenOptions::new() - .append(true) - .open(&lexicon_path) - .expect("open custom lexicon for legacy extras fixture"); - writeln!( - lexicon_file, - r#"{{"term":"Vetcoders","extras":{{"mispronunciations":["wet coders"]}}}}"# - ) - .expect("append legacy extras fixture"); let records = recent_quality_records(1).expect("recent records"); assert_eq!(records.len(), 1); @@ -2163,6 +2168,21 @@ mod tests { Some("send") ); + teach_span("raw one", "Junie", "lexicon_corrected").expect("explicit teach first pair"); + teach_span("raw two", "Loctree map", "lexicon_corrected") + .expect("explicit teach second pair"); + let lexicon_path = Config::config_dir().join("lexicon.custom.jsonl"); + let mut lexicon_file = OpenOptions::new() + .create(true) + .append(true) + .open(&lexicon_path) + .expect("open custom lexicon for legacy extras fixture"); + writeln!( + lexicon_file, + r#"{{"term":"Vetcoders","extras":{{"mispronunciations":["wet coders"]}}}}"# + ) + .expect("append legacy extras fixture"); + let lexicon = custom_lexicon_entries().expect("custom lexicon entries"); assert_eq!( lexicon, @@ -2233,10 +2253,10 @@ mod tests { assert_eq!(decoded.formatting_level, None); } - /// All levels append evidence; only Correction upserts lexicon candidates. + /// All overlay copy/close levels append evidence; none auto-teach lexicon. #[test] #[serial] - fn level_aware_commit_records_every_level_but_only_correction_teaches_lexicon() { + fn overlay_copy_records_every_level_and_never_teaches_lexicon() { let temp_dir = tempfile::tempdir().expect("temp quality root"); let _guard = EnvRestore::capture("CODESCRIBE_DATA_DIR"); let temp_root = temp_dir.path().canonicalize().unwrap(); @@ -2263,30 +2283,47 @@ mod tests { let records = recent_quality_records(10).expect("quality evidence rows"); let candidates = custom_lexicon_entries().expect("custom lexicon candidates"); assert_eq!(records.len(), 4, "every level appends quality evidence"); - assert_eq!(candidates.len(), 1, "only Correction emits a candidate"); - assert_eq!(candidates[0].variant, "korrvariant"); - assert_eq!(candidates[0].canonical, "CorrCanonical"); + assert!( + candidates.is_empty(), + "overlay copy must not write lexicon.custom.jsonl, got {candidates:?}" + ); } - /// Learning keys on raw STT text, not the formatter's delivered surface. + /// Live 2026-08-17: a human C-card correction must not invent "Meksyku" rules. #[test] #[serial] - fn correction_learning_uses_raw_stt_not_formatted_delivery() { + fn overlay_correction_of_garbled_take_is_evidence_only() { let temp_dir = tempfile::tempdir().expect("temp quality root"); let _guard = EnvRestore::capture("CODESCRIBE_DATA_DIR"); let temp_root = temp_dir.path().canonicalize().unwrap(); unsafe { std::env::set_var("CODESCRIBE_DATA_DIR", &temp_root) }; - let outcome = commit_overlay_correction_with_level( - "rawvariant", - "formattervariant", - "RawCanonical", + let outcome = commit_overlay_correction( + "A to jest pierwsze w oknie nie wybu słów tylko poprawiamy lokal power Meksyku.", + "A to jest pierwsze w oknie nie wybu słów tylko poprawiamy lokal power Meksyku.", + "Apple jest pierwszy, Whisper poprawia w oknie, nie wyjebujemy słów, tylko poprawiamy. Local power, leksykon.", "overlay", None, Some("copy"), - Some("correction"), ) - .expect("raw-source quality commit"); + .expect("quality evidence"); + assert_eq!(outcome.pairs_learned, 0); + assert!(outcome.evidence_only); + assert_eq!(outcome.acknowledgement_message(), "Saved as evidence"); + assert!(custom_lexicon_entries().expect("lexicon").is_empty()); + } + + /// Learning keys on raw STT text, not the formatter's delivered surface. + #[test] + #[serial] + fn correction_learning_uses_raw_stt_not_formatted_delivery() { + let temp_dir = tempfile::tempdir().expect("temp quality root"); + let _guard = EnvRestore::capture("CODESCRIBE_DATA_DIR"); + let temp_root = temp_dir.path().canonicalize().unwrap(); + unsafe { std::env::set_var("CODESCRIBE_DATA_DIR", &temp_root) }; + + let outcome = teach_span("rawvariant", "RawCanonical", "lexicon_corrected") + .expect("explicit teach from raw STT"); assert_eq!(outcome.pairs_learned, 1); let entries = custom_lexicon_entries().expect("custom lexicon"); @@ -2363,6 +2400,7 @@ mod tests { let lexicon_path = Config::config_dir().join("lexicon.custom.jsonl"); let mut duplicate = OpenOptions::new() + .create(true) .append(true) .open(&lexicon_path) .expect("open duplicate fixture"); diff --git a/core/quality/qube_report.rs b/core/quality/qube_report.rs index 868d6381..104b7610 100644 --- a/core/quality/qube_report.rs +++ b/core/quality/qube_report.rs @@ -466,14 +466,18 @@ fn prepare_cloud_jobs( CloudJobSet::Running(jobs) } -/// The cloud STT endpoint and key, or `None` when either is absent or blank. -/// Whitespace-only values count as absent — a half-filled setting is not -/// credentials, and treating it as such would fail later with a worse message. +/// The cloud STT endpoint and optional key, or `None` when required input is +/// absent. Loopback servers intentionally accept an empty key; remote owners +/// still require one. fn cloud_reference_credentials(app_config: &Config) -> Option<(String, String)> { let endpoint = app_config.stt_endpoint.as_deref()?.trim(); - let api_key = app_config.stt_api_key.as_deref()?.trim(); + let api_key = app_config.stt_api_key.as_deref().unwrap_or_default().trim(); - if endpoint.is_empty() || api_key.is_empty() { + if endpoint.is_empty() + || (crate::stt::tail_provider::stt_auth_mode(endpoint) + != crate::stt::tail_provider::SttAuthMode::Unauthenticated + && api_key.is_empty()) + { return None; } @@ -2025,6 +2029,15 @@ mod tests { config.stt_api_key = Some(" ".into()); assert_eq!(cloud_reference_credentials(&config), None); + + config.stt_endpoint = Some("http://127.0.0.1:8000/v1/audio/transcriptions".into()); + assert_eq!( + cloud_reference_credentials(&config), + Some(( + "http://127.0.0.1:8000/v1/audio/transcriptions".into(), + String::new(), + )) + ); } /// Spot-checks WER on a one-token substitution (ala ma kota → ala ma psa). diff --git a/core/quality/seal_atlas_html.rs b/core/quality/seal_atlas_html.rs new file mode 100644 index 00000000..33867244 --- /dev/null +++ b/core/quality/seal_atlas_html.rs @@ -0,0 +1,197 @@ +//! Seal Atlas HTML — the quality-report surface `codescribe-corpus` writes. +//! +//! Gold visual is take 01 (`docs/quality-reports/seal-atlas.take01.html`). +//! This renderer emits a handshake-valid atlas for a corpus profile when a +//! live dump is not attached. Waveform SVG is omitted until +//! `CODESCRIBE_SEAL_ATLAS_DUMP` is present; the page is still a Seal Atlas, +//! not a Qube WER table. + +use super::engine_contract::{ + ENGINE_CONTRACT_ID, QUALITY_REPORT_SURFACE, engine_contract_css, render_engine_contract_html, +}; + +/// Numbers Voice Lab lifts out of `.stat` cards. +#[derive(Debug, Clone)] +pub struct SealAtlasStats { + pub word_grain: String, + pub sealed_spans: String, + pub per_word_spans: String, + pub clock_lies: String, + pub silero_threshold: String, +} + +impl Default for SealAtlasStats { + fn default() -> Self { + Self { + word_grain: "n/a".into(), + sealed_spans: "n/a".into(), + per_word_spans: "n/a".into(), + clock_lies: "n/a".into(), + silero_threshold: "0.5".into(), + } + } +} + +/// One Seal Atlas HTML document. +#[derive(Debug, Clone)] +pub struct SealAtlasPage { + pub title: String, + pub lede: String, + pub stats: SealAtlasStats, + pub findings: Vec, + pub dump_present: bool, +} + +impl Default for SealAtlasPage { + fn default() -> Self { + Self { + title: "Seal Atlas".into(), + lede: + "One take, one PCM clock. Words from SealedSpan.words — not from the final string." + .into(), + stats: SealAtlasStats::default(), + findings: Vec::new(), + dump_present: false, + } + } +} + +/// Handshake-valid Seal Atlas HTML. Title contains `Seal Atlas`. +pub fn render_seal_atlas_html(page: &SealAtlasPage) -> String { + let title = if page.title.to_ascii_lowercase().contains("seal atlas") { + page.title.clone() + } else { + format!("Seal Atlas — {}", page.title) + }; + let stats = [ + (&page.stats.word_grain, "word-grain ≥75% speech"), + (&page.stats.sealed_spans, "sealed spans"), + (&page.stats.per_word_spans, "spans with per-word pins"), + (&page.stats.clock_lies, "clock-lie"), + (&page.stats.silero_threshold, "Silero threshold"), + ] + .into_iter() + .map(|(value, label)| { + format!( + "
{}{}
", + html_escape(value), + label + ) + }) + .collect::(); + + let findings = if page.findings.is_empty() { + "
  • No dump attached. This page is the contract surface; waveform waits on CODESCRIBE_SEAL_ATLAS_DUMP.
    • ".into() + } else { + page.findings + .iter() + .map(|line| format!("
  • {}
    • ", html_escape(line))) + .collect::() + }; + + let dump_note = if page.dump_present { + "

    Waveform from the live dump. Production Silero via vad_atlas_probe. Letter ticks = równomierna interpolacja, not measurement.

    " + } else { + "

    Waveform omitted — no CODESCRIBE_SEAL_ATLAS_DUMP. whisper_words still map backward onto pcm_time when a dump arrives. Per-word pins are real where they exist and not guaranteed. Utterance-grain vs word-grain stay labeled. Clock-lie is a finding.

    " + }; + + format!( + r#" + + + + + + +{title} + + + +
    +
    +

    {title}

    +

    {lede}

    +
    {stats}
    +
    +{plate} +
    +

    Lanes

    +

    Silero p(mowa) · word-grain · utterance-grain · clock-lie · whisper_words

    +{dump_note} +

    Words from SealedSpan.words / the live dump — never rebuilt from the final string. HQ / Cloud stay proposals.

    +
    +
    +

    Findings

    +
      {findings}
    +
    +
    + + +"#, + contract = ENGINE_CONTRACT_ID, + surface = QUALITY_REPORT_SURFACE, + title = html_escape(&title), + lede = html_escape(&page.lede), + stats = stats, + plate = render_engine_contract_html(), + dump_note = dump_note, + findings = findings, + css = engine_contract_css(), + ) +} + +fn html_escape(value: &str) -> String { + value + .replace('&', "&") + .replace('<', "<") + .replace('>', ">") + .replace('"', """) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::quality::engine_contract::validate_quality_html; + + #[test] + fn renderer_passes_the_html_handshake() { + let html = render_seal_atlas_html(&SealAtlasPage { + title: "profile apple-layer0".into(), + ..SealAtlasPage::default() + }); + let failures = validate_quality_html(&html); + assert!(failures.is_empty(), "{failures:?}"); + assert!(html.contains("Seal Atlas — profile apple-layer0")); + assert!(!html.contains("Avg WER")); + assert!(!html.contains("Codescribe Quality Report")); + } + + #[test] + fn renderer_escapes_page_content() { + let html = render_seal_atlas_html(&SealAtlasPage { + title: "Seal Atlas ".into(), + lede: "\"measured\" + + diff --git a/tests/cloud_transcribe_e2e.rs b/tests/cloud_transcribe_e2e.rs index df2a9db6..27a0a365 100644 --- a/tests/cloud_transcribe_e2e.rs +++ b/tests/cloud_transcribe_e2e.rs @@ -30,7 +30,8 @@ async fn contract_cloud_transcribe_success() { let success = server .mock("POST", "/v1/audio/transcriptions") - .match_header("x-api-key", "test-key") + .match_header("authorization", mockito::Matcher::Missing) + .match_header("x-api-key", mockito::Matcher::Missing) .with_status(200) .with_header("content-type", "application/json") .with_body(r#"{"text":"hello from cloud"}"#) @@ -39,10 +40,9 @@ async fn contract_cloud_transcribe_success() { .await; let audio = write_min_valid_audio_file(); - let verdict = - codescribe::client::transcribe_cloud(audio.path(), Some("en"), &endpoint, "test-key") - .await - .expect("cloud transcription should succeed"); + let verdict = codescribe::client::transcribe_cloud(audio.path(), Some("en"), &endpoint, "") + .await + .expect("cloud transcription should succeed"); success.assert_async().await; assert_eq!(verdict.text, "hello from cloud"); @@ -148,13 +148,14 @@ async fn test_cloud_transcribe_e2e() { return; } }; - let api_key = match std::env::var("STT_API_KEY") { - Ok(val) if !val.trim().is_empty() => val, - _ => { - eprintln!("Skipping cloud E2E (STT_API_KEY missing)"); - return; - } - }; + let api_key = std::env::var("STT_API_KEY").unwrap_or_default(); + if codescribe_core::stt::tail_provider::stt_auth_mode(&endpoint) + != codescribe_core::stt::tail_provider::SttAuthMode::Unauthenticated + && api_key.trim().is_empty() + { + eprintln!("Skipping cloud E2E (STT_API_KEY missing for remote endpoint)"); + return; + } let audio = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("tests/assets/1.fretka-Ziggy.mp3"); assert!(audio.exists(), "Missing test audio at {}", audio.display());