Skip to content

Rebuild and harden Lilac around Realtime Translation - #21

Draft
DexterStorey wants to merge 4 commits into
mainfrom
codex/realtime-translation
Draft

Rebuild and harden Lilac around Realtime Translation#21
DexterStorey wants to merge 4 commits into
mainfrom
codex/realtime-translation

Conversation

@DexterStorey

@DexterStorey DexterStorey commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

What changed

  • Rebuilt Lilac around OpenAI's dedicated Realtime Translation API over WebRTC.
  • Kept the permanent OpenAI key server-side; the browser receives only a short-lived Realtime client secret.
  • Made the data channel the single authority for connection state, tolerating transient mobile ICE disconnects while still failing closed on real channel/peer failure.
  • Preserved the source and target as independent append-only streams so either may lead without suppressing the other.
  • Gracefully closes translation sessions so OpenAI flushes pending source/target deltas before the transport is released.
  • Reworked Ukrainian into a one-at-a-time ordered suffix queue: each Russian prefix is translated once, streamed immediately, committed in order, and retried safely—even after the microphone stops.
  • Removed far-field audio preprocessing from the phone-microphone path.
  • Broke long target/source transcripts into readable sentence blocks, made the whole history reachable on mobile, and stopped auto-follow when the reader scrolls upward.
  • Improved contrast, touch targets, live-region semantics, and clear pending/retry states.

Root causes

The UI treated WebRTC's transient disconnected state and recoverable server events as fatal. It also tore down the transport immediately on Stop instead of sending session.close, which could drop translated output still draining from OpenAI. The Ukrainian bridge discarded valid in-flight work whenever a newer Russian delta arrived and was cancelled as soon as the microphone stopped. On mobile, end-aligned overflow clipped the beginning of long transcripts while headline-scale type made extended speech unreadable.

Validation

  • npm run check
  • npm run build
  • npm audit --audit-level=high — 0 vulnerabilities
  • Repeated concurrent 19-second real WebRTC audio sessions: every run delivered both source and translated streams with no server errors
  • Complete English → French live-media smoke
  • Complete English → Russian → Ukrainian live-media and bridge smoke
  • Graceful-close WebRTC smoke: session.close flushed the previously missing final translation before session.closed
  • Browser smoke at a true 390 × 844 viewport
  • Long-form mobile layout smoke: 4,228px of transcript remained reachable from the first target sentence through the last source sentence
  • Keyboard-focus, language-picker, microphone-failure/retry, and browser-console smoke

Deployment

The Vercel project under Rubric Labs is linked to this repository, has a sensitive server-only OPENAI_API_KEY for Preview and Production, and owns the verified lilac.chat domain.

@vercel

vercel Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
lilac Ready Ready Preview Jul 30, 2026 9:50pm

Request Review

@DexterStorey DexterStorey changed the title Rebuild Lilac around Realtime Translation Rebuild and harden Lilac around Realtime Translation Jul 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant