docs: design spec for multimodal stream governance - #107
Open
tishachawla-jg wants to merge 2 commits into
Open
tishachawla-jg wants to merge 2 commits into
tishachawla-jg wants to merge 2 commits into
Conversation
Draft design for governing continuous media streams (audio, video, realtime voice) on cost, provider-agnostic. Builds on primitives that already exist: the tick() time-based moment, stream_complete, and the declared-but-unimplemented CANCEL actuator (#67). Core idea is time-windowed accrual driven by tick, not per-frame pricing, so overhead is bounded regardless of frame rate. Pricing section grounded in how real 2026 streaming APIs bill (OpenAI Realtime per-token audio with cached-audio rates; Gemini Live fixed tokens-per-second audio and per-second video): the universal unit is seconds of stream, mapped per provider to a billable quantity, so one price(modality, direction, quantity, cached) signature covers both token-metered and time-metered providers. Status: draft. No implementation yet; phasing in the doc. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a section grounding the abstraction in the three real voice pipelines (speech-to-speech, speech-to-text, text-to-speech): how each flows, the unit it bills in, its distinct denial-of-wallet shape, and an approximate 2026 anchor price. Shows that three billing units (tokens, minutes, characters) and two directions collapse into one price() call and one accrual loop, and motivates why the detectors cover duration, spend, and rate rather than a single one. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a draft design spec (
docs/notes/multimodal-stream-governance-spec.md) for governing continuous media streams (audio, video, realtime voice) on cost, provider-agnostic. Safety/content moderation explicitly out of scope.Approach
Reuses primitives that already exist rather than building a new engine:
Governor.tick(now)(time-based moment) is the governance seamstream_completealready existsActionKind.CANCELis already declared (Actuators: CANCEL is declared but unimplemented, and the two connectors disagree #67) and is the streaming actuator; this spec gives it its purposeCore idea: time-windowed accrual. Cost accrues locally as frames arrive and flushes to the ledger on a fixed window (default 500ms); budget checks run at flush time, not per frame. Overhead is bounded regardless of frame rate.
Pricing, grounded in real 2026 APIs
Research into how streaming APIs actually bill shaped the abstraction:
The unifying unit is seconds of stream, mapped per provider to a billable quantity. One
price(modality, direction, quantity, cached)signature covers both token-metered and time-metered providers; a provider stream adapter does the mapping.Status
Draft / design. No implementation yet; the doc includes a phasing plan. Opening as a PR so the design is reviewable and citable.
🤖 Generated with Claude Code