Skip to content

Consolidate private Telegram identity, text, voice, and photo input #121

Open
Open
Consolidate private Telegram identity, text, voice, and photo input#121
Labels
P1ImportantassistantAssistant modules and jobsbackendBackend/APIdataData model, migration, storageenhancementNew or improved functionalityhumanCode done or issue blocked on human verificationinfraDeployment and infrastructuretestingTests and QA

Description

@alexeygrigorev
alexeygrigorev
opened on Jul 29, 2026

Consolidate private Telegram identity, text, voice, and photo input

Status: pending
Tags: enhancement, assistant, backend, infra, testing, data, P1
Parent epic: #122 (M3)
Depends on: #123, #125
Blocks: #126
Architecture: _docs/CONVERSATIONAL_AGENT_PLUGIN_ARCHITECTURE.md at 4e5402c

Scope

Turn POST /api/webhook/telegram into the single Telegram channel adapter for the new conversational runtime, behind a disabled-by-default feature flag.

This issue owns:

  • audited admin-managed binding between an enabled DataOps user and immutable Telegram numeric user ID;
  • private-chat-only authorization;
  • normalization of text, callback, and session-command updates into the channel-neutral runtime from Build the static plugin runtime and z.ai conversational adapter #125;
  • persistence through the ordered/idempotent conversation repositories from Define conversational state, proposal records, retention, and recovery #123;
  • bounded Telegram OGG voice-note transcription with Groq whisper-large-v3;
  • bounded Telegram photo/JPEG description and OCR with z.ai glm-4.6v through its native multimodal endpoint;
  • owner-private review/correction/discard of derived text before it becomes conversational input;
  • temporary-media isolation, finally deletion, bounded crash-orphan cleanup, managed secrets, and rollback behavior;
  • a static, public-safe private-chat redirect for group messages.

This issue does not register a todo/Typefully plugin, create a proposal, approve or execute an effect, process generic documents, or decommission the old podcast polling bot. It connects the channel to injected/fake runtime behavior; #126 supplies the first complete domain journey.

Current baseline and migration boundary

The current backend/src/routes/telegram.ts:

  • validates a webhook secret and chat allowlist;
  • handles messages but not callback queries;
  • treats telegram:<from.id> as a string rather than a verified DataOps identity;
  • routes /podcast and /social, and can create a task under a compatibility flag;
  • captures generic attachments into intake;
  • can call the social assistant and Typefully before the new exact approval boundary.

Chat allowlisting remains an ingress control, not authorization. Existing auth Session records remain separate from conversational state.

The read-only ../telegram-writing-assistant demonstrates Telegram file download, whisper-large-v3, and reply-with-transcript patterns. Do not copy its durable inbox/image storage, broad format handling, Groq vision model, success-only cleanup, polling topology, or chat-ID-only authorization. Do not modify that repository.

The new adapter is enabled by a runtime flag such as CONVERSATIONAL_TELEGRAM_ENABLED=false by default. While disabled, existing production behavior remains unchanged. When enabled:

Verified identity and admin API

Use IdentityBinding from #123. Telegram identity is from.id as a canonical decimal string. Username, display name, chat ID, and forwarded sender data are never identity keys.

Add authenticated admin-only endpoints following backend route conventions:

  • GET /api/conversational/identity-bindings?channel=telegram — list bounded binding metadata;
  • POST /api/conversational/identity-bindings with { userId, channel: "telegram", channelUserId } — create/reactivate only after the DataOps user exists and is enabled;
  • POST /api/conversational/identity-bindings/telegram/{channelUserId}/revoke — revoke with an audit event; do not hard-delete history.

Requirements:

  • only an authenticated enabled admin may list/create/revoke;
  • operator, anonymous, forged x-user-id, Telegram webhook, and disabled users receive no admin authority;
  • one active binding per Telegram numeric user; conflicting user linkage returns 409 and requires explicit admin resolution;
  • creation/revocation uses Define conversational state, proposal records, retention, and recovery #123 optimistic/conditional primitives and records admin actor/time without Telegram username dependence;
  • revoked/disabled bindings fail closed immediately for every later message/action;
  • API responses/logs do not include conversation content, bot tokens, secret references, provider keys, or raw media.

No identity-management UI or self-service Telegram linking is included.

Webhook ingress and authorization order

For every update:

  1. validate the rotated Telegram webhook secret with constant-time comparison;
  2. parse a bounded JSON body and reject malformed/oversized input;
  3. enforce configured chat ingress allowlist where retained;
  4. classify private versus group/supergroup/channel before any model/provider call;
  5. for private work, require from.id, resolve an active binding, load the enabled DataOps user, and bind the channel conversation to that identity;
  6. normalize and append the input with update_id plus subtype as the stable idempotency key through Define conversational state, proposal records, retention, and recovery #123;
  7. call the injected core/runtime boundary at the exact returned conversation revision;
  8. discard stale output if a newer event advanced the revision; bounded reprocessing follows Build the static plugin runtime and z.ai conversational adapter #125 and never commits/replies with stale model work;
  9. send only the normalized core interaction through a Telegram client abstraction.

The webhook secret and allowed chat do not substitute for the user binding. Unknown, revoked, disabled, missing-sender, malformed, or unsupported private input invokes no model/media provider/domain mutation and returns generic linking/unsupported guidance without revealing whether another identity/conversation exists.

Private and group behavior

MVP conversational behavior is private-chat only (chat.type=private). A group/supergroup/channel update must not:

  • resolve or reveal private conversation state;
  • append a private conversation event;
  • download media;
  • invoke Groq, z.ai, the conversational runtime, a plugin, or an executor;
  • render a proposal/private preview link.

When Telegram delivers a directed group mention/reply/command, the adapter may send one rate-limited static message: continue with the DataOps bot in private chat. Other group traffic is acknowledged without response. The text contains no user/resource-specific information.

Normalized input contract

Map Telegram updates to #125 core events without domain behavior:

  • private text message → message with bounded text and Telegram provenance;
  • callback query → button_action containing only the opaque action token/action reference plus trusted actor/chat/update provenance;
  • /new, /sessions, /continue, /cancel, /discard, /help → session_command/core action;
  • voice note → staged voice_note preview, then confirmed/corrected message;
  • photo plus optional caption → staged photo preview, then confirmed/corrected message.

Callback handling must call Telegram answerCallbackQuery promptly, reject missing/fabricated/oversized callback data safely, and never decode proposal content, permission, destination, or executable parameters from callback bytes. #124 owns approval-token semantics.

Core command meanings:

  • /new: explicitly end the active conversation and create a new one; executing attempts remain status-only.
  • /sessions: list only the actor's nonexpired resumable conversations with bounded public-safe labels.
  • /continue: present/select one of those conversations using server-side opaque action references; never accept another user's raw conversation ID.
  • /cancel: invoke core cancel_proposal, revoking unexecuted presentations while retaining the draft/conversation.
  • /discard: invoke core discard_draft; it cannot cancel an executing attempt or end the conversation.
  • /help: static explanation of text, voice, photo, review, approval, cancel/discard, and private-chat limits.

Plugins cannot define these transitions. Detached text such as “yes” remains a normal message, never approval.

Event deduplication, ordering, and stale turns

Use #123's atomic append primitive:

  • primary idempotency key is telegram:<update_id>:<normalized-kind>;
  • one accepted input advances the conversation revision once and receives one event sequence;
  • Telegram retries return the stored acknowledgment/result reference without redownloading media or invoking runtime/provider again;
  • concurrent different updates may commit only at their expected revision and must be processed in stored sequence order;
  • runtime work starts from revision N and may append/send output only if the conversation is still at N; otherwise discard it and use Build the static plugin runtime and z.ai conversational adapter #125's bounded stale-turn handling;
  • callback acknowledgments may be repeated safely, but no action event is applied twice.

No timestamp-based reordering, unbounded queue, or silent event merge.

Voice-note processing

Accept only Telegram message.voice whose declared media is OGG/Opus and duration is at most five minutes.

Limits and validation:

  • 20 MB maximum based on Telegram metadata and independently enforced while streaming;
  • authenticated Telegram getFile lookup using the configured bot client; never fetch an arbitrary URL supplied in update JSON;
  • validate returned Telegram file path, OGG magic/container, duration, and nonempty body before provider call;
  • one voice item per event; ordinary message.audio and voice-as-document are unsupported;
  • bounded configurable download/provider timeouts with safe upper limits below the Lambda timeout;
  • no automatic provider retry; on timeout/rate limit/invalid transcript the bot asks for text or a new voice note;
  • Groq request uses whisper-large-v3, response format text, bounded output, and the dedicated injected client;
  • blank/oversized/malformed/secret-like provider errors produce a safe code, not raw provider content.

Photo processing

Accept only Telegram message.photo; select the largest Telegram size that remains within configured bounds. Photos/images sent as documents are generic uploads and unsupported.

Limits and validation:

  • 10 MB and 20 megapixels maximum, checked from Telegram metadata where available and again after bounded streaming/decode;
  • authenticated getFile, Telegram-controlled path only;
  • JPEG magic/decode validation; reject corrupt, unsupported, or decompression-risk content before provider call;
  • one photo per event; preserve an optional bounded caption as separate provenance;
  • z.ai native multimodal request uses configured model glm-4.6v, bounded prompt/output/timeout, and an injected client;
  • output is a bounded plain-text description plus OCR; markdown/HTML from provider is treated as text and cannot create actions;
  • failure asks the operator for text and never guesses from the caption alone unless the operator explicitly submits that caption as text.

The former Groq Llama vision pattern from ../telegram-writing-assistant is not used.

Temporary media and derived-text lifecycle

Use a dedicated Lambda temp root such as /tmp/dataops-telegram-media, a random per-update directory with restrictive permissions, generated filenames, and no Telegram username/original filename.

  • stream downloads directly to the bounded temp file; do not buffer/base64 media into logs, DynamoDB, events, exports, or conversational-model context;
  • only the dedicated transcription/vision client receives the bytes;
  • wrap download, validation, provider call, and response handling in try/finally that removes the invocation directory on success, validation failure, timeout, provider error, reply failure, or stale-turn discard;
  • cleanup does not follow symlinks or touch other /tmp paths;
  • a lazy cold-start/before-processing reaper deletes only dedicated subdirectories older than 15 minutes, with a bounded entry/byte/time budget; excess work waits for a later invocation;
  • tests inject temp root, clock, filesystem, clients, and cleanup limits.

The transcript/photo description and caption are owner-private untrusted text. Store them only through #123's private payload/event contract with 30-day expiry; exclude them from application logs and audit payloads. First show a bounded preview with Use this text and Discard. The user may type a correction. Only explicit use/correction creates the downstream conversational message; discard removes staged derived text as soon as practical and invokes no runtime/plugin.

Telegram rendering and client boundary

Plugins never call Telegram. Provide an injected client supporting only the adapter needs:

  • getFile/bounded download;
  • send message;
  • send inline keyboard;
  • edit/disable obsolete controls;
  • answer callback query.

Render channel-neutral core interactions with bounded Telegram-safe text. Long content is chunked with stable ordering or linked only through a future authorized preview service; no public/private URL is invented here. Escape/avoid unsafe parse modes. Provider/user text must not become Telegram markup or callback data without encoding.

Telegram API failures map to safe retryable/nonretryable codes. Retry only idempotent reply/callback acknowledgment with a small fixed bound; never retry provider work or reappend an event to make a reply succeed.

Secrets and infrastructure

Keep the existing shared Telegram JSON secret (botToken, webhookSecret, allowedChatIds) and its rotated-secret cache behavior, but validate required fields when the new adapter is enabled.

Add out-of-band Secrets Manager ARN parameters and exact backend environment references for:

  • Groq transcription key, e.g. GroqTranscriptionApiKeySecretArn / GROQ_TRANSCRIPTION_API_KEY_SECRET_ARN;
  • z.ai vision key, e.g. ZaiVisionApiKeySecretArn / ZAI_VISION_API_KEY_SECRET_ARN.

The z.ai vision ARN may intentionally reference the same pre-created secret value as #125's conversational z.ai ARN, but the purpose-specific configuration/model/endpoint remains separate and no key is copied. Grant secretsmanager:GetSecretValue only for the exact configured ARNs. The OIDC deploy workflow passes ARNs and non-secret model/base-URL/timeout/limit parameters only; it never retrieves or prints keys.

Enabling text-only Telegram may not require media secrets. Voice/photo capabilities are independently disabled by default and fail closed at startup/config validation when enabled without their required secret/model settings. Feature flags:

  • global conversational Telegram;
  • voice input;
  • photo input.

Infrastructure/tests must prove flags disable provider lookup/download and preserve unrelated backend routes.

Acceptance criteria

  • Authenticated admin APIs create/list/revoke Telegram numeric-user bindings with uniqueness, enabled-user validation, optimistic/audit evidence, and correct 401/403/404/409 behavior; no self-service/UI is added.
  • Webhook secret and chat allowlist remain ingress controls, while every private event/action additionally requires an active binding to an enabled DataOps user.
  • Only private chats enter conversation/media processing; group/channel input receives at most a rate-limited static private-chat redirect and creates no private event/provider/model/mutation call.
  • Text, callback, core session commands, voice preview, and photo preview normalize to the channel-neutral contracts without plugin/domain logic in the adapter.
  • /new, /sessions, /continue, /cancel, /discard, and /help map to the stated core behavior; /todo, /social, /podcast, generic files, and detached “yes” cannot bypass approval in new mode.
  • Telegram update_id/kind deduplication, ordered append, expected revisions, and stale-turn rejection prevent duplicate events/provider calls/replies and stale runtime output.
  • Callback queries are promptly acknowledged, opaque/bounded, actor/chat-bound in trusted provenance, and contain no proposal/permission/executable data.
  • Voice accepts one ≤20 MB, ≤5-minute OGG/Opus note, uses Groq whisper-large-v3, and rejects arbitrary URLs, audio/doc formats, invalid containers, oversized media, and unsafe provider output.
  • Photo accepts one Telegram photo/JPEG ≤10 MB/20 MP, uses z.ai glm-4.6v, preserves bounded caption provenance, and rejects image documents, invalid JPEG/decode, oversized/decompression-risk media, and unsafe provider output.
  • Derived text is shown owner-privately before conversational use, can be corrected/discarded, is untrusted, is absent from logs/audit, and expires after 30 days.
  • Temporary bytes reach only the intended media provider, never persistence/log/model/export, and are deleted in finally on every exit; bounded orphan cleanup is symlink-safe and limited to the dedicated temp root.
  • Provider/Telegram failures are bounded/redacted, create no guessed conversation input, and do not trigger automatic media/provider retries.
  • Telegram, Groq, and z.ai secrets use exact managed ARN references and least-privilege SAM/OIDC wiring; no plaintext key or secret ARN appears in logs/responses/context/fixtures.
  • Global/voice/photo flags default off; disabled paths perform no provider secret lookup/network call and existing unrelated backend behavior remains compatible.
  • Current source repositories, podcast polling bot, plugins, proposal/approval/worker, and domain executors are not modified by this issue.

Test scenarios

Admin binding lifecycle and authorization

Given enabled/disabled DataOps users, admin/operator/anonymous requests, an existing binding, conflict, revoke, and reactivation
When identity API calls run
Then only admin succeeds, one Telegram numeric user maps to one active user, conflicts are explicit, revoked/disabled users fail immediately, audits are bounded, and no private content/secret is returned.

Private, unlinked, and group isolation

Given linked/unlinked/revoked senders in private chats plus group/supergroup/channel messages and directed/ordinary group traffic
When the webhook receives each update
Then only linked private senders append/invoke; unlinked private senders receive generic guidance; directed group traffic gets only the static redirect; all other group traffic is silently acknowledged; no group path resolves private context or calls providers/runtime.

Text, callbacks, and commands

Given text, duplicate callbacks, malformed/oversized callback data, every supported command, unsupported commands/files, and a fake core runtime
When updates are normalized
Then event kinds/provenance/actions are exact, callback acknowledgment is prompt/idempotent, commands invoke only core-owned actions, unsupported inputs create no mutation, and no channel-specific data leaks into plugin/domain contracts.

Duplicate, concurrent, and stale delivery

Given duplicate/out-of-order update IDs, concurrent different messages at one revision, and delayed fake runtime output from revision N after N+1 exists
When ingestion/runtime completion races
Then each input commits once in sequence, media/model work does not duplicate, stale output is discarded/not sent, and bounded retry/reprocessing never loops.

Voice success and rejection matrix

Given fake Telegram/Groq clients and valid OGG, lying metadata, stream-over-limit, >5-minute, wrong magic/container, empty/corrupt, arbitrary path/URL, timeout, rate limit, malformed/oversized transcript, and duplicate delivery
When voice handling runs
Then only one valid bounded file reaches whisper-large-v3 once, a private preview appears, explicit use/correction creates one message, every invalid/failure path asks for text safely, and no raw bytes/error secrets persist.

Photo success and rejection matrix

Given fake Telegram/z.ai clients and valid photo sizes/caption, >10 MB, >20 MP, invalid JPEG, decode bomb, image-as-document, arbitrary path/URL, timeout, malformed/oversized OCR, markup/action-like provider text, and duplicate delivery
When photo handling runs
Then only the valid bounded JPEG reaches glm-4.6v once, caption/description preview is private, explicit use/correction creates one untrusted message, invalid/failure paths create none, and provider text cannot create actions.

Cleanup on every exit and orphan reaping

Given injected temp storage with invocation directories, symlinks, fresh/old orphans, excessive files/bytes, and failures at download/validation/provider/persist/reply/stale-output stages
When processing and reaper run
Then the current directory is removed in every finally, only old dedicated orphans are removed within budget, symlinks/outside paths are untouched, and later invocations can finish deferred cleanup.

Secret redaction and configuration flags

Given disabled/enabled global/media flags, missing/malformed fake secrets, secret-like nested provider/Telegram errors, and captured logs/responses
When startup and failures run
Then disabled paths need no media key/call, enabled missing config fails closed, exact ARN IAM/env wiring is used, and no key/token/secret ARN/raw provider body appears.

Legacy rollback

Given new adapter flag off and on
When existing /status, intake, /social, /podcast, direct-task compatibility, and unrelated backend route tests run
Then flag-off preserves current behavior; flag-on prevents all mutation bypasses and gives safe compatibility guidance; switching the flag off restores the prior route without data migration or changing unrelated APIs.

HUMAN gates

  • [HUMAN] An AWS operator pre-creates/confirms Groq transcription and z.ai vision secrets, deploys exact SAM/OIDC ARN wiring, and verifies disabled flags perform no provider call/key lookup.
  • [HUMAN] An admin binds a real Telegram numeric user to the intended enabled DataOps account, then verifies revoke blocks the next private message without exposing prior context.
  • [HUMAN] A linked user sends private text and uses /new, /sessions, /continue, /cancel, /discard, and /help; duplicate callbacks/updates do not duplicate visible actions.
  • [HUMAN] A linked user sends one real multilingual ≤5-minute voice note; Groq whisper-large-v3 returns a useful private transcript, correction/discard works, and no raw audio remains.
  • [HUMAN] A linked user sends one real Telegram photo with caption; z.ai glm-4.6v returns useful description/OCR, correction/discard works, and no raw photo remains.
  • [HUMAN] An unlinked private user and a directed group mention both fail closed; the group receives only the static private-chat redirect and no provider/model/private event is observed.

Required verification

Run from repository root:

npm --prefix backend test
npm --prefix backend run typecheck
npm --prefix backend run build
make sam-validate
make sam-build

Tester must additionally:

  • run focused Telegram/identity/media/config tests using fake Telegram, Groq, z.ai vision, secret, clock, filesystem, and core-runtime clients;
  • assert no automated test opens an external provider/Telegram connection;
  • inspect generated SAM for exact secret ARN environment/IAM wiring and disabled-by-default flags;
  • inspect temp-root cleanup and captured logs/exports for raw bytes, transcripts/descriptions, bot token, provider keys, and secret ARNs;
  • report exact commands, exits, test counts, and whether Playwright was skipped. Playwright/screenshots are not required because no portal UI changes.

Assistant Engineer reviews the Telegram/media/provider and core-action boundary before Tester acceptance. After push, On-Call monitors OIDC deployment and redacted Telegram/media errors. If automated acceptance completes before HUMAN checks, use Refs #121, add human, and leave the issue open per process.

Rollback

The global adapter and voice/photo capabilities are independently disabled by configuration. Rollback disables new ingestion/provider calls without deleting #123 records, changing identity bindings, replaying callbacks, or reverting unrelated backend APIs. Existing route behavior remains available while the global flag is off until #128 performs final cutover.

Out of scope

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1ImportantassistantAssistant modules and jobsbackendBackend/APIdataData model, migration, storageenhancementNew or improved functionalityhumanCode done or issue blocked on human verificationinfraDeployment and infrastructuretestingTests and QA

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions