You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Status: pending
Tags: enhancement, assistant, work-engine, backend, infra, testing, data, P1
Parent: #122 (M6)
Depends on: #126 (code dependency satisfied at c47c525; HUMAN-only rollout remains open and non-blocking)
Blocks: #128
Architecture: _docs/CONVERSATIONAL_AGENT_PLUGIN_ARCHITECTURE.md at 4e5402c
Architect review: #127 (comment)
Related migration debt: #77, #80
Dependency contract — satisfied on main
#126 agent-verifiable implementation merged to main as c47c525 with Refs #126. The issue remains open with human only for the credentialed deployment and live Telegram gates. Under _docs/PROCESS.md Human Verification, those external checks do not block the next agent-verifiable implementation. #127 is ready and must consume two small shared primitives from #126 rather than copy todo-specific orchestration:
a channel-neutral proposal coordinator/adapter seam that accepts a validated plugin candidate, stores an immutable ProposalSpec, renders plugin-supplied content through core-owned controls, and reuses Add transactional approval and durable execution recovery #124 approval/worker semantics; and
a durable actor-private result-notification outbox that lets a worker publish a safe result plus an owner-bound private payload reference for later Telegram delivery.
If main regresses to todo-only orchestration or loses durable asynchronous result delivery, #127 becomes blocked again and the shared primitive must be corrected first. #127 adds one Typefully proposal adapter and one capability-scoped executor; it must not create a second conversational core, approval path, Telegram completion path, workflow DSL, or generic provider framework.
Use the existing plugin channel value channels: ['telegram']. #121 proves linked-user private-chat scope before core entry. Do not add a second telegram_private channel enum.
Product effect
After a complete deterministic preview and exact core approval, create exactly one saved Typefully draft in one authorized logical account:
plugin: typefully
action: propose_draft
operation: create
deliveryMode: operator_reconciliation_only
account: alexey | datatalksclub
platforms: one or both of x | linkedin
result: one unscheduled, unpublished, unshared saved draft
There is no update, delete, schedule, publish, queue/next-free-slot, preferred time, public share, tag, media, analytics, webhook, comment, account-management, or multi-draft capability. Scheduling and publication remain manual Typefully actions.
Public-source gate before z.ai
A Typefully request may use only:
the current typed operator message after the operator explicitly confirms that exact text is intended as public social-post source; and
opaque trusted source references already classified public by core policy.
The Typefully plugin is not exposed to the generic model catalog until that public-source grant exists. A bounded static intent recognizer may notice an ordinary typed request containing explicit social/Typefully/X/LinkedIn posting intent and present a core-owned Use this typed text as public source confirmation without invoking z.ai. Ambiguous text receives a local clarification. Only after confirmation may the first z.ai selection/load turn run.
For every Typefully z.ai turn, build a separate context containing only the confirmed current typed source, prior Typefully draft fields already covered by the same public grant, trusted public refs, and the Typefully instructions/schema. Never include ordinary recent conversation history, conversation summaries, todo context, private Telegram events, organization/user-private/restricted/credentials-adjacent content, or unconfirmed content. The model cannot see or choose a classification.
Voice transcripts, photo descriptions/captions, files, fetched URLs, and document ingestion are not eligible in this slice—even if they were confirmed for another plugin. Ask for a new typed public-safe summary. Do not fetch a supplied URL, silently sanitize/reclassify private text, or let generated copy downgrade source classification.
Core creates PublicSourceGrant/equivalent state with owner, private payload reference, source digest, classification=public, policy revision, confirmation event, and 30-day expiry. The model may return copy but cannot manufacture/modify sourceRefs; core attaches at most eight opaque trusted refs and their digests/policy revisions to the proposal. A stale, missing, cross-owner, non-public, or policy-drifted grant stops before z.ai/Typefully and requires fresh typed confirmation.
The adapter plugs into #126's shared coordinator. It owns only Typefully validation, clarification copy, immutable-spec construction, deterministic preview rendering, and capability registration. Core owns draft/proposal/presentation transitions, approve/request-changes/cancel/discard controls, expiry, exact approval claim, worker queueing, and result notification.
Missing or ambiguous logical account, platforms, purpose, or typed public source asks one concise question. It never asks for publication time. Request changes creates a complete new immutable proposal, revokes every old presentation, and requires a fresh full preview. Detached “yes”, group input, stale/old controls, assistant-job status, cancel, discard, and expiry perform no provider write.
Exact non-lossy action schema
Use additionalProperties: false at every object level. Normalize accepted strings with Unicode NFKC and trim surrounding whitespace; reject ASCII/Unicode controls, zero-width/bidi-control text, empty normalized text, invalid Unicode, and overflow. Never truncate, slice, summarize, reorder, merge, or drop model/operator content.
The exact candidate fields and bounds are:
account: exactly alexey or datatalksclub;
platforms: unique array, minItems: 1, maxItems: 2, values only x and linkedin;
xPosts: present iff x is selected; minItems: 1, maxItems: 25; each string 1–4,000 Unicode code points;
linkedinPosts: present iff linkedin is selected; exactly one item (minItems: 1, maxItems: 1); text 1–8,000 Unicode code points;
optional draftTitle: 1–120 Unicode code points when present;
optional scratchpadText: 1–1,000 Unicode code points when present;
core-attached sourceRefs: 1–8 unique opaque refs, each at most 160 ASCII characters and resolving to a current trusted public grant;
total canonical UTF-8 candidate payload: at most 64 KiB.
Reject a content array for an unselected platform or a selected platform without its required content. Reject duplicate platforms/refs/posts, empty items, arrays/objects in text positions, and all unknown fields. Recursively reject any schedule/publish/share/media/tag/comment/analytics/webhook/update/delete/target/provider field or semantic equivalent, including publish_at, publishAt, preferredTime, schedule, now, next-free-slot, share, share_url, media_ids, targetDraftId, draft_id, numeric social-set ID, provider URL, and existing-target/base-revision input.
Remove the existing Typefully client and social generator's silent 4,000/8,000/120/1,000 truncation. The same strict validator runs on model output, proposal construction, stored-spec load, preflight, and request serialization. Overflow is a validation/clarification result with zero provider calls.
Immutable spec and complete Telegram preview
The ProposalSpec binds:
plugin/action/operation/effect, exact build/schema/policy digests, and deliveryMode=operator_reconciliation_only plus its digest;
bounded public-source refs, source digests, classification, and policy revision;
exact account-configuration digest and account-scope authorization digest;
no provider/numeric social-set ID, target draft ID, or base revision because this is create-only.
The canonical payload and rendered-view hashes cover every field above. Telegram renders the full approved content without truncation: chunk long previews into deterministic numbered messages, show every ordered post and optional title/scratchpad, then show logical account, platforms, public-source proof summary, operator_reconciliation_only, and the exact effect “Create one unscheduled, unpublished, unshared Typefully saved draft.” Approval controls appear only after every preview chunk is durably delivered. The worker executes the stored spec byte-for-byte after provider serialization; it cannot regenerate or reinterpret content.
Account-scoped authorization and configuration
Extend the existing ApprovalPermission contract rather than create another authorization system. For typefully:create-saved-draft, store:
enabled and monotonically increasing revision;
a sorted unique allowedResourceKeys subset of typefully:account:alexey and typefully:account:datatalksclub (maximum two);
the current safe account-configuration digest;
the fixed delivery-mode digest for operator_reconciliation_only.
Bind all four to the proposal. At invocation/proposal creation, core verifies the active linked actor, permission, and selected resource key without revealing unauthorized account details. The #124 approval transaction must atomically condition-check enabled, exact permission revision, selected account membership, account-configuration digest, and delivery-mode digest while claiming the proposal. A read followed by an unscoped approval check is insufficient.
Immediately before dispatch, the worker repeats actor state, permission revision/scope, proposal/build/policy/payload/view hashes, account-config digest, and delivery-mode digest checks. Numeric Typefully social-set IDs are worker-only configuration. The model, plugin payload, Backend/Telegram Lambda, client APIs, proposals, Telegram messages, and long-lived receipts see only logical account keys and safe digests.
Changing actor scope, a numeric account mapping, configuration revision, or delivery mode invalidates the old proposal. It cannot be approved/dispatched; the operator must generate and approve a fresh preview.
Side-effect-free worker preflight
Add the smallest shared executor preflight hook and run it before #124 persists dispatchStartedAt or any provider-call marker. Typefully preflight may read trusted local/DynamoDB/AWS Secrets Manager state but makes no Typefully request and has no external business effect. It verifies:
CONVERSATIONAL_TYPEFULLY_EXECUTION_ENABLED=true;
the current actor/permission/account scope and exact permission revision;
the exact pre-created Typefully secret is readable and contains a non-empty token;
the attempt still has the current worker lease/envelope.
A missing/disabled flag, permission/config drift, missing mapping, unreadable/malformed secret, stale lease, or validation failure is failed_safe; dispatchStartedAt remains absent and Typefully call count is zero. Recheck the lease when conditionally recording dispatch intent after preflight so a stale worker cannot cross the boundary.
Use two independent default-off flags:
CONVERSATIONAL_TYPEFULLY_PLUGIN_ENABLED controls catalog visibility/public-source entry and lives with the backend safe configuration;
CONVERSATIONAL_TYPEFULLY_EXECUTION_ENABLED exists only on the execution worker and controls external dispatch.
Moving from preflight to dispatch is one conditional state transition. After dispatchStartedAt exists, operator_reconciliation_only forbids automatic provider replay.
Worker-only credential and provider capability
Keep z.ai behind #125's Anthropic-compatible model boundary. Move all Typefully write capability out of BackendFunction and into ConversationalExecutionWorkerFunction:
the exact pre-created Typefully token secret ARN/name environment reference;
TYPEFULLY_SOCIAL_SET_ALEXEY and TYPEFULLY_SOCIAL_SET_DATATALKSCLUB numeric mappings;
Typefully base URL/config revision and execution flag;
secretsmanager:GetSecretValue for only that exact secret ARN.
Remove Typefully secret/mapping environment variables and Typefully GetSecretValue permission from BackendFunction, Telegram/model/plugin paths, and other Lambdas. The execution worker resolves the bearer token during preflight/dispatch and never stores it in an attempt, private payload, event, receipt, error, or log. The capability exposes only createSavedDraft(approvedSpec) for one already-authorized logical account; no list/get/update/delete/schedule/publish/general fetch client is injected.
Exact provider request and validated success
After one persisted dispatch intent, send at most one request:
POST /v2/social-sets/{worker-resolved-social-set-id}/drafts
The JSON body contains only the selected platform content, optional approved draft_title/scratchpad_text, and share: false. It contains no publish_at, schedule/publish target, media, tags, comments, public sharing, provider target ID, or unselected platform. Serialization must equal the stored ordered proposal; it does not truncate or mutate it.
The first release uses operator_reconciliation_only; do not send or infer a provider idempotency/correlation key. Claim success only for a fully read HTTP 201 whose body validates all of:
non-empty draft ID and the exact intended numeric social-set ID;
status='draft' and publish_state=null;
scheduled_date, published_at, every platform/post published URL, and share_url are null/absent;
selected/unselected platform enablement corresponds exactly to the proposal;
every returned ordered selected-platform post text equals the approved normalized content;
any returned title/scratchpad equals the approved values;
a non-empty private editing URL has the expected HTTPS Typefully origin and contains no credential material.
A fully received, documented Typefully create rejection (400, 401, 402, 403, 404, 422, or 429) with a bounded parseable response is failed_safe. Transport/DNS/TLS failure, connection loss, timeout/abort, 5xx, any unlisted status, malformed/oversized body, contradictory or non-201 success, validation mismatch, or crash after dispatch intent/request send is outcome_unknown. Provider/account-level automatic settings are not assumed harmless: the live gate must compare Typefully's stored ordered content and unscheduled/unpublished/unshared fields to the approval.
Outcome, retention, and private result delivery
Never replay an attempt after dispatch intent. Duplicate callbacks, Stream deliveries, workers, lease recovery, and scheduled recovery find the dispatch marker and return/retain the current terminal or outcome_unknown state without another Typefully call.
On validated success:
the one-year safe effect receipt stores only draft ID, logical account, selected platforms, proposal/attempt IDs, canonical payload hash, safe response classification, and timestamps;
a separate owner-bound private payload stores the private editing URL and minimal display result, expires after 30 days, and is excluded from portable/shared status and safe receipt;
the shared outbox dispatcher—not the Typefully executor—loads the payload for the same active actor/channel and sends the completion in Telegram.
The Typefully executor receives no Telegram token/client. Shared attempt/status/reconciliation APIs return only the safe receipt; they never return the private URL or payload body. Cross-actor, wrong-channel, expired, deleted, or guessed payload refs return not found without disclosure. After 30 days the URL is unavailable while the safe receipt remains for one year. Raw prompts/completions, public-source text, proposal payloads, and private provider response payloads follow the architecture's 30-day retention; safe audit/effect metadata follows one year.
If #126's outbox cannot conditionally deduplicate worker result delivery, preserve private owner binding, and deliver asynchronous completion, #127 stays blocked.
operator_reconciliation_only
For outcome_unknown, show the exact safe message through the shared result outbox:
Typefully may have created this draft. Do not retry. An authorized operator must reconcile it in Typefully.
Disable retry/requeue/approve-again controls for the old attempt. Reconciliation is an authenticated admin-only action requiring the current attempt revision and an accepted private runbook. It records one bounded resolution:
found: exact logical account, unique draft ID, exact ordered approved content, unscheduled/unpublished/unshared state, and a time window consistent with the attempt were verified; or
not_found: the authorized operator completed the accepted account search and found no matching draft.
Store only sanitized proof classification, actor, draft ID when found, attempt/proposal/hash, and timestamp in the one-year receipt. A found private edit URL goes only in a fresh 30-day owner-private payload. Never store screenshots, account names/IDs beyond the logical key, raw provider exports, or runbook details in the public repo/issue.
Neither resolution retries, deletes, updates, schedules, or publishes. If not_found and the operator still wants a draft, create a new proposal/version from current public typed input and require a new exact approval; never replay the old attempt. Rollback never deletes an external draft.
Remove all legacy write bypasses
Conversational rollout eliminates, rather than conditionally preserves, every non-worker Typefully write path:
socialDraftAssistant may stage local copy only or be retired; remove its direct createSavedDraft(...) call and provider result/private URL artifacts;
POST /api/assistant-jobs/{id}/typefully-draft is removed or returns 410 Gone and cannot instantiate/inject a provider client regardless of assistant-job status;
/social with or without arguments returns static guidance to send a typed social request and confirm it as public source; it invokes neither z.ai nor Typefully;
waiting_approval, approved, succeeded, model output, assistant artifact state, webhook input, or a legacy feature flag can never authorize a provider call.
There is no temporary simultaneously enabled legacy mode. Reuse #77/#80 validation/client fixtures only after moving the strict non-truncating create serializer into the worker capability. Their status-only approval/retry semantics and long-lived private URL artifact are removed for this flow.
Enablement and rollback
Both flags default off. Automated implementation/tests use fake z.ai, Telegram, Secrets Manager, and Typefully adapters with no network. Agent-verifiable code may ship with execution disabled.
Rollback first disables plugin visibility, then disables worker execution before new dispatch. It preserves all proposals, attempts, safe receipts, private-payload TTLs, outbox events, and reconciliation evidence. It does not re-enable a legacy route, make an unknown attempt retryable, or delete a Typefully draft.
The production registry exposes only typefully.propose_draft with typefully:create-saved-draft, roles admin|operator, existing channel telegram, and create-only operator_reconciliation_only semantics.
Before either z.ai turn, core requires an owner-bound confirmation for the exact current typed public source. Typefully model context excludes ordinary/private history, summaries, derived media, files, fetched URLs, and non-public refs.
Model output cannot assert classification or source refs. Core binds at most eight current trusted public grants/digests/policy revisions; stale/cross-owner/private/derived refs stop before provider egress.
The exact schema enforces all numeric/item/total bounds, platform-content correspondence, Unicode/control rules, and recursive forbidden fields. Overflow or invalid data is rejected; no code path silently truncates/reorders/drops content.
Proposal and view hashes cover every ordered copy/account/source/auth/config/delivery field, and Telegram delivers a complete deterministic multi-message preview before controls become active.
Existing ApprovalPermission state carries bounded allowed logical accounts, exact revision, account-config digest, and delivery-mode digest. Invocation checks them, approval condition-checks them atomically, and worker preflight rechecks them.
Account/config/permission drift, unauthorized account selection, stale controls, detached text, cancel/discard/expiry, and request changes create no provider call; revision requires a fresh complete preview.
A side-effect-free executor preflight runs before dispatchStartedAt and proves execution flag, scope/config/delivery hashes, exact mapping, secret availability, validation, and lease. Any failure leaves no dispatch marker/call and is failed_safe.
Only ConversationalExecutionWorkerFunction receives Typefully secret reference, numeric social-set configuration, exact-secret GetSecretValue, execution flag, and create capability. Backend/model/Telegram/other Lambdas have none.
The worker sends at most one exact non-truncated POST after dispatch intent, with share:false, selected content only, and no update/schedule/publish/share/media/target field.
Success requires a fully validated 201 matching account, exact ordered content, draft/null publish fields, no public share/published URLs, and safe private edit URL. Contradictory/malformed success is never accepted.
The fixed operator_reconciliation_only matrix classifies proven pre-dispatch/documented rejection as failed_safe and all ambiguous post-dispatch cases as outcome_unknown; no automatic replay occurs after dispatch intent.
Duplicate approvals, callbacks, Stream records, workers, expired leases, and recovery cannot produce another Typefully call after the dispatch marker or overwrite terminal/unknown state.
Validated success stores a one-year redacted safe receipt and a separate 30-day actor-private edit-URL payload. Only Ship approved conversational todo creation #126's authorized outbox delivers the private result; shared status never exposes it and the executor has no Telegram capability.
outcome_unknown disables retry and supports only admin manual found|not_found reconciliation with sanitized evidence; either result preserves the old attempt and never mutates/recreates the provider draft.
/social, socialDraftAssistant, and POST /api/assistant-jobs/{id}/typefully-draft make zero model/provider writes and cannot bypass exact core approval under any flag/status.
Plugin visibility and worker execution flags are separate/default-off. Rollback preserves evidence/private TTL behavior and cannot convert unknown work into retryable work.
Secrets, numeric social-set IDs, private URLs, raw source/copy/provider bodies, Telegram IDs, prompts/completions, and auth headers are absent from shared status, one-year receipts, logs, errors, exports, and public artifacts.
Automated tests use the production registry/coordinator/approval/worker/outbox paths plus DynamoDB Local transactions; boolean-only call fakes do not substitute for atomic approval/recovery evidence.
Assistant Engineer reviews the prompt/context/source-egress and executor capability boundary before Tester review.
[HUMAN] In private configuration, an authorized operator confirms the real Typefully token, both logical-to-numeric account mappings, per-actor allowed account scopes, account-config digest, and accepts the operator_reconciliation_only runbook. No secret/account/runbook detail enters the public repo or issue.
[HUMAN] With both flags explicitly enabled in sandbox, an authorized operator approves one typed-public draft for each permitted account and verifies Typefully stored exactly the approved ordered content in the intended account with status=draft, null publish/schedule/share fields, and at most one provider effect. An unauthorized account attempt creates nothing.
[HUMAN] An operator exercises an accepted ambiguous-outcome reconciliation without blind retry, verifies the warning/private result access and 30-day URL behavior, and posts only sanitized gate evidence.
Test scenarios
Shared coordinator and private result delivery
Given #126's production coordinator/outbox, one complete Typefully candidate, and a worker completion after the webhook response
When the proposal is rendered, approved, executed, finalized, and delivered
Then the shared core owns controls/attempts, the adapter owns only Typefully fields, one deduplicated owner-bound Telegram result arrives asynchronously, and no Typefully-specific core/Telegram path exists
Public-source pre-model gate
Given typed public-intended text, ordinary private history, private/restricted refs, voice/photo derivations, URLs/files, stale grants, and cross-owner refs
When Typefully is requested before and after public confirmation
Then no z.ai call occurs before confirmation, each z.ai context contains only allowed typed/granted source, blocked inputs receive typed-summary guidance, and model-returned refs/classifications are ignored/rejected
Exact schema and preview
Given boundary-valid content plus 26 X posts, two LinkedIn posts, 4,001/8,001/121/1,001-character fields, >8 refs, >64-KiB payload, controls/zero-width text, duplicate items, cross-field mismatches, and recursively nested forbidden keys
When validation, model parsing, proposal load, rendering, preflight, and serialization run
Then only boundary-valid content survives unchanged, every overflow is rejected rather than truncated, and all exact fields appear in ordered Telegram preview chunks/hashes
Atomic account authorization drift
Given actors scoped to Alexey, DataTalksClub, both, neither, changed permission revision, changed mapping/config digest, and changed delivery digest
When invocation, atomic approval, and preflight race those changes
Then only the exact unchanged authorized account can queue/dispatch; all stale/unauthorized cases create no provider call and reveal no numeric/account-private detail
Preflight before dispatch
Given each flag state, missing/malformed secret, missing mapping, config mismatch, revoked actor, stale lease, invalid stored spec, and fake secret availability
When the worker runs
Then every invalid case becomes failed_safe with no dispatchStartedAt and zero Typefully calls; only a fully valid current lease conditionally crosses dispatch once
Exact create and response validation
Given selected X/LinkedIn content and fake responses covering exact 201, wrong account/content/order, scheduled/published/shared fields, malformed/oversized body, every definitive 4xx, 5xx, timeout, abort, connection loss, and unlisted/non-201 success
When the production serializer/client classifier runs
Then the request exactly equals approval with share:false; only exact 201 succeeds, documented complete 4xx is safe failure, and every ambiguous post-dispatch case is unknown with one call maximum
Duplicate/crash/lease matrix
Given 25 concurrent approvals, duplicate callbacks/Stream deliveries, concurrent workers, lease expiry, and injected crash before preflight, after preflight, after dispatch marker, after send, after response, after private payload/receipt write, and during outbox delivery
When DynamoDB Local runs production transaction branches and recovery
Then pre-dispatch cases safely retry/finish, no post-marker case calls Typefully twice, terminal/unknown state is not overwritten, and result notification is deduplicated
Private result and retention
Given sentinel edit URLs/secrets/account IDs, authorized and cross-actor status reads, result delivery, export/log/error paths, and clocks before/after 30 days and one year
When success/reconciliation is stored and read
Then only the owner-bound outbox can deliver the URL before expiry, shared receipts remain redacted for one year, expired private payloads are inaccessible, and no sentinel leaks
Manual reconciliation only
Given an unknown attempt and admin found, admin not_found, non-admin, stale revision, repeated resolution, and a request to retry
When reconciliation runs
Then only one current admin resolution is recorded, the old attempt never dispatches again, found produces bounded proof/private result, not_found requires a brand-new proposal for another create, and no provider delete/update occurs
Legacy bypass and rollback
Given /social, legacy social assistant execution, every assistant-job status, direct legacy endpoint calls, old flags, plugin-off, execution-off, and in-flight/unknown attempts
When each path and rollback run
Then only static/local staging remains, all legacy provider call counts are zero, new proposals/dispatch stop as configured, and existing evidence/unknown state remains non-retryable
HUMAN sandbox delivery
Given private real configuration and accepted reconciliation runbook
When authorized account drafts, an unauthorized attempt, and an ambiguous-outcome exercise are performed
Then the three [HUMAN] criteria above are satisfied with sanitized public evidence and no blind retry
Required verification
Software Engineer and Tester must run:
npm --prefix backend test
npm --prefix backend run typecheck
npm --prefix backend run build
npm --prefix backend run test:execution-transaction
make sam-validate
make sam-build
git diff --check
Add and run a focused DynamoDB Local harness for Typefully account-scope approval/preflight, 25-way approval/worker concurrency, dispatch-marker one-call behavior, crash/lease recovery, private payload/outbox deduplication, and retention. It must execute the production registry/coordinator/approval/worker transaction branches, not a boolean stand-in.
Run existing conversational state/runtime, Telegram identity/callback/media, todo coordinator compatibility, execution worker/recovery, portable export/restore, assistant-job/social legacy, Typefully client, and SAM/IAM tests in full. Automated tests use fake z.ai/Typefully/Telegram/Secrets Manager and no external network.
No portal UI changes are in scope, so Playwright/screenshots are not required unless implementation unexpectedly changes a web surface; Tester must state why. Telegram preview/result rendering is verified through deterministic adapter tests. Assistant Engineer must post a prompt/egress/capability verdict before Tester acceptance. Tester records exact commands, exit codes, counts, and criterion mapping.
With the #126 code dependency satisfied at c47c525, all automated criteria and reviews may ship with Refs #127; add human and leave the issue open until all three real-account gates pass.
Rollback
Disable plugin visibility to stop new Typefully public-source grants/proposals.
Disable worker execution before dispatch to stop new provider calls safely.
Preserve proposals, attempts, dispatch markers, safe receipts, private payload TTLs, outbox events, and reconciliation records.
Never re-enable a legacy path, retry an unknown attempt, or automatically delete/update a Typefully draft.
Out of scope
Typefully update/delete, scheduling/publishing, queue/next-free-slot, preferred time, public sharing, media, tags, analytics, webhooks, comments, account UI, provider search/list, or automatic reconciliation.
More than one draft per proposal, destinations beyond Alexey / Al_Grigor and DataTalksClub, splitting one proposal into multiple provider effects, or generic provider/execution DSL work.
Voice/photo-derived social source, URL/document fetching, files/uploads, private operational knowledge, silent sanitization/reclassification, history-wide retrieval, memory, groups, web approval, or cross-channel continuation.
Storing raw runbooks, private links, screenshots, credentials-adjacent setup, account details, prompts, or reconciliation artifacts in this public repo. These belong in private DataTalksClub/dataops-knowledge or another approved private store.
Retrofitting AssistantJob status as approval, preserving a legacy Typefully write endpoint, or allowing two Typefully execution paths.
Modifying ../dtc-operations, ../datatasks, or ../podcast-assistant.
Ship approved Typefully saved-draft creation
Status: pending
Tags:
enhancement,assistant,work-engine,backend,infra,testing,data,P1Parent: #122 (M6)
Depends on: #126 (code dependency satisfied at
c47c525; HUMAN-only rollout remains open and non-blocking)Blocks: #128
Architecture:
_docs/CONVERSATIONAL_AGENT_PLUGIN_ARCHITECTURE.mdat4e5402cArchitect review: #127 (comment)
Related migration debt: #77, #80
Dependency contract — satisfied on
main#126 agent-verifiable implementation merged to
mainasc47c525withRefs #126. The issue remains open withhumanonly for the credentialed deployment and live Telegram gates. Under_docs/PROCESS.mdHuman Verification, those external checks do not block the next agent-verifiable implementation. #127 is ready and must consume two small shared primitives from #126 rather than copy todo-specific orchestration:ProposalSpec, renders plugin-supplied content through core-owned controls, and reuses Add transactional approval and durable execution recovery #124 approval/worker semantics; andIf
mainregresses to todo-only orchestration or loses durable asynchronous result delivery, #127 becomes blocked again and the shared primitive must be corrected first. #127 adds one Typefully proposal adapter and one capability-scoped executor; it must not create a second conversational core, approval path, Telegram completion path, workflow DSL, or generic provider framework.Use the existing plugin channel value
channels: ['telegram']. #121 proves linked-user private-chat scope before core entry. Do not add a secondtelegram_privatechannel enum.Product effect
After a complete deterministic preview and exact core approval, create exactly one saved Typefully draft in one authorized logical account:
There is no update, delete, schedule, publish, queue/next-free-slot, preferred time, public share, tag, media, analytics, webhook, comment, account-management, or multi-draft capability. Scheduling and publication remain manual Typefully actions.
Public-source gate before z.ai
A Typefully request may use only:
publicby core policy.The Typefully plugin is not exposed to the generic model catalog until that public-source grant exists. A bounded static intent recognizer may notice an ordinary typed request containing explicit social/Typefully/X/LinkedIn posting intent and present a core-owned Use this typed text as public source confirmation without invoking z.ai. Ambiguous text receives a local clarification. Only after confirmation may the first z.ai selection/load turn run.
For every Typefully z.ai turn, build a separate context containing only the confirmed current typed source, prior Typefully draft fields already covered by the same public grant, trusted public refs, and the Typefully instructions/schema. Never include ordinary recent conversation history, conversation summaries, todo context, private Telegram events, organization/user-private/restricted/credentials-adjacent content, or unconfirmed content. The model cannot see or choose a classification.
Voice transcripts, photo descriptions/captions, files, fetched URLs, and document ingestion are not eligible in this slice—even if they were confirmed for another plugin. Ask for a new typed public-safe summary. Do not fetch a supplied URL, silently sanitize/reclassify private text, or let generated copy downgrade source classification.
Core creates
PublicSourceGrant/equivalent state with owner, private payload reference, source digest,classification=public, policy revision, confirmation event, and 30-day expiry. The model may return copy but cannot manufacture/modifysourceRefs; core attaches at most eight opaque trusted refs and their digests/policy revisions to the proposal. A stale, missing, cross-owner, non-public, or policy-drifted grant stops before z.ai/Typefully and requires fresh typed confirmation.Shared Typefully proposal adapter
Register one trusted static plugin:
The adapter plugs into #126's shared coordinator. It owns only Typefully validation, clarification copy, immutable-spec construction, deterministic preview rendering, and capability registration. Core owns draft/proposal/presentation transitions, approve/request-changes/cancel/discard controls, expiry, exact approval claim, worker queueing, and result notification.
Missing or ambiguous logical account, platforms, purpose, or typed public source asks one concise question. It never asks for publication time. Request changes creates a complete new immutable proposal, revokes every old presentation, and requires a fresh full preview. Detached “yes”, group input, stale/old controls, assistant-job status, cancel, discard, and expiry perform no provider write.
Exact non-lossy action schema
Use
additionalProperties: falseat every object level. Normalize accepted strings with Unicode NFKC and trim surrounding whitespace; reject ASCII/Unicode controls, zero-width/bidi-control text, empty normalized text, invalid Unicode, and overflow. Never truncate, slice, summarize, reorder, merge, or drop model/operator content.The exact candidate fields and bounds are:
account: exactlyalexeyordatatalksclub;platforms: unique array,minItems: 1,maxItems: 2, values onlyxandlinkedin;xPosts: present iffxis selected;minItems: 1,maxItems: 25; each string 1–4,000 Unicode code points;linkedinPosts: present ifflinkedinis selected; exactly one item (minItems: 1,maxItems: 1); text 1–8,000 Unicode code points;draftTitle: 1–120 Unicode code points when present;scratchpadText: 1–1,000 Unicode code points when present;sourceRefs: 1–8 unique opaque refs, each at most 160 ASCII characters and resolving to a current trusted public grant;Reject a content array for an unselected platform or a selected platform without its required content. Reject duplicate platforms/refs/posts, empty items, arrays/objects in text positions, and all unknown fields. Recursively reject any schedule/publish/share/media/tag/comment/analytics/webhook/update/delete/target/provider field or semantic equivalent, including
publish_at,publishAt,preferredTime,schedule,now,next-free-slot,share,share_url,media_ids,targetDraftId,draft_id, numeric social-set ID, provider URL, and existing-target/base-revision input.Remove the existing Typefully client and social generator's silent 4,000/8,000/120/1,000 truncation. The same strict validator runs on model output, proposal construction, stored-spec load, preflight, and request serialization. Overflow is a validation/clarification result with zero provider calls.
Immutable spec and complete Telegram preview
The
ProposalSpecbinds:deliveryMode=operator_reconciliation_onlyplus its digest;The canonical payload and rendered-view hashes cover every field above. Telegram renders the full approved content without truncation: chunk long previews into deterministic numbered messages, show every ordered post and optional title/scratchpad, then show logical account, platforms, public-source proof summary,
operator_reconciliation_only, and the exact effect “Create one unscheduled, unpublished, unshared Typefully saved draft.” Approval controls appear only after every preview chunk is durably delivered. The worker executes the stored spec byte-for-byte after provider serialization; it cannot regenerate or reinterpret content.Account-scoped authorization and configuration
Extend the existing
ApprovalPermissioncontract rather than create another authorization system. Fortypefully:create-saved-draft, store:enabledand monotonically increasingrevision;allowedResourceKeyssubset oftypefully:account:alexeyandtypefully:account:datatalksclub(maximum two);operator_reconciliation_only.Bind all four to the proposal. At invocation/proposal creation, core verifies the active linked actor, permission, and selected resource key without revealing unauthorized account details. The #124 approval transaction must atomically condition-check
enabled, exact permission revision, selected account membership, account-configuration digest, and delivery-mode digest while claiming the proposal. A read followed by an unscoped approval check is insufficient.Immediately before dispatch, the worker repeats actor state, permission revision/scope, proposal/build/policy/payload/view hashes, account-config digest, and delivery-mode digest checks. Numeric Typefully social-set IDs are worker-only configuration. The model, plugin payload, Backend/Telegram Lambda, client APIs, proposals, Telegram messages, and long-lived receipts see only logical account keys and safe digests.
Changing actor scope, a numeric account mapping, configuration revision, or delivery mode invalidates the old proposal. It cannot be approved/dispatched; the operator must generate and approve a fresh preview.
Side-effect-free worker preflight
Add the smallest shared executor
preflighthook and run it before #124 persistsdispatchStartedAtor any provider-call marker. Typefully preflight may read trusted local/DynamoDB/AWS Secrets Manager state but makes no Typefully request and has no external business effect. It verifies:CONVERSATIONAL_TYPEFULLY_EXECUTION_ENABLED=true;operator_reconciliation_onlyand its digest;A missing/disabled flag, permission/config drift, missing mapping, unreadable/malformed secret, stale lease, or validation failure is
failed_safe;dispatchStartedAtremains absent and Typefully call count is zero. Recheck the lease when conditionally recording dispatch intent after preflight so a stale worker cannot cross the boundary.Use two independent default-off flags:
CONVERSATIONAL_TYPEFULLY_PLUGIN_ENABLEDcontrols catalog visibility/public-source entry and lives with the backend safe configuration;CONVERSATIONAL_TYPEFULLY_EXECUTION_ENABLEDexists only on the execution worker and controls external dispatch.Moving from preflight to dispatch is one conditional state transition. After
dispatchStartedAtexists,operator_reconciliation_onlyforbids automatic provider replay.Worker-only credential and provider capability
Keep z.ai behind #125's Anthropic-compatible model boundary. Move all Typefully write capability out of
BackendFunctionand intoConversationalExecutionWorkerFunction:TYPEFULLY_SOCIAL_SET_ALEXEYandTYPEFULLY_SOCIAL_SET_DATATALKSCLUBnumeric mappings;secretsmanager:GetSecretValuefor only that exact secret ARN.Remove Typefully secret/mapping environment variables and Typefully
GetSecretValuepermission fromBackendFunction, Telegram/model/plugin paths, and other Lambdas. The execution worker resolves the bearer token during preflight/dispatch and never stores it in an attempt, private payload, event, receipt, error, or log. The capability exposes onlycreateSavedDraft(approvedSpec)for one already-authorized logical account; no list/get/update/delete/schedule/publish/general fetch client is injected.Exact provider request and validated success
After one persisted dispatch intent, send at most one request:
The JSON body contains only the selected platform content, optional approved
draft_title/scratchpad_text, andshare: false. It contains nopublish_at, schedule/publish target, media, tags, comments, public sharing, provider target ID, or unselected platform. Serialization must equal the stored ordered proposal; it does not truncate or mutate it.The first release uses
operator_reconciliation_only; do not send or infer a provider idempotency/correlation key. Claim success only for a fully read HTTP201whose body validates all of:status='draft'andpublish_state=null;scheduled_date,published_at, every platform/post published URL, andshare_urlare null/absent;A fully received, documented Typefully create rejection (
400,401,402,403,404,422, or429) with a bounded parseable response isfailed_safe. Transport/DNS/TLS failure, connection loss, timeout/abort,5xx, any unlisted status, malformed/oversized body, contradictory or non-201success, validation mismatch, or crash after dispatch intent/request send isoutcome_unknown. Provider/account-level automatic settings are not assumed harmless: the live gate must compare Typefully's stored ordered content and unscheduled/unpublished/unshared fields to the approval.Outcome, retention, and private result delivery
Never replay an attempt after dispatch intent. Duplicate callbacks, Stream deliveries, workers, lease recovery, and scheduled recovery find the dispatch marker and return/retain the current terminal or
outcome_unknownstate without another Typefully call.On validated success:
The Typefully executor receives no Telegram token/client. Shared attempt/status/reconciliation APIs return only the safe receipt; they never return the private URL or payload body. Cross-actor, wrong-channel, expired, deleted, or guessed payload refs return not found without disclosure. After 30 days the URL is unavailable while the safe receipt remains for one year. Raw prompts/completions, public-source text, proposal payloads, and private provider response payloads follow the architecture's 30-day retention; safe audit/effect metadata follows one year.
If #126's outbox cannot conditionally deduplicate worker result delivery, preserve private owner binding, and deliver asynchronous completion, #127 stays blocked.
operator_reconciliation_onlyFor
outcome_unknown, show the exact safe message through the shared result outbox:Disable retry/requeue/approve-again controls for the old attempt. Reconciliation is an authenticated admin-only action requiring the current attempt revision and an accepted private runbook. It records one bounded resolution:
found: exact logical account, unique draft ID, exact ordered approved content, unscheduled/unpublished/unshared state, and a time window consistent with the attempt were verified; ornot_found: the authorized operator completed the accepted account search and found no matching draft.Store only sanitized proof classification, actor, draft ID when found, attempt/proposal/hash, and timestamp in the one-year receipt. A found private edit URL goes only in a fresh 30-day owner-private payload. Never store screenshots, account names/IDs beyond the logical key, raw provider exports, or runbook details in the public repo/issue.
Neither resolution retries, deletes, updates, schedules, or publishes. If
not_foundand the operator still wants a draft, create a new proposal/version from current public typed input and require a new exact approval; never replay the old attempt. Rollback never deletes an external draft.Remove all legacy write bypasses
Conversational rollout eliminates, rather than conditionally preserves, every non-worker Typefully write path:
socialDraftAssistantmay stage local copy only or be retired; remove its directcreateSavedDraft(...)call and provider result/private URL artifacts;POST /api/assistant-jobs/{id}/typefully-draftis removed or returns410 Goneand cannot instantiate/inject a provider client regardless of assistant-job status;/socialwith or without arguments returns static guidance to send a typed social request and confirm it as public source; it invokes neither z.ai nor Typefully;waiting_approval,approved,succeeded, model output, assistant artifact state, webhook input, or a legacy feature flag can never authorize a provider call.There is no temporary simultaneously enabled legacy mode. Reuse #77/#80 validation/client fixtures only after moving the strict non-truncating create serializer into the worker capability. Their status-only approval/retry semantics and long-lived private URL artifact are removed for this flow.
Enablement and rollback
Both flags default off. Automated implementation/tests use fake z.ai, Telegram, Secrets Manager, and Typefully adapters with no network. Agent-verifiable code may ship with execution disabled.
Rollback first disables plugin visibility, then disables worker execution before new dispatch. It preserves all proposals, attempts, safe receipts, private-payload TTLs, outbox events, and reconciliation evidence. It does not re-enable a legacy route, make an unknown attempt retryable, or delete a Typefully draft.
Acceptance criteria
TodoConversationalCorecloning or a second approval/Telegram path.typefully.propose_draftwithtypefully:create-saved-draft, rolesadmin|operator, existing channeltelegram, and create-onlyoperator_reconciliation_onlysemantics.ApprovalPermissionstate carries bounded allowed logical accounts, exact revision, account-config digest, and delivery-mode digest. Invocation checks them, approval condition-checks them atomically, and worker preflight rechecks them.dispatchStartedAtand proves execution flag, scope/config/delivery hashes, exact mapping, secret availability, validation, and lease. Any failure leaves no dispatch marker/call and isfailed_safe.ConversationalExecutionWorkerFunctionreceives Typefully secret reference, numeric social-set configuration, exact-secretGetSecretValue, execution flag, and create capability. Backend/model/Telegram/other Lambdas have none.POSTafter dispatch intent, withshare:false, selected content only, and no update/schedule/publish/share/media/target field.201matching account, exact ordered content, draft/null publish fields, no public share/published URLs, and safe private edit URL. Contradictory/malformed success is never accepted.operator_reconciliation_onlymatrix classifies proven pre-dispatch/documented rejection asfailed_safeand all ambiguous post-dispatch cases asoutcome_unknown; no automatic replay occurs after dispatch intent.outcome_unknowndisables retry and supports only admin manualfound|not_foundreconciliation with sanitized evidence; either result preserves the old attempt and never mutates/recreates the provider draft./social,socialDraftAssistant, andPOST /api/assistant-jobs/{id}/typefully-draftmake zero model/provider writes and cannot bypass exact core approval under any flag/status.operator_reconciliation_onlyrunbook. No secret/account/runbook detail enters the public repo or issue.status=draft, null publish/schedule/share fields, and at most one provider effect. An unauthorized account attempt creates nothing.Test scenarios
Shared coordinator and private result delivery
Given #126's production coordinator/outbox, one complete Typefully candidate, and a worker completion after the webhook response
When the proposal is rendered, approved, executed, finalized, and delivered
Then the shared core owns controls/attempts, the adapter owns only Typefully fields, one deduplicated owner-bound Telegram result arrives asynchronously, and no Typefully-specific core/Telegram path exists
Public-source pre-model gate
Given typed public-intended text, ordinary private history, private/restricted refs, voice/photo derivations, URLs/files, stale grants, and cross-owner refs
When Typefully is requested before and after public confirmation
Then no z.ai call occurs before confirmation, each z.ai context contains only allowed typed/granted source, blocked inputs receive typed-summary guidance, and model-returned refs/classifications are ignored/rejected
Exact schema and preview
Given boundary-valid content plus 26 X posts, two LinkedIn posts, 4,001/8,001/121/1,001-character fields, >8 refs, >64-KiB payload, controls/zero-width text, duplicate items, cross-field mismatches, and recursively nested forbidden keys
When validation, model parsing, proposal load, rendering, preflight, and serialization run
Then only boundary-valid content survives unchanged, every overflow is rejected rather than truncated, and all exact fields appear in ordered Telegram preview chunks/hashes
Atomic account authorization drift
Given actors scoped to Alexey, DataTalksClub, both, neither, changed permission revision, changed mapping/config digest, and changed delivery digest
When invocation, atomic approval, and preflight race those changes
Then only the exact unchanged authorized account can queue/dispatch; all stale/unauthorized cases create no provider call and reveal no numeric/account-private detail
Preflight before dispatch
Given each flag state, missing/malformed secret, missing mapping, config mismatch, revoked actor, stale lease, invalid stored spec, and fake secret availability
When the worker runs
Then every invalid case becomes
failed_safewith nodispatchStartedAtand zero Typefully calls; only a fully valid current lease conditionally crosses dispatch onceExact create and response validation
Given selected X/LinkedIn content and fake responses covering exact
201, wrong account/content/order, scheduled/published/shared fields, malformed/oversized body, every definitive 4xx, 5xx, timeout, abort, connection loss, and unlisted/non-201 successWhen the production serializer/client classifier runs
Then the request exactly equals approval with
share:false; only exact201succeeds, documented complete 4xx is safe failure, and every ambiguous post-dispatch case is unknown with one call maximumDuplicate/crash/lease matrix
Given 25 concurrent approvals, duplicate callbacks/Stream deliveries, concurrent workers, lease expiry, and injected crash before preflight, after preflight, after dispatch marker, after send, after response, after private payload/receipt write, and during outbox delivery
When DynamoDB Local runs production transaction branches and recovery
Then pre-dispatch cases safely retry/finish, no post-marker case calls Typefully twice, terminal/unknown state is not overwritten, and result notification is deduplicated
Private result and retention
Given sentinel edit URLs/secrets/account IDs, authorized and cross-actor status reads, result delivery, export/log/error paths, and clocks before/after 30 days and one year
When success/reconciliation is stored and read
Then only the owner-bound outbox can deliver the URL before expiry, shared receipts remain redacted for one year, expired private payloads are inaccessible, and no sentinel leaks
Manual reconciliation only
Given an unknown attempt and admin
found, adminnot_found, non-admin, stale revision, repeated resolution, and a request to retryWhen reconciliation runs
Then only one current admin resolution is recorded, the old attempt never dispatches again,
foundproduces bounded proof/private result,not_foundrequires a brand-new proposal for another create, and no provider delete/update occursLegacy bypass and rollback
Given
/social, legacy social assistant execution, every assistant-job status, direct legacy endpoint calls, old flags, plugin-off, execution-off, and in-flight/unknown attemptsWhen each path and rollback run
Then only static/local staging remains, all legacy provider call counts are zero, new proposals/dispatch stop as configured, and existing evidence/unknown state remains non-retryable
HUMAN sandbox delivery
Given private real configuration and accepted reconciliation runbook
When authorized account drafts, an unauthorized attempt, and an ambiguous-outcome exercise are performed
Then the three
[HUMAN]criteria above are satisfied with sanitized public evidence and no blind retryRequired verification
Software Engineer and Tester must run:
npm --prefix backend test npm --prefix backend run typecheck npm --prefix backend run build npm --prefix backend run test:execution-transaction make sam-validate make sam-build git diff --checkAdd and run a focused DynamoDB Local harness for Typefully account-scope approval/preflight, 25-way approval/worker concurrency, dispatch-marker one-call behavior, crash/lease recovery, private payload/outbox deduplication, and retention. It must execute the production registry/coordinator/approval/worker transaction branches, not a boolean stand-in.
Run existing conversational state/runtime, Telegram identity/callback/media, todo coordinator compatibility, execution worker/recovery, portable export/restore, assistant-job/social legacy, Typefully client, and SAM/IAM tests in full. Automated tests use fake z.ai/Typefully/Telegram/Secrets Manager and no external network.
No portal UI changes are in scope, so Playwright/screenshots are not required unless implementation unexpectedly changes a web surface; Tester must state why. Telegram preview/result rendering is verified through deterministic adapter tests. Assistant Engineer must post a prompt/egress/capability verdict before Tester acceptance. Tester records exact commands, exit codes, counts, and criterion mapping.
With the #126 code dependency satisfied at
c47c525, all automated criteria and reviews may ship withRefs #127; addhumanand leave the issue open until all three real-account gates pass.Rollback
Out of scope
DataTalksClub/dataops-knowledgeor another approved private store.AssistantJobstatus as approval, preserving a legacy Typefully write endpoint, or allowing two Typefully execution paths.../dtc-operations,../datatasks, or../podcast-assistant.Dependencies
mainatc47c525:StaticProposalCoordinator/ProposalAdapterplusConversationalProposalCoreprovide the reusable selected-adapter proposal/control seam, andfinalizeAttempt(...resultNotification)plusdispatchOne/runResultDispatcherprovide the durable owner-private asynchronous result path. Ship approved conversational todo creation #126 remains open only for HUMAN rollout; that open state does not block Ship approved Typefully saved-draft creation #127 automated implementation. Ship approved Typefully saved-draft creation #127 must continue consuming these seams rather than clone them.operator_reconciliation_onlystates. Ship approved Typefully saved-draft creation #127 extends it only with a pre-dispatch executor hook and scoped permission conditions.telegramregistry value reaches core.