Skip to content

Allow OIDC deployment of the conversational result dispatcher #130

Description

@alexeygrigorev

Allow OIDC deployment of the conversational result dispatcher

Status: pending
Tags: enhancement, assistant, infra, testing, P1
Depends on: #129 source acceptance/merge — implementation may start from independently reviewed commit 9b5b888, but review/merge must use the finally accepted #129 source
Blocks: #126 production deployment and its HUMAN sandbox gate
Related: #124, DataTalksClub/aws-infra#18

Outcome and ownership

Update only the shared DataTalksClub/aws-infra source of truth at sandbox/dataops/template.github-actions.yaml and its focused tests so dataops-github-actions-deploy can deploy and roll back #126's exact result dispatcher. Do not change DataOps runtime code or apply AWS during agent work.

Fetch first and work in a fresh isolated worktree, suggested as /home/alexey/git/.worktrees/aws-infra-130 on issue-130-conversational-result-dispatcher-deploy-role. Preserve the dirty ../aws-infra checkout, merged #124 PR #18, and finally accepted #129 changes. Never stash/reset/overwrite the sibling checkout.

Authoritative #126 graph

Final #126 infra/template.full.yaml always declares:

  • AWS::Serverless::Function ConversationalResultDispatcherFunction;
  • generated role ConversationalResultDispatcherFunctionRole;
  • its generated Lambda log group;
  • one Schedule event named ResultDelivery, rate(1 minute), conditional on ConversationalResultDeliveryEnabled;
  • the schedule's generated EventBridge rule and Lambda permission.

There is no dispatcher event-source mapping, queue, EventInvokeConfig, Function URL, API, layer, or separately managed log resource. Default-false disables the schedule but does not remove the function/role.

Acceptance must compare the policy with final agent-approved #126 source, a fresh isolated sam build, and the deploy change-set inventory. Exact logical-ID/resource changes require exact policy/test changes; add nothing speculative.

Policy placement and quotas

The inline role policy is already about 9,341 compact characters, only 899 below IAM's aggregate 10,240 role-inline quota. Do not extend it.

Do not mix this unrelated domain into #129's Sponsor-only compute policy or weaken its negative boundary. Add one generated-name AWS::IAM::ManagedPolicy logical resource, ConversationalResultDispatcherDeployPolicy, attached only through Roles: [!Ref GitHubActionsDeployRole]. Do not set ManagedPolicyName. The OIDC role must not manage/version/attach its own policies; the credentialed shared-stack operator owns those operations.

Deterministic default/pseudo-value resolution plus compact JSON tests must prove:

If upstream breaks a ceiling, redesign/split rather than weaken assertions. Operator must inspect live external attachments before apply. Quota reference: https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_iam-quotas.html

Exact dispatcher policy

Lambda statement

Grant exactly:

lambda:AddPermission
lambda:CreateFunction
lambda:DeleteFunction
lambda:GetFunction
lambda:GetFunctionConfiguration
lambda:ListTags
lambda:RemovePermission
lambda:TagResource
lambda:UntagResource
lambda:UpdateFunctionCode
lambda:UpdateFunctionConfiguration

on exactly:

arn:${AWS::Partition}:lambda:${FullDocsRegion}:${AWS::AccountId}:function:${FullDocsStackName}-ConversationalResultDispatcherFunction-*
arn:${AWS::Partition}:lambda:${FullDocsRegion}:${AWS::AccountId}:function:${FullDocsStackName}-ConversationalResultDispatcherFunction-*:* 

Tests compare exact strings without trailing whitespace. Do not grant invoke, Function URL operations, lambda:*, ${FullDocsStackName}-Conversational*, or stack/account-wide functions. Add/RemovePermission cover only the generated schedule permission on this function.

Generated role and PassRole statement

Grant exactly:

iam:AttachRolePolicy
iam:CreateRole
iam:DeleteRole
iam:DeleteRolePolicy
iam:DetachRolePolicy
iam:GetRole
iam:GetRolePolicy
iam:ListAttachedRolePolicies
iam:ListRoleTags
iam:PassRole
iam:PutRolePolicy
iam:TagRole
iam:UntagRole
iam:UpdateAssumeRolePolicy

on only:

arn:${AWS::Partition}:iam::${AWS::AccountId}:role/${FullDocsStackName}-ConversationalResultDispatcherFunctionRole-*

No wildcard-role PassRole, iam:*, user/group, standalone policy administration, or unrelated role.

Log-group statement

Grant exactly:

logs:CreateLogGroup
logs:DeleteLogGroup
logs:DescribeLogGroups
logs:PutRetentionPolicy
logs:TagResource
logs:UntagResource

on exactly the base and stream-suffix forms:

arn:${AWS::Partition}:logs:${FullDocsRegion}:${AWS::AccountId}:log-group:/aws/lambda/${FullDocsStackName}-ConversationalResultDispatcherFunction-*
arn:${AWS::Partition}:logs:${FullDocsRegion}:${AWS::AccountId}:log-group:/aws/lambda/${FullDocsStackName}-ConversationalResultDispatcherFunction-*:* 

Tests compare exact strings without trailing whitespace. No log-event read/write, arbitrary log group, or broader retention access.

Schedule: preserve existing boundary

The one proven trigger is a SAM Schedule, not an event-source mapping. Existing inline EventBridgeDataOpsRules already grants rule lifecycle on only:

arn:${AWS::Partition}:events:${FullDocsRegion}:${AWS::AccountId}:rule/${FullDocsStackName}-*

Keep it unchanged. Exact Lambda Add/RemovePermission covers the generated permission. Add no EventBridge/Scheduler action, event-bus write, event-source mapping, async-invoke configuration, SQS, DynamoDB, Secrets Manager, Telegram, or runtime permission.

Acceptance criteria

Required tests

Parse YAML as an object graph and assert exact attachment/statements/resources plus absence from inline/Sponsor policies. Mutation tests must reject:

Resolve actual documents and enforce all size/count ceilings. Fresh-build #126 in an isolated output directory and assert exactly one dispatcher function, generated role/log family, one ResultDelivery schedule plus generated rule/permission, and no dispatcher mapping/queue/async config/URL. Compare the eventual CloudFormation change set.

Minimum commands:

uv run --with pytest python -m pytest sandbox/dataops/tests -q
sam validate --lint --template-file sandbox/dataops/template.github-actions.yaml
git diff --check <accepted-base>...HEAD

Also run project-supported #126 make sam-build and lint with fresh isolated output, never stale .aws-sam.

Pipeline and HUMAN apply

  1. Engineer implements/commits in isolated aws-infra with Refs DataTalksClub/dataops#130; no push/apply.
  2. Independent Tester/security reviewer verifies exact boundary, final Ship approved conversational todo creation #126 graph, quotas, and Add transactional approval and durable execution recovery #124/Extend the DataOps OIDC deploy role for reviewed sponsor communications #129 preservation.
  3. PM accepts reviewed diff/final SAM comparison.
  4. Integrator pushes and merges focused aws-infra PR, confirming merged source includes accepted Extend the DataOps OIDC deploy role for reviewed sponsor communications #129+Allow OIDC deployment of the conversational result dispatcher #130.
  5. Source merge does not apply AWS; keep Allow OIDC deployment of the conversational result dispatcher #130 open with human.
  6. [HUMAN] Credentialed operator creates/reviews change set, checks external attachments/operator managed-policy capability, then from sandbox/dataops runs:
aws cloudformation deploy \
  --stack-name dataops-github-actions \
  --template-file template.github-actions.yaml \
  --region eu-west-1 \
  --capabilities CAPABILITY_NAMED_IAM
  1. Require UPDATE_COMPLETE and inspect effective role before retrying DataOps CI.
  2. Deploy Ship approved conversational todo creation #126 only through normal GitHub Actions CI with conversational flags default-off. On-Call confirms function/role/log creation, generated result schedule disabled, no dispatch, and Add transactional approval and durable execution recovery #124/Extend the DataOps OIDC deploy role for reviewed sponsor communications #129 still effective.
  3. Enable/test delivery only in Ship approved conversational todo creation #126's controlled HUMAN rollout. Close Allow OIDC deployment of the conversational result dispatcher #130 only after apply and On-Call evidence pass.

Rollback

Use CloudFormation, never manual IAM edits. Retain the exact prior merged template/commit. A failed update must reach an explained terminal rollback state. For an unacceptable successful update, redeploy the prior accepted shared-infra template with CAPABILITY_NAMED_IAM; confirm dispatcher policy detaches/deletes and #124/#129 remain effective. Do not remove PR #18 while #124 resources exist, remove #129 coverage while Sponsor resources exist, or edit/delete DataOps app/runtime resources. Keep result delivery disabled during rollback verification.

Out of scope

DataOps code/SAM changes, conversational behavior, Telegram/z.ai calls, secrets, task writes, app deploy outside CI, #126's sandbox journey, broad role redesign, moving existing inline statements, permission boundaries, speculative access, live IAM edits, retained-resource deletion, or other stacks/repos.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1ImportantassistantAssistant modules and jobsenhancementNew or improved functionalityhumanCode done or issue blocked on human verificationinfraDeployment and infrastructuretestingTests and QA

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions