Skip to content

fix(iac/azure): move reservation role definition to bootstrap; runtime keeps only the assignment - #1095

Merged
cristim merged 1 commit into
feat/multicloud-web-frontendfrom
fix/azure-deploy-role-bootstrap
Jun 7, 2026
Merged

cristim merged 1 commit into
feat/multicloud-web-frontendfrom
fix/azure-deploy-role-bootstrap

Conversation

@cristim

@cristim cristim commented Jun 7, 2026

Copy link
Copy Markdown
Member

Summary

Fixes the deterministic 403 that blocks every Azure Container Apps deploy
(#1092). The runtime container-apps module instantiated the
cudly-reservation-role module, which creates an azurerm_role_definition.
Creating a role definition requires
Microsoft.Authorization/roleDefinitions/write, a bootstrap-class permission
the CI deploy service principal intentionally lacks (it holds only
roleDefinitions/read plus roleAssignments/{read,write,delete}). So the
runtime apply 403'd at the role-definition write every run, exactly as the
bootstrap-vs-runtime IAM split intends. The bug was a bootstrap-class resource
sitting in the runtime apply path.

Root cause and diagnosis: docs/code-review/21-azure-deploy-failures.md
(Finding A).

Fix (honors the split; does NOT grant roleDefinitions/write to the deploy SP)

  • Bootstrap (human-applied): create the host-side reservation role
    definition in terraform/environments/azure/ci-cd-permissions/
    (new reservation_role.tf), and output its resource ID + display name.
  • Runtime (modules/compute/azure/container-apps): stop creating the
    definition; resolve it via data.azurerm_role_definition by name +
    subscription scope and keep only the azurerm_role_assignment (the deploy SP
    has roleAssignments/write). A missing definition fails loud with a
    "not found" error, signalling the operator to re-run the bootstrap rather than
    widening the SP.
  • Shared module (modules/iam/azure/cudly-reservation-role): factor the
    role display name into a local (single source of truth) and expose it via a
    role_definition_name output so the runtime lookup stays in lockstep.
  • Secondary latent bug: drop the tenant-root
    /providers/Microsoft.Capacity from assignable_scopes (a subscription-scoped
    principal cannot register a scope above its subscription, a latent 403). It is
    now gated behind a new include_capacity_provider_scope variable
    (default false); the role is only ever assigned at subscription scope. This
    applies to both the host-side and customer-side
    (iac/federation/azure-target) consumers of the shared module.

REQUIRED MANUAL STEP (privileged human, before the next runtime deploy)

A privileged human must re-apply the terraform/environments/azure/ci-cd-permissions/
bootstrap module once
so the host-side reservation role definition exists:

cd terraform/environments/azure/ci-cd-permissions
terraform init -backend-config=backend.hcl
terraform plan    # should show the new module.cudly_reservation_role role definition
terraform apply

Until that bootstrap apply lands, the runtime data.azurerm_role_definition
lookup will fail loudly with "Role Definition ... was not found" - the intended
signal to run the bootstrap, NOT to widen the deploy SP's permissions. After the
bootstrap apply, the next Azure deploy resolves the role and creates only the
assignment, which the deploy SP is allowed to do.

Verification

Terraform apply / deploy was NOT run (staging-first; the bootstrap module is
applied manually by a privileged human). Static checks, run offline against
cached providers because the public Terraform registry was unreachable during
this work:

  • terraform fmt -recursive -check: clean on all touched paths.
  • terraform validate: clean on the shared module, the ci-cd-permissions
    bootstrap env, and the container-apps runtime module.
  • tflint: clean (no findings) on all three.

Notes / follow-ups

See docs/code-review/open-questions/fix-azure-deploy-1092.md (not committed).
One deliberate deferral: the shared module's provider constraint was left at
>= 3.0 (consumers pin ~> 3.0). Aligning it forces a regeneration of the
module's tracked .terraform.lock.hcl, which needs registry access that was
unavailable here; the deploy path is unaffected because the parent roots already
govern the provider version. A follow-up can align the constraint and relock
online.

Closes #1092

…e keeps only the assignment (closes #1092)

The runtime container-apps module instantiated the cudly-reservation-role
module, which creates an azurerm_role_definition. Creating a role definition
needs Microsoft.Authorization/roleDefinitions/write, a bootstrap-class
permission the CI deploy service principal intentionally lacks (it has only
roleDefinitions/read plus roleAssignments/{read,write,delete}). Every Azure
deploy therefore 403'd at the role-definition write, exactly as the
bootstrap-vs-runtime IAM split intends.

Fix, honoring the split (no roleDefinitions/write granted to the deploy SP):

- Bootstrap: create the host-side reservation role definition in the
  human-applied ci-cd-permissions module (new reservation_role.tf) and output
  its resource ID and display name.
- Runtime: stop creating the definition in container-apps; resolve it via
  data.azurerm_role_definition by name + subscription scope and keep only the
  azurerm_role_assignment. A missing definition fails loud ("not found"),
  signalling the operator to re-run the bootstrap rather than widening the SP.
- Shared module: factor the role display name into a local (single source of
  truth) and expose it via a role_definition_name output so the runtime lookup
  stays in lockstep.
- Drop the tenant-root "/providers/Microsoft.Capacity" assignable scope (a
  subscription-scoped principal cannot register a scope above its subscription,
  a latent 403). It is now gated behind include_capacity_provider_scope
  (default false); the role is only ever assigned at subscription scope.

REQUIRED MANUAL STEP: a privileged human must re-apply the ci-cd-permissions
bootstrap once to create the role definition before the next runtime deploy.

Verified offline: terraform fmt -check, terraform validate, and tflint are
clean on the shared module, the bootstrap env, and the container-apps module
(apply not run; staging-first).
@cristim cristim added bug Something isn't working triaged Item has been triaged priority/p1 Next up; this sprint severity/high Significant harm urgency/now Drop other things impact/internal Team-internal only effort/m Days type/bug Defect labels Jun 7, 2026
@coderabbitai

coderabbitai Bot commented Jun 7, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@cristim, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 10 minutes and 39 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 76383d66-e6bc-4de5-938c-b8073b72228f

📥 Commits

Reviewing files that changed from the base of the PR and between ee34869 and 7e95e46.

📒 Files selected for processing (7)
  • terraform/environments/azure/ci-cd-permissions/README.md
  • terraform/environments/azure/ci-cd-permissions/outputs.tf
  • terraform/environments/azure/ci-cd-permissions/reservation_role.tf
  • terraform/modules/compute/azure/container-apps/main.tf
  • terraform/modules/iam/azure/cudly-reservation-role/main.tf
  • terraform/modules/iam/azure/cudly-reservation-role/outputs.tf
  • terraform/modules/iam/azure/cudly-reservation-role/variables.tf
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/azure-deploy-role-bootstrap

Comment @coderabbitai help to get the list of available commands and usage tips.

@cristim

cristim commented Jun 7, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 7, 2026 •

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@cristim
cristim merged commit 2276bce into feat/multicloud-web-frontend Jun 7, 2026
4 checks passed
@cristim
cristim deleted the fix/azure-deploy-role-bootstrap branch July 27, 2026 11:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working effort/m Days impact/internal Team-internal only priority/p1 Next up; this sprint severity/high Significant harm triaged Item has been triaged type/bug Defect urgency/now Drop other things

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant