-
Notifications
You must be signed in to change notification settings - Fork 0
Reference deployment for a workload trust zone cluster in aws #18
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
alec-w
wants to merge
1
commit into
main
Choose a base branch
from
trust-zone-aws-deployment-reference
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,87 @@ | ||
| # Trust Zone Infrastructure | ||
|
|
||
| Terragrunt stack that provisions the AWS infrastructure for a Cofide Connect trust zone workload cluster. The SPIRE server is deployed onto this cluster and registered as a trust zone with the Connect control plane. | ||
|
|
||
| The stack reuses the VPC, jump instance, and Route53 hosted zone from the control-plane stack by default. Cross-stack dependencies are optional — set the relevant values in a unit's `common.local.hcl` to deploy against different infrastructure. | ||
|
|
||
| ## Configuration | ||
|
|
||
| ### Root configuration | ||
|
|
||
| Create the root `common.local.hcl` before running any unit: | ||
|
|
||
| ```sh | ||
| cp common.local.hcl.example common.local.hcl | ||
| ``` | ||
|
|
||
| Edit it to set your AWS account, region, and S3 remote state bucket: | ||
|
|
||
| ```hcl | ||
| locals { | ||
| aws_account_id = "123456789012" | ||
| aws_region = "eu-west-2" | ||
| tf_state_bucket_region = "eu-west-2" | ||
| tf_state_bucket_name = "my-tf-state-bucket" | ||
| tf_state_key_prefix = "connect-trust-zone" | ||
| } | ||
| ``` | ||
|
|
||
| Use a different `tf_state_key_prefix` from the control-plane stack to avoid state key collisions. | ||
|
|
||
| ### Per-unit configuration | ||
|
|
||
| Each unit directory contains a `common.local.hcl.example` documenting the available configuration. Copy it to `common.local.hcl` and edit it before applying. At minimum, set `cluster_admin_role_arns` in `eks-cluster/cluster/common.local.hcl`. | ||
|
|
||
| ### IAM mode | ||
|
|
||
| The `spire-server/iam-role` unit supports two authentication modes, configured via `iam_mode` in `common.local.hcl`: | ||
|
|
||
| - **`pod_identity`** (default) — EKS Pod Identity. Requires the `eks-pod-identity-agent` add-on, which is enabled by default in the cluster unit. | ||
| - **`irsa`** — IAM Roles for Service Accounts. Requires `enable_irsa = true` in the cluster unit's `common.local.hcl` and `oidc_provider_arn` set in the controller unit's `common.local.hcl`. | ||
|
|
||
| --- | ||
|
|
||
| ## Deployment | ||
|
|
||
| Before running any Terragrunt command, create and populate the root configuration file: | ||
|
|
||
| ```sh | ||
| cp common.local.hcl.example common.local.hcl | ||
| # Set aws_account_id, aws_region, tf_state_bucket_region, tf_state_bucket_name, tf_state_key_prefix | ||
| ``` | ||
|
|
||
| ### EKS cluster | ||
|
|
||
| The cluster unit creates the EKS control plane, node group, and add-ons. Apply it first; the controller and SPIRE units depend on its outputs. | ||
|
|
||
| ```sh | ||
| cd eks-cluster/cluster | ||
| cp common.local.hcl.example common.local.hcl | ||
| # Set cluster_admin_role_arns to your IAM role ARN. | ||
| terragrunt apply | ||
| ``` | ||
|
|
||
| See [`eks-cluster/cluster/README.md`](eks-cluster/cluster/README.md) for how to access the cluster. | ||
|
|
||
| ### SPIRE server IAM role | ||
|
|
||
| Creates the IAM role for the SPIRE server with KMS permissions. Apply this after the cluster unit. See [`spire-server/README.md`](spire-server/README.md) for details. | ||
|
|
||
| ```sh | ||
| cd spire-server/iam-role && terragrunt apply | ||
| ``` | ||
|
|
||
| ### Connect registration | ||
|
|
||
| Registers the trust zone, cluster, and trust zone server with Cofide Connect in a single apply. See [`connect/README.md`](connect/README.md) for details. | ||
|
|
||
| ```sh | ||
| export COFIDE_API_TOKEN="<your-api-token>" | ||
|
|
||
| cd connect/trust-zone | ||
| cp common.local.hcl.example common.local.hcl | ||
| # Set trust_domain. | ||
| terragrunt apply | ||
| ``` | ||
|
|
||
| Once applied, run `generate-local-values.sh` in `k8s/spire-server/spire/` to populate the SPIRE server configuration — see the [k8s guide](../../k8s/README.md). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| locals { | ||
| aws_account_id = "1234567890" | ||
| aws_region = "eu-west-2" | ||
| tf_state_bucket_region = "eu-west-2" | ||
| tf_state_bucket_name = "example-bucket-name" | ||
| tf_state_key_prefix = "connect-trust-zone" | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| # Connect Registration | ||
|
|
||
| Terragrunt unit that registers the trust zone and cluster with Cofide Connect using the [`cofide/cofide`](https://registry.terraform.io/providers/cofide/cofide/latest/docs) Terraform provider. | ||
|
|
||
| This unit is applied after the AWS infrastructure is provisioned and before deploying the SPIRE server onto Kubernetes. The IDs and PSAT configuration it outputs are used to populate the SPIRE server's datastore configuration. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| - `COFIDE_API_TOKEN` — API token for the Connect API; must be set as an environment variable. | ||
| - The control-plane `base/dns` unit must be applied — the Connect gRPC address (`<zone>:443`) is derived from its `zone_name` output. The provider prepends the `connect.` subdomain automatically. Set `connect_url` in `common.local.hcl` to override (format: `host:port`, no scheme). | ||
| - The `eks-cluster/cluster` unit must be applied — the OIDC issuer URL is read from its outputs. Set `oidc_issuer_url` in `common.local.hcl` to override. | ||
|
|
||
| ## `trust-zone/` | ||
|
|
||
| Registers a trust zone and cluster with Cofide Connect. The cluster is registered in the same apply as the trust zone because the SPIRE server for this trust zone lives on this cluster. | ||
|
|
||
| **Required values in `common.local.hcl`:** | ||
|
|
||
| | Local | Description | | ||
| |-------|-------------| | ||
| | `trust_domain` | SPIFFE trust domain for the trust zone (e.g. `trust-zone.example.cofide.dev`) | | ||
|
|
||
| **Outputs:** | ||
|
|
||
| | Name | Description | | ||
| |------|-------------| | ||
| | `trust_zone_id` | Trust zone ID — used in the SPIRE server datastore config | | ||
| | `cluster_id` | Cluster ID — used in the SPIRE server datastore config | | ||
| | `bundle_endpoint_url` | Bundle endpoint URL assigned by Connect | | ||
|
|
||
| ## Deployment | ||
|
|
||
| ```sh | ||
| export COFIDE_API_TOKEN="<your-api-token>" | ||
|
|
||
| cd trust-zone | ||
| cp common.local.hcl.example common.local.hcl | ||
| # Set trust_domain. | ||
| terragrunt apply | ||
| ``` | ||
|
|
||
| Once applied, run `generate-local-values.sh` in `k8s/spire-server/spire/` to populate the SPIRE server configuration — see the [k8s guide](../../../k8s/README.md). |
42 changes: 42 additions & 0 deletions
42
workload/deployment/aws/infra/stack/connect/trust-zone/.terraform.lock.hcl
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
14 changes: 14 additions & 0 deletions
14
workload/deployment/aws/infra/stack/connect/trust-zone/common.local.hcl.example
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| locals { | ||
| # Name for the trust zone in Cofide Connect. | ||
| trust_zone_name = "connect-trust-zone-aws-reference-arch" | ||
|
|
||
| # SPIFFE trust domain for this trust zone. Required — no default. | ||
| trust_domain = "trust-zone.example.cofide.dev" | ||
|
|
||
| # Name for the cluster in Cofide Connect. | ||
| cluster_name = "connect-trust-zone-aws-reference-arch" | ||
|
|
||
| # Optional: override values if not reading from their respective dependency units. | ||
| # connect_url = "example.cofide.dev:443" | ||
| # oidc_issuer_url = "https://oidc.eks.eu-west-2.amazonaws.com/id/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" | ||
| } |
17 changes: 17 additions & 0 deletions
17
workload/deployment/aws/infra/stack/connect/trust-zone/main.tf
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| resource "cofide_connect_trust_zone" "this" { | ||
| name = var.trust_zone_name | ||
| trust_domain = var.trust_domain | ||
| } | ||
|
|
||
| resource "cofide_connect_cluster" "this" { | ||
| name = var.cluster_name | ||
| trust_zone_id = cofide_connect_trust_zone.this.id | ||
| profile = "kubernetes" | ||
| oidc_issuer_url = var.oidc_issuer_url | ||
| external_server = false | ||
| kubernetes_context = "" | ||
|
|
||
| trust_provider = { | ||
| kind = "kubernetes" | ||
| } | ||
| } |
24 changes: 24 additions & 0 deletions
24
workload/deployment/aws/infra/stack/connect/trust-zone/outputs.tf
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| output "trust_domain" { | ||
| description = "The SPIFFE trust domain for this trust zone." | ||
| value = cofide_connect_trust_zone.this.trust_domain | ||
| } | ||
|
|
||
| output "trust_zone_id" { | ||
| description = "The ID of the trust zone registered with Cofide Connect." | ||
| value = cofide_connect_trust_zone.this.id | ||
| } | ||
|
|
||
| output "bundle_endpoint_url" { | ||
| description = "The bundle endpoint URL for this trust zone, as assigned by Cofide Connect." | ||
| value = cofide_connect_trust_zone.this.bundle_endpoint_url | ||
| } | ||
|
|
||
| output "cluster_id" { | ||
| description = "The ID of the cluster registered with Cofide Connect." | ||
| value = cofide_connect_cluster.this.id | ||
| } | ||
|
|
||
| output "cluster_name" { | ||
| description = "The name of the cluster as registered with Cofide Connect." | ||
| value = cofide_connect_cluster.this.name | ||
| } |
4 changes: 4 additions & 0 deletions
4
workload/deployment/aws/infra/stack/connect/trust-zone/providers.tf
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| provider "cofide" { | ||
| connect_url = var.connect_url | ||
| # api_token is configured via the COFIDE_API_TOKEN environment variable. | ||
| } |
54 changes: 54 additions & 0 deletions
54
workload/deployment/aws/infra/stack/connect/trust-zone/terragrunt.hcl
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,54 @@ | ||
| include "root" { | ||
| path = find_in_parent_folders("root.hcl") | ||
| expose = true | ||
| } | ||
|
|
||
| dependency "cluster" { | ||
| config_path = "../../eks-cluster/cluster" | ||
|
|
||
| mock_outputs_allowed_terraform_commands = ["apply", "destroy", "plan", "validate"] | ||
| mock_outputs = { | ||
| oidc_issuer_url = "https://oidc.eks.eu-west-2.amazonaws.com/id/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" | ||
| } | ||
| } | ||
|
|
||
| dependency "cp_dns" { | ||
| config_path = "${get_repo_root()}/control-plane/deployment/aws/infra/stack/base/dns" | ||
|
|
||
| mock_outputs_allowed_terraform_commands = ["apply", "destroy", "plan", "validate"] | ||
| mock_outputs = { | ||
| zone_name = "example.cofide.dev" | ||
| } | ||
| } | ||
|
|
||
| locals { | ||
| default_config = { | ||
| trust_zone_name = "connect-trust-zone-aws-reference-arch" | ||
| cluster_name = "connect-trust-zone-aws-reference-arch" | ||
| } | ||
|
|
||
| unit_config_path = "${get_terragrunt_dir()}/common.local.hcl" | ||
| has_local_config = fileexists(local.unit_config_path) | ||
| user_config = local.has_local_config ? read_terragrunt_config(local.unit_config_path).locals : {} | ||
| merged_config = merge(local.default_config, local.user_config) | ||
|
|
||
| # trust_domain has no default — it must be set in common.local.hcl. | ||
| trust_domain = local.merged_config.trust_domain | ||
|
|
||
| user_connect_url = try(local.merged_config.connect_url, null) | ||
| user_oidc_issuer_url = try(local.merged_config.oidc_issuer_url, null) | ||
| } | ||
|
|
||
| terraform { | ||
| source = "." | ||
| } | ||
|
|
||
| inputs = { | ||
| connect_url = local.user_connect_url != null ? local.user_connect_url : "${dependency.cp_dns.outputs.zone_name}:443" | ||
|
|
||
| trust_zone_name = local.merged_config.trust_zone_name | ||
| trust_domain = local.trust_domain | ||
| cluster_name = local.merged_config.cluster_name | ||
|
|
||
| oidc_issuer_url = local.user_oidc_issuer_url != null ? local.user_oidc_issuer_url : dependency.cluster.outputs.oidc_issuer_url | ||
| } | ||
24 changes: 24 additions & 0 deletions
24
workload/deployment/aws/infra/stack/connect/trust-zone/variables.tf
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| variable "connect_url" { | ||
| type = string | ||
| description = "Cofide Connect API gRPC address in host:port form (e.g. example.cofide.dev:443). The provider prepends the connect. subdomain. No scheme." | ||
| } | ||
|
|
||
| variable "trust_zone_name" { | ||
| type = string | ||
| description = "The name of the trust zone to register with Cofide Connect." | ||
| } | ||
|
|
||
| variable "trust_domain" { | ||
| type = string | ||
| description = "The SPIFFE trust domain for this trust zone (e.g. trust-zone.example.cofide.dev)." | ||
| } | ||
|
|
||
| variable "cluster_name" { | ||
| type = string | ||
| description = "The name of the cluster to register with Cofide Connect." | ||
| } | ||
|
|
||
| variable "oidc_issuer_url" { | ||
| type = string | ||
| description = "OIDC issuer URL of the EKS cluster (e.g. https://oidc.eks.<region>.amazonaws.com/id/<id>)." | ||
| } |
14 changes: 14 additions & 0 deletions
14
workload/deployment/aws/infra/stack/connect/trust-zone/versions.tf
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| terraform { | ||
| required_version = ">= 1.10" | ||
|
|
||
| required_providers { | ||
| aws = { | ||
| source = "hashicorp/aws" | ||
| version = "~> 6.0" | ||
| } | ||
| cofide = { | ||
| source = "cofide/cofide" | ||
| version = "~> 0.9" | ||
| } | ||
| } | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If
trust_domainis not defined incommon.local.hcl, accessinglocal.merged_config.trust_domaindirectly will cause a fatal Terragrunt parsing error ("Unsupported attribute") instead of letting Terraform handle the missing required variable cleanly. Usingtry()allows it to safely fall back tonullso that Terraform can raise a standard validation error.