Skip to content

refactor(multi-runner): decouple EC2 Terraform logic into provider modules #5252

Description

@edersonbrilhante

Summary

Decouple EC2-specific Terraform logic from the common multi-runner orchestration so future lane providers can plug into the module without copying or bending EC2 assumptions.

The current Terraform layout mixes common runner, queue, pool, webhook, IAM, and EC2 implementation details. As we add experimental multi_runner_config_v2 and prepare for providers such as microVM or CodeBuild, the module should have a provider-neutral common layer plus provider-specific modules.

Proposed Direction

  • Keep common lane orchestration in a shared multi-runner layer.
  • Move EC2-specific runner logic into a provider module, for example modules/multi-runner/providers/ec2 or a similar repo-convention-aligned path.
  • Split runner concerns into common inputs/outputs and provider-specific inputs/outputs.
  • Split pool logic the same way: common scheduling/pool contract in the shared layer, provider-specific pool implementation in the EC2 provider module.
  • Split webhook matcher/routing config from provider execution details.
  • Split IAM/policies between common responsibilities and per-provider/per-lane permissions.
  • Keep queue ownership and lane keys stable so existing Terraform addresses do not churn unnecessarily.
  • Preserve EC2 behavior while moving implementation behind the provider boundary.

Scope Notes

  • This issue is about Terraform module structure, not implementing microVM or CodeBuild Terraform resources yet.
  • EC2 should remain the only active Terraform-managed provider until new provider modules are implemented.
  • The experimental multi_runner_config_v2 shape can guide the boundary, but it should remain flexible until the module split settles.

Acceptance Criteria

  • Common multi-runner logic no longer directly owns EC2-specific runner and pool implementation details.
  • EC2 provider module owns EC2 runner resources, EC2 pool resources, and EC2-specific policies.
  • Common module owns shared queue, webhook matcher/routing, lane normalization, and shared policy pieces.
  • Existing multi_runner_config EC2 lanes keep working without behavior changes.
  • multi_runner_config_v2 can route an EC2 lane through the provider module without introducing active microVM or CodeBuild support.
  • Terraform validation and tflint pass for the affected modules.
  • Documentation explains the common/provider split and marks non-EC2 providers as future work.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions