Skip to content

fix(apigee): retry "the resource is locked by another operation" - #18649

Open
alec-pinson wants to merge 1 commit into
GoogleCloudPlatform:mainfrom
alec-pinson:fix/apigee-retry-resource-locked
Open

fix(apigee): retry "the resource is locked by another operation"#18649
alec-pinson wants to merge 1 commit into
GoogleCloudPlatform:mainfrom
alec-pinson:fix/apigee-retry-resource-locked

Conversation

@alec-pinson

Copy link
Copy Markdown

Apigee serialises long-running operations per organization. A mutation issued while another operation is in flight against the same org is rejected immediately with HTTP 400 the resource is locked by another operation rather than being queued.

google_apigee_instance already retries this via transport_tpg.IsApigeeRetryableError (added for hashicorp/terraform-provider-google#10979), but every other org-scoped asynchronous Apigee resource can hit the same error and does not.

This is easy to reproduce with a for_each that creates two of them concurrently in a single apply — the first succeeds, the second fails outright:

google_apigee_endpoint_attachment.this["euw2"]: Creation complete after 1m5s
Error: Error creating EndpointAttachment: googleapi: Error 400: the resource is
locked by another operation where endpoint attachment swp-euw2 is currently being
created for organization <org> by operation: 1c83301f-195f-4d20-b6f4-7d5c8f6b66e8

There is no practical config-side workaround: Terraform builds dependency edges at the config-node level rather than per instance, so neither a time_sleep stagger nor a child-module-per-item stagger serialises a dynamic for_each — every instance still starts after the longest sleep. That leaves -parallelism=1 for the whole apply, which also serialises the 30–45 minute google_apigee_instance creates.

This PR applies the existing predicate to the remaining org-scoped async Apigee resources: EndpointAttachment, InstanceAttachment, EnvgroupAttachment, Environment, Envgroup and NatAddress.

Note that InstanceAttachment already carries mutex: apigeeInstanceAttachments. That mutex only covers a single provider process and does not prevent collisions against the same org from a different resource type, so the predicate is complementary rather than redundant.

Contributes to hashicorp/terraform-provider-google#12369, hashicorp/terraform-provider-google#11445 and hashicorp/terraform-provider-google#10084.

apigee: fixed an issue where concurrently creating `google_apigee_endpoint_attachment`, `google_apigee_instance_attachment`, `google_apigee_envgroup_attachment`, `google_apigee_environment`, `google_apigee_envgroup` or `google_apigee_nat_address` resources could fail with a 400 error stating the resource is locked by another operation

Apigee serialises long-running operations per organization. Any mutation
issued while another operation is in flight against the same org is
rejected immediately with HTTP 400 "the resource is locked by another
operation" rather than being queued.

google_apigee_instance already retries this via
transport_tpg.IsApigeeRetryableError (GoogleCloudPlatform#10979), but every other org-scoped
asynchronous Apigee resource can hit the same error and does not. This is
easy to reproduce with a for_each that creates two of them concurrently,
e.g. two google_apigee_endpoint_attachment resources in one apply: the
first succeeds, the second fails outright.

Apply the existing predicate to the remaining org-scoped async resources:
EndpointAttachment, InstanceAttachment, EnvgroupAttachment, Environment,
Envgroup and NatAddress.

Reported for endpoint attachments in hashicorp/terraform-provider-google#12369
and for instance attachments in hashicorp/terraform-provider-google#11445
and hashicorp/terraform-provider-google#10084.
@modular-magician modular-magician added the awaiting-approval Pull requests that need reviewer's approval to run presubmit tests label Aug 14, 2026
@github-actions

Copy link
Copy Markdown

Googlers: For automatic test runs see go/terraform-auto-test-runs.

@c2thorn, a repository maintainer, has been assigned to review your changes. If you have not received review feedback within 2 business days, please leave a comment on this PR asking them to take a look.

You can help make sure that review is quick by doing a self-review and by running impacted tests locally.

@github-actions
github-actions Bot requested a review from c2thorn August 14, 2026 08:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting-approval Pull requests that need reviewer's approval to run presubmit tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants