Skip to content

registry pull retries - #2443

Merged
cgwalters merged 3 commits into
bootc-dev:mainfrom
HarshwardhanPatil07:fix/ostree-registry-pull-retries
Sep 22, 2026
Merged

cgwalters merged 3 commits into
bootc-dev:mainfrom
HarshwardhanPatil07:fix/ostree-registry-pull-retries

Conversation

@HarshwardhanPatil07

Copy link
Copy Markdown
Member

What

Add bounded retries for registry-backed OSTree image pulls when the containers-image proxy reports a GetBlob request-initiation failure.

This also routes the normal, non-unified OSTree installation path through the shared retrying pull function.

Related: #2177

Why

CI has intermittently failed during commands such as bootc install to-filesystem when registries like Quay return transient errors, including 502 Bad Gateway and network timeouts.

The existing CI retry wrapper covers preparatory Podman operations, but it does not cover image pulls performed internally by bootc. Consequently, one temporary registry failure can fail the entire integration job.

Retries are appropriate here because registry availability is outside bootc's control and repeating an image pull is safe.

The proxy error does not expose the underlying HTTP status. Therefore, bootc cannot distinguish a transient 502 from every permanent GetBlob initiation failure. The retry count bounds the worst-case additional delay to 60 seconds.

@github-actions github-actions Bot added the area/install Issues related to `bootc install` label Sep 8, 2026
@bootc-bot
bootc-bot Bot requested a review from cgwalters September 8, 2026 09:44
Comment thread crates/lib/src/install.rs Outdated
Comment thread crates/lib/src/deploy.rs
Comment thread crates/lib/src/deploy.rs
Comment thread crates/lib/src/deploy.rs Outdated
Comment thread crates/lib/src/deploy.rs Outdated
Comment thread crates/lib/src/deploy.rs
Comment thread crates/lib/src/deploy.rs Outdated
Comment thread crates/lib/src/deploy.rs Outdated
Comment thread crates/lib/src/install.rs Outdated
@github-actions github-actions Bot added the area/ostree Issues related to ostree label Sep 15, 2026
@HarshwardhanPatil07
HarshwardhanPatil07 force-pushed the fix/ostree-registry-pull-retries branch 2 times, most recently from 1485745 to a2a0ced Compare September 15, 2026 11:36
Direct image pulls can fail when a registry transiently rejects the GetBlob request. Retry that narrowly classified failure at the whole-pull boundary with a bounded attempt count, while allowing non-registry and unrelated failures to return immediately.

Related: bootc-dev#2177

Assisted-by: AI
Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>
@HarshwardhanPatil07
HarshwardhanPatil07 force-pushed the fix/ostree-registry-pull-retries branch from a2a0ced to 7bd3c43 Compare September 15, 2026 12:11
@HarshwardhanPatil07

Copy link
Copy Markdown
Member Author

PTAL @Johan-Liebert1 @cgwalters

Comment thread crates/lib/src/deploy.rs
Comment thread crates/lib/src/deploy.rs Outdated
Comment thread crates/lib/src/install.rs
The non-unified installation path prepared and pulled images directly, bypassing the shared retry boundary and leaving bootc install to-filesystem exposed to transient registry failures. Route that path through the retrying pull while preserving unified-storage behavior.

Related: bootc-dev#2177

Assisted-by: AI
Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>
@HarshwardhanPatil07
HarshwardhanPatil07 force-pushed the fix/ostree-registry-pull-retries branch 2 times, most recently from 496fb9a to 09be175 Compare September 16, 2026 11:43
cgwalters
cgwalters previously approved these changes Sep 16, 2026
Comment thread crates/lib/src/deploy.rs
Share the pull progress renderer across attempts and print retry notices through indicatif so tracing output cannot interfere with active progress bars.

Assisted-by: AI
Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>
@HarshwardhanPatil07

Copy link
Copy Markdown
Member Author

PTAL @Johan-Liebert1 @cgwalters

use super::*;

#[test]
fn test_join_fetch_preserves_typed_driver_error() {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not quite sure what we're trying to test here

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's that we can traverse the error chain?

Comment thread crates/lib/src/deploy.rs
retry_delay.as_secs()
);
progress
.send(Event::ProgressSteps {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I am not totally sure this matches the current concept of the progress steps which I think assumes a fixed number of steps?

Of course...the API is currently experimental anyways, but we may need to ensure it supports dynamism

use super::*;

#[test]
fn test_join_fetch_preserves_typed_driver_error() {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's that we can traverse the error chain?

@cgwalters
cgwalters merged commit 38a99e5 into bootc-dev:main Sep 22, 2026
69 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/install Issues related to `bootc install` area/ostree Issues related to ostree

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants