Skip to content

fix(EC-1778): use go-gather WithTransport options for HTTP transports#3327

Open
robnester-rh wants to merge 9 commits into
conforma:mainfrom
robnester-rh:EC-1778
Open

fix(EC-1778): use go-gather WithTransport options for HTTP transports#3327
robnester-rh wants to merge 9 commits into
conforma:mainfrom
robnester-rh:EC-1778

Conversation

@robnester-rh
Copy link
Copy Markdown
Contributor

Summary

  • Update go-gather dependency from v1.1.0 to v1.2.0
  • Replace removed goci.Transport/ghttp.Transport global mutations with WithTransport functional options on constructed gatherer instances
  • gatherFunc now dispatches OCI/HTTP sources via Matcher before falling back to the registry for git/file sources
  • Update all downloader tests for the new API surface

Test plan

  • go test -race -tags=unit ./internal/downloader/ — all 6 tests pass
  • make test — all unit, integration, and generative tests pass across the CLI
  • Local code review — no critical or important issues
  • CodeRabbit review — 0 findings

resolves: EC-1778

🤖 Generated with Claude Code

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Jun 1, 2026

📝 Walkthrough

Walkthrough

This PR migrates the CLI downloader to use go-gather v1.2.0's functional transport options, replacing global transport variable mutation with gatherer construction via WithTransport options. The change introduces global ociGatherer and httpGatherer variables, rewrites initialization to build retry-wrapped transports, and updates dispatch logic to route sources via matcher-based selection with registry fallback.

Changes

Downloader WithTransport Migration to go-gather v1.2.0

Layer / File(s) Summary
Design and Migration Specification
docs/superpowers/specs/2026-05-28-ec1778-downloader-withtransport-design.md, docs/superpowers/plans/2026-05-29-ec1778-downloader-withtransport.md
Design spec and implementation plan document the migration to go-gather v1.2.0, including functional-option-based gatherer construction, matcher-based dispatch routing with registry fallback, transport wrapping for retry and optional tracing, and corresponding test adjustments.
Dependency Update
go.mod
Bump github.com/conforma/go-gather to v1.2.0 and github.com/go-git/go-git/v5 to v5.18.0.
Downloader implementation
internal/downloader/downloader.go
Introduce package-level ociGatherer and httpGatherer, change imports to build net/http RoundTrippers, rewrite _initialize to construct a base RoundTripper (optional tracing), create retry-enabled transports per gatherer, instantiate gatherers with WithTransport and store them in globals, and change gatherFunc to dispatch via Matcher then registry.GetGatherer fallback.
Unit Test Updates
internal/downloader/downloader_test.go
Update tests to reset ociGatherer/httpGatherer in cleanup, call _initialize() explicitly where needed, rewrite TestOCIClientConfiguration to assert gatherer type, and adjust HTTP tests to inspect httpGatherer.Client.Transport for retry assertions.

Sequence Diagram

sequenceDiagram
  participant gatherFunc
  participant initialize
  participant ociGatherer
  participant httpGatherer
  participant registry
  gatherFunc->>initialize: call _initialize()
  initialize->>initialize: build base RoundTripper (with optional tracing)
  initialize->>ociGatherer: NewOCIGatherer(WithTransport(retry))
  initialize->>httpGatherer: NewHTTPGatherer(WithTransport(retry))
  gatherFunc->>ociGatherer: Matcher(source)
  alt OCI match
    ociGatherer->>ociGatherer: Gather(source)
  else HTTP match
    gatherFunc->>httpGatherer: Matcher(source)
    httpGatherer->>httpGatherer: Gather(source)
  else fallback
    gatherFunc->>registry: GetGatherer(source)
  end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: migrating from removed Transport globals to WithTransport functional options in go-gather, which is the primary focus across all modified files.
Description check ✅ Passed The description is directly related to the changeset, detailing the dependency bump, API migration strategy, dispatch logic changes, test updates, and providing a comprehensive test plan with passing results.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@qodo-for-conforma
Copy link
Copy Markdown

Review Summary by Qodo

Migrate downloader to go-gather v1.2.0 WithTransport API

✨ Enhancement 🧪 Tests

Grey Divider

Walkthroughs

Description
• Migrate from removed goci.Transport/ghttp.Transport globals to WithTransport functional
  options
• Update go-gather dependency from v1.1.0 to v1.2.0
• Implement Matcher-based dispatch in gatherFunc for OCI/HTTP sources
• Refactor _initialize to construct gatherer instances with transport stack
• Update all downloader tests to work with new gatherer variable pattern
Diagram
flowchart LR
  A["go-gather v1.2.0"] -->|"WithTransport options"| B["_initialize"]
  B -->|"constructs"| C["ociGatherer"]
  B -->|"constructs"| D["httpGatherer"]
  E["gatherFunc"] -->|"Matcher dispatch"| C
  E -->|"Matcher dispatch"| D
  E -->|"fallback"| F["registry"]
  G["Transport Stack"] -->|"tracing + retry"| B

Loading

Grey Divider

File Changes

1. internal/downloader/downloader.go ✨ Enhancement +28/-10

Replace transport globals with WithTransport gatherers

• Add net_http import alias for stdlib net/http to avoid conflict with internal/http
• Introduce package-level ociGatherer and httpGatherer variables initialized via sync.OnceFunc
• Rewrite _initialize to build shared base transport (with optional tracing), create independent
 retry transports, and construct gatherer instances via WithTransport functional options
• Replace gatherFunc single registry lookup with Matcher-based dispatch: OCI first, HTTP second,
 registry fallback for git/file sources
• Remove all references to removed goci.Transport and ghttp.Transport globals

internal/downloader/downloader.go


2. internal/downloader/downloader_test.go 🧪 Tests +23/-15

Update tests for new gatherer variable pattern

• Update TestOCITracing cleanup to reset ociGatherer and httpGatherer vars instead of removed
 transport globals
• Update TestHTTPTracing cleanup similarly; add direct _initialize() call and
 httpGatherer.Gather() invocation to test HTTP path specifically (avoiding OCI Matcher dispatch on
 localhost)
• Simplify TestOCIClientConfiguration to assert gatherer type only, since OCIGatherer.transport
 is private; transport wiring verified end-to-end by TestOCITracing
• Rewrite TestHTTPClientConfiguration to inspect httpGatherer.Client.Transport instead of
 removed ghttp.Transport global; add cleanup for gatherer vars

internal/downloader/downloader_test.go


3. go.mod Dependencies +2/-2

Update go-gather and go-git dependencies

• Update github.com/conforma/go-gather from v1.1.0 to v1.2.0
• Update github.com/go-git/go-git/v5 from v5.17.1 to v5.18.0

go.mod


View more (3)
4. go.sum Dependencies +4/-4

Update dependency checksums

• Update checksums for github.com/conforma/go-gather v1.2.0
• Update checksums for github.com/go-git/go-git/v5 v5.18.0

go.sum


5. docs/superpowers/specs/2026-05-28-ec1778-downloader-withtransport-design.md 📝 Documentation +158/-0

Add design specification for WithTransport migration

• Add comprehensive design specification for the WithTransport migration
• Document architecture: package-level gatherer vars with sync.OnceFunc, Matcher-based dispatch,
 registry fallback
• Specify production changes to _initialize and gatherFunc with code examples
• Detail test strategy for each test function, including rationale for TestOCIClientConfiguration
 simplification (private transport field)
• Document import alias conventions and acceptance criteria

docs/superpowers/specs/2026-05-28-ec1778-downloader-withtransport-design.md


6. docs/superpowers/plans/2026-05-29-ec1778-downloader-withtransport.md 📝 Documentation +532/-0

Add implementation plan with task-by-task guidance

• Add detailed 7-task implementation plan with step-by-step instructions
• Task 1: Update go-gather to v1.2.0 with verification steps
• Tasks 2-6: Rewrite production code and update each test function with exact code replacements and
 test commands
• Task 7: Run full test suite and verify compilation
• Include commit message templates and expected test outcomes for each task

docs/superpowers/plans/2026-05-29-ec1778-downloader-withtransport.md


Grey Divider

Qodo Logo

@qodo-for-conforma
Copy link
Copy Markdown

qodo-for-conforma Bot commented Jun 1, 2026

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0)

Grey Divider


Action required

1. HTTP scheme routed to OCI ✓ Resolved 🐞 Bug ≡ Correctness
Description
gatherFunc checks ociGatherer.Matcher(source) before httpGatherer.Matcher(source) without
first respecting an explicit http:///https:// scheme, so URLs like localhost/127.0.0.1 can be
routed to the OCI gatherer. This can prevent intended HTTPS file downloads from
localhost/registry-like hosts because they will be handled by OCIGatherer rather than the HTTP
gatherer.
Code

internal/downloader/downloader.go[R55-60]

Relevance

⭐⭐ Medium

No historical review evidence about URL scheme vs OCI/HTTP matcher ordering in downloader routing.

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The new dispatch in gatherFunc checks OCI before HTTP. The PR’s own tests document that
127.0.0.1 matches the OCI registry pattern and that gatherFunc would therefore select OCI
instead of HTTP, and the design doc explicitly states OCI matcher includes localhost. Since
Download allows https://... sources, this routing affects real CLI downloads for HTTPS
localhost/registry-like hosts.

internal/downloader/downloader.go[52-66]
internal/downloader/downloader_test.go[231-238]
docs/superpowers/specs/2026-05-28-ec1778-downloader-withtransport-design.md[91-94]
internal/downloader/downloader.go[139-141]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`gatherFunc` dispatches using OCI matcher first, which can match `localhost`/`127.0.0.1` even when the input is an explicit `http(s)://...` URL. This causes explicit HTTP(S) URLs to be handled by the OCI gatherer.

### Issue Context
The design doc claims “no overlap” between matchers, but the test suite had to bypass `gatherFunc` for `127.0.0.1` because it would route to OCI.

### Fix Focus Areas
- internal/downloader/downloader.go[52-66]

### Suggested fix
Add an explicit scheme-based dispatch before the Matcher-based switch, e.g.:
- If `source` starts with `"oci://"` or `"oci::"` -> OCI gatherer
- Else if `source` starts with `"http://"` or `"https://"` -> HTTP gatherer
- Else fall back to current Matcher ordering (OCI first, then HTTP, then registry)

Optionally, update/extend unit coverage so `TestHTTPTracing` can call `gatherFunc(...)` directly once explicit scheme routing is in place.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Qodo Logo

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@internal/downloader/downloader.go`:
- Around line 55-59: The current routing checks ociGatherer.Matcher(source)
before httpGatherer.Matcher(source), which can misroute HTTP URLs (e.g.,
127.0.0.1) to the OCI gatherer; modify the gather routing so HTTP(s) URLs are
matched first—either by placing the httpGatherer.Matcher(source) case before
ociGatherer.Matcher(source) in the switch or by adding an explicit scheme check
(strings.HasPrefix(source,"http://") || strings.HasPrefix(source,"https://"))
that routes to httpGatherer.Gather(ctx, source, destination) before falling back
to ociGatherer.Gather(ctx, source, destination); update the switch in the gather
function where ociGatherer.Matcher and httpGatherer.Matcher are referenced to
implement this change.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Enterprise

Run ID: 99df4c82-897b-45b9-b2b8-021d7cf07f26

📥 Commits

Reviewing files that changed from the base of the PR and between 1026da7 and 357d4c3.

⛔ Files ignored due to path filters (1)
  • go.sum is excluded by !**/*.sum
📒 Files selected for processing (5)
  • docs/superpowers/plans/2026-05-29-ec1778-downloader-withtransport.md
  • docs/superpowers/specs/2026-05-28-ec1778-downloader-withtransport-design.md
  • go.mod
  • internal/downloader/downloader.go
  • internal/downloader/downloader_test.go

Comment thread internal/downloader/downloader.go Outdated
Comment thread internal/downloader/downloader.go Outdated
@codecov
Copy link
Copy Markdown

codecov Bot commented Jun 1, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.

Flag Coverage Δ
acceptance 55.59% <78.57%> (-0.01%) ⬇️
generative 17.81% <0.00%> (-0.02%) ⬇️
integration 26.54% <0.00%> (-0.02%) ⬇️
unit 69.02% <85.71%> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
internal/downloader/downloader.go 95.74% <100.00%> (+0.87%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

robnester-rh and others added 8 commits June 2, 2026 12:59
reference: EC-1778

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Resolve three gaps: explicit sync.OnceFunc wiring, concrete
TestOCIClientConfiguration strategy, import alias conventions.

reference: EC-1778

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
7 tasks: go.mod update, production rewrite, 4 test updates, final
verification. TDD-sequenced with exact code and commands.

reference: EC-1778

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
reference: EC-1778

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace removed goci.Transport/ghttp.Transport global mutations with
constructed gatherer instances using WithTransport functional options.
gatherFunc now dispatches via Matcher (OCI, HTTP) before falling back
to the registry for git/file sources.

reference: EC-1778

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace references to removed goci.Transport/ghttp.Transport globals
with gatherer var cleanup. TestOCIClientConfiguration simplified to
type assertion (transport is private). TestHTTPClientConfiguration
inspects httpGatherer.Client.Transport instead of removed global.
TestHTTPTracing calls httpGatherer.Gather directly since 127.0.0.1
now matches the OCI registry pattern in the new dispatch order.

reference: EC-1778

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
reference: EC-1778

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…names

The OCI Matcher matches localhost/127.0.0.1 as known registries, which
misroutes git-over-HTTP sources hosted on localhost in acceptance tests.
Dispatch to custom gatherers only for explicit oci:// or http(s)://
scheme prefixes; bare hostnames fall through to the registry where
git matchers are checked before OCI matchers.

reference: EC-1778

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/superpowers/plans/2026-05-29-ec1778-downloader-withtransport.md`:
- Around line 190-208: Update the documented gatherFunc to match the real
implementation in downloader.go: replace the Matcher-based dispatch
(ociGatherer.Matcher / httpGatherer.Matcher) with explicit scheme prefix checks
using strings.HasPrefix for OCI ("oci://" or "oci::") and HTTP ("https://" or
"http://"), and preserve the additional conditional call to httpGatherer.Matcher
inside the HTTP branch before delegating to httpGatherer.Gather; keep the
registry fallback using registry.GetGatherer and its error handling unchanged.

In `@docs/superpowers/specs/2026-05-28-ec1778-downloader-withtransport-design.md`:
- Around line 72-88: The spec's dispatch snippet and "Matcher ordering
rationale" must be updated to match the implemented gatherFunc logic: describe
that dispatch uses explicit scheme-prefix checks (e.g., "oci://" routed to
ociGatherer) plus a conditional httpGatherer.Matcher guard for "http(s)://"
while leaving bare hostnames (like "quay.io/repo:tag") to fall through to
registry.GetGatherer; replace the Matcher-only examples (ociGatherer.Matcher,
httpGatherer.Matcher) with the actual scheme-check logic, explain why this
prevents bare hostnames from being captured by the OCI gatherer, and update any
ordering rationale to reflect this implementation detail.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Enterprise

Run ID: 67b02627-4777-4cbf-9d91-0426a8aba259

📥 Commits

Reviewing files that changed from the base of the PR and between a3ec50f and 4f665de.

⛔ Files ignored due to path filters (1)
  • go.sum is excluded by !**/*.sum
📒 Files selected for processing (5)
  • docs/superpowers/plans/2026-05-29-ec1778-downloader-withtransport.md
  • docs/superpowers/specs/2026-05-28-ec1778-downloader-withtransport-design.md
  • go.mod
  • internal/downloader/downloader.go
  • internal/downloader/downloader_test.go
🚧 Files skipped from review as they are similar to previous changes (2)
  • go.mod
  • internal/downloader/downloader_test.go

Comment thread docs/superpowers/plans/2026-05-29-ec1778-downloader-withtransport.md Outdated
Comment thread docs/superpowers/specs/2026-05-28-ec1778-downloader-withtransport-design.md Outdated
These were working artifacts for the implementation process,
not deliverables.

reference: EC-1778

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added size: M and removed size: XXL labels Jun 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant