Skip to content

make test passes when the envtest asset fetch fails #17

Open

Description

@shreemaan-abhishek

Problem

The test target runs KUBEBUILDER_ASSETS="$(setup-envtest use ...)" go test ./... (Makefile#L20). The exit status is go test's, so a failed fetch is swallowed, TestMain skips the envtest suites, and the job is green.

Reproduce

make test SETUP_ENVTEST=/usr/bin/false: the envtest suites print "skipping" and the target exits 0. PR #2's CI did run the suites (about 9 s for internal/controller), so this hasn't hit yet.

Scores

Reproducibility Likelihood Severity Existing mitigations Detectability Test coverage Fix cost vs benefit
10 2 4 3 3 0 9

Scores are out of 10. Higher is worse for likelihood and severity. Higher is better for mitigations, detectability, test coverage and cost vs benefit (10 = cheap fix, high value).

Fix

  • Makefile: assets="$$($(SETUP_ENVTEST) use ...)" && REQUIRE_ENVTEST=1 KUBEBUILDER_ASSETS="$$assets" go test ./....
  • TestMain: exit 1 when REQUIRE_ENVTEST is set and the assets are missing. Set it only in that target, so make test-aisix keeps skipping envtest.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions