From f1eefd9d69cbac2a7122e368849b715b09b8ca8f Mon Sep 17 00:00:00 2001 From: Jan Rose Date: Fri, 6 Nov 2026 13:05:38 +0000 Subject: [PATCH 1/2] Run ruff via uv tool run in acceptance tests and install Pythons in bump-sdk-post The acceptance harness required a `ruff` binary on PATH for every run, although only the pydabs check-formatting test uses it. The SDK bumper's image has uv but no ruff, so `test-update` there stopped at TestAccept/prerequisites and bumper PRs never refreshed goldens such as acceptance/help/output.txt (#7277). Run ruff through `uv tool run ruff@0.9.1` instead: the harness fetches that release into the uv cache before it turns on UV_OFFLINE, and the test script runs the same command offline from the cache. The prefetch lives in testAccept rather than the prerequisites subtest, which a -run filter for a test directory skips. Once the ruff check no longer stops it, test-update in the bumper hits the next gap: its image has only Python 3.12, while bundle tests pin 3.10 and one template matrix needs 3.9-3.13, and uv is offline during tests. Run install-pythons at the start of bump-sdk-post, as CI does in setup-build-environment, so a missing interpreter is not baked into the goldens as a uv error. Co-authored-by: Isaac --- .agents/skills/bump-sdk/SKILL.md | 4 +-- Taskfile.yml | 6 ++++ acceptance/acceptance_test.go | 12 ++++++-- .../pydabs/check-formatting/output.txt | 4 +-- .../templates/pydabs/check-formatting/script | 7 +++-- acceptance/internal/ruff.go | 29 +++++-------------- acceptance/internal/ruff_test.go | 15 ---------- 7 files changed, 32 insertions(+), 45 deletions(-) delete mode 100644 acceptance/internal/ruff_test.go diff --git a/.agents/skills/bump-sdk/SKILL.md b/.agents/skills/bump-sdk/SKILL.md index 8ca4b0c4c6f..6541be3e9ca 100644 --- a/.agents/skills/bump-sdk/SKILL.md +++ b/.agents/skills/bump-sdk/SKILL.md @@ -20,7 +20,7 @@ The SDK bumper (eng-dev-ecosystem automation) does steps 1-4 for every new stabl It opens a PR from the `automation/go-sdk-vX.Y.Z` branch of `databricks/cli`, titled `Update Go SDK to vX.Y.Z`, containing the `go get` + `go mod tidy` result, `.codegen/_openapi_sha`, `.codegen/cli.json` (genkit, followed by `./task generate-clijson-post`), and a changelog fragment without a PR link. `generate-clijson-post` ends with `bump-sdk-post` (step 4), whose error it ignores so that the bumper, which aborts on any error there, still opens a PR. The PR therefore carries whatever got regenerated, and stays red while the bump needs hand fixes (step 5). Nothing logs the task's output in the bumper, so judge from the PR what got regenerated. -The bumper's environment has `uv` and Python but no `ruff`, which the acceptance tests require, so `test-update` stops at `TestAccept/prerequisites` there and bumper PRs carry no golden updates. +The bumper's image has `uv` and Python 3.12 but no `ruff`; the acceptance tests run ruff through `uv tool run` (prefetched by the harness) and `bump-sdk-post` installs the managed Pythons, so `test-update` can run there. Bumper PRs up to #7277 predate that and carry no golden updates. Bumper PRs opened before `generate-clijson-post` ran `bump-sdk-post` (e.g. #7138) have nothing downstream of `cli.json`: their stale command stubs usually fail to compile, which fails lint and every test job. The bumper does not merge CLI PRs, and it reuses an open PR for the same release on later runs, keeping commits pushed to its branch. @@ -58,7 +58,7 @@ Run directly, it fails on errors: `generate-refschema` and `test-update` build t Run it in CI's environment, or the goldens pick up local drift: - `uv` index: the acceptance tests build wheels offline from the vendored packages (`UV_OFFLINE`, `UV_FIND_LINKS`), but `uv` still reads its default index's pages from the shared cache. If a `uv.toml` (e.g. `/etc/uv/uv.toml`, which applies even under the tests' temporary `HOME`) makes an internal PyPI proxy the default index, `uv` picks versions from cached proxy pages that it cannot download offline and fails with `No solution found when resolving: hatchling`; `-update` then bakes that error into dozens of template goldens and deletes their `output/` trees. The same failure on `main` tells it apart from bump fallout. Set `UV_DEFAULT_INDEX=https://pypi.org/simple` for the acceptance runs only, so `uv` stops reading cached proxy pages (CI's test jobs point `UV_INDEX_URL` at a JFrog mirror instead): the generators fetch tools such as `ruff` and need the proxy where `pypi.org` is unreachable. On such a machine, run the task's two halves separately: `./task generate-check`, then `UV_DEFAULT_INDEX=https://pypi.org/simple ./task test-update` (and the same prefix for `./task test` later). If `generate-clijson` already ran `test-update` without it, first revert the `acceptance/` churn (`git checkout -- acceptance`, plus any stray untracked files it left). -- Python: CI installs the managed interpreters with `./task install-pythons`; do the same. +- Python: the managed interpreters from `./task install-pythons`, which `bump-sdk-post` runs first (CI runs it in setup-build-environment). - Go: the `toolchain` in `go.mod`. A newer local `go` also changes goldens and generated `gofmt` output; use `GOTOOLCHAIN=` if they differ. Before going on, account for every changed file outside the generated set, including the bumper's (its environment is not CI's either): a bump without fallout changes `cmd/**`, `bundle/schema/jsonschema.json`, `bundle/internal/validation/generated/*`, `python/databricks/bundles/**`, `acceptance/bundle/refschema/output` and help-text goldens (`acceptance/help/output.txt`, `acceptance/cmd/account/account-help/output.txt`). diff --git a/Taskfile.yml b/Taskfile.yml index 17c508c0594..f1f747de84b 100644 --- a/Taskfile.yml +++ b/Taskfile.yml @@ -967,6 +967,12 @@ tasks: bump-sdk-post: desc: Regenerate everything downstream of an SDK bump and refresh acceptance goldens cmds: + # The acceptance tests run uv offline, so the managed interpreters they + # pin (3.10 for bundle tests, 3.9-3.13 in one template matrix) must exist + # up front. CI installs them in setup-build-environment; the SDK bumper's + # image ships only Python 3.12, and a missing version would otherwise be + # baked into the goldens as a uv error. + - task: install-pythons - task: generate-check - task: test-update diff --git a/acceptance/acceptance_test.go b/acceptance/acceptance_test.go index 3805300a8e7..22e9ec8f8be 100644 --- a/acceptance/acceptance_test.go +++ b/acceptance/acceptance_test.go @@ -226,9 +226,6 @@ func requirePrerequisites(t *testing.T) bool { // uv builds the databricks-bundles wheel and provides the test interpreter // via `uv python find`, which landed in the 0.3 line. internal.RequireUV(t, "0.4") - // ruff 0.9.1 is pinned across the repo (python/pyproject.toml, Taskfile.yml); - // the check-formatting test's golden output assumes its formatter behavior. - internal.RequireRuff(t, "0.9.1") // Acceptance scripts import the stdlib tomllib module, added in Python 3.11. internal.RequirePython(t, "3.11") }) @@ -295,6 +292,15 @@ func testAccept(t *testing.T, inprocessMode bool, selectedTests []string, skipTo // change survives for the rest of the run. internal.ConfigurePython(t, "3.11") + // The check-formatting script runs `uv tool run ruff@0.9.1` (its golden + // assumes that release's formatter; keep the two pins in sync). Fetch it + // into the uv cache before UV_OFFLINE below cuts the sandbox off. Not inside + // requirePrerequisites: a -run filter for a test directory skips that + // subtest. DBR has no network, and no test that runs there uses ruff. + if !WorkspaceTmpDir { + internal.PrefetchRuff(t, "0.9.1") + } + buildDir := getBuildDir(t, cwd, runtime.GOOS, runtime.GOARCH) // Download old CLI versions in parallel, overlapping with the builds below. diff --git a/acceptance/bundle/templates/pydabs/check-formatting/output.txt b/acceptance/bundle/templates/pydabs/check-formatting/output.txt index 29db0fde3c5..4c063bf16d5 100644 --- a/acceptance/bundle/templates/pydabs/check-formatting/output.txt +++ b/acceptance/bundle/templates/pydabs/check-formatting/output.txt @@ -13,9 +13,9 @@ Note that [DATABRICKS_URL] is used for initialization. To get started, refer to the project README.md file and the documentation at https://docs.databricks.com/dev-tools/bundles/python/index.html. Checking code formatting with ruff format --line-length 88 ->>> ruff format --isolated --line-length 88 --quiet --diff --check my_pydabs +>>> uv tool run ruff@0.9.1 format --isolated --line-length 88 --quiet --diff --check my_pydabs Checking code formatting with ruff format --line-length 120 ->>> ruff format --isolated --line-length 120 --quiet --diff --check my_pydabs +>>> uv tool run ruff@0.9.1 format --isolated --line-length 120 --quiet --diff --check my_pydabs >>> yamlcheck.py diff --git a/acceptance/bundle/templates/pydabs/check-formatting/script b/acceptance/bundle/templates/pydabs/check-formatting/script index 03bd395cc37..474b238cdc8 100644 --- a/acceptance/bundle/templates/pydabs/check-formatting/script +++ b/acceptance/bundle/templates/pydabs/check-formatting/script @@ -9,12 +9,15 @@ trace $CLI bundle init pydabs --config-file ./input.json # # You can use trailing comma to force line breaks to make formatting consistent # with different line-length settings. +# +# ruff comes from the uv cache: the harness prefetches this exact release +# (PrefetchRuff in acceptance_test.go) because the sandbox is offline. echo "Checking code formatting with ruff format --line-length 88" -trace ruff format --isolated --line-length 88 --quiet --diff --check my_pydabs +trace uv tool run ruff@0.9.1 format --isolated --line-length 88 --quiet --diff --check my_pydabs echo "Checking code formatting with ruff format --line-length 120" -trace ruff format --isolated --line-length 120 --quiet --diff --check my_pydabs +trace uv tool run ruff@0.9.1 format --isolated --line-length 120 --quiet --diff --check my_pydabs trace yamlcheck.py diff --git a/acceptance/internal/ruff.go b/acceptance/internal/ruff.go index 907eada2800..af780663c7c 100644 --- a/acceptance/internal/ruff.go +++ b/acceptance/internal/ruff.go @@ -2,30 +2,17 @@ package internal import ( "os/exec" - "strings" "testing" - - "golang.org/x/mod/semver" ) -// RequireRuff fails the run if ruff is missing from PATH or older than -// minVersion (e.g. "0.9.1"). See the call site for why the minimum is what it is. -func RequireRuff(t *testing.T, minVersion string) { - out, err := exec.Command("ruff", "--version").Output() +// PrefetchRuff runs `uv tool run ruff@ --version` so that release +// lands in the uv cache before the harness sets UV_OFFLINE. Test scripts run +// ruff through the same command, offline, from that cache: not every +// environment the suite runs in has a ruff binary on PATH (the SDK bumper's +// image does not), while uv is a prerequisite anyway. +func PrefetchRuff(t *testing.T, version string) { + out, err := exec.Command("uv", "tool", "run", "ruff@"+version, "--version").CombinedOutput() if err != nil { - t.Fatalf("ruff not found on PATH (acceptance tests require ruff >= %s): %v", minVersion, err) - } - version := strings.TrimSpace(string(out)) - if !ruffVersionOK(version, minVersion) { - t.Fatalf("acceptance tests require ruff >= %s (found %q); install a newer ruff", minVersion, version) - } -} - -// ruffVersionOK reports whether `ruff --version` output (e.g. "ruff 0.9.1") is >= minVersion. -func ruffVersionOK(versionOutput, minVersion string) bool { - fields := strings.Fields(versionOutput) - if len(fields) < 2 { - return false + t.Fatalf("uv could not fetch ruff %s for the acceptance tests: %v\n%s", version, err, out) } - return semver.Compare("v"+fields[1], "v"+minVersion) >= 0 } diff --git a/acceptance/internal/ruff_test.go b/acceptance/internal/ruff_test.go deleted file mode 100644 index ddef9021950..00000000000 --- a/acceptance/internal/ruff_test.go +++ /dev/null @@ -1,15 +0,0 @@ -package internal - -import ( - "testing" - - "github.com/stretchr/testify/assert" -) - -func TestRuffVersionOK(t *testing.T) { - assert.True(t, ruffVersionOK("ruff 0.9.1", "0.9.1")) - assert.True(t, ruffVersionOK("ruff 0.9.2", "0.9.1")) - assert.True(t, ruffVersionOK("ruff 0.11.0", "0.9.1")) - assert.False(t, ruffVersionOK("ruff 0.9.0", "0.9.1")) - assert.False(t, ruffVersionOK("ruff 0.8.5", "0.9.1")) -} From bc633bd4e81445d6018a83d8c35fc01d579a1dce Mon Sep 17 00:00:00 2001 From: Jan Rose Date: Fri, 6 Nov 2026 13:25:26 +0000 Subject: [PATCH 2/2] Share uv's effective cache dir between the ruff prefetch and the tests CI's setup-uv exports UV_CACHE_DIR, so the ruff prefetch filled that cache while the harness pinned test scripts to the HOME-derived default: every check-formatting variant failed with "ruff was not found in the cache". Ask uv for its cache directory (as getUVPythonInstallDir already does for the Python install dir) instead of deriving the default from HOME. Pass -q to uv in the check-formatting script: the first variant to build the tool environment for the test interpreter otherwise prints "Installed 1 package" to stderr, which the other variants do not. Co-authored-by: Isaac --- acceptance/acceptance_test.go | 21 ++++++++----------- .../pydabs/check-formatting/output.txt | 4 ++-- .../templates/pydabs/check-formatting/script | 8 ++++--- 3 files changed, 16 insertions(+), 17 deletions(-) diff --git a/acceptance/acceptance_test.go b/acceptance/acceptance_test.go index 22e9ec8f8be..db58b46223f 100644 --- a/acceptance/acceptance_test.go +++ b/acceptance/acceptance_test.go @@ -433,7 +433,7 @@ func testAccept(t *testing.T, inprocessMode bool, selectedTests []string, skipTo t.Setenv("PATH", strings.Join(paths, string(os.PathListSeparator))) // Make use of uv cache; since we set HomeEnvVar to temporary directory, it is not picked up automatically - uvCache := getUVDefaultCacheDir(t) + uvCache := getUVCacheDir(t) t.Setenv("UV_CACHE_DIR", uvCache) // UV_CACHE_DIR only applies to packages but not Python installations. @@ -1811,17 +1811,14 @@ func ListDir(t *testing.T, src string) []string { return files } -func getUVDefaultCacheDir(t *testing.T) string { - // According to uv docs https://docs.astral.sh/uv/concepts/cache/#caching-in-continuous-integration - // the default cache directory is - // "A system-appropriate cache directory, e.g., $XDG_CACHE_HOME/uv or $HOME/.cache/uv on Unix and %LOCALAPPDATA%\uv\cache on Windows" - cacheDir, err := os.UserCacheDir() - require.NoError(t, err) - if runtime.GOOS == "windows" { - return cacheDir + "\\uv\\cache" - } else { - return cacheDir + "/uv" - } +// getUVCacheDir returns uv's effective cache directory. Must be called before +// HOME is overridden in tests. Asking uv, rather than deriving the default from +// HOME, honors UV_CACHE_DIR (CI's setup-uv points it at a per-job directory), +// so test scripts read the cache that PrefetchRuff filled. +func getUVCacheDir(t *testing.T) string { + out, err := exec.Command("uv", "cache", "dir").Output() + require.NoError(t, err, "uv cache dir") + return strings.TrimSpace(string(out)) } // getUVPythonInstallDir returns the directory where uv stores managed Python installations. diff --git a/acceptance/bundle/templates/pydabs/check-formatting/output.txt b/acceptance/bundle/templates/pydabs/check-formatting/output.txt index 4c063bf16d5..fec60202db9 100644 --- a/acceptance/bundle/templates/pydabs/check-formatting/output.txt +++ b/acceptance/bundle/templates/pydabs/check-formatting/output.txt @@ -13,9 +13,9 @@ Note that [DATABRICKS_URL] is used for initialization. To get started, refer to the project README.md file and the documentation at https://docs.databricks.com/dev-tools/bundles/python/index.html. Checking code formatting with ruff format --line-length 88 ->>> uv tool run ruff@0.9.1 format --isolated --line-length 88 --quiet --diff --check my_pydabs +>>> uv tool run -q ruff@0.9.1 format --isolated --line-length 88 --quiet --diff --check my_pydabs Checking code formatting with ruff format --line-length 120 ->>> uv tool run ruff@0.9.1 format --isolated --line-length 120 --quiet --diff --check my_pydabs +>>> uv tool run -q ruff@0.9.1 format --isolated --line-length 120 --quiet --diff --check my_pydabs >>> yamlcheck.py diff --git a/acceptance/bundle/templates/pydabs/check-formatting/script b/acceptance/bundle/templates/pydabs/check-formatting/script index 474b238cdc8..63a1cd38ca7 100644 --- a/acceptance/bundle/templates/pydabs/check-formatting/script +++ b/acceptance/bundle/templates/pydabs/check-formatting/script @@ -11,13 +11,15 @@ trace $CLI bundle init pydabs --config-file ./input.json # with different line-length settings. # # ruff comes from the uv cache: the harness prefetches this exact release -# (PrefetchRuff in acceptance_test.go) because the sandbox is offline. +# (PrefetchRuff in acceptance_test.go) because the sandbox is offline. uv's -q +# hides its "Installed 1 package" line, printed only by the first variant that +# builds the tool environment for the test interpreter. echo "Checking code formatting with ruff format --line-length 88" -trace uv tool run ruff@0.9.1 format --isolated --line-length 88 --quiet --diff --check my_pydabs +trace uv tool run -q ruff@0.9.1 format --isolated --line-length 88 --quiet --diff --check my_pydabs echo "Checking code formatting with ruff format --line-length 120" -trace uv tool run ruff@0.9.1 format --isolated --line-length 120 --quiet --diff --check my_pydabs +trace uv tool run -q ruff@0.9.1 format --isolated --line-length 120 --quiet --diff --check my_pydabs trace yamlcheck.py