From d1aad7e7d313ac4e8a61710d9c00da520220c506 Mon Sep 17 00:00:00 2001 From: Grigory Panov Date: Tue, 1 Sep 2026 17:01:28 +0200 Subject: [PATCH 1/4] Add orthogonal setup-local flags and a "skipped" phase status MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit *Why* `databricks environments setup-local` had a single `--constraints-only` mode that bundled "skip databricks-connect" together with the rest of the setup. A caller (the VS Code extension, agents) that wants to write project files but defer provisioning, or manage its own dependency pins, had no way to express that. The setup steps are independent, so the flags that control them should be too. *What* Adds three orthogonal, composable negative flags: - `--no-constraints` skips writing the remote Python-version and dependency pins (requires-python and the [tool.uv] constraint block); any existing values are left untouched, and provisioning still installs the resolved Python (the flag governs only what is written). - `--no-dbconnect` skips the databricks-connect dependency. Equivalent to the existing `--constraints-only`, which stays as-is for now. - `--no-provision` writes the project files through the merge phase, then stops: no Python download, uv sync, or validation. Because it never invokes uv, its preflight no longer requires or installs uv either — a files-only run works on a machine without uv. Introduces a new `skipped` phase status (distinct from `pending`, which means an earlier phase failed): the provision and validate phases report `skipped` under `--no-provision`, `venvPath` is omitted, and the dry-run plan drops `wouldInstallPython`. The text summary gains a dedicated "provisioning skipped" variant so it no longer prints an empty venv path or a broken activation hint. The `--no-constraints` "unmanaged" signal is a nil ConstraintDeps / empty requires-python; parseConstraints now normalizes a missing [tool.uv].constraint-dependencies to a non-nil empty slice so that nil uniquely means the flag, not merely an artifact that omits the section. Default runs (no new flags) are byte-for-byte unchanged; the JSON schemaVersion stays at 1 since the new status only appears when a new flag is passed. *Verification* - Unit tests (libs/localenv): no-provision writes files then skips provision/validate without invoking uv, dry-run plus no-provision marks them skipped and drops wouldInstallPython, no-constraints leaves existing pins untouched and omits them greenfield, parseConstraints normalizes missing constraint-dependencies, and the merge/render skip guards. - Acceptance goldens: no-provision (real run), no-constraints, no-dbconnect, no-provision-dry-run, no-provision-text, plus the refreshed help output. - gofmt, go vet, and full go build ./... clean. Co-authored-by: Isaac --- .../cli/setup-local-orthogonal-flags.md | 1 + acceptance/localenv/help/output.txt | 3 + .../localenv/no-constraints/out.test.toml | 2 + acceptance/localenv/no-constraints/output.txt | 60 ++++++++ acceptance/localenv/no-constraints/script | 15 ++ acceptance/localenv/no-constraints/test.toml | 23 +++ .../localenv/no-dbconnect/out.test.toml | 2 + acceptance/localenv/no-dbconnect/output.txt | 59 ++++++++ acceptance/localenv/no-dbconnect/script | 15 ++ acceptance/localenv/no-dbconnect/test.toml | 23 +++ .../no-provision-dry-run/out.test.toml | 2 + .../localenv/no-provision-dry-run/output.txt | 53 +++++++ .../localenv/no-provision-dry-run/script | 4 + .../localenv/no-provision-dry-run/test.toml | 20 +++ .../localenv/no-provision-text/out.test.toml | 2 + .../localenv/no-provision-text/output.txt | 10 ++ acceptance/localenv/no-provision-text/script | 3 + .../localenv/no-provision-text/test.toml | 24 ++++ .../localenv/no-provision/out.test.toml | 2 + acceptance/localenv/no-provision/output.txt | 49 +++++++ acceptance/localenv/no-provision/script | 5 + acceptance/localenv/no-provision/test.toml | 24 ++++ cmd/environments/output.go | 79 ++++++++--- cmd/environments/sync.go | 15 +- libs/localenv/constraints.go | 8 ++ libs/localenv/constraints_test.go | 17 +++ libs/localenv/merge.go | 34 ++++- libs/localenv/merge_test.go | 50 +++++++ libs/localenv/pipeline.go | 99 +++++++++++-- libs/localenv/pipeline_test.go | 132 ++++++++++++++++++ libs/localenv/result.go | 11 +- 31 files changed, 813 insertions(+), 33 deletions(-) create mode 100644 .nextchanges/cli/setup-local-orthogonal-flags.md create mode 100644 acceptance/localenv/no-constraints/out.test.toml create mode 100644 acceptance/localenv/no-constraints/output.txt create mode 100644 acceptance/localenv/no-constraints/script create mode 100644 acceptance/localenv/no-constraints/test.toml create mode 100644 acceptance/localenv/no-dbconnect/out.test.toml create mode 100644 acceptance/localenv/no-dbconnect/output.txt create mode 100644 acceptance/localenv/no-dbconnect/script create mode 100644 acceptance/localenv/no-dbconnect/test.toml create mode 100644 acceptance/localenv/no-provision-dry-run/out.test.toml create mode 100644 acceptance/localenv/no-provision-dry-run/output.txt create mode 100644 acceptance/localenv/no-provision-dry-run/script create mode 100644 acceptance/localenv/no-provision-dry-run/test.toml create mode 100644 acceptance/localenv/no-provision-text/out.test.toml create mode 100644 acceptance/localenv/no-provision-text/output.txt create mode 100644 acceptance/localenv/no-provision-text/script create mode 100644 acceptance/localenv/no-provision-text/test.toml create mode 100644 acceptance/localenv/no-provision/out.test.toml create mode 100644 acceptance/localenv/no-provision/output.txt create mode 100644 acceptance/localenv/no-provision/script create mode 100644 acceptance/localenv/no-provision/test.toml diff --git a/.nextchanges/cli/setup-local-orthogonal-flags.md b/.nextchanges/cli/setup-local-orthogonal-flags.md new file mode 100644 index 00000000000..f71f68acb61 --- /dev/null +++ b/.nextchanges/cli/setup-local-orthogonal-flags.md @@ -0,0 +1 @@ +Added orthogonal `--no-constraints`, `--no-dbconnect`, and `--no-provision` flags to `databricks environments setup-local`. The flags compose: `--no-constraints` skips writing the remote Python-version and dependency pins, `--no-dbconnect` skips the databricks-connect dependency, and `--no-provision` writes the project files but stops before creating the virtual environment. Phases that a flag opts out of report a new `skipped` status in `--output json`. diff --git a/acceptance/localenv/help/output.txt b/acceptance/localenv/help/output.txt index 4ff2f8ac15b..7d455e00d42 100644 --- a/acceptance/localenv/help/output.txt +++ b/acceptance/localenv/help/output.txt @@ -22,6 +22,9 @@ Flags: --dry-run compute the plan without writing files or provisioning -h, --help help for setup-local --job-task string job task to use as the compute target, as . (the task key is required) + --no-constraints skip writing the remote Python version and dependency constraints + --no-dbconnect skip adding the databricks-connect dependency + --no-provision write the project files but skip creating the virtual environment (no uv sync, Python download, or validation) --serverless-version string serverless version to use as the compute target (e.g. 5) Global Flags: diff --git a/acceptance/localenv/no-constraints/out.test.toml b/acceptance/localenv/no-constraints/out.test.toml new file mode 100644 index 00000000000..0938e678987 --- /dev/null +++ b/acceptance/localenv/no-constraints/out.test.toml @@ -0,0 +1,2 @@ +Cloud = false +EnvMatrix.DATABRICKS_BUNDLE_ENGINE = ["direct"] diff --git a/acceptance/localenv/no-constraints/output.txt b/acceptance/localenv/no-constraints/output.txt new file mode 100644 index 00000000000..6010c343f5a --- /dev/null +++ b/acceptance/localenv/no-constraints/output.txt @@ -0,0 +1,60 @@ + +>>> [CLI] environments setup-local --serverless-version 4 --no-constraints --dry-run --output json +{ + "schemaVersion": 1, + "command": "environments setup-local", + "ok": true, + "mode": "default", + "dryRun": true, + "compute": { + "source": "serverless", + "serverlessVersion": "v4", + "envKey": "serverless/serverless-v4" + }, + "resolved": { + "pythonVersion": "3.12", + "dbconnectVersion": "17.2.0", + "artifactSource": "network" + }, + "greenfield": false, + "plan": { + "wouldWrite": "[TEST_TMP_DIR]/pyproject.toml", + "wouldBackup": "[TEST_TMP_DIR]/pyproject.toml.bak", + "wouldInstallPython": "3.12", + "diff": "--- pyproject.toml\n+++ pyproject.toml.new\n@@ -3,4 +3,7 @@\n requires-python = \"\u003e=3.10\"\n \n [dependency-groups]\n-dev = [\"databricks-connect~=16.0\"]\n+dev = [\"databricks-connect~=17.2.0\"]\n+\n+[tool.databricks.environment]\n+environment_version = \"4\"\n" + }, + "phases": [ + { + "phase": "preflight", + "status": "ok" + }, + { + "phase": "resolve", + "status": "ok" + }, + { + "phase": "fetch", + "status": "ok" + }, + { + "phase": "merge", + "status": "ok" + }, + { + "phase": "provision", + "status": "ok" + }, + { + "phase": "validate", + "status": "ok" + } + ], + "warnings": [ + { + "code": "W_DBCONNECT_PIN_OVERRIDDEN", + "message": "databricks-connect \"databricks-connect~=16.0\" is replaced by the environment's \"databricks-connect~=17.2.0\"" + } + ], + "error": null, + "durationMs": [DURATION_MS] +} diff --git a/acceptance/localenv/no-constraints/script b/acceptance/localenv/no-constraints/script new file mode 100644 index 00000000000..f4711b38dd5 --- /dev/null +++ b/acceptance/localenv/no-constraints/script @@ -0,0 +1,15 @@ +# --no-constraints leaves the remote Python version and dependency pins +# unmanaged: the plan writes neither requires-python nor the [tool.uv] constraint +# block, and the user's existing requires-python is left untouched. The +# databricks-connect dependency (orthogonal to --no-constraints) is still managed. +# The JSON plan shows the diff. +cat > pyproject.toml <<'PY' +[project] +name = "demo" +requires-python = ">=3.10" + +[dependency-groups] +dev = ["databricks-connect~=16.0"] +PY + +trace $CLI environments setup-local --serverless-version 4 --no-constraints --dry-run --output json diff --git a/acceptance/localenv/no-constraints/test.toml b/acceptance/localenv/no-constraints/test.toml new file mode 100644 index 00000000000..6377a54fca8 --- /dev/null +++ b/acceptance/localenv/no-constraints/test.toml @@ -0,0 +1,23 @@ +EnvMatrix.DATABRICKS_BUNDLE_ENGINE = ["direct"] + +# The script writes pyproject.toml as the merge input; --dry-run leaves it unchanged. +Ignore = ["pyproject.toml"] + +Env.DATABRICKS_LOCALENV_CONSTRAINT_SOURCE_URL_TEST_OVERRIDE = "$DATABRICKS_HOST" + +[[Server]] +Pattern = "GET /serverless/serverless-v4/pyproject.toml" +Response.Body = ''' +[project] +requires-python = ">=3.12" + +[dependency-groups] +dev = ["databricks-connect~=17.2.0"] + +[tool.uv] +constraint-dependencies = ["pyarrow<19"] +''' + +[[Repls]] +Old = 'uv uv \S+(?: \([^)]+\))?' +New = 'uv [UV_VERSION]' diff --git a/acceptance/localenv/no-dbconnect/out.test.toml b/acceptance/localenv/no-dbconnect/out.test.toml new file mode 100644 index 00000000000..0938e678987 --- /dev/null +++ b/acceptance/localenv/no-dbconnect/out.test.toml @@ -0,0 +1,2 @@ +Cloud = false +EnvMatrix.DATABRICKS_BUNDLE_ENGINE = ["direct"] diff --git a/acceptance/localenv/no-dbconnect/output.txt b/acceptance/localenv/no-dbconnect/output.txt new file mode 100644 index 00000000000..677a6fbf26b --- /dev/null +++ b/acceptance/localenv/no-dbconnect/output.txt @@ -0,0 +1,59 @@ + +>>> [CLI] environments setup-local --serverless-version 4 --no-dbconnect --dry-run --output json +{ + "schemaVersion": 1, + "command": "environments setup-local", + "ok": true, + "mode": "constraints-only", + "dryRun": true, + "compute": { + "source": "serverless", + "serverlessVersion": "v4", + "envKey": "serverless/serverless-v4" + }, + "resolved": { + "pythonVersion": "3.12", + "artifactSource": "network" + }, + "greenfield": false, + "plan": { + "wouldWrite": "[TEST_TMP_DIR]/pyproject.toml", + "wouldBackup": "[TEST_TMP_DIR]/pyproject.toml.bak", + "wouldInstallPython": "3.12", + "diff": "--- pyproject.toml\n+++ pyproject.toml.new\n@@ -1,7 +1,17 @@\n [project]\n name = \"demo\"\n-requires-python = \"\u003e=3.10\"\n+requires-python = \"\u003e=3.12\"\n dependencies = [\"databricks-connect==15.1.*\"]\n \n [dependency-groups]\n dev = [\"databricks-connect~=16.0\"]\n+\n+[tool.databricks.environment]\n+environment_version = \"4\"\n+\n+# managed by databricks environments setup-local — do not edit\n+[tool.uv]\n+constraint-dependencies = [\n+ \"pyarrow\u003c19\",\n+]\n+# end managed by databricks environments setup-local\n" + }, + "phases": [ + { + "phase": "preflight", + "status": "ok" + }, + { + "phase": "resolve", + "status": "ok" + }, + { + "phase": "fetch", + "status": "ok" + }, + { + "phase": "merge", + "status": "ok" + }, + { + "phase": "provision", + "status": "ok" + }, + { + "phase": "validate", + "status": "ok" + } + ], + "warnings": [ + { + "code": "W_REQUIRES_PYTHON_OVERRIDDEN", + "message": "requires-python \"\u003e=3.10\" is replaced by the environment's \"\u003e=3.12\"" + } + ], + "error": null, + "durationMs": [DURATION_MS] +} diff --git a/acceptance/localenv/no-dbconnect/script b/acceptance/localenv/no-dbconnect/script new file mode 100644 index 00000000000..63b38e08faa --- /dev/null +++ b/acceptance/localenv/no-dbconnect/script @@ -0,0 +1,15 @@ +# --no-dbconnect is the orthogonal spelling of --constraints-only: it omits the +# databricks-connect dependency (mode "constraints-only") while still managing +# requires-python and the [tool.uv] constraints. Existing databricks-connect +# requirements the user already had are left untouched. +cat > pyproject.toml <<'PY' +[project] +name = "demo" +requires-python = ">=3.10" +dependencies = ["databricks-connect==15.1.*"] + +[dependency-groups] +dev = ["databricks-connect~=16.0"] +PY + +trace $CLI environments setup-local --serverless-version 4 --no-dbconnect --dry-run --output json diff --git a/acceptance/localenv/no-dbconnect/test.toml b/acceptance/localenv/no-dbconnect/test.toml new file mode 100644 index 00000000000..6377a54fca8 --- /dev/null +++ b/acceptance/localenv/no-dbconnect/test.toml @@ -0,0 +1,23 @@ +EnvMatrix.DATABRICKS_BUNDLE_ENGINE = ["direct"] + +# The script writes pyproject.toml as the merge input; --dry-run leaves it unchanged. +Ignore = ["pyproject.toml"] + +Env.DATABRICKS_LOCALENV_CONSTRAINT_SOURCE_URL_TEST_OVERRIDE = "$DATABRICKS_HOST" + +[[Server]] +Pattern = "GET /serverless/serverless-v4/pyproject.toml" +Response.Body = ''' +[project] +requires-python = ">=3.12" + +[dependency-groups] +dev = ["databricks-connect~=17.2.0"] + +[tool.uv] +constraint-dependencies = ["pyarrow<19"] +''' + +[[Repls]] +Old = 'uv uv \S+(?: \([^)]+\))?' +New = 'uv [UV_VERSION]' diff --git a/acceptance/localenv/no-provision-dry-run/out.test.toml b/acceptance/localenv/no-provision-dry-run/out.test.toml new file mode 100644 index 00000000000..0938e678987 --- /dev/null +++ b/acceptance/localenv/no-provision-dry-run/out.test.toml @@ -0,0 +1,2 @@ +Cloud = false +EnvMatrix.DATABRICKS_BUNDLE_ENGINE = ["direct"] diff --git a/acceptance/localenv/no-provision-dry-run/output.txt b/acceptance/localenv/no-provision-dry-run/output.txt new file mode 100644 index 00000000000..a0dc1aafdc7 --- /dev/null +++ b/acceptance/localenv/no-provision-dry-run/output.txt @@ -0,0 +1,53 @@ + +>>> [CLI] environments setup-local --serverless-version 4 --no-provision --dry-run --output json +{ + "schemaVersion": 1, + "command": "environments setup-local", + "ok": true, + "mode": "default", + "dryRun": true, + "compute": { + "source": "serverless", + "serverlessVersion": "v4", + "envKey": "serverless/serverless-v4" + }, + "resolved": { + "pythonVersion": "3.12", + "dbconnectVersion": "17.2.0", + "artifactSource": "network" + }, + "greenfield": true, + "plan": { + "wouldWrite": "[TEST_TMP_DIR]/pyproject.toml", + "diff": "--- pyproject.toml\n+++ pyproject.toml\n@@ -1 +1,20 @@\n+[project]\n+name = \"001\"\n+version = \"0.0.0\"\n+requires-python = \"\u003e=3.12\"\n+\n+[dependency-groups]\n+dev = [\n+ \"databricks-connect~=17.2.0\",\n+]\n+\n+[tool.databricks.environment]\n+environment_version = \"4\"\n+\n+# managed by databricks environments setup-local — do not edit\n+[tool.uv]\n+constraint-dependencies = [\n+ \"pyarrow\u003c19\",\n+ \"pandas\u003c3\",\n+]\n+# end managed by databricks environments setup-local\n" + }, + "phases": [ + { + "phase": "preflight", + "status": "ok" + }, + { + "phase": "resolve", + "status": "ok" + }, + { + "phase": "fetch", + "status": "ok" + }, + { + "phase": "merge", + "status": "ok" + }, + { + "phase": "provision", + "status": "skipped" + }, + { + "phase": "validate", + "status": "skipped" + } + ], + "warnings": [], + "error": null, + "durationMs": [DURATION_MS] +} diff --git a/acceptance/localenv/no-provision-dry-run/script b/acceptance/localenv/no-provision-dry-run/script new file mode 100644 index 00000000000..de74336fbf3 --- /dev/null +++ b/acceptance/localenv/no-provision-dry-run/script @@ -0,0 +1,4 @@ +# --dry-run --no-provision: nothing is provisioned (as with any dry run), but the +# plan reflects the real run's intent, so provision and validate report "skipped" +# rather than the "ok" a plain dry run reports. +trace $CLI environments setup-local --serverless-version 4 --no-provision --dry-run --output json diff --git a/acceptance/localenv/no-provision-dry-run/test.toml b/acceptance/localenv/no-provision-dry-run/test.toml new file mode 100644 index 00000000000..467508f4a03 --- /dev/null +++ b/acceptance/localenv/no-provision-dry-run/test.toml @@ -0,0 +1,20 @@ +EnvMatrix.DATABRICKS_BUNDLE_ENGINE = ["direct"] + +Env.DATABRICKS_LOCALENV_CONSTRAINT_SOURCE_URL_TEST_OVERRIDE = "$DATABRICKS_HOST" + +[[Server]] +Pattern = "GET /serverless/serverless-v4/pyproject.toml" +Response.Body = ''' +[project] +requires-python = ">=3.12" + +[dependency-groups] +dev = ["databricks-connect~=17.2.0"] + +[tool.uv] +constraint-dependencies = ["pyarrow<19", "pandas<3"] +''' + +[[Repls]] +Old = 'uv uv \S+(?: \([^)]+\))?' +New = 'uv [UV_VERSION]' diff --git a/acceptance/localenv/no-provision-text/out.test.toml b/acceptance/localenv/no-provision-text/out.test.toml new file mode 100644 index 00000000000..0938e678987 --- /dev/null +++ b/acceptance/localenv/no-provision-text/out.test.toml @@ -0,0 +1,2 @@ +Cloud = false +EnvMatrix.DATABRICKS_BUNDLE_ENGINE = ["direct"] diff --git a/acceptance/localenv/no-provision-text/output.txt b/acceptance/localenv/no-provision-text/output.txt new file mode 100644 index 00000000000..f42fbf13dc6 --- /dev/null +++ b/acceptance/localenv/no-provision-text/output.txt @@ -0,0 +1,10 @@ + +>>> [CLI] environments setup-local --serverless-version 4 --no-provision +✔ Project files written (provisioning skipped) + + Compute target serverless 4 + Python 3.12 + databricks-connect 17.2.0 + pyproject.toml created + +No virtual environment was created (--no-provision). Re-run without --no-provision to create it. diff --git a/acceptance/localenv/no-provision-text/script b/acceptance/localenv/no-provision-text/script new file mode 100644 index 00000000000..0baf8a6d58f --- /dev/null +++ b/acceptance/localenv/no-provision-text/script @@ -0,0 +1,3 @@ +# Text-mode --no-provision: the files are written but no virtual environment is +# created, so the summary must not claim a venv or print activation hints for one. +trace $CLI environments setup-local --serverless-version 4 --no-provision diff --git a/acceptance/localenv/no-provision-text/test.toml b/acceptance/localenv/no-provision-text/test.toml new file mode 100644 index 00000000000..12ca584b223 --- /dev/null +++ b/acceptance/localenv/no-provision-text/test.toml @@ -0,0 +1,24 @@ +EnvMatrix.DATABRICKS_BUNDLE_ENGINE = ["direct"] + +# A real --no-provision run creates pyproject.toml through the merge phase; the +# assertion is the JSON output, so ignore the written file. +Ignore = ["pyproject.toml"] + +Env.DATABRICKS_LOCALENV_CONSTRAINT_SOURCE_URL_TEST_OVERRIDE = "$DATABRICKS_HOST" + +[[Server]] +Pattern = "GET /serverless/serverless-v4/pyproject.toml" +Response.Body = ''' +[project] +requires-python = ">=3.12" + +[dependency-groups] +dev = ["databricks-connect~=17.2.0"] + +[tool.uv] +constraint-dependencies = ["pyarrow<19", "pandas<3"] +''' + +[[Repls]] +Old = 'uv uv \S+(?: \([^)]+\))?' +New = 'uv [UV_VERSION]' diff --git a/acceptance/localenv/no-provision/out.test.toml b/acceptance/localenv/no-provision/out.test.toml new file mode 100644 index 00000000000..0938e678987 --- /dev/null +++ b/acceptance/localenv/no-provision/out.test.toml @@ -0,0 +1,2 @@ +Cloud = false +EnvMatrix.DATABRICKS_BUNDLE_ENGINE = ["direct"] diff --git a/acceptance/localenv/no-provision/output.txt b/acceptance/localenv/no-provision/output.txt new file mode 100644 index 00000000000..9a120ccbd64 --- /dev/null +++ b/acceptance/localenv/no-provision/output.txt @@ -0,0 +1,49 @@ + +>>> [CLI] environments setup-local --serverless-version 4 --no-provision --output json +{ + "schemaVersion": 1, + "command": "environments setup-local", + "ok": true, + "mode": "default", + "dryRun": false, + "compute": { + "source": "serverless", + "serverlessVersion": "v4", + "envKey": "serverless/serverless-v4" + }, + "resolved": { + "pythonVersion": "3.12", + "dbconnectVersion": "17.2.0", + "artifactSource": "network" + }, + "greenfield": true, + "phases": [ + { + "phase": "preflight", + "status": "ok" + }, + { + "phase": "resolve", + "status": "ok" + }, + { + "phase": "fetch", + "status": "ok" + }, + { + "phase": "merge", + "status": "ok" + }, + { + "phase": "provision", + "status": "skipped" + }, + { + "phase": "validate", + "status": "skipped" + } + ], + "warnings": [], + "error": null, + "durationMs": [DURATION_MS] +} diff --git a/acceptance/localenv/no-provision/script b/acceptance/localenv/no-provision/script new file mode 100644 index 00000000000..d538ad5cc0e --- /dev/null +++ b/acceptance/localenv/no-provision/script @@ -0,0 +1,5 @@ +# --no-provision writes the project files (a real run, not a dry run) but stops +# before creating the venv: no Python download, uv sync, or validation. The +# provision and validate phases report "skipped" (distinct from "pending", which +# would mean an earlier phase failed) and venvPath is omitted. +trace $CLI environments setup-local --serverless-version 4 --no-provision --output json diff --git a/acceptance/localenv/no-provision/test.toml b/acceptance/localenv/no-provision/test.toml new file mode 100644 index 00000000000..12ca584b223 --- /dev/null +++ b/acceptance/localenv/no-provision/test.toml @@ -0,0 +1,24 @@ +EnvMatrix.DATABRICKS_BUNDLE_ENGINE = ["direct"] + +# A real --no-provision run creates pyproject.toml through the merge phase; the +# assertion is the JSON output, so ignore the written file. +Ignore = ["pyproject.toml"] + +Env.DATABRICKS_LOCALENV_CONSTRAINT_SOURCE_URL_TEST_OVERRIDE = "$DATABRICKS_HOST" + +[[Server]] +Pattern = "GET /serverless/serverless-v4/pyproject.toml" +Response.Body = ''' +[project] +requires-python = ">=3.12" + +[dependency-groups] +dev = ["databricks-connect~=17.2.0"] + +[tool.uv] +constraint-dependencies = ["pyarrow<19", "pandas<3"] +''' + +[[Repls]] +Old = 'uv uv \S+(?: \([^)]+\))?' +New = 'uv [UV_VERSION]' diff --git a/cmd/environments/output.go b/cmd/environments/output.go index aa2d124f371..4b1935a11d9 100644 --- a/cmd/environments/output.go +++ b/cmd/environments/output.go @@ -80,19 +80,65 @@ func renderResult(ctx context.Context, cmd *cobra.Command, res *libslocalenv.Res return nil } + if provisionSkipped(res) { + renderProvisionSkippedSuccess(ctx, res) + return nil + } + renderSuccess(ctx, res) return nil } -// renderSuccess prints the friendly post-provision summary (DECO-27977). +// provisionSkipped reports whether the run wrote the project files but skipped +// creating the virtual environment (--no-provision), so the text summary must +// not claim a venv or print activation hints for one. +func provisionSkipped(res *libslocalenv.Result) bool { + for _, ph := range res.Phases { + if ph.Phase == libslocalenv.PhaseProvision { + return ph.Status == libslocalenv.StatusSkipped + } + } + return false +} + +// renderSuccess prints the friendly post-provision summary. // -// It runs only on a non-dry-run success (renderResult returns earlier for JSON, -// failures, and dry runs), so res.VenvPath is always set: the validate phase — the -// last thing a successful run does — assigns it unconditionally (see Pipeline.validate). +// It runs only on a non-dry-run success that actually provisioned (renderResult +// returns earlier for JSON, failures, dry runs, and --no-provision), so +// res.VenvPath is always set: the validate phase — the last thing such a run does +// — assigns it unconditionally (see Pipeline.validate). func renderSuccess(ctx context.Context, res *libslocalenv.Result) { cmdio.LogString(ctx, "✔ Local environment ready") cmdio.LogString(ctx, "") + renderComputeAndResolved(ctx, res) + cmdio.LogString(ctx, fmt.Sprintf(" %-20s%s", "Virtual env", res.VenvPath)) + cmdio.LogString(ctx, fmt.Sprintf(" %-20s%s", "pyproject.toml", pyprojectDetail(res))) + + cmdio.LogString(ctx, "") + cmdio.LogString(ctx, "Next steps:") + cmdio.LogString(ctx, " • Activate it: "+activateHint(res.VenvPath)) + cmdio.LogString(ctx, " • Or select "+res.VenvPath+" as the Python interpreter in VS Code / Cursor") +} + +// renderProvisionSkippedSuccess prints the summary for a --no-provision run: the +// project files were written but no virtual environment was created. It omits the +// venv line and activation hints renderSuccess prints (res.VenvPath is empty +// here), and tells the user how to finish the setup. +func renderProvisionSkippedSuccess(ctx context.Context, res *libslocalenv.Result) { + cmdio.LogString(ctx, "✔ Project files written (provisioning skipped)") + cmdio.LogString(ctx, "") + + renderComputeAndResolved(ctx, res) + cmdio.LogString(ctx, fmt.Sprintf(" %-20s%s", "pyproject.toml", pyprojectDetail(res))) + + cmdio.LogString(ctx, "") + cmdio.LogString(ctx, "No virtual environment was created (--no-provision). Re-run without --no-provision to create it.") +} + +// renderComputeAndResolved prints the shared compute-target and resolved-version +// rows used by both the provisioned and --no-provision success summaries. +func renderComputeAndResolved(ctx context.Context, res *libslocalenv.Result) { if res.Compute != nil { cmdio.LogString(ctx, fmt.Sprintf(" %-20s%s", "Compute target", res.Compute.Label())) } @@ -102,20 +148,19 @@ func renderSuccess(ctx context.Context, res *libslocalenv.Result) { cmdio.LogString(ctx, fmt.Sprintf(" %-20s%s", "databricks-connect", res.Resolved.DBConnectVersion)) } } - cmdio.LogString(ctx, fmt.Sprintf(" %-20s%s", "Virtual env", res.VenvPath)) - // pyproject.toml was created (greenfield) or updated in place (with a backup). - pyprojectDetail := "updated" - if res.Greenfield { - pyprojectDetail = "created" - } else if res.BackupPath != "" { - pyprojectDetail = "updated (backup: " + res.BackupPath + ")" - } - cmdio.LogString(ctx, fmt.Sprintf(" %-20s%s", "pyproject.toml", pyprojectDetail)) +} - cmdio.LogString(ctx, "") - cmdio.LogString(ctx, "Next steps:") - cmdio.LogString(ctx, " • Activate it: "+activateHint(res.VenvPath)) - cmdio.LogString(ctx, " • Or select "+res.VenvPath+" as the Python interpreter in VS Code / Cursor") +// pyprojectDetail describes what happened to pyproject.toml: created (greenfield) +// or updated in place (noting the backup when one was written). +func pyprojectDetail(res *libslocalenv.Result) string { + switch { + case res.Greenfield: + return "created" + case res.BackupPath != "": + return "updated (backup: " + res.BackupPath + ")" + default: + return "updated" + } } // activateHint returns the shell command to activate the virtual environment, diff --git a/cmd/environments/sync.go b/cmd/environments/sync.go index 5505e6834cf..b7c6935c460 100644 --- a/cmd/environments/sync.go +++ b/cmd/environments/sync.go @@ -60,6 +60,12 @@ func addComputeFlags(cmd *cobra.Command) { cmd.Flags().String("serverless-version", "", "serverless version to use as the compute target (e.g. 5)") cmd.Flags().String("job-task", "", "job task to use as the compute target, as . (the task key is required)") cmd.Flags().Bool("constraints-only", false, "apply the Python version and constraints without adding the databricks-connect dependency") + // The negative flags (--no-constraints, --no-dbconnect, --no-provision) are + // orthogonal and compose. --no-dbconnect and the older --constraints-only are + // equivalent (both skip the databricks-connect dependency). + cmd.Flags().Bool("no-constraints", false, "skip writing the remote Python version and dependency constraints") + cmd.Flags().Bool("no-dbconnect", false, "skip adding the databricks-connect dependency") + cmd.Flags().Bool("no-provision", false, "write the project files but skip creating the virtual environment (no uv sync, Python download, or validation)") cmd.Flags().Bool("dry-run", false, "compute the plan without writing files or provisioning") // The mutual exclusivity of the target flags is enforced in the pipeline's // preflight (as E_USAGE) rather than via cmd.MarkFlagsMutuallyExclusive, so @@ -122,6 +128,9 @@ func runPipeline(cmd *cobra.Command) error { serverless, _ := cmd.Flags().GetString("serverless-version") jobTask, _ := cmd.Flags().GetString("job-task") constraintsOnly, _ := cmd.Flags().GetBool("constraints-only") + noConstraints, _ := cmd.Flags().GetBool("no-constraints") + noDBConnect, _ := cmd.Flags().GetBool("no-dbconnect") + noProvision, _ := cmd.Flags().GetBool("no-provision") check, _ := cmd.Flags().GetBool("dry-run") computeFlags := libslocalenv.ComputeFlags{ @@ -134,8 +143,10 @@ func runPipeline(cmd *cobra.Command) error { // preflight, so a conflict is reported as E_USAGE through the phase/JSON // contract rather than as a bare error here. + // --no-dbconnect is the orthogonal spelling of --constraints-only; either skips + // the databricks-connect dependency, which the pipeline models as the mode. mode := libslocalenv.ModeDefault - if constraintsOnly { + if constraintsOnly || noDBConnect { mode = libslocalenv.ModeConstraintsOnly } @@ -177,6 +188,8 @@ func runPipeline(cmd *cobra.Command) error { p := &libslocalenv.Pipeline{ Mode: mode, Check: check, + SkipConstraints: noConstraints, + SkipProvision: noProvision, ProjectDir: projectDir, ConstraintBaseURL: constraintBaseURL, CacheDir: cacheDir, diff --git a/libs/localenv/constraints.go b/libs/localenv/constraints.go index 36357facddb..5c3b9e64b0c 100644 --- a/libs/localenv/constraints.go +++ b/libs/localenv/constraints.go @@ -284,7 +284,15 @@ func parseConstraints(data []byte) (requiresPython, dbconnect string, deps []str } } + // Normalize a missing [tool.uv].constraint-dependencies to a non-nil empty + // slice. A nil ConstraintDeps is reserved as the --no-constraints "leave the + // constraint block unmanaged" signal (mergeToolUv and RenderFreshPyproject skip + // on nil); without this, an artifact that simply omits the key would be + // indistinguishable from the flag and would silently stop being managed. deps = p.Tool.UV.ConstraintDependencies + if deps == nil { + deps = []string{} + } return requiresPython, dbconnect, deps, nil } diff --git a/libs/localenv/constraints_test.go b/libs/localenv/constraints_test.go index 2590ce0c643..6eef3e639d6 100644 --- a/libs/localenv/constraints_test.go +++ b/libs/localenv/constraints_test.go @@ -259,3 +259,20 @@ func TestFetchConstraintsUnusableBodyDoesNotPoisonCache(t *testing.T) { assert.True(t, c.FromCache) assert.Equal(t, "==3.12.*", c.RequiresPython) } + +func TestParseConstraintsNormalizesMissingConstraintDepsToEmpty(t *testing.T) { + // An artifact without [tool.uv].constraint-dependencies yields a non-nil empty + // slice, not nil, so a nil ConstraintDeps is reserved as the --no-constraints + // "leave the constraint block unmanaged" signal (mergeToolUv / RenderFreshPyproject + // treat nil as skip). Without this, a normal artifact that simply omits the key + // would be indistinguishable from the flag. + _, _, deps, err := parseConstraints([]byte(`[project] +requires-python = ">=3.12" + +[dependency-groups] +dev = ["databricks-connect~=17.2.0"] +`)) + require.NoError(t, err) + require.NotNil(t, deps) + assert.Empty(t, deps) +} diff --git a/libs/localenv/merge.go b/libs/localenv/merge.go index e89c05ccef1..78b8080b288 100644 --- a/libs/localenv/merge.go +++ b/libs/localenv/merge.go @@ -217,6 +217,15 @@ func tableBounds(lines []string, name string) (header, end int, found bool) { // the line's leading whitespace. If the key is absent, it is inserted directly under the // [project] header. Returns whether the line slice changed. func mergeRequiresPython(lines []string, value string) ([]string, bool) { + // An empty value means requires-python is unmanaged (--no-constraints): leave + // the user's line untouched rather than overwrite it with a blank pin. This + // mirrors how an empty DatabricksConnect / EnvironmentVersion is a no-op; a + // real fetched artifact always carries a requires-python, so empty only ever + // reaches here when the caller deliberately cleared it. + if value == "" { + return lines, false + } + header, end, found := tableBounds(lines, "[project]") if !found { return lines, false @@ -861,6 +870,15 @@ func arrayLineSpan(lines []string, start, limit int) (last int, multiline bool) // marker-bracketed block already exists, its contents are replaced in place. Otherwise any // plain [tool.uv] table is removed and a fresh marker-bracketed block is appended at EOF. func mergeToolUv(lines, deps []string) ([]string, bool) { + // A nil deps slice means the [tool.uv] constraint region is unmanaged + // (--no-constraints): leave any existing block untouched and write none. + // Distinct from a non-nil empty slice, which still renders an empty managed + // block; a real fetched artifact always carries constraint-dependencies, so + // nil only reaches here when the caller deliberately cleared it. + if deps == nil { + return lines, false + } + start, stop, found := markerBounds(lines) if found { // Replace the existing managed region in place. Whether it owns a [tool.uv] @@ -1232,7 +1250,11 @@ func RenderFreshPyproject(projectName string, c Constraints) []byte { fmt.Fprintf(&b, "name = %q\n", projectName) // uv requires project.version when a [project] table is present. fmt.Fprintf(&b, "version = %q\n", freshProjectVersion) - fmt.Fprintf(&b, "requires-python = %q\n", c.RequiresPython) + // requires-python is omitted when unmanaged (--no-constraints); an empty pin + // would be invalid, and a fresh project without it lets uv pick the interpreter. + if c.RequiresPython != "" { + fmt.Fprintf(&b, "requires-python = %q\n", c.RequiresPython) + } b.WriteString("\n") b.WriteString("[dependency-groups]\n") if c.DatabricksConnect != "" { @@ -1250,9 +1272,13 @@ func RenderFreshPyproject(projectName string, c Constraints) []byte { fmt.Fprintf(&b, "environment_version = %q\n", c.EnvironmentVersion) b.WriteString("\n") } - for _, line := range renderToolUvBlock(c.ConstraintDeps, true) { - b.WriteString(line) - b.WriteString("\n") + // The [tool.uv] constraint block is omitted when unmanaged (--no-constraints, + // signalled by a nil slice); a non-nil empty slice still renders an empty block. + if c.ConstraintDeps != nil { + for _, line := range renderToolUvBlock(c.ConstraintDeps, true) { + b.WriteString(line) + b.WriteString("\n") + } } return []byte(b.String()) } diff --git a/libs/localenv/merge_test.go b/libs/localenv/merge_test.go index 048b401d617..67f67a69778 100644 --- a/libs/localenv/merge_test.go +++ b/libs/localenv/merge_test.go @@ -1155,3 +1155,53 @@ constraint-dependencies = ["old~=1.0"] func countOccurrences(s, substr string) int { return strings.Count(s, substr) } + +func TestMergeManagedSkipsRequiresPythonWhenEmpty(t *testing.T) { + // An empty RequiresPython is the --no-constraints signal: the merge must leave + // the user's requires-python untouched rather than overwrite it with "". + in := []byte(`[project] +name = "demo" +requires-python = ">=3.9" + +[dependency-groups] +dev = [] +`) + c := testConstraints() + c.RequiresPython = "" + out, regions, err := MergeManaged(in, c) + require.NoError(t, err) + assert.Contains(t, string(out), `requires-python = ">=3.9"`) + assert.NotContains(t, regions, regionRequiresPython) +} + +func TestMergeManagedSkipsToolUvWhenNil(t *testing.T) { + // A nil ConstraintDeps is the --no-constraints signal: no managed [tool.uv] + // constraint block is written and the region is not reported. + in := []byte(`[project] +name = "demo" +requires-python = "==3.12.*" + +[dependency-groups] +dev = [] +`) + c := testConstraints() + c.ConstraintDeps = nil + out, regions, err := MergeManaged(in, c) + require.NoError(t, err) + assert.NotContains(t, string(out), "constraint-dependencies") + assert.NotContains(t, regions, regionToolUv) +} + +func TestRenderFreshPyprojectOmitsConstraintsWhenEmpty(t *testing.T) { + // Greenfield --no-constraints: neither the Python pin nor the [tool.uv] + // constraint block is rendered, but databricks-connect (orthogonal) still is. + c := testConstraints() + c.RequiresPython = "" + c.ConstraintDeps = nil + out := RenderFreshPyproject("demo", c) + s := string(out) + assert.NotContains(t, s, "requires-python") + assert.NotContains(t, s, "constraint-dependencies") + assert.Contains(t, s, `"databricks-connect~=17.2.0",`) + requireValidTOML(t, out) +} diff --git a/libs/localenv/pipeline.go b/libs/localenv/pipeline.go index 258645aba30..bfa834ecb24 100644 --- a/libs/localenv/pipeline.go +++ b/libs/localenv/pipeline.go @@ -59,6 +59,16 @@ type Pipeline struct { Compute ComputeClient PM PackageManager + // SkipConstraints (--no-constraints) leaves the remote Python version and + // dependency pins unmanaged: the merge writes neither requires-python nor the + // [tool.uv] constraint block, and any existing values are left untouched. It is + // orthogonal to Mode (the databricks-connect axis). + SkipConstraints bool + // SkipProvision (--no-provision) writes the project files through the merge + // phase, then stops: no Python install, uv sync, or validation. The provision + // and validate phases report StatusSkipped and venvPath is omitted. + SkipProvision bool + // Progress, when non-nil, receives a PhaseStarted call as each phase begins. // Left nil by callers that don't render progress (e.g. --output json). Progress Reporter @@ -172,9 +182,18 @@ func (p *Pipeline) run(ctx context.Context) error { // - PackageManager.EnsureAvailable may install the manager (uv) if missing. // Both exist to fail fast before real writes, which --dry-run never performs, so // they are skipped in a dry run. Neither result is needed to compute the plan. - if p.Check { + switch { + case p.Check: p.markOK(PhasePreflight, "check") - } else { + case p.SkipProvision: + // --no-provision writes files but never invokes uv, so it must not require + // or install it — a files-only run succeeds on a machine without uv. The + // project must still be writable, since the merge phase writes to it. + if err := ensureWritable(p.ProjectDir); err != nil { + return p.fail(PhasePreflight, false, NewError(ErrNotWritable, err, "project directory %s is not writable", filepath.ToSlash(p.ProjectDir))) + } + p.markOK(PhasePreflight, "no-provision") + default: if err := ensureWritable(p.ProjectDir); err != nil { return p.fail(PhasePreflight, false, NewError(ErrNotWritable, err, "project directory %s is not writable", filepath.ToSlash(p.ProjectDir))) } @@ -234,8 +253,7 @@ func (p *Pipeline) run(ctx context.Context) error { // Check mode stops after planning — nothing below mutates disk. if p.Check { p.markOK(PhaseMerge, "") - p.markOK(PhaseProvision, "") - p.markOK(PhaseValidate, "") + p.markProvisionAndValidate() return nil } @@ -245,6 +263,14 @@ func (p *Pipeline) run(ctx context.Context) error { } p.markOK(PhaseMerge, "") + // --no-provision writes the files but stops here: no Python install, uv sync, + // or validation. The provision and validate phases are skipped and venvPath is + // left empty (omitted), since no venv is created. + if p.SkipProvision { + p.markProvisionAndValidate() + return nil + } + // Phase: provision — ensure Python, run uv sync, seed pip. p.report(ctx, PhaseProvision) if err := p.provision(ctx, pyMinor); err != nil { @@ -384,13 +410,35 @@ func (p *Pipeline) mergePlan(_ context.Context, pyMinor string, c *Constraints, effective := *c effective.DatabricksConnect = dbcPin effective.EnvironmentVersion = envVersion + if p.SkipConstraints { + // --no-constraints: leave the remote Python version and dependency pins + // unmanaged. The empty/nil values signal both the merge (mergeRequiresPython, + // mergeToolUv) and the fresh render to skip those regions, leaving any + // existing values untouched. + // + // The flag governs only what is *written*. A provisioning run still installs + // and validates the resolved Python (pyMinor), so if the user's kept + // requires-python is disjoint from the target, uv surfaces it as a normal + // E_PROVISION rather than this command guessing an alternative. The + // --no-constraints --no-provision pairing skips provisioning entirely and + // avoids that tension. + effective.RequiresPython = "" + effective.ConstraintDeps = nil + } var changedRegions []string if greenfield { // No existing pyproject.toml — render a fresh one. The project name comes - // from the directory name as a reasonable default. + // from the directory name as a reasonable default. Only the regions actually + // rendered are reported (requires-python and tool.uv are omitted under + // --no-constraints). merged = RenderFreshPyproject(projectName(p.ProjectDir), effective) - changedRegions = []string{regionRequiresPython, regionToolUv} + if effective.RequiresPython != "" { + changedRegions = append(changedRegions, regionRequiresPython) + } + if effective.ConstraintDeps != nil { + changedRegions = append(changedRegions, regionToolUv) + } if dbcPin != "" { changedRegions = append(changedRegions, regionDatabricksConnect) } @@ -427,10 +475,14 @@ func (p *Pipeline) mergePlan(_ context.Context, pyMinor string, c *Constraints, diff := fmt.Sprint(gotextdiff.ToUnified(oldName, newName, oldStr, edits)) plan := &Plan{ - WouldWrite: filepath.ToSlash(pyproject), - Diff: diff, - ChangedRegions: changedRegions, - WouldInstallPython: pyMinor, + WouldWrite: filepath.ToSlash(pyproject), + Diff: diff, + ChangedRegions: changedRegions, + } + // --no-provision stops before installing Python, so the plan must not claim + // a Python install; every other path would provision it. + if !p.SkipProvision { + plan.WouldInstallPython = pyMinor } // Report a backup only when the run would actually write one (i.e. it changes // the file); a no-op re-run writes none. @@ -611,6 +663,33 @@ func (p *Pipeline) markOK(name PhaseName, detail string) { } } +// markSkipped marks a phase skipped: a flag opted out of running it. Unlike +// markOK it carries no detail — there is nothing to report about work not done — +// and unlike a pending phase it is a successful outcome, not a stopped one. +func (p *Pipeline) markSkipped(name PhaseName) { + for i := range p.res.Phases { + if p.res.Phases[i].Phase == name { + p.res.Phases[i].Status = StatusSkipped + p.res.Phases[i].Detail = "" + return + } + } +} + +// markProvisionAndValidate records the provision and validate outcomes for a run +// that stops before executing them. Under --no-provision they are skipped; under +// a plain --dry-run they are ok (the plan through merge succeeded and nothing +// below would run). It is the shared tail of both non-provisioning paths. +func (p *Pipeline) markProvisionAndValidate() { + if p.SkipProvision { + p.markSkipped(PhaseProvision) + p.markSkipped(PhaseValidate) + return + } + p.markOK(PhaseProvision, "") + p.markOK(PhaseValidate, "") +} + // fail marks the given phase as errored, attaches the error (with its phase and // disk-mutation flag) to the Result, and returns it. Phases after the failing // one remain pending. diff --git a/libs/localenv/pipeline_test.go b/libs/localenv/pipeline_test.go index fdf11c2322c..32b2735f9e4 100644 --- a/libs/localenv/pipeline_test.go +++ b/libs/localenv/pipeline_test.go @@ -1201,3 +1201,135 @@ func TestPipelineReportsPhaseStarts(t *testing.T) { // A full successful run enters every phase exactly once in canonical order. assert.Equal(t, allPhases, rep.started) } + +func TestPipelineNoProvisionWritesFilesButSkipsProvisionAndValidate(t *testing.T) { + dir := writeProject(t) + srv := newTestServer(t) + defer srv.Close() + + // --no-provision writes the project files (a real merge, not a dry run) but + // stops before creating the venv: provision and validate are "skipped". It also + // never invokes uv — not even the preflight availability probe — so a files-only + // run succeeds on a machine without uv. noProvisionPM fails every PackageManager + // method, so this passing proves none of them (EnsureAvailable included) is called. + p := &Pipeline{ + Mode: ModeDefault, SkipProvision: true, ProjectDir: dir, + ConstraintBaseURL: srv.URL, CacheDir: t.TempDir(), + Flags: ComputeFlags{Serverless: "v4"}, + Compute: stubCompute{}, PM: noProvisionPM{}, + } + res, err := p.Run(t.Context()) + require.NoError(t, err) + assert.True(t, res.OK) + // Files were written through the merge phase. + data, readErr := os.ReadFile(filepath.Join(dir, "pyproject.toml")) + require.NoError(t, readErr) + assert.Contains(t, string(data), `"databricks-connect~=17.2.0"`) + // Merge succeeded; provision and validate are skipped (distinct from pending, + // which would mean an earlier phase failed). + assert.Equal(t, StatusOK, phaseStatus(res, PhaseMerge)) + assert.Equal(t, StatusSkipped, phaseStatus(res, PhaseProvision)) + assert.Equal(t, StatusSkipped, phaseStatus(res, PhaseValidate)) + // No venv was provisioned, so venvPath is omitted. + assert.Empty(t, res.VenvPath) +} + +func TestPipelineNoProvisionUnderDryRunMarksProvisionValidateSkipped(t *testing.T) { + dir := writeProject(t) + srv := newTestServer(t) + defer srv.Close() + + // --dry-run --no-provision: nothing is provisioned (as with any dry run), but + // the plan reflects the real run's intent, so provision/validate are "skipped" + // rather than the "ok" a plain dry run reports. + p := &Pipeline{ + Mode: ModeDefault, Check: true, SkipProvision: true, ProjectDir: dir, + ConstraintBaseURL: srv.URL, CacheDir: t.TempDir(), + Flags: ComputeFlags{Serverless: "v4"}, + Compute: stubCompute{}, PM: noProvisionPM{}, + } + res, err := p.Run(t.Context()) + require.NoError(t, err) + assert.True(t, res.OK) + require.NotNil(t, res.Plan) + assert.Equal(t, StatusSkipped, phaseStatus(res, PhaseProvision)) + assert.Equal(t, StatusSkipped, phaseStatus(res, PhaseValidate)) + assert.Empty(t, res.VenvPath) +} + +func TestPipelineNoConstraintsLeavesExistingPinsUntouched(t *testing.T) { + dir := t.TempDir() + // An existing project with the user's own requires-python and no managed + // [tool.uv] constraint block. + require.NoError(t, os.WriteFile(filepath.Join(dir, "pyproject.toml"), []byte(`[project] +name = "demo" +requires-python = ">=3.9" + +[dependency-groups] +dev = ["databricks-connect~=16.0.0"] +`), 0o644)) + srv := newTestServer(t) + defer srv.Close() + + p := &Pipeline{ + Mode: ModeDefault, SkipConstraints: true, ProjectDir: dir, + ConstraintBaseURL: srv.URL, CacheDir: t.TempDir(), + Flags: ComputeFlags{Serverless: "v4"}, + Compute: stubCompute{}, PM: fakePM{py: "3.12", dbc: "17.2.0"}, + } + res, err := p.Run(t.Context()) + require.NoError(t, err) + assert.True(t, res.OK) + data, _ := os.ReadFile(filepath.Join(dir, "pyproject.toml")) + s := string(data) + // requires-python keeps the user's value; the artifact's ==3.12.* is not written. + assert.Contains(t, s, `requires-python = ">=3.9"`) + assert.NotContains(t, s, "==3.12.*") + // No managed [tool.uv] constraint-dependencies block is written. + assert.NotContains(t, s, "constraint-dependencies") + // databricks-connect is still managed: --no-constraints is orthogonal to it. + assert.Contains(t, s, "databricks-connect~=17.2.0") +} + +func TestPipelineNoConstraintsGreenfieldOmitsPins(t *testing.T) { + dir := t.TempDir() + srv := newTestServer(t) + defer srv.Close() + + p := &Pipeline{ + Mode: ModeDefault, SkipConstraints: true, ProjectDir: dir, + ConstraintBaseURL: srv.URL, CacheDir: t.TempDir(), + Flags: ComputeFlags{Serverless: "v4"}, + Compute: stubCompute{}, PM: fakePM{py: "3.12", dbc: "17.2.0"}, + } + res, err := p.Run(t.Context()) + require.NoError(t, err) + assert.True(t, res.OK) + assert.True(t, res.Greenfield) + data, _ := os.ReadFile(filepath.Join(dir, "pyproject.toml")) + s := string(data) + // The artifact's Python pin and constraint-dependencies are not written. + assert.NotContains(t, s, "==3.12.*") + assert.NotContains(t, s, "constraint-dependencies") + // databricks-connect (orthogonal) is still added. + assert.Contains(t, s, "databricks-connect~=17.2.0") +} + +func TestPipelineNoProvisionDryRunPlanOmitsWouldInstallPython(t *testing.T) { + dir := writeProject(t) + srv := newTestServer(t) + defer srv.Close() + + // --no-provision would not install Python, so the dry-run plan must not claim + // it would (wouldInstallPython is omitted). + p := &Pipeline{ + Mode: ModeDefault, Check: true, SkipProvision: true, ProjectDir: dir, + ConstraintBaseURL: srv.URL, CacheDir: t.TempDir(), + Flags: ComputeFlags{Serverless: "v4"}, + Compute: stubCompute{}, PM: noProvisionPM{}, + } + res, err := p.Run(t.Context()) + require.NoError(t, err) + require.NotNil(t, res.Plan) + assert.Empty(t, res.Plan.WouldInstallPython) +} diff --git a/libs/localenv/result.go b/libs/localenv/result.go index 236d845d685..f6e164d89a2 100644 --- a/libs/localenv/result.go +++ b/libs/localenv/result.go @@ -21,7 +21,11 @@ const ( CommandName = CommandGroup + " " + CommandVerb // SchemaVersion is the version of the --json output contract (spec §6). - // Bump it on any breaking change to the JSON shape. + // Bump it on any breaking change to the JSON shape — the set of keys and their + // types. Adding a new value to an existing enum field is not such a change: it + // stays at 1. The "skipped" phase status is an example — it only appears when a + // new opt-in flag (--no-provision) is passed, so a default run's output is + // byte-for-byte unchanged and existing consumers see exactly what they did before. SchemaVersion = 1 ) @@ -68,6 +72,11 @@ const ( StatusOK = "ok" StatusError = "error" StatusPending = "pending" + // StatusSkipped marks a phase the run deliberately did not perform because a + // flag opted out of it (provision + validate under --no-provision). It is + // distinct from StatusPending, which means an earlier phase failed before this + // one could run: skipped is a successful outcome, pending is a stopped one. + StatusSkipped = "skipped" ) // ErrorCode is a stable failure-class identifier surfaced in --json error.code From 1521ec14a4c84d5985431493704bdf8c678f06dc Mon Sep 17 00:00:00 2001 From: Grigory Panov Date: Wed, 2 Sep 2026 10:14:10 +0200 Subject: [PATCH 2/4] Emit E_PROVISION_CONFLICT for uv sync resolution failures MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `databricks environments setup-local` wrapped every `uv sync` failure as E_PROVISION. The extension's recovery flow needs to tell a dependency-resolution failure — the remote pins just written can't be satisfied against the user's local dependencies — apart from a generic sync failure it can't fix by adjusting constraints. Classify a `uv sync` failure as the new E_PROVISION_CONFLICT when uv's stderr carries its resolver banner ("No solution found when resolving dependencies"); every other sync failure keeps E_PROVISION. The failing phase (provision), diskMutated=true, and the E_CANCELED reclassification are unchanged. Adds the matching telemetry enum value and an acceptance golden that drives a real setup-local run against a fake uv failing sync with the resolver banner. Co-authored-by: Isaac --- .../cli/setup-local-provision-conflict.md | 1 + .../localenv/provision-conflict/fake-uv.sh | 31 ++++++++++ .../localenv/provision-conflict/out.test.toml | 3 + .../localenv/provision-conflict/output.txt | 62 +++++++++++++++++++ .../provision-conflict/pyproject.toml | 7 +++ acceptance/localenv/provision-conflict/script | 19 ++++++ .../localenv/provision-conflict/test.toml | 26 ++++++++ cmd/environments/telemetry.go | 2 + cmd/environments/telemetry_test.go | 1 + libs/localenv/pipeline.go | 8 +-- libs/localenv/result.go | 3 +- libs/localenv/uv.go | 30 ++++++++- libs/localenv/uv_test.go | 28 +++++++++ libs/telemetry/protos/setup_local.go | 1 + 14 files changed, 216 insertions(+), 6 deletions(-) create mode 100644 .nextchanges/cli/setup-local-provision-conflict.md create mode 100755 acceptance/localenv/provision-conflict/fake-uv.sh create mode 100644 acceptance/localenv/provision-conflict/out.test.toml create mode 100644 acceptance/localenv/provision-conflict/output.txt create mode 100644 acceptance/localenv/provision-conflict/pyproject.toml create mode 100644 acceptance/localenv/provision-conflict/script create mode 100644 acceptance/localenv/provision-conflict/test.toml diff --git a/.nextchanges/cli/setup-local-provision-conflict.md b/.nextchanges/cli/setup-local-provision-conflict.md new file mode 100644 index 00000000000..451d0506002 --- /dev/null +++ b/.nextchanges/cli/setup-local-provision-conflict.md @@ -0,0 +1 @@ +`databricks environments setup-local` now reports a distinct `E_PROVISION_CONFLICT` error code in `--output json` when `uv sync` fails because the dependency pins cannot be resolved (for example, the remote constraints conflict with the project's own dependencies). Other sync failures continue to report `E_PROVISION`. diff --git a/acceptance/localenv/provision-conflict/fake-uv.sh b/acceptance/localenv/provision-conflict/fake-uv.sh new file mode 100755 index 00000000000..faaf79c86e2 --- /dev/null +++ b/acceptance/localenv/provision-conflict/fake-uv.sh @@ -0,0 +1,31 @@ +#!/usr/bin/env bash +# +# Fake `uv` for the provision-conflict acceptance test. A real `uv sync` version +# conflict needs a specific managed Python installed offline, which no CI machine +# guarantees, so this stub stands in for uv on PATH and drives the provision phase +# deterministically: +# +# uv --version -> a version banner (EnsureAvailable preflight) +# uv python install ... -> succeeds (EnsurePython) +# uv sync ... -> exits non-zero, printing uv's real resolver-conflict +# banner to stderr, which setup-local classifies as +# E_PROVISION_CONFLICT +# +# The `uv sync` stderr mirrors genuine uv output; the isUvResolutionConflict unit +# test pins the same "No solution found when resolving dependencies" marker against +# real uv stderr, so this stub and the classifier stay in agreement. +set -euo pipefail + +if [[ "${1:-}" == "--version" ]]; then + echo "uv 0.0.0-fake" + exit 0 +fi + +if [[ "${1:-}" == "sync" ]]; then + echo " × No solution found when resolving dependencies:" >&2 + echo " ╰─▶ Because your project depends on pip==24.0 and pip<24, we can conclude that your project's requirements are unsatisfiable." >&2 + exit 1 +fi + +# Every other invocation (notably `uv python install `) succeeds silently. +exit 0 diff --git a/acceptance/localenv/provision-conflict/out.test.toml b/acceptance/localenv/provision-conflict/out.test.toml new file mode 100644 index 00000000000..4cc2d5fdf7e --- /dev/null +++ b/acceptance/localenv/provision-conflict/out.test.toml @@ -0,0 +1,3 @@ +Cloud = false +GOOS.windows = false +EnvMatrix.DATABRICKS_BUNDLE_ENGINE = ["direct"] diff --git a/acceptance/localenv/provision-conflict/output.txt b/acceptance/localenv/provision-conflict/output.txt new file mode 100644 index 00000000000..bc8797e7975 --- /dev/null +++ b/acceptance/localenv/provision-conflict/output.txt @@ -0,0 +1,62 @@ +{ + "schemaVersion": 1, + "command": "environments setup-local", + "ok": false, + "mode": "default", + "dryRun": false, + "compute": { + "source": "serverless", + "serverlessVersion": "v4", + "envKey": "serverless/serverless-v4" + }, + "resolved": { + "pythonVersion": "3.12", + "dbconnectVersion": "17.2.0", + "artifactSource": "network" + }, + "greenfield": false, + "phases": [ + { + "phase": "preflight", + "status": "ok" + }, + { + "phase": "resolve", + "status": "ok" + }, + { + "phase": "fetch", + "status": "ok" + }, + { + "phase": "merge", + "status": "ok" + }, + { + "phase": "provision", + "status": "error" + }, + { + "phase": "validate", + "status": "pending" + } + ], + "warnings": [ + { + "code": "W_DBCONNECT_PIN_OVERRIDDEN", + "message": "databricks-connect \"databricks-connect~=16.0.0\" is replaced by the environment's \"databricks-connect~=17.2.0\"" + }, + { + "code": "W_USER_CONSTRAINT_CONFLICT", + "message": "dependency \"pip==24.0\" conflicts with the environment constraint \"pip\u003c24\"" + } + ], + "error": { + "code": "E_PROVISION_CONFLICT", + "failurePhase": "provision", + "message": "uv sync failed: × No solution found when resolving dependencies:\n ╰─▶ Because your project depends on pip==24.0 and pip\u003c24, we can conclude that your project's requirements are unsatisfiable.: [TEST_TMP_DIR]/uv sync --python 3.12: exit status 1", + "diskMutated": true + }, + "backupPath": "[TEST_TMP_DIR]/pyproject.toml.bak", + "durationMs": [DURATION_MS] +} diff --git a/acceptance/localenv/provision-conflict/pyproject.toml b/acceptance/localenv/provision-conflict/pyproject.toml new file mode 100644 index 00000000000..269a5c16a14 --- /dev/null +++ b/acceptance/localenv/provision-conflict/pyproject.toml @@ -0,0 +1,7 @@ +[project] +name = "demo" +requires-python = ">=3.12" +dependencies = ["pip==24.0"] + +[dependency-groups] +dev = ["databricks-connect~=16.0.0"] diff --git a/acceptance/localenv/provision-conflict/script b/acceptance/localenv/provision-conflict/script new file mode 100644 index 00000000000..5f491dc70bb --- /dev/null +++ b/acceptance/localenv/provision-conflict/script @@ -0,0 +1,19 @@ +# A `uv sync` that fails because the merged dependency pins can't be resolved is +# reported as E_PROVISION_CONFLICT — distinct from a generic E_PROVISION — with the +# constraints already written to disk (diskMutated=true), the contract the +# extension's recovery flow depends on. Here the user pins pip==24.0 while the +# remote constraints pin pip<24, so the merged project is unsatisfiable. +# +# A real `uv sync` version conflict needs a specific managed Python installed +# offline, which CI cannot guarantee, so put a fake `uv` on PATH that fails +# `uv sync` with uv's real resolver-conflict banner (see fake-uv.sh). +mv fake-uv.sh uv + +cleanup() { + rm -f uv +} +trap cleanup EXIT + +export PATH="$(pwd):$PATH" + +musterr $CLI environments setup-local --serverless-version 4 --output json diff --git a/acceptance/localenv/provision-conflict/test.toml b/acceptance/localenv/provision-conflict/test.toml new file mode 100644 index 00000000000..1fb0355752e --- /dev/null +++ b/acceptance/localenv/provision-conflict/test.toml @@ -0,0 +1,26 @@ +EnvMatrix.DATABRICKS_BUNDLE_ENGINE = ["direct"] + +# The fake `uv` on PATH is a bash script; the psql acceptance tests document that +# this binary-on-PATH technique does not work on the Windows GitHub runner. +GOOS.windows = false + +# setup-local writes pyproject.toml through the merge phase (backing up the +# committed one) before provision runs; the assertion is the JSON output, so ignore +# the mutated file and its backup. No .venv is created because the fake `uv sync` +# fails before provisioning one. +Ignore = ["pyproject.toml", "pyproject.toml.bak"] + +Env.DATABRICKS_LOCALENV_CONSTRAINT_SOURCE_URL_TEST_OVERRIDE = "$DATABRICKS_HOST" + +[[Server]] +Pattern = "GET /serverless/serverless-v4/pyproject.toml" +Response.Body = ''' +[project] +requires-python = ">=3.12" + +[dependency-groups] +dev = ["databricks-connect~=17.2.0"] + +[tool.uv] +constraint-dependencies = ["pip<24"] +''' diff --git a/cmd/environments/telemetry.go b/cmd/environments/telemetry.go index c8c653a8060..67fd56f58fa 100644 --- a/cmd/environments/telemetry.go +++ b/cmd/environments/telemetry.go @@ -105,6 +105,8 @@ func errorCodeType(code libslocalenv.ErrorCode) protos.SetupLocalErrorCode { return protos.SetupLocalErrorCodePythonInstall case libslocalenv.ErrProvision: return protos.SetupLocalErrorCodeProvision + case libslocalenv.ErrProvisionConflict: + return protos.SetupLocalErrorCodeProvisionConflict case libslocalenv.ErrValidate: return protos.SetupLocalErrorCodeValidate case libslocalenv.ErrCanceled: diff --git a/cmd/environments/telemetry_test.go b/cmd/environments/telemetry_test.go index a458821c0a5..6362d984cf8 100644 --- a/cmd/environments/telemetry_test.go +++ b/cmd/environments/telemetry_test.go @@ -96,6 +96,7 @@ func TestErrorCodeCoversLocalenv(t *testing.T) { libslocalenv.ErrMerge, libslocalenv.ErrPythonInstall, libslocalenv.ErrProvision, + libslocalenv.ErrProvisionConflict, libslocalenv.ErrValidate, libslocalenv.ErrCanceled, } diff --git a/libs/localenv/pipeline.go b/libs/localenv/pipeline.go index bfa834ecb24..14c2d345ef0 100644 --- a/libs/localenv/pipeline.go +++ b/libs/localenv/pipeline.go @@ -418,10 +418,10 @@ func (p *Pipeline) mergePlan(_ context.Context, pyMinor string, c *Constraints, // // The flag governs only what is *written*. A provisioning run still installs // and validates the resolved Python (pyMinor), so if the user's kept - // requires-python is disjoint from the target, uv surfaces it as a normal - // E_PROVISION rather than this command guessing an alternative. The - // --no-constraints --no-provision pairing skips provisioning entirely and - // avoids that tension. + // requires-python is disjoint from the target, uv fails to resolve it + // (surfaced as E_PROVISION_CONFLICT) rather than this command guessing an + // alternative. The --no-constraints --no-provision pairing skips provisioning + // entirely and avoids that tension. effective.RequiresPython = "" effective.ConstraintDeps = nil } diff --git a/libs/localenv/result.go b/libs/localenv/result.go index f6e164d89a2..5962bb0ffbe 100644 --- a/libs/localenv/result.go +++ b/libs/localenv/result.go @@ -103,7 +103,8 @@ const ( ErrWrite ErrorCode = "E_WRITE" // merge: greenfield write failed ErrMerge ErrorCode = "E_MERGE" // merge: existing-project merge failed ErrPythonInstall ErrorCode = "E_PYTHON_INSTALL" // provision: uv python install failed - ErrProvision ErrorCode = "E_PROVISION" // provision: uv sync failed + ErrProvision ErrorCode = "E_PROVISION" // provision: uv sync failed (generic) + ErrProvisionConflict ErrorCode = "E_PROVISION_CONFLICT" // provision: uv sync could not resolve dependencies ErrValidate ErrorCode = "E_VALIDATE" // validate: post-provision version mismatch // ErrCanceled is not in the spec's error-code table: it reports a user/parent diff --git a/libs/localenv/uv.go b/libs/localenv/uv.go index 7e4ca82c746..4be5a123f70 100644 --- a/libs/localenv/uv.go +++ b/libs/localenv/uv.go @@ -104,11 +104,39 @@ func (m *uvManager) EnsurePython(ctx context.Context, minor string) error { func (m *uvManager) Provision(ctx context.Context, projectDir, pyMinor string) error { args := append([]string{m.bin}, m.syncArgs(pyMinor)...) if err := m.runUv(ctx, args, projectDir); err != nil { - return uvFailure(ErrProvision, err, "uv sync") + // A resolution failure (the remote pins just written can't be satisfied + // against the user's local dependencies) gets a distinct code so a caller can + // react to it — relax the written constraints / requires-python and retry — + // rather than treat it like any other uv sync failure (build backend, + // permissions, transport), which E_PROVISION still covers. + code := ErrProvision + if isUvResolutionConflict(err) { + code = ErrProvisionConflict + } + return uvFailure(code, err, "uv sync") } return nil } +// uvResolverConflictMarker is the stable header uv prints to stderr whenever +// `uv sync` cannot satisfy the project's requirements. Every unsatisfiable +// resolution surfaces under it — a disjoint version pin, an incompatible +// requires-python, or an unavailable dependency — and uv exposes no exit code or +// machine-readable field distinguishing them, so this marker classifies the whole +// resolution-failure class. That is exactly the class a caller can act on, so the +// coarse match is the right granularity, not a limitation. Matching uv's own +// output is the only classification uv offers; there is no typed error to compare. +const uvResolverConflictMarker = "No solution found when resolving dependencies" + +// isUvResolutionConflict reports whether a failed uv invocation failed because uv +// could not resolve dependencies (see uvResolverConflictMarker). It reads the +// process stderr; an error that is not a *process.ProcessError, or one whose +// stderr lacks the marker, is not a resolution conflict. +func isUvResolutionConflict(err error) bool { + perr, ok := errors.AsType[*process.ProcessError](err) + return ok && strings.Contains(perr.Stderr, uvResolverConflictMarker) +} + // venvPython returns the path to the virtualenv's Python interpreter, // accounting for the Windows (Scripts/python.exe) vs Unix (bin/python) layout. func venvPython(projectDir string) string { diff --git a/libs/localenv/uv_test.go b/libs/localenv/uv_test.go index 8fcce4376a7..d229f3ec7fe 100644 --- a/libs/localenv/uv_test.go +++ b/libs/localenv/uv_test.go @@ -236,6 +236,34 @@ func TestUvFailureIncludesStderr(t *testing.T) { }) } +func TestIsUvResolutionConflict(t *testing.T) { + t.Run("resolver_no_solution_stderr_is_conflict", func(t *testing.T) { + // uv prints this banner for any unsatisfiable resolution — a disjoint version + // pin, an incompatible requires-python, or an unavailable dependency. + err := &process.ProcessError{ + Command: "uv sync", + Err: errors.New("exit status 1"), + Stderr: " × No solution found when resolving dependencies:\n ╰─▶ Because your project depends on pip==24.0 and pip<24, we can conclude that your project's requirements are unsatisfiable.\n", + } + assert.True(t, isUvResolutionConflict(err)) + }) + + t.Run("generic_failure_stderr_is_not_conflict", func(t *testing.T) { + // A transport error is a generic provision failure, not a resolution conflict. + err := &process.ProcessError{ + Command: "uv sync", + Err: errors.New("exit status 2"), + Stderr: "error: Connection refused\n", + } + assert.False(t, isUvResolutionConflict(err)) + }) + + t.Run("non_process_error_is_not_conflict", func(t *testing.T) { + // Without a process error there is no stderr to classify. + assert.False(t, isUvResolutionConflict(errors.New("some other error"))) + }) +} + func TestConfirmUvInstall(t *testing.T) { t.Run("opt_in_env_var_consents_without_prompt", func(t *testing.T) { // Non-interactive context, but the opt-in env var grants consent. diff --git a/libs/telemetry/protos/setup_local.go b/libs/telemetry/protos/setup_local.go index f359abf8664..7d3b5c92d87 100644 --- a/libs/telemetry/protos/setup_local.go +++ b/libs/telemetry/protos/setup_local.go @@ -41,6 +41,7 @@ const ( SetupLocalErrorCodeMerge SetupLocalErrorCode = "E_MERGE" SetupLocalErrorCodePythonInstall SetupLocalErrorCode = "E_PYTHON_INSTALL" SetupLocalErrorCodeProvision SetupLocalErrorCode = "E_PROVISION" + SetupLocalErrorCodeProvisionConflict SetupLocalErrorCode = "E_PROVISION_CONFLICT" SetupLocalErrorCodeValidate SetupLocalErrorCode = "E_VALIDATE" SetupLocalErrorCodeCanceled SetupLocalErrorCode = "E_CANCELED" ) From ac944575d74d6284b0bb7a6b95df2d915dd07888 Mon Sep 17 00:00:00 2001 From: Grigory Panov Date: Wed, 2 Sep 2026 10:27:42 +0200 Subject: [PATCH 3/4] Review: gate E_PROVISION_CONFLICT on the merge conflict signal MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three independent reviewers converged: matching uv's "No solution found" stderr classified every resolution failure — including an unavailable package, which relaxing constraints can't fix — as a conflict, broader than the ticket's "remote pins conflict with the user's dependencies" and misleading to consumers. Gate the code on the CLI's own detection instead: emit E_PROVISION_CONFLICT when a uv sync failure coincides with the merge phase's W_USER_CONSTRAINT_CONFLICT warning (a provable disjoint-version conflict), else keep E_PROVISION. Removes the stderr string-matching (and its brittleness / the errors-string-matching concern) entirely. Adds pipeline unit tests for the conflict and generic-failure paths; the acceptance golden and telemetry mapping are unchanged in outcome. Co-authored-by: Isaac --- .../cli/setup-local-provision-conflict.md | 2 +- .../localenv/provision-conflict/fake-uv.sh | 10 +- libs/localenv/pipeline.go | 35 ++++++- libs/localenv/pipeline_test.go | 92 +++++++++++++++++++ libs/localenv/result.go | 2 +- libs/localenv/uv.go | 30 +----- libs/localenv/uv_test.go | 28 ------ 7 files changed, 130 insertions(+), 69 deletions(-) diff --git a/.nextchanges/cli/setup-local-provision-conflict.md b/.nextchanges/cli/setup-local-provision-conflict.md index 451d0506002..9ca360343ff 100644 --- a/.nextchanges/cli/setup-local-provision-conflict.md +++ b/.nextchanges/cli/setup-local-provision-conflict.md @@ -1 +1 @@ -`databricks environments setup-local` now reports a distinct `E_PROVISION_CONFLICT` error code in `--output json` when `uv sync` fails because the dependency pins cannot be resolved (for example, the remote constraints conflict with the project's own dependencies). Other sync failures continue to report `E_PROVISION`. +`databricks environments setup-local` now reports a distinct `E_PROVISION_CONFLICT` error code in `--output json` when `uv sync` fails and the merge phase detected a version conflict between the project's dependencies and the pins written for the target environment (the same conflict surfaced as a `W_USER_CONSTRAINT_CONFLICT` warning). Other sync failures continue to report `E_PROVISION`. diff --git a/acceptance/localenv/provision-conflict/fake-uv.sh b/acceptance/localenv/provision-conflict/fake-uv.sh index faaf79c86e2..54dee7b2a8f 100755 --- a/acceptance/localenv/provision-conflict/fake-uv.sh +++ b/acceptance/localenv/provision-conflict/fake-uv.sh @@ -8,12 +8,12 @@ # uv --version -> a version banner (EnsureAvailable preflight) # uv python install ... -> succeeds (EnsurePython) # uv sync ... -> exits non-zero, printing uv's real resolver-conflict -# banner to stderr, which setup-local classifies as -# E_PROVISION_CONFLICT +# banner to stderr # -# The `uv sync` stderr mirrors genuine uv output; the isUvResolutionConflict unit -# test pins the same "No solution found when resolving dependencies" marker against -# real uv stderr, so this stub and the classifier stay in agreement. +# setup-local reports E_PROVISION_CONFLICT because the merge phase already flagged +# the pip==24.0 vs pip<24 conflict (W_USER_CONSTRAINT_CONFLICT); the stderr here is +# realistic uv output that ends up in the error message, not the classification +# signal. set -euo pipefail if [[ "${1:-}" == "--version" ]]; then diff --git a/libs/localenv/pipeline.go b/libs/localenv/pipeline.go index 14c2d345ef0..333490223e3 100644 --- a/libs/localenv/pipeline.go +++ b/libs/localenv/pipeline.go @@ -418,10 +418,10 @@ func (p *Pipeline) mergePlan(_ context.Context, pyMinor string, c *Constraints, // // The flag governs only what is *written*. A provisioning run still installs // and validates the resolved Python (pyMinor), so if the user's kept - // requires-python is disjoint from the target, uv fails to resolve it - // (surfaced as E_PROVISION_CONFLICT) rather than this command guessing an - // alternative. The --no-constraints --no-provision pairing skips provisioning - // entirely and avoids that tension. + // requires-python is disjoint from the target, uv surfaces it as a normal + // E_PROVISION rather than this command guessing an alternative. The + // --no-constraints --no-provision pairing skips provisioning entirely and + // avoids that tension. effective.RequiresPython = "" effective.ConstraintDeps = nil } @@ -549,7 +549,19 @@ func (p *Pipeline) provision(ctx context.Context, pyMinor string) error { return p.fail(PhaseProvision, true, asPipelineError(err, ErrPythonInstall, "ensure python %s failed", pyMinor)) } if err := p.PM.Provision(ctx, p.ProjectDir, pyMinor); err != nil { - return p.fail(PhaseProvision, true, asPipelineError(err, ErrProvision, "provision failed")) + pe := asPipelineError(err, ErrProvision, "provision failed") + // When the merge phase already flagged a provable version conflict between + // the user's dependencies and the pins this command wrote + // (W_USER_CONSTRAINT_CONFLICT), a failing uv sync is that conflict surfacing. + // Report the distinct E_PROVISION_CONFLICT so a caller can act on it — relax + // the written pins and retry — instead of the generic E_PROVISION every other + // sync failure (build backend, permissions, transport, an unavailable package) + // keeps. Gating on the merge signal rather than uv's stderr keeps the code + // precise: it fires only when a conflict the CLI itself detected is present. + if p.hasConstraintConflictWarning() { + pe.Code = ErrProvisionConflict + } + return p.fail(PhaseProvision, true, pe) } if err := p.PM.PostProvision(ctx, p.ProjectDir); err != nil { return p.fail(PhaseProvision, true, asPipelineError(err, ErrProvision, "post-provision failed")) @@ -558,6 +570,19 @@ func (p *Pipeline) provision(ctx context.Context, pyMinor string) error { return nil } +// hasConstraintConflictWarning reports whether the merge phase recorded a provable +// user/environment version conflict (W_USER_CONSTRAINT_CONFLICT). It reads the +// warnings already accumulated on the Result, which the merge phase populates +// before provision runs, so it is only meaningful once merge has completed. +func (p *Pipeline) hasConstraintConflictWarning() bool { + for _, w := range p.res.Warnings { + if w.Code == WarnUserConstraintConflict { + return true + } + } + return false +} + // validate reads the Python and databricks-connect versions from the venv and // populates the venv path. dbcPin is "" in constraints-only mode, where the DB // Connect assertion is skipped. diff --git a/libs/localenv/pipeline_test.go b/libs/localenv/pipeline_test.go index 32b2735f9e4..82ba91b9109 100644 --- a/libs/localenv/pipeline_test.go +++ b/libs/localenv/pipeline_test.go @@ -97,6 +97,16 @@ func (c cancelPM) Provision(ctx context.Context, _, _ string) error { }, "uv sync") } +// provisionFailPM fails `uv sync` with the given error, so a test can drive the +// provision phase to failure without a real uv. EnsurePython succeeds first, as it +// does in a real run before sync. Validate is never reached. +type provisionFailPM struct { + fakePM + err error +} + +func (p provisionFailPM) Provision(context.Context, string, string) error { return p.err } + func writeProject(t *testing.T) string { dir := t.TempDir() require.NoError(t, os.WriteFile(filepath.Join(dir, "pyproject.toml"), []byte(`[project] @@ -432,6 +442,88 @@ func TestPipelineProvisionsAndValidatesExisting(t *testing.T) { assert.FileExists(t, filepath.Join(dir, "pyproject.toml.bak")) } +// syncFailure builds the *PipelineError a failing uv sync returns (uvFailure with +// ErrProvision), so the provision tests exercise the same shape the real +// uvManager produces. +func syncFailure(stderr string) error { + return uvFailure(ErrProvision, &process.ProcessError{ + Command: "uv sync", + Err: errors.New("exit status 1"), + Stderr: stderr, + }, "uv sync") +} + +func TestPipelineProvisionConflictWhenMergeFlagsConflict(t *testing.T) { + // The user pins pip==24.0 while the environment's constraint-dependencies pin + // pip<24 — a provably disjoint range — so the merge records + // W_USER_CONSTRAINT_CONFLICT. A failing uv sync in that state is that conflict + // surfacing, so it is reported as E_PROVISION_CONFLICT (not generic E_PROVISION), + // at the provision phase, with disk already mutated by the merge. + dir := t.TempDir() + require.NoError(t, os.WriteFile(filepath.Join(dir, "pyproject.toml"), []byte(`[project] +name = "demo" +requires-python = ">=3.12" +dependencies = ["pip==24.0"] + +[dependency-groups] +dev = ["databricks-connect~=16.0.0"] +`), 0o644)) + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + _, _ = w.Write([]byte(`[project] +requires-python = ">=3.12" + +[dependency-groups] +dev = ["databricks-connect~=17.2.0"] + +[tool.uv] +constraint-dependencies = ["pip<24"] +`)) + })) + defer srv.Close() + + p := &Pipeline{ + Mode: ModeDefault, ProjectDir: dir, + ConstraintBaseURL: srv.URL, CacheDir: t.TempDir(), + Flags: ComputeFlags{Serverless: "v4"}, + Compute: stubCompute{}, PM: provisionFailPM{err: syncFailure("No solution found when resolving dependencies")}, + } + res, err := p.Run(t.Context()) + var pe *PipelineError + require.ErrorAs(t, err, &pe) + assert.Equal(t, ErrProvisionConflict, pe.Code) + assert.Equal(t, PhaseProvision, pe.FailurePhase) + assert.True(t, pe.DiskMutated, "the merge wrote the pins before provision failed") + require.NotNil(t, res.Error) + assert.Equal(t, ErrProvisionConflict, res.Error.Code) + // The merge conflict warning that gates the code must be present. + assert.Contains(t, codes(res.Warnings), WarnUserConstraintConflict) +} + +func TestPipelineProvisionGenericFailureWithoutConflictWarning(t *testing.T) { + // writeProject declares no dependency that conflicts with the environment + // constraints, so the merge records no W_USER_CONSTRAINT_CONFLICT. A failing uv + // sync — even one whose stderr looks like a resolver error — is then a generic + // E_PROVISION: the code is gated on the CLI's own conflict detection, not on uv + // output. + dir := writeProject(t) + srv := newTestServer(t) + defer srv.Close() + + p := &Pipeline{ + Mode: ModeDefault, ProjectDir: dir, + ConstraintBaseURL: srv.URL, CacheDir: t.TempDir(), + Flags: ComputeFlags{Serverless: "v4"}, + Compute: stubCompute{}, PM: provisionFailPM{err: syncFailure("No solution found when resolving dependencies")}, + } + res, err := p.Run(t.Context()) + var pe *PipelineError + require.ErrorAs(t, err, &pe) + assert.Equal(t, ErrProvision, pe.Code) + assert.Equal(t, PhaseProvision, pe.FailurePhase) + assert.True(t, pe.DiskMutated) + assert.NotContains(t, codes(res.Warnings), WarnUserConstraintConflict) +} + func TestPipelineDryRunOmitsFabricatedDBConnectVersion(t *testing.T) { // A major-only pin like ~=17.0 (serverless, environments#15) is not a concrete // version. Under --dry-run validate never corrects the reported value, so it diff --git a/libs/localenv/result.go b/libs/localenv/result.go index 5962bb0ffbe..75c73d1432b 100644 --- a/libs/localenv/result.go +++ b/libs/localenv/result.go @@ -104,7 +104,7 @@ const ( ErrMerge ErrorCode = "E_MERGE" // merge: existing-project merge failed ErrPythonInstall ErrorCode = "E_PYTHON_INSTALL" // provision: uv python install failed ErrProvision ErrorCode = "E_PROVISION" // provision: uv sync failed (generic) - ErrProvisionConflict ErrorCode = "E_PROVISION_CONFLICT" // provision: uv sync could not resolve dependencies + ErrProvisionConflict ErrorCode = "E_PROVISION_CONFLICT" // provision: uv sync failed on a merge-detected version conflict ErrValidate ErrorCode = "E_VALIDATE" // validate: post-provision version mismatch // ErrCanceled is not in the spec's error-code table: it reports a user/parent diff --git a/libs/localenv/uv.go b/libs/localenv/uv.go index 4be5a123f70..7e4ca82c746 100644 --- a/libs/localenv/uv.go +++ b/libs/localenv/uv.go @@ -104,39 +104,11 @@ func (m *uvManager) EnsurePython(ctx context.Context, minor string) error { func (m *uvManager) Provision(ctx context.Context, projectDir, pyMinor string) error { args := append([]string{m.bin}, m.syncArgs(pyMinor)...) if err := m.runUv(ctx, args, projectDir); err != nil { - // A resolution failure (the remote pins just written can't be satisfied - // against the user's local dependencies) gets a distinct code so a caller can - // react to it — relax the written constraints / requires-python and retry — - // rather than treat it like any other uv sync failure (build backend, - // permissions, transport), which E_PROVISION still covers. - code := ErrProvision - if isUvResolutionConflict(err) { - code = ErrProvisionConflict - } - return uvFailure(code, err, "uv sync") + return uvFailure(ErrProvision, err, "uv sync") } return nil } -// uvResolverConflictMarker is the stable header uv prints to stderr whenever -// `uv sync` cannot satisfy the project's requirements. Every unsatisfiable -// resolution surfaces under it — a disjoint version pin, an incompatible -// requires-python, or an unavailable dependency — and uv exposes no exit code or -// machine-readable field distinguishing them, so this marker classifies the whole -// resolution-failure class. That is exactly the class a caller can act on, so the -// coarse match is the right granularity, not a limitation. Matching uv's own -// output is the only classification uv offers; there is no typed error to compare. -const uvResolverConflictMarker = "No solution found when resolving dependencies" - -// isUvResolutionConflict reports whether a failed uv invocation failed because uv -// could not resolve dependencies (see uvResolverConflictMarker). It reads the -// process stderr; an error that is not a *process.ProcessError, or one whose -// stderr lacks the marker, is not a resolution conflict. -func isUvResolutionConflict(err error) bool { - perr, ok := errors.AsType[*process.ProcessError](err) - return ok && strings.Contains(perr.Stderr, uvResolverConflictMarker) -} - // venvPython returns the path to the virtualenv's Python interpreter, // accounting for the Windows (Scripts/python.exe) vs Unix (bin/python) layout. func venvPython(projectDir string) string { diff --git a/libs/localenv/uv_test.go b/libs/localenv/uv_test.go index d229f3ec7fe..8fcce4376a7 100644 --- a/libs/localenv/uv_test.go +++ b/libs/localenv/uv_test.go @@ -236,34 +236,6 @@ func TestUvFailureIncludesStderr(t *testing.T) { }) } -func TestIsUvResolutionConflict(t *testing.T) { - t.Run("resolver_no_solution_stderr_is_conflict", func(t *testing.T) { - // uv prints this banner for any unsatisfiable resolution — a disjoint version - // pin, an incompatible requires-python, or an unavailable dependency. - err := &process.ProcessError{ - Command: "uv sync", - Err: errors.New("exit status 1"), - Stderr: " × No solution found when resolving dependencies:\n ╰─▶ Because your project depends on pip==24.0 and pip<24, we can conclude that your project's requirements are unsatisfiable.\n", - } - assert.True(t, isUvResolutionConflict(err)) - }) - - t.Run("generic_failure_stderr_is_not_conflict", func(t *testing.T) { - // A transport error is a generic provision failure, not a resolution conflict. - err := &process.ProcessError{ - Command: "uv sync", - Err: errors.New("exit status 2"), - Stderr: "error: Connection refused\n", - } - assert.False(t, isUvResolutionConflict(err)) - }) - - t.Run("non_process_error_is_not_conflict", func(t *testing.T) { - // Without a process error there is no stderr to classify. - assert.False(t, isUvResolutionConflict(errors.New("some other error"))) - }) -} - func TestConfirmUvInstall(t *testing.T) { t.Run("opt_in_env_var_consents_without_prompt", func(t *testing.T) { // Non-interactive context, but the opt-in env var grants consent. From f0d3bbcffc914ca798b6b43c490e827b8282c2ac Mon Sep 17 00:00:00 2001 From: Grigory Panov Date: Wed, 2 Sep 2026 10:35:45 +0200 Subject: [PATCH 4/4] Review: document the co-occurring-failure tradeoff in the conflict gate MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Make explicit in provision() that E_PROVISION_CONFLICT gating on the merge's W_USER_CONSTRAINT_CONFLICT signal can coincide with an unrelated sync failure — and that this is intentional: the warning proves a real conflict exists, so relaxing the pins is a necessary step regardless, and uv stays the source of truth for whether sync fails while the CLI's own detection classifies why. No behavior change. Raised in review (Codex/Claude), kept as-is per decision. Co-authored-by: Isaac --- libs/localenv/pipeline.go | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/libs/localenv/pipeline.go b/libs/localenv/pipeline.go index 333490223e3..289b44acede 100644 --- a/libs/localenv/pipeline.go +++ b/libs/localenv/pipeline.go @@ -558,6 +558,14 @@ func (p *Pipeline) provision(ctx context.Context, pyMinor string) error { // sync failure (build backend, permissions, transport, an unavailable package) // keeps. Gating on the merge signal rather than uv's stderr keeps the code // precise: it fires only when a conflict the CLI itself detected is present. + // + // The warning means the merged pins are provably unsatisfiable, so a real + // conflict exists to fix regardless of what uv reported. If an unrelated + // failure (e.g. a transport error) happens to surface first, the code still + // truthfully says a conflict is present — relaxing the pins is then a + // necessary step, even if not the whole fix. We deliberately do not also parse + // uv's stderr to disambiguate: uv is left as the source of truth for whether + // sync fails, and the CLI's own detection classifies why. if p.hasConstraintConflictWarning() { pe.Code = ErrProvisionConflict }