Repository navigation
Use .workspace.URL for the host in built-in templates - #7350
Merged
Merged
Conversation
pavloKozlov
force-pushed
the
workspace-context-builtin-templates
branch
from
November 4, 2026 15:30
abf32b2 to
d6dcf5e
Compare
Contributor
|
Commit: Regression test reportSelected 3 changed tests (limit=20, 0 not selected) 4 tests: regression=4 unreleased=0 coverage=0 failing=0
TestAccept/bundle/templates-machinery/workspace_variable/runtime/DMS=: branch ✅ | main (663e1f7) ❌ | latest (v1.24.0) ❌main (663e1f7): latest (v1.24.0): TestAccept/bundle/templates-machinery/workspace_variable/runtime/DMS=true: branch ✅ | main (663e1f7) ❌ | latest (v1.24.0) ❌main (663e1f7): latest (v1.24.0): TestAccept/bundle/templates/dbt-sql-workspace-id/DMS=: branch ✅ | main (663e1f7) ❌ | latest (v1.24.0) ❌main (663e1f7): latest (v1.24.0): TestAccept/bundle/templates/default-minimal/profiles/DMS=: branch ✅ | main (663e1f7) ❌ | latest (v1.24.0) ❌main (663e1f7): latest (v1.24.0): |
pavloKozlov
force-pushed
the
workspace-context-template-data
branch
from
November 4, 2026 15:41
b931f85 to
5f2f6db
Compare
pavloKozlov
force-pushed
the
workspace-context-builtin-templates
branch
from
November 4, 2026 15:41
d6dcf5e to
e702534
Compare
Collaborator
✅ Integration test reportCommit: 684c444
|
pavloKozlov
force-pushed
the
workspace-context-builtin-templates
branch
4 times, most recently
from
November 5, 2026 14:04
93ee68b to
494b311
Compare
pavloKozlov
marked this pull request as ready for review
November 5, 2026 14:06
denik
approved these changes
Nov 6, 2026
andrewnester
reviewed
Nov 6, 2026
| default: {{.http_path}} | ||
| {{- /* dbt selects the workspace on a host shared by several workspaces with ?o=<id> in the | ||
| HTTP path. Use the ID from the workspace URL, which has it when the profile sets it. */}} | ||
| {{- $id := (regexp `[?&]w=([^&#]+)`).FindStringSubmatch .workspace.URL}} |
Contributor
There was a problem hiding this comment.
Parsing the ID back out of the URL we just rendered feels roundabout: workspaceContext already knows which ID URL adds (urlWorkspaceID in libs/template/workspace_context.go:51). Can we expose that on .workspace instead of re-deriving it with a regex here? Could go into #7349. I assume .workspace.ID doesn't fit because it also returns the host-metadata ID?
Contributor
Author
There was a problem hiding this comment.
Agree. Switched to .workspace.URLID.
| @@ -0,0 +1,33 @@ | |||
| # The test server is reachable as both 127.0.0.1 and localhost. The second | |||
| # stands in for a host shared by several workspaces. | |||
| CUSTOM_HOST=${DATABRICKS_HOST/127.0.0.1/localhost} | |||
Contributor
There was a problem hiding this comment.
This setup (localhost swap, .well-known/databricks-config, hand-written profiles) is now the fourth copy after acceptance/bundle/strict_profile_match/script.prepare and acceptance/bundle/templates-machinery/workspace_variable/script.prepare, and dbt-sql-workspace-id/script repeats it again. Maybe it should be a helper in acceptance/bin instead?
| # when the profile sets the ID, like ?w= in the bundle host. | ||
| for profile in no_workspace_id custom_host; do | ||
| title "Profile $profile\n" | ||
| $CLI bundle init dbt-sql --config-file ./input.json --profile $profile > /dev/null 2>&1 |
Contributor
There was a problem hiding this comment.
nit: why hide the init output here? If init fails we only see grep failing on a missing file, and the default-minimal test traces it.
Contributor
Author
There was a problem hiding this comment.
Good catch, it traces bundle init now
- Add .workspace.URLID: the workspace ID that .workspace.URL adds as ?w=, or empty. Unlike .workspace.ID, it doesn't fail when the ID is unknown and doesn't return an ID from host metadata. - Schema rendering keeps the default of an input that has a value when the default fails to render, instead of failing. bundle init doesn't use that default, so the same input works there. - Add setup_custom_host to acceptance/script.prepare for the test server reached as a host shared by several workspaces, and use it in workspace_variable. - The runtime test covers URLID per profile, a failing default of an input with a value, the w and workspace_id selectors and a protocol-relative path. - Doc comments name where URLID comes from, and the missing ID error names the bundle's workspace_id. Co-authored-by: Isaac <no-reply@databricks.com>
Built-in templates, except dbt-sql, write the workspace URL to workspace.host. It has ?w=<id> when the profile or an environment variable sets the workspace ID. Other profiles keep the same host as before. The value is quoted so the YAML stays valid for any URL. Template schemas keep workspace_host. Co-authored-by: Isaac <no-reply@databricks.com>
The welcome messages showed workspace_host, which has no workspace ID. They now show the URL that the bundle uses, so they match workspace.host in the generated databricks.yml. Messages that printed "workspace_host:" now print "workspace_url:". Co-authored-by: Isaac <no-reply@databricks.com>
The bundle host now comes from .workspace.URL like in the other built-in templates. dbt jobs get their host from Databricks, so they are not affected. The dbt profile default uses .workspace.Hostname, which gives the same value as stripping the scheme from workspace_host. No built-in template uses workspace_host now. Co-authored-by: Isaac <no-reply@databricks.com>
The dbt profile default no longer has the port of the test server. Databricks hosts don't have an explicit port, so for them the default is the same as before. Co-authored-by: Isaac <no-reply@databricks.com>
The host is no longer quoted. A URL with ?w= is a valid plain YAML value, and the quotes were only needed for test outputs, which no longer have ?w=. Welcome messages keep their "workspace_host:" label. The ?w= targets in strict_profile_match/host and the extra-workspace-input test are removed: the profiles test and the input tests in templates-machinery cover the same cases. Template outputs are now the same as before for auth without a workspace ID. Co-authored-by: Isaac <no-reply@databricks.com>
The bundle host comes from .workspace.URL like in the other built-in templates. Deployed dbt jobs get their host from Databricks, so they don't read it. The HTTP path suggested for dbt init gets ?o=<id> when the workspace URL has the ID, because dbt selects the workspace on a host shared by several workspaces with ?o= in the HTTP path. The host suggested for dbt init is .workspace.Hostname. Co-authored-by: Isaac <no-reply@databricks.com>
The dbt-sql profile default for dbt init is host:port again, so its output is the same as before. Co-authored-by: Isaac <no-reply@databricks.com>
pavloKozlov
force-pushed
the
workspace-context-builtin-templates
branch
from
November 6, 2026 14:14
850ed27 to
684b87b
Compare
pavloKozlov
changed the base branch from
main
to
workspace-context-review-fixes
November 6, 2026 14:14
dbt-sql adds ?o= from .workspace.URLID, escaped, instead of parsing the rendered URL with a regex. The profiles and dbt-sql-workspace-id tests use setup_custom_host instead of their own copy of that setup, and the dbt-sql-workspace-id test traces bundle init so a failed init shows its error. Co-authored-by: Isaac <no-reply@databricks.com>
pavloKozlov
force-pushed
the
workspace-context-builtin-templates
branch
from
November 6, 2026 14:18
684b87b to
684c444
Compare
andrewnester
approved these changes
Nov 9, 2026
deco-sdk-tagging Bot
added a commit
that referenced
this pull request
Nov 9, 2026
## Release v1.25.0 ### CLI * `databricks aitools install --output json` now reports a more specific `error_category` (`MANIFEST_NOT_FOUND`, `PERMISSION_DENIED`, `FILESYSTEM`) for some failures it reported as `UNCATEGORIZED`. ([#7380](#7380)) * Suggest `databricks auth logout --profile <profile-name> --delete` when multiple profiles match the same workspace in `databricks auth docker` commands. ([#7363](#7363)) * Fix `%pip` in Jupyter notebooks opened over `databricks ssh connect` on serverless environment versions 5 and 6 and on dedicated clusters running recent Databricks Runtime versions, which fell back to the stock IPython magic instead of the Databricks one. ([#7263](#7263)) ### AI Runtime * `databricks air run` now validates the config server-side while it uploads the run artifacts, instead of before, which shortens submission. ([#7368](#7368)) * Remove `CRITICAL` as a valid `compute.priority_class` value; the AI Runtime API no longer accepts it. ([#7371](#7371)) ### Bundles * Built-in templates add the workspace ID to `workspace.host` (`?w=<id>`) when the profile or an environment variable sets it, so generated bundles select the right workspace on hosts shared by several workspaces. Welcome messages show the same URL, and `dbt-sql` also adds `?o=<id>` to the HTTP path it suggests for `dbt init`. ([#7350](#7350)) * Fix grants of a renamed or recreated catalog, schema, volume or other UC resource staying recorded under the old name, which made the next `bundle plan` propose to apply them again. ([#7345](#7345)) * Fix permissions of a recreated pipeline, app, dashboard, model serving endpoint or other resource staying recorded under the old ID, which made every later `bundle plan` propose to apply them again. ([#7361](#7361)) * Add the `postgres_data_apis` resource (Private Preview) for enabling and configuring the Data API of a Lakebase Postgres database. ([#7160](#7160)) * Fixes [#4012](#4012): registered model `aliases` are now set in Unity Catalog on deploy instead of being silently ignored. A model without an `aliases` section keeps the aliases set outside the bundle (for example by MLflow), and `aliases: []` removes them. ([#7163](#7163)) * Include variable declarations and the source of each resolved value in the `--include-locations` output of `bundle validate` and `bundle summary`. ([#7351](#7351)) * Add `.workspace.URLID` to bundle templates: the workspace ID that `.workspace.URL` adds, or empty, so templates can use it without failing when the ID is unknown. ([#7388](#7388)) * Expose the workspace of the selected profile to bundle templates as `.workspace`, with `Hostname`, `ID`, and `URL`, so templates can address a workspace on a host shared by several workspaces. A template input named `workspace` takes precedence. ([#7349](#7349)) ### Dependency Updates * Bump dependencies with known vulnerabilities. ([#7394](#7394)) * Upgrade `github.com/mattn/go-runewidth` to v0.0.31, which removes ~30ms of startup time from every command. ([#7391](#7391)) * Update the Go SDK to [v0.194.0](https://github.com/databricks/databricks-sdk-go/releases/tag/v0.194.0). ([#7371](#7371)) * Update the Go SDK to [v0.195.0](https://github.com/databricks/databricks-sdk-go/releases/tag/v0.195.0). ([#7384](#7384))
Collaborator
❌ Integration test reportCommit: b61373f
37 interesting tests: 37 FAIL
Top 50 slowest tests (at least 2 minutes):
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #7388.
Changes
Built-in templates write
host: {{.workspace.URL}}instead of{{workspace_host}}. With a profile that setsworkspace_id, the host gets?w=<id>. Other profiles get the same host as before. Welcome messages show.workspace.URL. Indbt-sql, the HTTP path suggested fordbt initgets?o=<id>from.workspace.URLIDin the same case, because dbt reads the workspace fromo.Why
On a host shared by several workspaces, the bundle needs the workspace ID in the host to deploy to the right workspace, and local dbt needs
?o=in the HTTP path to connect. Deployed dbt jobs get their host from Databricks and connect without?o=, so their profile doesn't change.Tests
New acceptance tests create bundles with a profile without
workspace_idand with one that has it. One validates with strict profile matching and deploys, the other checks thedbt-sqlhost and HTTP path. Other template outputs don't change. I also connected with dbt-databricks 1.12 and 1.13 to a workspace through a shared host and through its own host, with and without?o=.This pull request and its description were written by Isaac.