Update Databricks CLI to v1.8.0

[eng-dev-ecosystem-bot]

Update Databricks CLI to v1.8.0

[github-actions]

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/vscode

Inputs:

• PR number: 2017
• Commit SHA: 4a148152e94798914265f537d2adce8f72fb8bc9

Checks will be approved automatically on success.

[rugpanov]

CLI 1.8.0 vs 1.7.0 — what's new (and what's interesting for the extension)

This bumps the bundled CLI straight from 1.7.0 → 1.8.0. There were no 1.7.x patch releases in between, so this is exactly the v1.8.0 changelog. Highlights below, ranked by how much they touch the extension.

🔴 Most relevant to the extension

1. Auto-migrate terraform → direct engine on deploy (the headline "Notable" change)

Auto-migrate a bundle from terraform to the direct engine when bundle.engine is "direct" (or DATABRICKS_BUNDLE_ENGINE=direct) and the post-deploy dry-run migration is clean; a warning is emitted if the dry-run surfaces errors or warnings so the automatic migration is skipped.

The extension shells out to bundle deploy (CliWrapper.bundleDeploy), so any project whose databricks.yml sets engine: direct will now actually migrate its state on deploy through the extension — where 1.6/1.7 only dry-ran the migration and reported it via telemetry. It's opt-in (gated on engine: direct), but ⚠️ worth a backward-compat sanity check: confirm the extension's deploy path — and our terraform-state regeneration in bundleSummarise — still behaves once a project auto-migrates off terraform.

2. auth describe no longer mislabels a DATABRICKS_CONFIG_PROFILE profile as (from bundle) inside a bundle root (#5904)

The extension always injects DATABRICKS_CONFIG_PROFILE (AuthProvider) and runs inside bundle roots — precisely the scenario this fixes. Anywhere we surface or reason about auth-describe output, profile attribution is now correct.

🟡 Nice-to-have

3. bundle init template-URL improvements — recognizes ssh:// template URLs (#5891) and returns an actionable error for unsupported protocols (http://, git://, ftp(s)://) instead of a confusing "not a bundle template" message (#5902). Our BundleInitWizard runs bundle init in a terminal, so users get clearer failures and one more supported scheme.

4. New env: section under scripts.<name> for env vars that reference ${bundle.*} / ${workspace.*} / ${var.*} (#5299). New bundle-schema surface — it'll show up in the extension's YAML completions once BundleSchema is regenerated from 1.8.

5. bundle generate warns when the generated file isn't matched by any include: pattern (#5868) — nice guardrail for users generating resources from the extension.

⚪ Probably not relevant here

• Experimental ssh connect interpreter/shell changes (#5888) — the extension doesn't use ssh connect.
• The new "run databricks aitools install" hint on stderr is gated to Claude Code callers only and never fires for human callers, so it won't pollute the stderr we collect in waitForProcess.
• Assorted direct-engine drift/deploy fixes for Lakebase / UC / model-serving resources — good for reliability, but not extension-specific.