[eng-dev-ecosystem-bot]

Update Databricks CLI to v1.24.0

[github-actions]

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/vscode

Inputs:

• PR number: 2286
• Commit SHA: e80daff3a026bea2b69ccb780e2718ccb5825dea

Checks will be approved automatically on success.

[rugpanov]

Executive summary — Databricks CLI v1.23.0 → v1.24.0

Notable / behavior change

• Variables that hold secrets are now sensitive. A variable used as a secret's value is sensitive by default. If any non-sensitive field references it (for example a job parameter or a secret's comment), you get an error. You can mark other variables with sensitive: true. (#7288)

CLI

• auth login / configure: saving a dotted profile such as team.prod no longer overwrites the host and token of an existing [team] profile. (#7266)
• auth login / auth token: after "Create a new profile", the name of an existing profile is now rejected. (#7196)
• auth login / auth token: a host with no hostname (/, ?o=123, https://) is now rejected. An empty manual host no longer starts discovery login. (#7195)
• Prompts that validate input no longer show a red cross before you type. (#7314)
• ssh connect: --environment-version is now documented. Each version gets its own default-named server, and the flag cannot be used with --cluster. The default connection name changes even without the flag, so old local .databricks SSH state (keys, known hosts, secret scope) is orphaned. (#7310)

AI Runtime (air)

• air run -h config.compute.accelerator_type now shows the accelerator types that the workspace supports. (#7275)
• An empty air list now prints No active runs found. (#7276)
• air run --watch no longer prints a retry notice on every run. (#7270)

Bundles

• New sensitive: true for variables. bundle validate -o json now redacts only sensitive variables, not every string variable. (#7288)
• New resources:
  • data_quality_monitors (direct engine only). (#7094)
  • materialization_sets (Feature Store). (#7164)
• bundle deploy now asks for confirmation before it recreates or deletes a Feature View. deploy and destroy warn that FeatureSpecs using a Feature View are not checked. (#7164)
• Fix: bundle plan / bundle deploy no longer show an app config.env diff and redeploy the app on every run. (#7326)
• Grant privileges values that the SDK enum does not list yet are now accepted. validate no longer warns, and --strict no longer errors. (#7315)
• Notebook detection now uses the workspace rule: the first line contains Databricks notebook source (for example after a BOM). Before, removing such a file locally left the remote notebook behind. (#7290)

Dependency updates

• Two bumps for dependencies with known vulnerabilities. (#7305, #7327)
• Go SDK updated to v0.191.1 (#7277), then to v0.193.0 (#7303).

What could be integrated into the VS Code extension experience

1. Bundle Variables view: values change under the new redaction rule (cleanup → small feature)

• How the view gets its values. The extension runs bundle validate with DATABRICKS_OUTPUT_FORMAT=json (packages/databricks-vscode/src/cli/CliWrapper.ts:1041, packages/databricks-vscode/src/utils/envVarGenerators.ts:183). It parses stdout in packages/databricks-vscode/src/bundle/models/BundleValidateModel.ts:93-103.
• Where a redacted value shows up. BundleVariableModel sets valueInTarget from validateConfig.variables[key].value (packages/databricks-vscode/src/bundle/models/BundleVariableModel.ts:174-207). This applies when the variable is not a literal in the target and is not "required". The Variables tree then shows that string as the item description (packages/databricks-vscode/src/ui/bundle-variables/VariableTreeNode.ts:13-20,43).
• What changes after the bump.
  • Non-sensitive string variables now show their resolved value instead of a redacted placeholder. This is an improvement that needs no code change.
  • Sensitive variables still show the placeholder text as if it were the value.
• Opportunity.
  1. Read sensitive from the definition and show a lock icon or a "sensitive" label instead of the raw placeholder.
  2. Warn when a user types an override for a sensitive variable. Overrides go in plain text to .databricks/bundle/<target>/vscode.bundlevars.json (BundleVariableModel.ts:116-123).
• Effort. Small feature, but it needs one prerequisite. The static BundleSchema.ts types are stale: they have no sensitive key, and apps is still typed as any at packages/databricks-vscode/src/ui/bundle-resource-explorer/types.ts:43. Regenerate them first with yarn package:bundle-schema:write (packages/databricks-vscode/package.json:1994).

2. Resource Explorer icons for data_quality_monitors / materialization_sets (near flag-flip)

• Current behavior. A type that is not in KNOWN_ICON_RESOURCE_TYPES (types.ts:40-53) falls back to a generic folder icon (packages/databricks-vscode/src/ui/bundle-resource-explorer/ResourceTypeHeaderTreeNode.ts:49-50). Icons load from resources/{dark,light}/resource-explorer/<type>.svg (ResourceTypeHeaderTreeNode.ts:59-67).
• Opportunity. quality_monitors.svg already exists, so data_quality_monitors needs only a list entry plus copied SVGs. materialization_sets needs one new icon.
• Effort. Small cleanup.
• Out of scope. Rich tree nodes (like the job and pipeline nodes behind KNOWN_RESOURCE_TYPES, types.ts:36-39) would be net-new work. Nothing in this release asks for them.

3. SSH: serverless environment version (existing plumbing + net-new picker UI), and a release-note item

• Current behavior. getSshConnectCommand (packages/databricks-vscode/src/cli/CliWrapper.ts:152-166) builds ssh connect --ide=… --auto-approve. It adds --cluster or --accelerator, and nothing else. The picker offers plain serverless and two GPU accelerators (packages/databricks-vscode/src/ssh/SshCommands.ts:47-63).
• Opportunity. Add an optional environmentVersion to the serverless variant of SshConnectCompute (CliWrapper.ts:184-186). Then push --environment-version in the serverless branch. The existing if/else already keeps it away from --cluster, which the CLI now rejects.
• Effort. The arg change is trivial. A version picker step in SshCommands.ts is net-new UI.
• Must call out now (no code change). The extension never passes a connection name, so it uses the CLI default. After this bump, the first serverless connect from the extension creates a new default-named server and new local key/known-hosts state. Old entries under .databricks are orphaned. This is worth a line in the extension's release notes.

4. Feature View deploy confirmation vs. non-interactive deploy (existing gap, wider in this release)

• Current behavior.
  • Extension deploys run bundle deploy --target <t> --verbose [--force-lock --force] with no --auto-approve (CliWrapper.ts:1211-1216). No code handles a CLI confirmation request (no approve / recreat match outside ssh connect and destroy).
  • bundle destroy already passes --auto-approve (CliWrapper.ts:1258). There, the new FeatureSpec warning only goes to the bundle logs channel.
• Opportunity. Deploys that recreate or delete a Feature View will now hit the CLI's confirmation path inside the extension, like other destructive changes already do. A "review destructive changes → Approve" flow would be net-new feature work. It could detect the confirmation error, show a modal, and re-run with --auto-approve. It would cover every destructive-change prompt, not only Feature Views.
• Not verified. I did not check here how the CLI's non-TTY confirmation error appears in ProcessError.showErrorMessage (packages/databricks-vscode/src/ui/bundle-resource-explorer/BundleCommands.ts:159-160). Check that before you scope this work.

5. Login profile names with dots (cleanup candidate, not a flag-flip)

• Current behavior. The new-profile input rejects names that contain . (packages/databricks-vscode/src/configuration/LoginWizard.ts:278-283). The CLI fix (#7266) removes one reason for this rule.
• Why it is not a flag-flip. The wizard writes the profile with ini.stringify + appendFile (LoginWizard.ts:410-430), not through the CLI. That writer has its own dotted-section behavior. Lifting the rule means changing or checking that writer first.

No-ops for the extension (transparent on bump)

• #7196 / #7195 (auth login validation). The extension always passes a real --host from authProvider.host, plus --profile when set (packages/databricks-vscode/src/configuration/auth/DatabricksCliCheck.ts:138-152). It never uses the interactive "Create a new profile" or "Enter a host URL manually" pickers.
• #7314 (prompt red cross). Only interactive TTY prompts change. The extension runs the CLI non-interactively.
• #7326 (app config.env diff). This is a pure win: extension deploys of apps stop redeploying every time. The extension does not call bundle plan.
• #7315 (unknown grant privileges). Transparent. YAML completion comes from the runtime databricks bundle schema (CliWrapper.ts:802-808, packages/databricks-vscode/src/bundle/bundleAutocompleteProvider.ts:13). The new resources and sensitive therefore also appear in completion with no change.
• #7290 (notebook detection). CLI sync gets this automatically. The extension has its own detector, isNotebook, for run/init-script handling (packages/databricks-vscode/src/utils/fileUtils.ts:12-38).
  • It already removes a leading BOM, because TextDecoder does that by default.
  • It uses startsWith rather than "contains", so it is now slightly stricter than the CLI. Switching to includes would be an optional one-line cleanup.
• AI Runtime (#7275, #7276, #7270). The extension does not call databricks air.
• Dependency and Go SDK bumps (#7305, #7327, #7277, #7303). No extension-facing surface changes.

_{🤖 Auto-generated executive summary of the CLI v1.23.0 → v1.24.0 changelog. Integration notes are opportunities, not commitments — verify before acting.}