Map a project folder after -e in environment dependencies to its deployed folder by tharunbhuk-db · Pull Request #7427 · databricks/cli

tharunbhuk-db · 2026-11-10T00:23:45Z

🥞 Stacked PR

stack/pipelines-test-b1-editable-folder [Files changed] ← this PR
- stack/pipelines-test-pr4-bundles [Files changed]
  - stack/pipelines-test-b3-testing-paths [Files changed]
    - stack/pipelines-test-b4-testing-deploy [Files changed]
      - stack/pipelines-test-j1-testing-sources [Files changed]
        
        stack/pipelines-test-changelog [Files changed]

With DATABRICKS_BUNDLE_ENABLE_EXPERIMENTAL_PIPELINE_TESTING=true, a local project folder after -e, such as -e ./project[test], maps to its deployed folder. This applies to job and pipeline environment dependencies.
With the variable off, the behavior is the same as on main: a folder after -e is an error.
Adds the DATABRICKS_BUNDLE_ENABLE_EXPERIMENTAL_PIPELINE_TESTING variable, which the next PRs also use.

Why: test dependencies use the project's test extra, such as -e ./project[test].

Tests: TestTranslatePathEditableProjectFolder (variable on and off); acceptance bundle/environments/editable-folder (variable on and off).

This pull request and its description were written by Isaac.

github-actions · 2026-11-10T00:32:51Z

Commit: 9a2cc3ca6 | Workflow run

Regression test report

Selected 1 changed tests (limit=20, 0 not selected)

2 tests: regression=2 unreleased=0 coverage=0 failing=0

test	branch	main (`688c228`)	latest (v1.25.0)
TestAccept/bundle/environments/editable-folder/DMS=	✅	❌	❌
TestAccept/bundle/environments/editable-folder/DMS=true	✅	❌	❌

TestAccept/bundle/environments/editable-folder/DMS=: branch ✅ | main (688c228) ❌ | latest (v1.25.0) ❌

main (688c228):

acceptance_test.go:1345: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1345
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1086
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:660
        	Error:      	Not equal: 
        	            	expected: "\n=== With pipeline testing, a folder after -e maps to its deployed folder\n\n>>> [CLI] bundle validate -o json\n{\n  \"job\": [\n    \"-e /Workspace/Users/[USERNAME]/.bundle/editable-folder/default/files/project\"\n  ],\n  \"pipeline\": [\n    \"-e /Workspace/Users/[USERNAME]/.bundle/editable-folder/default/files/project[test]\"\n  ]\n}\n\n=== Without pipeline testing, a folder after -e is an error, as before\nError: unable to determine if [TEST_TMP_DIR]/project is not a notebook\n"
        	            	actual  : "\n=== With pipeline testing, a folder after -e maps to its deployed folder\n\n>>> [CLI] bundle validate -o json\nError: unable to determine if [TEST_TMP_DIR]/project is not a notebook: read [TEST_TMP_DIR]/project: is a directory\n\n{\n  \"job\": [\n    \"-e project\"\n  ],\n  \"pipeline\": [\n    \"-e project[test]\"\n  ]\n}\n\nExit code: 1\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -4,8 +4,10 @@
        	            	 >>> [CLI] bundle validate -o json
        	            	+Error: unable to determine if [TEST_TMP_DIR]/project is not a notebook: read [TEST_TMP_DIR]/project: is a directory
        	            	+
        	            	 {
        	            	   "job": [
        	            	-    "-e /Workspace/Users/[USERNAME]/.bundle/editable-folder/default/files/project"
        	            	+    "-e project"
        	            	   ],
        	            	   "pipeline": [
        	            	-    "-e /Workspace/Users/[USERNAME]/.bundle/editable-folder/default/files/project[test]"
        	            	+    "-e project[test]"
        	            	   ]
        	            	@@ -13,4 +15,3 @@
        	            	 
        	            	-=== Without pipeline testing, a folder after -e is an error, as before
        	            	-Error: unable to determine if [TEST_TMP_DIR]/project is not a notebook
        	            	+Exit code: 1
        	            	 
        	Test:       	TestAccept/bundle/environments/editable-folder/DMS=
        	Messages:   	bundle/environments/editable-folder/output.txt vs /tmp/TestAcceptbundleenvironmentseditable-folderDMS=3911196911/002/output.txt

latest (v1.25.0):

acceptance_test.go:1345: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1345
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1086
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:660
        	Error:      	Not equal: 
        	            	expected: "\n=== With pipeline testing, a folder after -e maps to its deployed folder\n\n>>> [CLI] bundle validate -o json\n{\n  \"job\": [\n    \"-e /Workspace/Users/[USERNAME]/.bundle/editable-folder/default/files/project\"\n  ],\n  \"pipeline\": [\n    \"-e /Workspace/Users/[USERNAME]/.bundle/editable-folder/default/files/project[test]\"\n  ]\n}\n\n=== Without pipeline testing, a folder after -e is an error, as before\nError: unable to determine if [TEST_TMP_DIR]/project is not a notebook\n"
        	            	actual  : "\n=== With pipeline testing, a folder after -e maps to its deployed folder\n\n>>> [CLI] bundle validate -o json\nError: unable to determine if [TEST_TMP_DIR]/project is not a notebook: read [TEST_TMP_DIR]/project: is a directory\n\n{\n  \"job\": [\n    \"-e project\"\n  ],\n  \"pipeline\": [\n    \"-e project[test]\"\n  ]\n}\n\nExit code: 1\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -4,8 +4,10 @@
        	            	 >>> [CLI] bundle validate -o json
        	            	+Error: unable to determine if [TEST_TMP_DIR]/project is not a notebook: read [TEST_TMP_DIR]/project: is a directory
        	            	+
        	            	 {
        	            	   "job": [
        	            	-    "-e /Workspace/Users/[USERNAME]/.bundle/editable-folder/default/files/project"
        	            	+    "-e project"
        	            	   ],
        	            	   "pipeline": [
        	            	-    "-e /Workspace/Users/[USERNAME]/.bundle/editable-folder/default/files/project[test]"
        	            	+    "-e project[test]"
        	            	   ]
        	            	@@ -13,4 +15,3 @@
        	            	 
        	            	-=== Without pipeline testing, a folder after -e is an error, as before
        	            	-Error: unable to determine if [TEST_TMP_DIR]/project is not a notebook
        	            	+Exit code: 1
        	            	 
        	Test:       	TestAccept/bundle/environments/editable-folder/DMS=
        	Messages:   	bundle/environments/editable-folder/output.txt vs /tmp/TestAcceptbundleenvironmentseditable-folderDMS=2473631374/002/output.txt
FAIL

TestAccept/bundle/environments/editable-folder/DMS=true: branch ✅ | main (688c228) ❌ | latest (v1.25.0) ❌

main (688c228):

acceptance_test.go:1345: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1345
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1086
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:660
        	Error:      	Not equal: 
        	            	expected: "\n=== With pipeline testing, a folder after -e maps to its deployed folder\n\n>>> [CLI] bundle validate -o json\n{\n  \"job\": [\n    \"-e /Workspace/Users/[USERNAME]/.bundle/editable-folder/default/files/project\"\n  ],\n  \"pipeline\": [\n    \"-e /Workspace/Users/[USERNAME]/.bundle/editable-folder/default/files/project[test]\"\n  ]\n}\n\n=== Without pipeline testing, a folder after -e is an error, as before\nError: unable to determine if [TEST_TMP_DIR]/project is not a notebook\n"
        	            	actual  : "\n=== With pipeline testing, a folder after -e maps to its deployed folder\n\n>>> [CLI] bundle validate -o json\nError: unable to determine if [TEST_TMP_DIR]/project is not a notebook: read [TEST_TMP_DIR]/project: is a directory\n\n{\n  \"job\": [\n    \"-e project\"\n  ],\n  \"pipeline\": [\n    \"-e project[test]\"\n  ]\n}\n\nExit code: 1\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -4,8 +4,10 @@
        	            	 >>> [CLI] bundle validate -o json
        	            	+Error: unable to determine if [TEST_TMP_DIR]/project is not a notebook: read [TEST_TMP_DIR]/project: is a directory
        	            	+
        	            	 {
        	            	   "job": [
        	            	-    "-e /Workspace/Users/[USERNAME]/.bundle/editable-folder/default/files/project"
        	            	+    "-e project"
        	            	   ],
        	            	   "pipeline": [
        	            	-    "-e /Workspace/Users/[USERNAME]/.bundle/editable-folder/default/files/project[test]"
        	            	+    "-e project[test]"
        	            	   ]
        	            	@@ -13,4 +15,3 @@
        	            	 
        	            	-=== Without pipeline testing, a folder after -e is an error, as before
        	            	-Error: unable to determine if [TEST_TMP_DIR]/project is not a notebook
        	            	+Exit code: 1
        	            	 
        	Test:       	TestAccept/bundle/environments/editable-folder/DMS=true
        	Messages:   	bundle/environments/editable-folder/output.txt vs /tmp/TestAcceptbundleenvironmentseditable-folderDMS=true2717439549/002/output.txt
FAIL

latest (v1.25.0):

acceptance_test.go:1345: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1345
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1086
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:660
        	Error:      	Not equal: 
        	            	expected: "\n=== With pipeline testing, a folder after -e maps to its deployed folder\n\n>>> [CLI] bundle validate -o json\n{\n  \"job\": [\n    \"-e /Workspace/Users/[USERNAME]/.bundle/editable-folder/default/files/project\"\n  ],\n  \"pipeline\": [\n    \"-e /Workspace/Users/[USERNAME]/.bundle/editable-folder/default/files/project[test]\"\n  ]\n}\n\n=== Without pipeline testing, a folder after -e is an error, as before\nError: unable to determine if [TEST_TMP_DIR]/project is not a notebook\n"
        	            	actual  : "\n=== With pipeline testing, a folder after -e maps to its deployed folder\n\n>>> [CLI] bundle validate -o json\nError: unable to determine if [TEST_TMP_DIR]/project is not a notebook: read [TEST_TMP_DIR]/project: is a directory\n\n{\n  \"job\": [\n    \"-e project\"\n  ],\n  \"pipeline\": [\n    \"-e project[test]\"\n  ]\n}\n\nExit code: 1\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -4,8 +4,10 @@
        	            	 >>> [CLI] bundle validate -o json
        	            	+Error: unable to determine if [TEST_TMP_DIR]/project is not a notebook: read [TEST_TMP_DIR]/project: is a directory
        	            	+
        	            	 {
        	            	   "job": [
        	            	-    "-e /Workspace/Users/[USERNAME]/.bundle/editable-folder/default/files/project"
        	            	+    "-e project"
        	            	   ],
        	            	   "pipeline": [
        	            	-    "-e /Workspace/Users/[USERNAME]/.bundle/editable-folder/default/files/project[test]"
        	            	+    "-e project[test]"
        	            	   ]
        	            	@@ -13,4 +15,3 @@
        	            	 
        	            	-=== Without pipeline testing, a folder after -e is an error, as before
        	            	-Error: unable to determine if [TEST_TMP_DIR]/project is not a notebook
        	            	+Exit code: 1
        	            	 
        	Test:       	TestAccept/bundle/environments/editable-folder/DMS=true
        	Messages:   	bundle/environments/editable-folder/output.txt vs /tmp/TestAcceptbundleenvironmentseditable-folderDMS=true1301042383/002/output.txt

eng-dev-ecosystem-bot · 2026-11-10T00:48:21Z

✅ Integration test report

Commit: 9a2cc3c

Run: 42864420304

	Env	✅pass	🙈skip	Time
✅	aws linux-2core-8gb	271	16	4:38
✅	aws-windows-latest-4core-16gb	273	14	2:58
✅	azure linux-2core-8gb	270	16	4:42
✅	azure-windows-latest-4core-16gb	272	14	3:41
✅	gcp linux-2core-8gb	271	16	5:11
✅	gcp-windows-latest-4core-16gb	273	14	4:10

Top 3 slowest tests (at least 2 minutes):

duration	env	testname
2:59	gcp-windows-latest-4core-16gb	TestAccept
2:54	azure-windows-latest-4core-16gb	TestAccept
2:54	aws-windows-latest-4core-16gb	TestAccept

…oyed folder With DATABRICKS_BUNDLE_ENABLE_EXPERIMENTAL_PIPELINE_TESTING=true, a local project folder after -e, with optional extras such as -e ./project[test], maps to its deployed folder instead of failing. This applies to job and pipeline environment dependencies. With the variable off, the old behavior stays. Co-authored-by: Isaac <no-reply@databricks.com>

github-actions Bot added the DABs DABs related issues label Nov 10, 2026

tharunbhuk-db force-pushed the tharun/pipelines-test-b1-editable-folder branch from abc52a2 to 6092358 Compare November 10, 2026 00:25

tharunbhuk-db force-pushed the tharun/pipelines-test-b1-editable-folder branch from 6092358 to 7e4c017 Compare November 10, 2026 10:06

This was referenced Nov 10, 2026

Translate the testing paths of pipelines and upload local test wheels #7437

Open

Deploy the testing property of pipelines and show remote changes in bundle plan #7438

Open

Add the changelog entries for pipeline testing #7439

Open

tharunbhuk-db force-pushed the tharun/pipelines-test-b1-editable-folder branch 2 times, most recently from bf6225d to e98a9f7 Compare November 10, 2026 16:49

tharunbhuk-db requested a review from pranshupand-db November 10, 2026 17:22

tharunbhuk-db force-pushed the tharun/pipelines-test-b1-editable-folder branch from e98a9f7 to 9a2cc3c Compare November 10, 2026 17:22

tharunbhuk-db marked this pull request as ready for review November 10, 2026 17:22

tharunbhuk-db requested review from a team as code owners November 10, 2026 17:22

tharunbhuk-db requested review from shreyas-goenka and removed request for shreyas-goenka November 10, 2026 17:22

Navigation Menu

Repository navigation

Map a project folder after -e in environment dependencies to its deployed folder - #7427

Map a project folder after -e in environment dependencies to its deployed folder#7427
tharunbhuk-db wants to merge 1 commit into
mainfrom
tharun/pipelines-test-b1-editable-folder

tharunbhuk-db commented Nov 10, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Nov 10, 2026 •

edited

Loading

Uh oh!

eng-dev-ecosystem-bot commented Nov 10, 2026 •

edited

Loading

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

tharunbhuk-db commented Nov 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🥞 Stacked PR

Uh oh!

github-actions Bot commented Nov 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Regression test report

Uh oh!

eng-dev-ecosystem-bot commented Nov 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Integration test report

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

tharunbhuk-db commented Nov 10, 2026 •

edited

Loading

github-actions Bot commented Nov 10, 2026 •

edited

Loading

eng-dev-ecosystem-bot commented Nov 10, 2026 •

edited

Loading