test(e2e): quarantine the run_dbconnect magic-comments test (headless-harness stall)#2023
test(e2e): quarantine the run_dbconnect magic-comments test (headless-harness stall)#2023rugpanov wants to merge 1 commit into
Conversation
|
🤖 Integration tests triggered for |
d0102cd to
a67a15b
Compare
|
🤖 Integration tests ❌ 2 of 35 test jobs failed for |
a67a15b to
d36cc1a
Compare
|
🤖 Integration tests ❌ 4 of 35 test jobs failed for |
d36cc1a to
4b842af
Compare
|
🤖 Integration tests ❌ 2 of 35 test jobs failed for |
…-harness stall) *Why* "should run a databricks notebook with dbconnect and handle magic comments" consistently fails in the headless CI harness: "Jupyter: Run All Cells" runs the first DBConnect cell and does not advance to the rest, so the `%sql`/`%run` cells stay `<not run>` and their output files are never written. This is a webdriver/xvfb-harness artifact, NOT a product bug — verified on a real machine that Run All runs every cell, and running the cells one-by-one through the kernel produces all outputs (magic-notebook-output.json / databricks-run-notebook-output.json). Attempts to make the test drive the cells explicitly did not resolve it: - The `.py` "Databricks notebook source" file runs in an Interactive Window, which can only be driven through its input box (interactive.execute always appends+runs a new cell) — existing IW cells can't be executed by index. - Rewriting as a real `.ipynb` and driving cells via `notebook.cell.execute` got the first cell (run by Run All) to complete, but a subsequently-driven cell never reached a terminal executionSummary in the harness, regardless of cell content (reproduced with a plain `spark.sql(...)` cell — so it is not the `%sql` display formatter). The failure is invisible from the test (no cell state, no output) and could not be reproduced locally (no headless renderer). *What* - Mark the test `it.skip` with a detailed comment explaining the harness-only root cause and the ruled-out approaches, pending a harness-side fix. - The `%sql`/`%run` transform logic remains covered by the unit test databricks_magics_transformer_test.py. *Verification* - `tsc --noEmit -p src/test/e2e/tsconfig.json`: 0 errors; eslint + prettier: clean. - Diff vs main is limited to skipping this one test; the other run_dbconnect tests are unchanged. Co-authored-by: Isaac
4b842af to
4382121
Compare
|
If integration tests don't run automatically, an authorized user can run them manually by following the instructions below: Trigger: Inputs:
Checks will be approved automatically on success. |
rugpanov
changed the title
|
🤖 Integration tests ❌ 1 of 35 test jobs failed for |
|
Closing in favor of @anton-107's #2022, which covers the same run_dbconnect harness stall and is more complete (it skips both affected tests — the magic test AND |
1 participant
Summary
Quarantines (
it.skip) the e2e testshould run a databricks notebook with dbconnect and handle magic comments, which consistently fails in the headless CI harness. Closes the loop on the last still-failingrun_dbconnecttest after the merged de-flake work (#2019/#2020/#2021 + diagnostics #2013).Tracking issue: #2026.
Why quarantine rather than fix
The failure is a headless webdriver/xvfb harness artifact, not a product bug — verified on a real machine that
Run Allruns every cell, and running the cells one-by-one through the kernel produces all outputs. In the harness,Run Allruns the first DBConnect cell and doesn't advance; the%sql/%runcells stay<not run>.I attempted real fixes before quarantining (details in #2026 and the code comment):
.ipynband driving cells vianotebook.cell.execute— the first cell completes, but a subsequently-driven cell never reaches a terminalexecutionSummaryin the harness, regardless of cell content (reproduced with plainspark.sql(...), ruling out the%sqldisplay formatter). The failure is invisible from the test and can't be reproduced locally (no headless renderer). This needs someone who can debug the harness interactively.What changed
it.skipthe one magic-comments test, with a detailed comment (root cause + ruled-out approaches + [BUG] e2e run_dbconnect magic-comments test stalls in headless CI (Run All does not advance past first DBConnect cell) #2026).run_dbconnecttest changed. Diff vs main is limited to this skip.%sql/%runtransform coverage remains in the unit testdatabricks_magics_transformer_test.py.Verification
tsc/eslint/prettier clean.run_dbconnectno longer contributes the magic-test failure; remaining red shards in validation runs were unrelated pre-existing flakes (deploy_and_run_pipeline,bundle_init,unity_catalog)./cc @anton-107 — final piece on the run_dbconnect stack; defer to you on whether to take this or the deeper harness fix in #2026.
This pull request and its description were written by Isaac.