Skip to content

fix(kernel): keep local imports and DEEPNOTE_PROJECT_ID working outside Deepnote Cloud - #155

Draft
tkislan wants to merge 1 commit into
mainfrom
tk/fix-set-notebook-path-detached
Draft

tkislan wants to merge 1 commit into
mainfrom
tk/fix-set-notebook-path-detached

Conversation

@tkislan

@tkislan tkislan commented Nov 18, 2026

Copy link
Copy Markdown
Contributor

@coderabbitai ignore

Closes #153.

Why

The VS Code extension and the deepnote CLI start the toolkit through runtime-core, in detached mode. Jupyter and the kernel start in the .deepnote file's folder. set_notebook_path still rewrote sys.path and the working directory against ~/work there. That removed the notebook's folder from sys.path, so a module next to the .deepnote file could not be imported.

Imports only worked when parsing the project ID from the session name crashed first. A notebook name with two colons (sales:q1:report) parsed into a garbage ID, and the rewrite broke imports again. Because of this, deepnote/vscode-deepnote#617 stopped setting DEEPNOTE_PROJECT_ID, so user code no longer saw it locally.

What changes

set_notebook_path now reads the runtime config before the Jupyter session lookup:

  • Keeps the kernel's working directory and sys.path when the toolkit runs detached, outside dev mode, and with neither paths.notebook_root nor paths.home_dir set. In that setup the session name is no longer parsed.
  • Exports runtime.project_id as DEEPNOTE_PROJECT_ID in detached or dev mode. The config loader already resolves DEEPNOTE_PROJECT_ID and DEEPNOTE_RUNTIME__PROJECT_ID and ignores empty values. The session-name parse remains the fallback when no ID is configured.

Attached runs (Deepnote Cloud), dev mode and configs with explicit paths keep the rewrite.

There is one intended change in dev mode: when DEEPNOTE_RUNTIME__PROJECT_ID or DEEPNOTE_PROJECT_ID is set, the configured ID now takes priority over the ID parsed from the session name. That is the value every other reader of the project ID already uses.

docs/user/configuration.md and the function's docstring describe the new behavior.

Tests

New tests in tests/unit/test_set_notebook_path_config.py load the config through the real loader:

Test On main
Detached, no paths, ID configured: cwd and sys.path kept, ID exported fails
Detached, no paths, no ID, two-colon session name: cwd and sys.path kept, nothing exported fails
Dev mode, ID configured: the configured ID wins over the session name fails
Attached, dev mode, and detached + dev mode still rewrite to ~/work/<dir> or /work/<dir> passes (guards today's behavior)
  • _cfg() in tests/unit/test_set_notebook_path.py gains project_id=None, because the function now reads it.
  • A mutation check over the gate and the export confirms the tests catch realistic mistakes. Each of these makes a test fail:
    • the gate includes dev mode;
    • it ignores detached mode;
    • it ignores home_dir;
    • there is no early return;
    • the ID is exported only on the early-return path.
  • tests/unit gives 1468 passed and 4 skipped, with the same 13 warnings as main.
  • An end-to-end run used a real python -m deepnote_toolkit server launched the way runtime-core does it, with session name sales:q1:report. The kernel stays in the project folder, import helper_mod works, and DEEPNOTE_PROJECT_ID is the configured ID. On main, the import fails and the ID is report.

Follow-ups (not in this PR)

  • vscode-deepnote: in the same change that bumps the toolkit pin, remove the interim workaround that serves DEEPNOTE_PROJECT_ID from the integration env endpoint. Remove it no earlier, because kernels on the currently pinned toolkit rely on it.
  • DataFrame storage location: resolve_project_root() in deepnote_toolkit/artifacts/_dataframe_storage.py still returns $HOME/work for runtime-core kernels, not their working directory. Storage lands in the same place as on main. Whether it should move to the kernel's folder is a separate decision.
  • Existing test leak: test_set_notebook_path_updates_chdir_and_env leaves DEEPNOTE_PROJECT_ID set after it runs. This is already the case on main.

…de Deepnote Cloud

runtime-core, which the VS Code extension and the deepnote CLI use,
starts the toolkit in detached mode, with Jupyter and the kernel in the
.deepnote file's folder. set_notebook_path still rewrote sys.path and
the working directory against ~/work there. That removed the
notebook's folder from sys.path, so a module next to the .deepnote
file could not be imported. Imports only worked when parsing the
project ID from the session name crashed first; a notebook name with
two colons parsed into a garbage ID and broke them again.

In detached mode, outside dev mode, and with neither
paths.notebook_root nor paths.home_dir set, the kernel now keeps the
working directory and sys.path it starts with. In detached or dev mode,
runtime.project_id is exported as DEEPNOTE_PROJECT_ID before the
Jupyter session lookup, so user code sees the same variable as in
Cloud. In the detached case above, the session name is no longer
parsed. Attached runs, dev mode and configs with explicit paths keep
the rewrite.

Closes #153
@github-actions

Copy link
Copy Markdown

📦 Python package built successfully!

  • Version: 2.9.1.dev4+6b18934
  • Wheel: deepnote_toolkit-2.9.1.dev4+6b18934-py3-none-any.whl
  • Install:
    pip install "deepnote-toolkit @ https://deepnote-staging-runtime-artifactory.s3.amazonaws.com/deepnote-toolkit-packages/2.9.1.dev4%2B6b18934/deepnote_toolkit-2.9.1.dev4%2B6b18934-py3-none-any.whl"

@codecov

codecov Bot commented Nov 18, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 83.33333% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.54%. Comparing base (0b0d0ab) to head (9d1fb43).
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
deepnote_toolkit/set_notebook_path.py 83.33% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #155      +/-   ##
==========================================
+ Coverage   78.43%   78.54%   +0.10%     
==========================================
  Files         121      121              
  Lines        6920     6927       +7     
  Branches     1005     1007       +2     
==========================================
+ Hits         5428     5441      +13     
+ Misses       1182     1179       -3     
+ Partials      310      307       -3     
Flag Coverage Δ
combined 78.54% <83.33%> (+0.10%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

@deepnote-bot

Copy link
Copy Markdown

🚀 Review App Deployment Started

📝 Description 🌐 Link / Info
🌍 Review application ra-155
🔑 Sign-in URL Click to sign-in
📊 Application logs View logs
🔄 Actions Click to redeploy
🚀 ArgoCD deployment View deployment
⏰ Last deployed 2026-11-18 11:56:59 (UTC)
📜 Deployed commit 1f7874bbd9d6b50166cbd489cd22fbb72c0f76c5
🛠️ Toolkit version 6b18934

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

set_notebook_path breaks local imports outside Deepnote Cloud, and DEEPNOTE_PROJECT_ID goes missing

2 participants