Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,10 @@ Run `Deepnote: Preview App` from the Command Palette, or right-click a folder in
- A run keeps going until its notebook finishes, even after you close the preview. Reloading or closing the VS Code window stops it.
- For a legacy `.deepnote` file that holds several notebooks, the first run shows the prompt to split the file, as opening the file does.
- `appData()` answers from that notebook's newest successful `run()` in the preview: the first JSON object the run displayed (an `application/json` output), the first agent block's answer, and when the run started. Until a run succeeds, it rejects with Deepnote's message. The preview keeps run results in memory, so reloading or closing the VS Code window clears them.
- `dataframe(name, { pageIndex, pageSize })` pages through the frame stored under that name in `deepnote_dataframes/<name>/` in the app's project folder, the folder of the `.deepnote` file nearest at or above the app folder. Pages and values match Deepnote's, but the preview does not apply Deepnote's limits on how much one page may hold.
- Deepnote's client does not have `dataframe()` yet, so the preview adds it to the bundled copy. A published app cannot call it until Deepnote's client does.
- The deepnote-toolkit version the extension installs does not store frames there yet: `store_dataframe(df, name)` writes them to `<home>/work/deepnote_dataframes/`, which every project shares. The preview reads a frame from there when the project folder has none under that name.
- Reading a frame that deepnote-toolkit compressed with zstd needs an editor whose Node.js has zstd (22.15 or later).
- Previewing needs a trusted workspace and the desktop app.

### Database integrations
Expand Down
11 changes: 9 additions & 2 deletions build/esbuild/build.ts
Original file line number Diff line number Diff line change
Expand Up @@ -211,6 +211,7 @@ function createConfig(
): SameShape<BuildOptions, BuildOptions> {
const inject: string[] = [];
const isWebTestSource = source.endsWith(path.join('web', 'index.ts'));
const isAppPreviewScript = source.endsWith(path.join('appPreview', 'dataframeShim.ts'));
const plugins: Plugin[] = [];
let define: SameShape<BuildOptions, BuildOptions>['define'] = undefined;
if (target === 'web') {
Expand Down Expand Up @@ -286,14 +287,15 @@ function createConfig(
);
}
// Desktop builds use CommonJS for VS Code/Cursor compatibility
// Web builds use ESM for browser compatibility
// Web builds use ESM for browser compatibility, except the dataframe() shim: app pages load it as a classic script,
// where an ESM bundle's top-level names would become globals.
const config: SameShape<BuildOptions, BuildOptions> = {
entryPoints: [source],
outfile,
bundle: true,
external,
alias,
format: target === 'desktop' ? 'cjs' : 'esm',
format: target === 'desktop' ? 'cjs' : isAppPreviewScript ? 'iife' : 'esm',
metafile: isDevbuild && !watch,
define,
target: target === 'desktop' ? 'node18' : 'es2018',
Expand Down Expand Up @@ -447,6 +449,11 @@ async function buildAll() {
path.join(extensionFolder, 'dist', 'webviews', 'webview-side', 'bigNumberComparisonSettings', 'index.js'),
{ target: 'web', watch: watchAll }
),
build(
path.join(extensionFolder, 'src', 'webviews', 'webview-side', 'appPreview', 'dataframeShim.ts'),
path.join(extensionFolder, 'dist', 'webviews', 'webview-side', 'appPreview', 'dataframeShim.js'),
{ target: 'web', watch: watchAll }
),
// Notebook renderers (integrated from jupyter-renderers)
build(
path.join(extensionFolder, 'src', 'renderers', 'client', 'index.tsx'),
Expand Down
3 changes: 2 additions & 1 deletion cspell.json
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,8 @@
"xanchor",
"xaxis",
"yanchor",
"yaxis"
"yaxis",
"žluťoučký"
],
"useGitignore": true
}
38 changes: 37 additions & 1 deletion package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -2818,6 +2818,7 @@
"@vscode/jupyter-ipywidgets8": "^1.0.9",
"@vscode/python-extension": "^1.0.4",
"ansi-to-html": "^0.6.7",
"apache-arrow": "^21.2.0",
"bootstrap": "^5.0.0",
"bootstrap-less": "^3.3.8",
"clsx": "^2.1.1",
Expand Down
70 changes: 53 additions & 17 deletions src/notebooks/deepnote/appPreview/appPreviewApi.node.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,14 @@ import { isObject } from '../../../platform/common/utils/sysTypes';
import { generateUuid } from '../../../platform/common/uuid';
import type { DeepnoteNotebook } from '../../../platform/deepnote/deepnoteTypes';
import { logger } from '../../../platform/logging';
import type { AppPreviewProject } from './appPreviewSession.node';
import { blocksInNotebookOrder, toNotebookInputs, type WorkspaceNotebook } from './appPreviewWorkspace.node';
import {
DATAFRAME_NAME_PATTERN,
DATAFRAME_NOT_FOUND_MESSAGE,
DataframePageError,
readStoredDataframePage
} from './storedDataframePage.node';
import type {
AppPreviewInputValue,
AppPreviewRunOutcome,
Expand All @@ -22,12 +29,11 @@ import type {
SnapshotBlock
} from './types';

export type AppPreviewOperation = 'dataframe()';

export interface AppPreviewApiOptions {
token: string;
findNotebook(notebookId: string): Promise<WorkspaceNotebook | undefined>;
onUnsupported(operation: AppPreviewOperation): void;
/** The project the previewed app belongs to, read again on every dataframe request. */
getProject(): AppPreviewProject | undefined;
runner: IAppPreviewRunner;
}

Expand Down Expand Up @@ -67,16 +73,22 @@ type AppViewerNotebook = operations['getNotebook']['responses'][200]['content'][

// Each finished run holds all its outputs, and a page may start one on every reload.
const MAX_FINISHED_RUNS = 20;
const MAX_DATAFRAME_PAGE_SIZE = 1000;

const createRunBodySchema = z.strictObject({
notebookId: z.string().min(1),
inputs: z.record(z.string().min(1), z.union([z.string(), z.boolean(), z.array(z.string())])).optional(),
detachedRunStorageMode: z.enum(['read_write', 'readonly']).optional()
});

function notSupportedMessage(operation: AppPreviewOperation): string {
return `${operation} is not supported in the local preview yet.`;
}
// Deepnote also requires a UUID `projectId`, but a `.deepnote` file's project id can be any string.
const dataframeParamsSchema = z.object({
projectId: z.string(),
name: z.string().regex(DATAFRAME_NAME_PATTERN)
});
const dataframeQuerySchema = z.strictObject({
pageIndex: z.coerce.number().int().min(0).default(0),
pageSize: z.coerce.number().int().min(1).max(MAX_DATAFRAME_PAGE_SIZE).default(10)
});

/** The 4xx status of an error body-parser raises for the request itself, such as malformed JSON. */
function clientErrorStatus(error: unknown): number | undefined {
Expand Down Expand Up @@ -227,21 +239,17 @@ function toAppViewerNotebook({ file, notebook }: WorkspaceNotebook): AppViewerNo
* The local stand-in for Deepnote's app viewer API, mounted at `/v2` on the preview server.
* Requests without the preview's bearer token get 401. `GET /notebooks/:notebookId` answers from the workspace's
* notebooks, and `POST /runs` runs one with `runner`, which `GET /runs/:runId` then reports on.
* `GET /notebooks/:notebookId/app-data` answers from the notebook's newest successful run. The dataframe route, which
* the version of Deepnote's client that adds `dataframe()` calls, gets 501, and every other route gets 403, as Deepnote
* answers app viewer tokens. A body that cannot be read, such as malformed JSON, gets its 4xx; any other failure is
* logged and gets a JSON 500.
* `GET /notebooks/:notebookId/app-data` answers from the notebook's newest successful run, and
* `GET /projects/:projectId/dataframes/:name` from the dataframes stored in the folder of `getProject()`'s project, or
* under `$HOME/work` for a name that folder has none under.
* Every other route gets 403, as Deepnote answers app viewer tokens. A body that cannot be read, such as malformed
* JSON, gets its 4xx; any other failure is logged and gets a JSON 500.
*/
export function createAppPreviewApi(options: AppPreviewApiOptions): AppPreviewApi {
const router = express.Router();
const runs = new Map<string, PreviewRun>();
const unfinished = new Set<CancellationTokenSource>();

const unsupported = (operation: AppPreviewOperation) => (_req: Request, res: Response) => {
options.onUnsupported(operation);
res.status(501).json({ message: notSupportedMessage(operation) });
};

const startRun = (run: PreviewRun, request: AppPreviewRunRequest) => {
const cancellation = new CancellationTokenSource();
unfinished.add(cancellation);
Expand Down Expand Up @@ -358,7 +366,35 @@ export function createAppPreviewApi(options: AppPreviewApiOptions): AppPreviewAp
}
res.json({ run: { ...run, snapshotExpiresAt: null, snapshotContent: null, snapshotDownloadUrl: null } });
});
router.get('/projects/:projectId/dataframes/:name', unsupported('dataframe()'));
router.get('/projects/:projectId/dataframes/:name', async (req: Request, res: Response) => {
const params = dataframeParamsSchema.safeParse(req.params);
if (!params.success) {
res.status(400).json({ message: readableErrors(params.error) });

return;
}
const query = dataframeQuerySchema.safeParse(req.query);
if (!query.success) {
res.status(400).json({ message: readableErrors(query.error) });

return;
}
const project = options.getProject();
if (!project || project.id !== params.data.projectId) {
res.status(404).json({ message: DATAFRAME_NOT_FOUND_MESSAGE });

return;
}

try {
res.json(await readStoredDataframePage(project.folder, params.data.name, query.data));
} catch (error) {
if (!(error instanceof DataframePageError)) {
throw error;
}
res.status(error.status).json({ message: error.message });
}
});

router.use((_req: Request, res: Response) => {
res.status(403).json({ message: NOT_AVAILABLE_MESSAGE });
Expand Down
Loading
Loading