Project-driven simulations: persistent library, runs with provenance, notebook-first analysis

[andeplane]

Project-driven simulations: persistent library, runs with provenance, notebook-first analysis

Implements the architecture in docs/adr/001–003 (written, agent-reviewed, and iterated in this branch) and the visual design from the Claude Design project (Atomify Projects.dc.html).

What this is

Atomify becomes project-driven: a persistent Projects library (IndexedDB), where each project is a working tree of files plus an append-only history of runs — each run holding a snapshot of the inputs it ran with, its outputs, and metadata (run.json). The Jupyter notebook operates on the same filesystem the app writes, so app runs, notebook analysis, and Python-scripted runs all see one project directory.

Architecture (ADR-001)

• Single project filesystem: projects live directly in JupyterLite's contents IndexedDB ("JupyterLite Storage") using its record schema — "sync with the notebook" holds by construction, not by a sync protocol.
• Swappable storage: everything goes through the ProjectStorage interface (src/storage/). One implementation (ContentsProjectStorage) over two key-value backends: localforage/IndexedDB (the library) and in-memory (quick runs, embeds, tests). A future API backend is a third implementation — the interface mirrors Jupyter's contents API shape.
• Run outputs data path: the engine's wasm FS lives in a worker whose main-thread bridge was write-only; a new snapshotWorkdir worker command reads outputs out (ASYNCIFY-safe — MEMFS reads are pure JS), streamed to storage on a 3 s throttle mid-run and fully at completion.
• Runs execute through the untouched engine pipeline: each run materializes a Simulation with id = <dirName>/runs/<runId>, so kokkos preprocessing, vars wrapper, and all writes land inside the run directory.
• Reconciliation: zombie running runs become interrupted on open (notebook-origin runs get a grace window); metadata-less runs/* dirs list as external runs.

Notebook (ADR-002)

• Per-project analysis.ipynb, generated once, discovers runs by glob (runs/*/log.lammps) and reads sweep variables from each run's run.json — cross-run analysis works without regeneration.
• lammps-js Python package bundled (wheel vendored in pypi/, indexed by piplite) — notebooks can run LAMMPS in the kernel and script parameter sweeps in Python; outputs land in the project.
• lammps-logfile now ships as a wheel (the old content-dir copy broke under project-scoped notebook cwds).
• make jupyter builds the full site locally (JupyterLite + lammps.js client + COI service-worker patch ported from lammps.js); CI uses the same target.

Product & design (ADR-003)

• Home (continue-where-you-left-off, recent projects, start-from-example), Example library ("Use as project" / "Quick run"), project workspace (Files | Runs | Notebook tabs, editor with autosave states, run detail with live viewport + following console + stats/outputs panel).
• Input script is explicit project state — the Run button prompts to choose when ambiguous; Files shows the chip, per-script "Set as input" / "Run this file".
• Parameter sweeps: the New Run modal detects variable X equal <number> parameters, per-variable value/range sweep, cartesian product runs sequentially with grouped rows and cancel-remaining; overrides recorded in run.json.vars.
• Quick runs are ephemeral scratch projects with a "Save as project" banner — tire-kicking never litters the library.
• Delete flows show run count + size and are blocked mid-run; Settings has a Storage tab (usage meter, per-project sizes, persist toggle); dark + light themes from the design token system.
• Embedded/share-link mode keeps the legacy pipeline untouched.

Testing

• Unit tests: storage layer (schema fidelity, checkpoints cleanup, rename/delete semantics), runs module (allocation, snapshot contract, reconciliation, sweep expansion), notebook template, script-variable parsing.
• Store-level integration tests: the full create → edit → run → sweep → persist loop against real storage with a controllable fake engine.
• Playwright end-to-end: real browser + real WASM engine — create project, edit with autosave surviving reload, run to completion, sweep, quick-run → save-as-project, delete.
• CI: vitest + build + jupyter build.

Known limitations / follow-ups

• Shared ?data=/?script= links open in the legacy editor shell (exactly as on main); importing shares as quick-run projects with the "Save as project" banner is the follow-up. The legacy shell no longer has a Notebook pane (nothing writes JupyterLite storage on that path anymore).

• Kernel-side engine covers the MOLECULE (+KOKKOS) package set until an upstream lammps.js variant: "atomify" option lands; two small upstream PRs planned (wheel publication, variant option). Vendored wheel is 1.5.0 vs npm 1.5.1 (tracked in commit message).

• Per-run thread-count picker omitted (thread pool is fixed at wasm load); the multithreading toggle maps to the script-level KOKKOS opt-in.

• Run snapshots byte-duplicate unchanged files (size-capped); content-addressed dedup is future work behind the storage interface.

• Trajectory replay for historical runs is out of scope; run detail shows the final frame capture + persisted log.

🤖 Generated with Claude Code

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request transforms Atomify from a single-simulation tool into a project-driven platform. By unifying the application's working directory with JupyterLite's storage, it enables persistent project management, reproducible run history, and notebook-first analysis. The architecture is supported by a new swappable storage interface and a worker-to-main-thread data streaming protocol, ensuring a seamless experience for parameter sweeps and long-running simulations.

Highlights

• Project-driven architecture: Atomify now features a persistent library of projects using IndexedDB, where each project maintains a working tree of files and an append-only history of simulation runs with full provenance.
• JupyterLite integration: The Jupyter notebook now operates directly on the project's filesystem, eliminating the need for sync protocols. Project-scoped notebooks are generated automatically and discover simulation runs via glob patterns.
• UI Shell Redesign: The application has been migrated to a new, project-centric UI shell featuring a library view, run history, and a detailed run-view with live viewport and console following, replacing the legacy modal-based console.
• Parameter Sweeps: Added support for parameter sweeps that execute sequentially, with variable overrides recorded in run metadata, enabling cross-run analysis directly within the notebook.
• Worker Data Path: Implemented a new worker protocol (snapshotWorkdir) that allows the main thread to stream simulation outputs from the WASM engine to storage mid-run, ensuring outputs are available for analysis even before completion.

New Features

🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Ignored Files

• Ignored by pattern: .github/workflows/** (1)
  • .github/workflows/deploy.yaml

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request implements a major architectural redesign of Atomify, introducing a persistent project library, run history, and deep integration with JupyterLite's contents store, alongside a comprehensive Playwright E2E test suite. The review feedback is highly constructive and identifies several critical improvements: implementing a try-finally block in the run execution thunk to prevent lockups on storage errors, resolving a potential memory leak on HomeScreen unmount, ensuring parent directories are created during directory renames, recursively walking subdirectories during working tree snapshots, avoiding call stack limits in base64 conversion, fixing glob matching and file encoding in the python patch script, and adding a fallback for crypto.subtle in non-secure contexts.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

If any storage operation (such as writeRunMeta, readRunMeta, or copyOutputs) throws an error (e.g., due to IndexedDB quota limits), the thunk execution will stop immediately. This leaves activeRun permanently set, locking up the run loop and preventing any future runs from starting. Wrapping the execution in a try-finally block ensures that activeRun is always cleared and the queue advances.

    try {
      let stopReason: string = "failed";
      let errorMessage: string | undefined;
      const interval = setInterval(() => {
        void copyOutputs(MID_RUN_COPY_CAP);
      }, MID_RUN_COPY_INTERVAL_MS);
      try {
        await storeActions.simulation.newSimulation(simulation);
        for (const file of binaryFiles) {
          getWasm().FS.writeFile("/" + simulation.id + "/" + file.fileName, file.bytes);
        }
        const result = (await storeActions.simulation.run()) as
          | { stopReason: string; errorMessage?: string }
          | undefined;
        stopReason = result?.stopReason ?? "failed";
        errorMessage =
          result?.errorMessage ??
          (result
            ? undefined
            : ((getStoreState() as State<StoreModel>).simulation.lastError ??
              "The engine did not start this run."));
      } catch (error) {
        errorMessage = error instanceof Error ? error.message : String(error);
      } finally {
        clearInterval(interval);
      }

      await copyOutputs(undefined);
      const lammpsState = (getStoreState() as State<StoreModel>).simulation;
      try {
        const logPath = RUNS_DIR + "/" + runId + "/log.lammps";
        const engineLog = await storage.stat(request.dirName, logPath);
        if (!engineLog && lammpsState.lammpsOutput.length > 0) {
          await storage.write(
            request.dirName,
            logPath,
            lammpsState.lammpsOutput.join("\n") + "\n",
          );
        }
      } catch {
        // The log is a convenience copy; never fail the run over it.
      }
      const existing = await readRunMeta(storage, request.dirName, runId);
      runMeta = {
        ...(existing ?? runMeta),
        status:
          stopReason === "canceled"
            ? "canceled"
            : stopReason === "failed"
              ? "failed"
              : "completed",
        finishedAt: new Date().toISOString(),
        error: stopReason === "failed" ? errorMessage : undefined,
        stats: {
          timesteps: lammps.getTimesteps(),
          numAtoms: lammps.getNumAtoms(),
          wallSeconds:
            (Date.now() - new Date(runMeta.startedAt).getTime()) / 1000,
        },
      };
      await writeRunMeta(storage, request.dirName, runMeta);
      const frame = await captureViewport();
      if (frame) {
        try {
          await storage.write(
            request.dirName,
            RUNS_DIR + "/" + runId + "/.atomify/frame.png",
            frame,
          );
        } catch {
          // Frame capture is decorative; never fail the run over it.
        }
      }
    } catch (error) {
      console.error("Run execution failed:", error);
    } finally {
      actions.setActiveRun(undefined);
      await actions.refreshActive().catch(() => {});
      const lammpsState = (getStoreState() as State<StoreModel>).simulation;
      if (!lammpsState.running) {
        await actions.executeNextRun().catch(() => {});
      }
    }

[gemini-code-assist]

If the component unmounts while readFileRaw is pending, mounted is set to false. However, URL.createObjectURL is still called when the promise resolves, and since the cleanup function has already run, this Object URL is leaked. Guarding the block with if (!mounted) return; prevents this memory leak.

        if (frame instanceof Uint8Array && frame.length > 0) {
          if (!mounted) {
            return;
          }
          const bytes = new Uint8Array(frame);
          objectUrl = URL.createObjectURL(
            new Blob([bytes.buffer as ArrayBuffer], { type: "image/png" }),
          );
          if (mounted) {
            setFrameUrl(objectUrl);
          }
          return;
        }

[gemini-code-assist]

When renaming a directory, we must ensure that the parent directories of the new target directory exist in the store. Currently, ensureParents is only called for file renames (line 219). Moving await this.ensureParents(toFull) to the top of the rename method ensures that directory renames also create any missing ancestor directory records, preventing JupyterLite from throwing errors when accessing the renamed path.

Suggested change

	if (record.type === "directory") {
	const prefix = `${fromFull}/`;
	const keys = await this.files.keys();
	for (const key of keys) {
	if (key === fromFull || key.startsWith(prefix)) {
	const target = toFull + key.slice(fromFull.length);
	await this.moveKey(key, target);
	}
	}
	return;
	await this.ensureParents(toFull);
	if (record.type === "directory") {
	const prefix = fromFull + "/";
	const keys = await this.files.keys();
	for (const key of keys) {
	if (key === fromFull || key.startsWith(prefix)) {
	const target = toFull + key.slice(fromFull.length);
	await this.moveKey(key, target);
	}
	}
	return;
	}

[gemini-code-assist]

storage.list(dirName) only lists the direct children of the project root and is not recursive. If the working tree contains subdirectories (e.g., uploaded force field or structure folders), they will be ignored and not snapshotted, causing the simulation to fail if it references files inside them. Implementing a recursive walk ensures all nested files are correctly snapshotted.

  const copied: string[] = [];
  const gaps: NonNullable<RunMeta["snapshotGaps"]> = [];

  const walk = async (subdir?: string): Promise<void> => {
    const entries = await storage.list(dirName, subdir);
    for (const entry of entries) {
      if (entry.type === "directory") {
        if (entry.path === "runs" || entry.path === ".atomify") {
          continue;
        }
        await walk(entry.path);
      } else if (isSnapshotted(entry)) {
        if (entry.size > sizeCap) {
          const content = await storage.read(dirName, entry.path);
          gaps.push({
            path: entry.path,
            size: entry.size,
            sha256: await sha256Hex(content),
          });
        } else {
          const content = await storage.read(dirName, entry.path);
          await storage.write(
            dirName,
            RUNS_DIR + "/" + runId + "/" + entry.path,
            content,
          );
          copied.push(entry.path);
        }
      }
    }
  };
  await walk();

[gemini-code-assist]

Using String.fromCharCode with the spread operator on large subarrays can exceed the maximum call stack size in some JavaScript engines. A cleaner, faster, and safer approach in modern environments is to decode the bytes using TextDecoder with the 'latin1' encoding, which maps bytes 0-255 directly to codepoints 0-255.

export function bytesToBase64(bytes: Uint8Array): string {
  return btoa(new TextDecoder("latin1").decode(bytes));
}

[gemini-code-assist]

out.glob("*/index.html") only matches index.html files that are exactly one directory level deep. If there is an index.html at the root of the built JupyterLite site, it will be missed. Using out.rglob("index.html") recursively finds all index.html files, and the subsequent jupyter-config-data check will safely filter out any non-app pages.

Suggested change

	for page in sorted(out.glob("*/index.html")):
	for page in sorted(out.rglob("index.html")):

[gemini-code-assist]

crypto.subtle is only available in secure contexts (HTTPS or localhost). If Atomify is loaded over HTTP (e.g., in some local network deployments), crypto.subtle will be undefined and calling crypto.subtle.digest will throw a TypeError. Adding a fallback prevents runtime crashes in non-secure contexts.

async function sha256Hex(content: string | Uint8Array): Promise<string> {
  const bytes =
    typeof content === "string" ? new TextEncoder().encode(content) : content;
  if (typeof crypto === "undefined" || !crypto.subtle) {
    return "unsafe-context-" + bytes.length + "-" + Date.now();
  }
  const digest = await crypto.subtle.digest(
    "SHA-256",
    bytes.buffer instanceof ArrayBuffer && bytes.byteOffset === 0 &&
      bytes.byteLength === bytes.buffer.byteLength
      ? bytes.buffer
      : bytes.slice().buffer,
  );
  return [...new Uint8Array(digest)]
    .map((byte) => byte.toString(16).padStart(2, "0"))
    .join("");
}

[gemini-code-assist]

Reading and writing files without specifying an encoding can lead to platform-dependent encoding issues (e.g., on Windows where the default encoding is cp1252). Specifying encoding="utf-8" ensures consistent behavior across all platforms.

Suggested change

	sw = sw_path.read_text()
	sw = sw_path.read_text(encoding="utf-8")