fix(cli,driver-sql): os migrate plan stops creating a database on a fresh project (#6743)

.changeset/lucky-pugs-repeat.md

@@ -0,0 +1,34 @@
+---
+'@objectstack/driver-sql': patch
+'@objectstack/service-datasource': patch
+'@objectstack/runtime': patch
+'@objectstack/cli': patch
+---
+
+`os migrate plan` no longer creates a database on a project that has never been started (#6743)
+
+`migrate plan` is a dry run, and since #3917 it has reported the boot-time
+create-table DDL and the artifact seed instead of performing them. It still
+brought the database file itself into existence, though: SQLite creates the
+file at open, so a `plan` in a fresh project left behind a 0-table
+`.objectstack/data/objectstack.db` — a write side effect from a read-only
+command, and one that erased the only signal ("no database file yet") by which
+the next command can tell a never-started project from a started one.
+
+A missing SQLite target is now opened as an empty in-memory database instead of
+being created. **The plan output is unchanged**, deliberately: a database with
+zero tables is exactly what a freshly created empty file is, so "every table
+needs creating" — the true and useful answer for a new project — still prints,
+and the `Database:` line still names the real target path rather than the
+in-memory stand-in.
+
+New driver capability, additive and off by default:
+`SqlDriverConfig.sqliteAbsentFile` (`'create'` | `'empty-in-memory'`, default
+`'create'`). Every existing caller keeps SQLite's own create-if-absent
+behaviour. It is threaded to the driver as a host-composition option
+(`createDefaultDatasourceDriverFactory`, `DefaultDatasourcePlugin`,
+`createStandaloneStack`), not as an authorable `datasource.config` key — a
+datasource must not be able to declare itself into never persisting.
+
+`os migrate apply` deliberately does **not** use it: it boots deferred too, but
+flushes the deferred DDL after confirmation and needs a real file to flush into.

packages/cli/src/commands/migrate/plan.ts

@@ -40,6 +40,16 @@ import {
  * printed a single line is now REPORTED as pending work instead of performed.
  * A database another process is using is reported too — as a warning, not a
  * refusal, since a plan writes nothing either way.
+ *
+ * Since #6743 the dry run also stops short of CREATING the database. The
+ * deferral #3917 introduced covered the DDL and the seed but not the open
+ * itself, so a `plan` in a never-started project still left a 0-table
+ * `.objectstack/data/objectstack.db` behind — a write side effect from a
+ * read-only command, and one that made "this project has no database yet"
+ * unobservable to whatever ran next. A missing sqlite target is now opened as
+ * an empty in-memory database: the plan is unchanged (an empty database is an
+ * empty database, and "every table needs creating" is the true answer for a
+ * fresh project), and nothing is written to disk.
  */
 export default class MigratePlan extends Command {
   static override description =
@@ -77,7 +87,19 @@ export default class MigratePlan extends Command {
 
     let stack;
     try {
-      stack = await bootSchemaStack({ jsonOutput: flags.json, databaseUrl: flags['database-url'], deferSchemaDdl: true });
+      stack = await bootSchemaStack({
+        jsonOutput: flags.json,
+        databaseUrl: flags['database-url'],
+        deferSchemaDdl: true,
+        // A plan writes nothing — so it must not bring a database file into
+        // existence either (#6743). On a never-started project the target is
+        // opened as an empty in-memory database: the plan below is unchanged
+        // (an empty database is an empty database), and no file, `-wal` or
+        // `-shm` is left behind. `os migrate apply` deliberately does NOT set
+        // this — it flushes the deferred DDL after confirmation and needs a
+        // real file to flush into.
+        readOnlyProbe: true,
+      });
     } catch (error: any) {
       if (flags.json) { await emitJson({ error: error.message }, 0, { compact: true }); this.exit(1); }
       printError(error.message || String(error));

packages/cli/src/utils/schema-migrate.readonly-probe.integration.test.ts

@@ -0,0 +1,215 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+
+/**
+ * End-to-end acceptance for #6743: `os migrate plan` on a project that has
+ * never been started must leave NO database behind — and must print exactly
+ * the plan it printed before.
+ *
+ * #3917 made the boot stop WRITING (no create-table DDL, no seed rows), and
+ * `schema-migrate.deferred-ddl.integration.test.ts` pins that half against a
+ * database that already exists. The half left over — and the one this file
+ * pins — is the OPEN itself: SQLite creates a database file the moment a
+ * connection issues its first statement, so a `plan` in a fresh project still
+ * produced a 0-table `.objectstack/data/objectstack.db`. A dry run that leaves
+ * a file behind also destroys the one signal "this project has no database
+ * yet", which every following command reads by asking whether the file exists.
+ *
+ * Two assertions, and BOTH are load-bearing (the `domain:cli` ruling on the
+ * issue is explicit that the fix must not be paid for with the report):
+ *
+ *   1. **No file.** Not just `objectstack.db` — the `-wal` and `-shm`
+ *      companions too, and the `data/` directory itself. A pin that names only
+ *      the `.db` path misses two of the three files an interrupted WAL
+ *      connection can strand.
+ *   2. **Same report.** On a fresh project "every table needs creating" is not
+ *      a false report; it is the true answer, and it is what a user most wants
+ *      from the first `plan` of a new project. So the pending-work set is
+ *      compared against a control boot that DOES create the file, and the
+ *      database label is asserted to still name the real target path rather
+ *      than the `:memory:` stand-in the probe actually read.
+ */
+
+import { describe, it, expect, beforeEach, afterEach } from 'vitest';
+import { mkdtempSync, writeFileSync, mkdirSync, rmSync, existsSync, readdirSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import { bootSchemaStack, type PendingSchemaWork } from './schema-migrate.js';
+
+const ARTIFACT = {
+  id: 'probe_smoke',
+  name: 'Probe Smoke',
+  objects: [
+    { name: 'probe_widget', fields: { name: { type: 'text', required: true }, colour: { type: 'text' } } },
+    { name: 'probe_gadget', fields: { label: { type: 'text' } } },
+  ],
+};
+
+/**
+ * The env vars that outrank the unified project default (#6469). Every one of
+ * them must be absent or the boot resolves somewhere else entirely and this
+ * test silently stops testing anything.
+ */
+const OVERRIDING_ENV = [
+  'OS_DATABASE_URL',
+  'DATABASE_URL',
+  'TURSO_DATABASE_URL',
+  'OS_DATABASE_DRIVER',
+  'OS_HOME',
+] as const;
+
+describe('os migrate plan on a fresh project — no database is created (#6743)', () => {
+  let dir: string;
+  let dataDir: string;
+  let dbFile: string;
+  const savedEnv: Record<string, string | undefined> = {};
+
+  beforeEach(() => {
+    dir = mkdtempSync(join(tmpdir(), 'os-probe-'));
+    mkdirSync(join(dir, 'dist'), { recursive: true });
+    writeFileSync(join(dir, 'dist', 'objectstack.json'), JSON.stringify(ARTIFACT));
+    // Deliberately NOT created — the whole point is that the boot must not
+    // bring it into existence. This is the unified default of #6469.
+    dataDir = join(dir, '.objectstack', 'data');
+    dbFile = join(dataDir, 'objectstack.db');
+
+    for (const key of OVERRIDING_ENV) {
+      savedEnv[key] = process.env[key];
+      delete process.env[key];
+    }
+    savedEnv.OS_ARTIFACT_PATH = process.env.OS_ARTIFACT_PATH;
+    savedEnv.NODE_ENV = process.env.NODE_ENV;
+    process.env.OS_ARTIFACT_PATH = join(dir, 'dist', 'objectstack.json');
+    process.env.NODE_ENV = 'production'; // no dev auto-reconcile, no wasm step-down
+  });
+
+  afterEach(() => {
+    for (const key of OVERRIDING_ENV) {
+      if (savedEnv[key] === undefined) delete process.env[key];
+      else process.env[key] = savedEnv[key];
+    }
+    process.env.OS_ARTIFACT_PATH = savedEnv.OS_ARTIFACT_PATH;
+    process.env.NODE_ENV = savedEnv.NODE_ENV;
+    try { rmSync(dir, { recursive: true, force: true }); } catch { /* ignore */ }
+  });
+
+  /** Every path the sqlite target could have brought into existence. */
+  function databaseArtifactsOnDisk(): string[] {
+    const found: string[] = [];
+    for (const path of [dbFile, `${dbFile}-wal`, `${dbFile}-shm`]) {
+      if (existsSync(path)) found.push(path);
+    }
+    // Anything else the boot dropped in `data/` counts too — a companion this
+    // list does not name is still debris.
+    if (existsSync(dataDir)) {
+      for (const entry of readdirSync(dataDir)) {
+        const full = join(dataDir, entry);
+        if (!found.includes(full)) found.push(full);
+      }
+    }
+    return found.sort();
+  }
+
+  /** Comparable projection of the plan's "New (additive)" section. */
+  function pendingShape(pending: PendingSchemaWork[]): Array<{ table: string; kind: string; columns: number }> {
+    return pending
+      .map((p) => ({ table: p.table, kind: p.kind, columns: p.columns.length }))
+      .sort((a, b) => a.table.localeCompare(b.table));
+  }
+
+  it('leaves no .objectstack/data/ at all — file, -wal and -shm alike', async () => {
+    expect(existsSync(dataDir)).toBe(false);
+
+    const stack = await bootSchemaStack({
+      jsonOutput: false,
+      deferSchemaDdl: true,
+      readOnlyProbe: true,
+      projectRoot: dir,
+    });
+    try {
+      // The plan is computed and non-empty — this is a real run, not a boot
+      // that failed early and left nothing behind for uninteresting reasons.
+      expect(stack.pendingSchemaWork.length).toBeGreaterThan(0);
+      expect(await stack.driver!.detectManagedDrift()).toEqual([]);
+    } finally {
+      await stack.shutdown();
+    }
+
+    expect(databaseArtifactsOnDisk()).toEqual([]);
+    // The directory itself is the other half of the write side effect: a
+    // `mkdir -p` here would recreate exactly the state whose absence tells the
+    // next command that this project has never been started.
+    expect(existsSync(dataDir)).toBe(false);
+  }, 60_000);
+
+  it('prints the same plan as a boot that does create the file, and still names the real target', async () => {
+    const probe = await bootSchemaStack({
+      jsonOutput: false,
+      deferSchemaDdl: true,
+      readOnlyProbe: true,
+      projectRoot: dir,
+    });
+    const probePending = pendingShape(probe.pendingSchemaWork);
+    const probeLabel = probe.dbLabel;
+    const probeExamined = probe.managedTableCount;
+    await probe.shutdown();
+
+    expect(existsSync(dataDir)).toBe(false);
+
+    // Control: the pre-#6743 behaviour, on the same fresh project.
+    const control = await bootSchemaStack({
+      jsonOutput: false,
+      deferSchemaDdl: true,
+      projectRoot: dir,
+    });
+    const controlPending = pendingShape(control.pendingSchemaWork);
+    const controlLabel = control.dbLabel;
+    const controlExamined = control.managedTableCount;
+    await control.shutdown();
+
+    // The control is what proves the fixture is a genuine fresh project: it
+    // creates the file this issue is about.
+    expect(existsSync(dbFile)).toBe(true);
+
+    expect(probePending).toEqual(controlPending);
+    expect(probeExamined).toBe(controlExamined);
+    // The one line that would betray the stand-in. `describeDb` reads the
+    // driver's DECLARED config, which the driver keeps pointing at the real
+    // file even while its Knex instance holds `:memory:`.
+    expect(probeLabel).toBe(controlLabel);
+    expect(probeLabel).toContain('objectstack.db');
+    expect(probeLabel).not.toContain(':memory:');
+  }, 60_000);
+
+  it('does NOT arm itself for a boot that will write — os migrate apply keeps its file', async () => {
+    // `apply` boots with `deferSchemaDdl` too and then flushes the deferred DDL
+    // after the operator confirms. If the absent-file redirect ever became
+    // implied by `deferSchemaDdl`, that flush would land in an ephemeral
+    // database and the operator's migration would silently evaporate.
+    const stack = await bootSchemaStack({
+      jsonOutput: false,
+      deferSchemaDdl: true,
+      projectRoot: dir,
+    });
+    try {
+      const created = await stack.flushSchemaDdl();
+      expect(created.length).toBeGreaterThan(0);
+    } finally {
+      await stack.shutdown();
+    }
+
+    expect(existsSync(dbFile)).toBe(true);
+
+    // And the tables really are in the FILE, not in a stand-in that vanished.
+    const after = await bootSchemaStack({
+      jsonOutput: false,
+      deferSchemaDdl: true,
+      readOnlyProbe: true,
+      projectRoot: dir,
+    });
+    try {
+      expect(after.pendingSchemaWork).toEqual([]);
+    } finally {
+      await after.shutdown();
+    }
+  }, 60_000);
+});

packages/cli/src/utils/schema-migrate.ts

@@ -189,6 +189,28 @@ export async function bootSchemaStack(
      * tables to exist.
      */
     deferSchemaDdl?: boolean;
+    /**
+     * Boot WITHOUT BRINGING A DATABASE INTO EXISTENCE (#6743).
+     *
+     * `deferSchemaDdl` stopped the boot from writing DDL and seed rows, but the
+     * sqlite driver still opened its target in SQLite's default create-if-absent
+     * mode — so `os migrate plan` on a never-started project left a 0-table
+     * `.objectstack/data/objectstack.db` (plus its `-wal`/`-shm` on an unclean
+     * exit) behind: a write side effect from a command that calls itself a dry
+     * run, and one that makes "this project has no database yet" unobservable
+     * to the next command.
+     *
+     * With this set, a missing sqlite file is opened as an empty `:memory:`
+     * database instead. A database with zero tables is exactly what a freshly
+     * created empty file is, so the plan is byte-for-byte the one printed
+     * before — the report was never the defect and must not pay for the fix.
+     *
+     * ⚠️ NOT implied by `deferSchemaDdl`, and it must not become so:
+     * `os migrate apply` also boots deferred, then FLUSHES the deferred DDL
+     * once the operator confirms. Writes into the `:memory:` stand-in would be
+     * discarded at disconnect, so `apply` keeps the default.
+     */
+    readOnlyProbe?: boolean;
     /**
      * Project root the booted stack scopes its on-disk state to — the default
      * sqlite database and the metadata FileSystemRepository
@@ -216,6 +238,7 @@ export async function bootSchemaStack(
     projectRoot: opts.projectRoot ?? process.cwd(),
     ...(opts.databaseUrl ? { databaseUrl: opts.databaseUrl } : {}),
     ...(defer ? { skipSeedData: true } : {}),
+    ...(opts.readOnlyProbe ? { sqliteAbsentFile: 'empty-in-memory' as const } : {}),
   });
 
   // No HTTP, no cluster — this is a one-shot schema operation.

packages/drivers/driver-sql/src/index.ts

@@ -3,9 +3,14 @@
 import { SqlDriver } from './sql-driver.js';
 
 export { SqlDriver };
+// The absent-file open decision (#6743). Exported as a VALUE because the
+// service layer's native→wasm step-down resolves the same answer for its
+// fallback rung — one judgement, two call sites, no second `existsSync`.
+export { resolveSqliteAbsentFileTarget } from './sql-driver.js';
 export type {
   SqlDriverConfig,
   SqliteJournalMode,
+  SqliteAbsentFileMode,
   IntrospectedSchema,
   IntrospectedTable,
   IntrospectedColumn,