diff --git a/CHANGELOG.md b/CHANGELOG.md index 7fb591c..c862a81 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,6 +17,9 @@ Every entry says what changed as a caller sees it, why, and what to watch for ### Added +- Drive research acceptance selects exact version-specific export and memory-search contracts, + and records aggregate read-only Google import/recovery results without publishing source data. + - The installed Drive research check now verifies person links, stale-note revision refusal, scoped source-backed memories and local source retirement through the released CLI. It checks persisted memory staleness and owner-note preservation with synthetic data only. diff --git a/performance/documents/offline-catalog/README.md b/performance/documents/offline-catalog/README.md index 73e33ac..0b84d53 100644 --- a/performance/documents/offline-catalog/README.md +++ b/performance/documents/offline-catalog/README.md @@ -8,8 +8,61 @@ The release check also asks whether the new JSON envelope and config migration p workflows for installed-package consumers. The [controlled Google checklist](live-checklist.md#use-the-prepared-checker) now includes a -reusable isolated acceptance runner. Google checks remain unrun until the owner connects a -dedicated test account; its synthetic and installed-package validation is recorded below. +reusable isolated acceptance runner. The owner later explicitly authorized read-only checks with +an existing profile; aggregate results are recorded below, alongside synthetic package validation. + +## Read-only Google acceptance + +The owner asked to validate an existing selected folder using its already connected profile. +On 2026-10-15, Linux/Node 24.19.0 and installed Drive 0.7.0 (Core 0.278.0/Base 0.28.2) +read Google while keeping all content, database, checkpoints and model-cache paths in a new +private workspace. Original authentication was reused; the owner's database was not opened. +Google files, sharing and permissions were never modified. + +| Check | Observed result | +| --- | --- | +| Complete selected catalog | 443 entries: 65 folders and 378 files | +| Full import | 197 text-indexed files; 181 metadata-only; zero pending jobs | +| Metadata-only reasons | 179 unsupported types; one scanned PDF without a text layer; one unreadable XLSX | +| Cached hierarchy and metadata | Complete folder ancestry/tree and retained provider metadata | +| Initial offline batch | 20 files; 56 assertions passed with network/browser/keyring denied | +| Reader coverage | Exact word search and byte-exact Markdown export passed for one file each in eight categories | +| Live Google Docs comparison | Five single-tab Docs; 937 fragments and two tables matched; provider versions stayed stable | +| Interruption and resume | Own CLI exited 130 after SIGINT; resumed and drained pending jobs | +| Local document context | Temporary tag, note and proposed source-backed memory survived interruption and repeated sync | +| Repeat import | Zero new text saves; zero pending jobs; source references preserved | + +The eight categories were Google Docs, Google Sheets, PDF, DOCX, XLSX, CSV, Markdown and TXT. +Exported files had mode 0600. The TXT test initially normalized CRLF while comparing Python +strings; comparing UTF-8 bytes established exact preservation. This was a harness issue. + +Private report and scripts: `cli-private/reports/drive/2026-10-15-live-read/`. Queries, source IDs, +per-file outcomes and local workspace paths stay there. These scripts describe historical staged +checks on that authorized workspace; they are not an invitation to use arbitrary owner accounts. +The reusable synthetic runner below remains independent of this live run. + +This establishes reading and offline retrieval for this selected tree. It does not certify fresh +OAuth consent, Google verification, all fields on every account, remote writes, AI extraction, +embeddings or OCR. No footnotes or multiple/nested tabs occurred in the five-Doc live sample. + +## Fresh-install compatibility + +Drive 0.8.0 changes export to `--output` and memory search to a list envelope. The version-aware +checker accepts exact reviewed pins and selects the corresponding contract, retaining 0.7.0's +`--out` and rich memory-search result. Ten synthetic guard tests pass; 0.7.0's 8 initial, 8 repeat +and 7 knowledge lifecycle groups still pass after these checker changes. + +A fresh npm 0.8.0 install passed the same lifecycle groups but failed broader discovery checks: +Core 0.281.0 asks for Base 0.28.x, while Drive pins Base 0.31.0. npm installs two instances and +generated API commands lose their operation identity/effects. Do not interpret the lifecycle +pass as full standards acceptance. The producer correction is +[Core PR 1257](https://github.com/WireCatLabs/cli-core/pull/1257). + +The aligned candidate (Core 0.283.0/Base 0.31.1) passes 41 CLI invocations and three MCP +validation/audit calls. Its installed Node and Bun smoke checks require generated read/write +operation identities and correct remote effects. Drive also preserves its existing one-time +camelCase config migration after the producer stopped converting old settings. Candidate +results must be distinguished from the eventual published patch's installation results. ## Live-checker preparation @@ -288,8 +341,7 @@ and the [controlled live checklist](live-checklist.md) for the separately author ## Limits -This is a functional contract check, not a latency benchmark or Google API integration test. -It does not establish extraction fidelity, live metadata completeness, Google verification or -live user sign-in. Metadata-only backfill never refreshes old Google responses, and -catalog retirement differs from regular sync's indexed-content removal. No owner account data -is used or published. +The synthetic runner is a functional contract check, not a latency benchmark or Google API +integration test. It does not establish extraction fidelity, live metadata completeness, Google +verification or live user sign-in; the bounded real-data observations above are reported separately. Metadata-only backfill never refreshes old Google responses, and +catalog retirement differs from regular sync's indexed-content removal. Synthetic fixtures contain no owner data. The separately authorized live run publishes totals only. diff --git a/performance/documents/offline-catalog/knowledge-installed.mjs b/performance/documents/offline-catalog/knowledge-installed.mjs index 394800a..da38e99 100644 --- a/performance/documents/offline-catalog/knowledge-installed.mjs +++ b/performance/documents/offline-catalog/knowledge-installed.mjs @@ -2,7 +2,7 @@ import assert from "node:assert/strict" import { existsSync } from "node:fs" import { join } from "node:path" -export const verifyKnowledge = async ({ invoke, openStore, env, workspace, reference }) => { +export const verifyKnowledge = async ({ invoke, openStore, env, workspace, reference, release }) => { const checks = [] const item = (words, options) => { const answer = invoke(words, options) @@ -50,8 +50,9 @@ export const verifyKnowledge = async ({ invoke, openStore, env, workspace, refer checks.push("memory-evidence-and-review") assert.equal(list(["memories", "list", "doc", "--scope", "personal"]).length, 0) assert.ok(list(["memories", "list", "doc", "--scope", "work"]).some((entry) => entry.ref === memory.ref)) - const found = item(["memories", "search", "doc", "lifecyclemarker", "--scope", "work"]) - assert.ok(found.items.some((entry) => entry.ref === memory.ref)) + const search = ["memories", "search", "doc", "lifecyclemarker", "--scope", "work"] + const found = release.memory_search === "items" ? list(search) : item(search).items + assert.ok(found.some((entry) => entry.ref === memory.ref)) checks.push("memory-scope-and-source-search") item(["folders", "remove", "root"]) diff --git a/performance/documents/offline-catalog/live-check.mjs b/performance/documents/offline-catalog/live-check.mjs index 85f1da6..00b1145 100644 --- a/performance/documents/offline-catalog/live-check.mjs +++ b/performance/documents/offline-catalog/live-check.mjs @@ -1,6 +1,7 @@ import assert from "node:assert/strict" import { mkdtempSync, readFileSync } from "node:fs" import { join } from "node:path" +import { releaseOf } from "./live-input.mjs" const tag = "wirecat-acceptance" const note = "Synthetic acceptance note; preserve across rename and resync." @@ -9,6 +10,7 @@ const annotated = (context) => context.tags.some((entry) => entry.tag === tag) && context.notes.items.some((entry) => entry.text === note) export const checkFixture = ({ manifest, invoke, workspace, refresh = false, annotate = false, baseline }) => { + const release = releaseOf(manifest.drive_version) const checks = [] const check = (name, fn) => { fn() @@ -74,7 +76,7 @@ export const checkFixture = ({ manifest, invoke, workspace, refresh = false, ann check("old-marker-removed", () => assert.ok(search.hits.every((hit) => hit.ref !== document.reference))) } const out = join(exports, `${index}.md`) - item(["documents", "export", entry.id, "--out", out]) + item(["documents", "export", entry.id, release.export_flag, out]) check("markdown-export", () => { const markdown = readFileSync(out, "utf8") for (const text of entry.markdown_markers ?? []) assert.ok(markdown.includes(text)) diff --git a/performance/documents/offline-catalog/live-input.mjs b/performance/documents/offline-catalog/live-input.mjs index 31221ff..304fb5f 100644 --- a/performance/documents/offline-catalog/live-input.mjs +++ b/performance/documents/offline-catalog/live-input.mjs @@ -3,6 +3,16 @@ import { mkdirSync, readFileSync, realpathSync, writeFileSync } from "node:fs" import { join, resolve } from "node:path" const marker = "drive-acceptance-workspace.json" +const releases = { + "0.7.0": { export_flag: "--out", memory_search: "item", core: "0.278.0", framework: "0.28.2" }, + "0.8.1": { export_flag: "--output", memory_search: "items", core: "0.283.0", framework: "0.31.1" }, + "0.8.0": { export_flag: "--output", memory_search: "items", core: "0.281.0", framework: "0.31.0" }, +} +export const releaseOf = (version) => { + assert.equal(typeof version, "string") + assert.ok(Object.hasOwn(releases, version), "unreviewed Drive release") + return releases[version] +} const object = (value, keys) => { assert.ok(value && typeof value === "object" && !Array.isArray(value), "invalid manifest object") assert.ok( @@ -20,7 +30,7 @@ const strings = (value) => { export const validateManifest = (value) => { object(value, ["schema_version", "drive_version", "folder_id", "folders", "documents", "absent_ids"]) assert.equal(value.schema_version, 1) - assert.equal(value.drive_version, "0.7.0", "review a new package before changing the accepted version") + releaseOf(value.drive_version) id(value.folder_id) assert.ok(Array.isArray(value.folders) && value.folders.length > 0) assert.ok(Array.isArray(value.documents) && value.documents.length > 0) diff --git a/performance/documents/offline-catalog/live-installed.mjs b/performance/documents/offline-catalog/live-installed.mjs index 821cf0e..c03b430 100644 --- a/performance/documents/offline-catalog/live-installed.mjs +++ b/performance/documents/offline-catalog/live-installed.mjs @@ -6,13 +6,15 @@ import { tmpdir } from "node:os" import { join, resolve } from "node:path" import { fileURLToPath, pathToFileURL } from "node:url" import { verifyKnowledge } from "./knowledge-installed.mjs" -import { initialize, workspaceEnvironment } from "./live-input.mjs" +import { initialize, releaseOf, workspaceEnvironment } from "./live-input.mjs" const [input, ...rest] = process.argv.slice(2) -assert.ok(input && rest.length === 0, "usage: node live-installed.mjs ") +assert.ok(input && rest.length === 0, "usage: node live-installed.mjs ") const pkg = resolve(input) const metadata = JSON.parse(readFileSync(join(pkg, "package.json"), "utf8")) -assert.equal(metadata.version, "0.7.0") +const release = releaseOf(metadata.version) +assert.equal(metadata.dependencies["@wirecat/cli-core"], release.core) +assert.equal(metadata.dependencies["@wirecat/cli-base"], release.framework) const require = createRequire(join(pkg, "package.json")) const { openStore } = await import(pathToFileURL(require.resolve("@wirecat/cli-core/store"))) const root = mkdtempSync(join(tmpdir(), "drive-live-installed-")) @@ -104,7 +106,7 @@ writeFileSync( manifest, JSON.stringify({ schema_version: 1, - drive_version: "0.7.0", + drive_version: metadata.version, folder_id: "root", folders: [{ id: "root", parents: [] }], documents: [ @@ -161,7 +163,7 @@ const invoke = (words, { offline = true, failure = false } = {}) => { if (!failure) assert.equal(answer.meta.complete, true) return answer } -const knowledge = await verifyKnowledge({ invoke, openStore, env, workspace, reference: note.ref }) +const knowledge = await verifyKnowledge({ invoke, openStore, env, workspace, reference: note.ref, release }) process.stdout.write( `${JSON.stringify({ passed: true, data: "synthetic", live_google: false, drive: metadata.version, core: metadata.dependencies["@wirecat/cli-core"], first_checks: annotated.checks.length, repeat_checks: repeated.checks.length, knowledge_checks: knowledge })}\n`, ) diff --git a/performance/documents/offline-catalog/live.mjs b/performance/documents/offline-catalog/live.mjs index fb80c43..687eed6 100644 --- a/performance/documents/offline-catalog/live.mjs +++ b/performance/documents/offline-catalog/live.mjs @@ -4,7 +4,7 @@ import { closeSync, openSync, readFileSync, writeFileSync } from "node:fs" import { join, resolve } from "node:path" import { parseArgs } from "node:util" import { checkFixture } from "./live-check.mjs" -import { initialize, validateManifest, workspaceEnvironment } from "./live-input.mjs" +import { initialize, releaseOf, validateManifest, workspaceEnvironment } from "./live-input.mjs" let report let stage = "preflight" @@ -64,6 +64,9 @@ try { const pkg = JSON.parse(readFileSync(join(values.drive, "package.json"), "utf8")) assert.equal(pkg.name, "@wirecat/drive-cli") assert.equal(pkg.version, manifest.drive_version) + const release = releaseOf(pkg.version) + assert.equal(pkg.dependencies?.["@wirecat/cli-core"], release.core) + assert.equal(pkg.dependencies?.["@wirecat/cli-base"], release.framework) const baseline = values.baseline ? JSON.parse(readFileSync(values.baseline, "utf8")) : undefined if (baseline) { assert.equal(baseline.passed, true) diff --git a/performance/documents/offline-catalog/live.test.mjs b/performance/documents/offline-catalog/live.test.mjs index bb4320d..4cfe764 100644 --- a/performance/documents/offline-catalog/live.test.mjs +++ b/performance/documents/offline-catalog/live.test.mjs @@ -9,6 +9,11 @@ import { checkFixture } from "./live-check.mjs" import { initialize, validateManifest, workspaceEnvironment } from "./live-input.mjs" const runner = fileURLToPath(new URL("./live.mjs", import.meta.url)) +const testPackage = { + name: "@wirecat/drive-cli", + version: "0.7.0", + dependencies: { "@wirecat/cli-core": "0.278.0", "@wirecat/cli-base": "0.28.2" }, +} const fixture = () => ({ schema_version: 1, drive_version: "0.7.0", @@ -57,7 +62,9 @@ const setup = (change = {}) => { if (group === "documents" && action === "show") return item({ metadata: { id: "doc" }, text: "acceptance-new", tabs: ["one", "two"] }) if (group === "documents" && action === "export") { - writeFileSync(words[words.indexOf("--out") + 1], "# Acceptance") + const offset = words.indexOf(change.version && change.version !== "0.7.0" ? "--output" : "--out") + assert.ok(offset >= 0) + writeFileSync(words[offset + 1], "# Acceptance") return item({}) } if (group === "documents" && action === "context") @@ -69,7 +76,12 @@ const setup = (change = {}) => { }) throw new Error("unexpected fixture command") } - return { workspace, calls, invoke, manifest: validateManifest(fixture()) } + return { + workspace, + calls, + invoke, + manifest: validateManifest({ ...fixture(), drive_version: change.version ?? "0.7.0" }), + } } test("manifest rejects unknown fields, wrong release, duplicate ids and undeclared parents", () => { @@ -78,7 +90,7 @@ test("manifest rejects unknown fields, wrong release, duplicate ids and undeclar f.extra = true }, (f) => { - f.drive_version = "0.8.0" + f.drive_version = "0.9.0" }, (f) => { f.documents[0].id = "root" @@ -124,6 +136,12 @@ test("cached verification checks hierarchy, own-source search, tabs, exports and assert.ok(args.calls.every((call) => call.offline)) assert.equal(result.references.length, 3) }) +test("reviewed 0.8 uses --output while 0.7 keeps --out", () => { + const args = setup({ version: "0.8.0" }) + checkFixture(args) + assert.ok(args.calls.some((call) => call.words.includes("--output"))) + assert.ok(args.calls.every((call) => !call.words.includes("--out"))) +}) test("wrong root refuses before any Google access or local annotation", () => { const args = setup({ extraRoot: true }) assert.throws(() => checkFixture({ ...args, refresh: true, annotate: true })) @@ -158,10 +176,7 @@ test("entry point refuses unknown flags, missing profile and existing report bef assert.ok(!existsSync(join(workspace, "missing-profile"))) initialize(join(workspace, "state"), "synthetic") mkdirSync(join(workspace, "package")) - writeFileSync( - join(workspace, "package/package.json"), - JSON.stringify({ name: "@wirecat/drive-cli", version: "0.7.0" }), - ) + writeFileSync(join(workspace, "package/package.json"), JSON.stringify(testPackage)) writeFileSync(join(workspace, "manifest.json"), JSON.stringify(fixture())) const output = join(workspace, "report.json") writeFileSync(output, "preserve") @@ -187,7 +202,7 @@ test("failed CLI output never leaks through stdout, diagnostics or a private fai initialize(join(root, "workspace"), "synthetic") const pkg = join(root, "package") mkdirSync(join(pkg, "dist/bin"), { recursive: true }) - writeFileSync(join(pkg, "package.json"), JSON.stringify({ name: "@wirecat/drive-cli", version: "0.7.0" })) + writeFileSync(join(pkg, "package.json"), JSON.stringify(testPackage)) writeFileSync( join(pkg, "dist/bin/drive.js"), 'process.stdout.write(JSON.stringify({error:{code:"synthetic_failure",message:"SYNTHETIC_TOKEN_CANARY"}})); process.stderr.write("SYNTHETIC_TOKEN_CANARY"); process.exitCode=1;',