[Windows] Repeated incremental re-index leaves an oversized/stuck SQLite WAL — node count frozen while HEAD pointer advances; delete_project returns "Permission denied"

[breiti35]

Environment

• OS: Windows 11 Pro (10.0.26200)
• codebase-memory-mcp version: 0.9.0
• Store location: C:\Users\<user>\.cache\codebase-memory-mcp\
• Two projects in the same store: one healthy, one affected

Summary

On Windows, after a project has been re-indexed many times (incrementally) — including some runs interrupted mid-write (host process closed, MCP server reinstalled/reconnected) — the project's index stops reflecting new code. index_repository reports status: "indexed" and advances the stored HEAD sha, but the node/edge counts stay frozen and newly added symbols never appear in the graph. A second, untouched project in the same store keeps working normally.

Observed evidence

1. Graph frozen while pointer advances. index_status for the affected project shows the new head_sha (latest commit) but nodes stays at the old value (e.g. 6023) across repeated index_repository (both moderate and full).
2. New symbols missing from the graph but present on disk.
   • search_graph(name_pattern=".*NewSymbol.*") → total: 0.
   • search_code("NewSymbol") (live grep) → finds it in the working-tree files.
     So extraction sees the code, but the graph write does not persist.
3. expected_nodes != nodes on one incremental run: response returned nodes: 5979 but expected_nodes: 5998 — i.e. the indexer computed the correct new total but the persisted graph did not change; on the next run the pointer had advanced and it hard-skipped (nodes == old, no expected_* shown).
4. Oversized / stale WAL. In the cache dir, the affected project's write-ahead log is huge and frozen in time, while the healthy project's WAL is checkpointed:

   <healthy-project>.db-wal          0 bytes        (today)      <- healthy
   <affected-project>.db-wal    45,039,752 bytes    (yesterday, frozen)  <- affected
   

   The main .db and -shm have today's mtime; only the -wal is stuck at an old timestamp.
5. delete_project fails. Calling delete_project on the affected project returns:

   {"project":"<affected-project>","status":"delete_failed","error":"Permission denied"}
   

   Deleting a different (freshly created) project name in the same store succeeded, so it is specific to this project's on-disk store — consistent with the file being locked / the stuck WAL.

Steps to reproduce (approximate)

1. On Windows, index a repo (index_repository, full) — works, graph correct.
2. Make commits and re-run index_repository (incremental) repeatedly; occasionally interrupt the MCP server mid-run (close host / reinstall).
3. After a while: new commits' symbols stop appearing. index_status shows the new head_sha but frozen node counts; search_graph returns 0 for new symbols while search_code finds them; the project's .db-wal has grown large and stopped checkpointing.
4. delete_project on that project → delete_failed: Permission denied.

Impact

• The graph silently serves stale results after code changes (no error surfaced — status: "indexed" is misleading).
• Reinstalling the tool does not fix it, because the reinstall replaces the binary but leaves the broken ~/.cache/codebase-memory-mcp/<project>.db* files in place.
• The only recovery is manual deletion of the project's store files while the server is stopped, then a from-scratch index.

Workaround

1. Stop the MCP server (release the file lock).
2. Delete <cache>\<project>.db, <project>.db-wal, <project>.db-shm.
3. Restart the server and run index_repository (full) once — the from-scratch build persists correctly and creates a fresh, checkpointed WAL.

Suggestions

• Force a WAL checkpoint (PRAGMA wal_checkpoint(TRUNCATE)) after each index run, and/or open the DB with a bounded journal_size_limit so the WAL cannot grow unbounded / get orphaned on Windows.
• Make index_repository detect a persist mismatch (expected_nodes != nodes) and surface an error instead of reporting status: "indexed".
• Have delete_project return a clearer error (and a --force/retry path) when the store file is locked, since a stuck store is exactly when users need to delete it.

Possibly related

• Incremental re-index is O(changed_files × total_nodes) — gbuf.delete_by_file full-scans the whole node buffer per file; a single day's changes make it slower than a full rebuild #1172 (incremental re-index is unreliable; community workaround is delete_project + full re-index)
• docs: add a Windows known-issues / troubleshooting section #1170 (Windows known-issues / troubleshooting doc)
• Can't upgrade codebase-memory-mcp on windows platform. Can't uninstall. #1168 (can't upgrade / uninstall on Windows — reinstall does not clear state)

[DeusData]

Thanks for the detailed consistency evidence and recovery procedure. This is now a high-priority 0.9.1-rc bug under the silent index-corruption epic #391. Review needs to cover checkpoint/locking behavior, commit-pointer advancement ordering, persisted-count validation, and reliable deletion recovery on Windows.