feat: implement high-performance atom cache and maintenance tools (Issue #29)

[c-ferrier]

This PR implements Phase 1: The Immutable Atom Cache for #29, providing a foundational performance boost for all Lore queries.

Key Improvements:

• ~52% Speedup: Confirmed by eliminating the N+1 Git process bottleneck during repository scanning.
• Sharded Storage: Implements an immutable, sharded plain-text cache in .lore/cache/atoms.
• Setup Automation: lore init now automatically ensures .lore/cache is ignored by your .gitignore.
• Maintenance Tools: Added lore cache --clean to safely manage or reset local storage.
• Full Verification: 444 passing unit tests and updated architectural documentation.

This unified implementation replaces the draft approach in the previously closed PR #47.

[c-ferrier]

Ready for feedback or merging.

[Ian-stetsenko]

Nice work on this one. Architecture looks good — clean interface, Null Object done right, files-only as we discussed. Few things to address:

1. process.exit(1) in cache.ts — Project convention is process.exitCode = 1; return; (see CLAUDE.md). process.exit() kills the process immediately and can lose pending I/O output.

2. error: any in cache.ts and atom-cache.ts — Should be error: unknown with type narrowing (error instanceof Error ? error.message : String(error) for messages, (error as NodeJS.ErrnoException).code for filesystem errors). Matches the pattern in config-loader.ts and head-lore-id-reader.ts.

3. Corrupted cache file returns garbage — If a cache file is partially written or contains bad data, content.split('\n') returns garbage strings that silently corrupt query results. getFiles should return null (treat as cache miss) when content looks wrong (e.g., contains NUL bytes or is empty when it shouldn't be).

4. No test for corrupted cache — Add a test that writes bad content to a cache path and verifies getFiles returns null instead of garbage.

[c-ferrier]

Updates in a new commit in the pull request. Let me know what you think.

[Ian-stetsenko]

Pushed fixes for all review findings:

Security:

• Added hex hash validation (/^[0-9a-f]{7,64}$/i) in AtomCache — invalid hashes return null/no-op instead of path traversal. Added test for ../../../etc/passwd rejection.

DIP compliance:

• Removed concrete NullAtomCache import from AtomRepository — constructor now requires IAtomCache explicitly. Composition root handles the wiring.

Type safety:

• Replaced filesPerCommit[index] as readonly string[] cast with null guard that throws on BUG condition.

Convention fixes:

• Added pre-parse documentation to shouldBypassCache
• Clarified ensureCacheIgnored uses cwd intentionally (matches where lore init creates .lore/)
• Removed unused CONFIG_DIR import from cache.ts
• Fixed duplicate step "8" numbering in main.ts
• Fixed blank lines in atom-cache.ts and atom-repository.ts

451 tests pass, typecheck clean.

[c-ferrier]

I've pushed two new commits to address the final feedback items:

1. Architectural Refactor: Root Resolver

Moved root-resolver.ts from src/util/ to src/services/. This aligns with the project convention that util/ should be for pure leaf nodes, while services handle I/O and more complex orchestration. All internal references and tests have been updated and verified.

2. Robustness Verification: Self-Healing Cache

Added a dedicated unit test in tests/unit/services/atom-cache.test.ts to empirically verify the "self-healing" mechanism we discussed. The test simulates binary corruption (NUL bytes) and empty files, confirming they are correctly detected as cache misses. It then verifies that the repository successfully overwrites the corrupted file with fresh data via an atomic rename.

---

Discussion: .gitignore and Monorepo Strategy

Regarding the .gitignore location in subdirectories, I wanted to clarify the intended design before making changes.

The current implementation follows a Monorepo Override pattern. The Lore configuration system is designed to walk up the directory tree and merge configurations, allowing sub-projects to have their own .lore/config.toml (e.g., for project-specific custom trailers or stricter validation).

By creating the .gitignore and .lore folder in the current directory (where lore init is run):

• We ensure that a sub-project's local cache remains ignored by its local .gitignore.
• We support isolated configuration boundaries within a larger repository.

However, I agree this could lead to "accidental fragmentation" in a standard single-project repository if run from a subdirectory.

How would you like to handle this duality?

1. Status Quo: Keep as-is to prioritize monorepo flexibility.
2. Explicit Intent Lore: Add a check to detect if we are already inside a Lore-enabled project. If so, lore init could require a --force flag or prompt for confirmation before creating a nested configuration.
3. Explicit Intent GIT: Add a check to detect if we are not at a git root. If so, lore init could require a --force flag or prompt for confirmation before creating a nested configuration.
4. Strict Root: Force lore init to always resolve to the Git root.

I'd love your take on the preferred architectural direction for monorepos before finalizing this part.