Reloading a database after new events land costs about 0.45 s even with the incremental path in place. The expensive part is no longer decoding (segments are cached, and only newly listed ones are decoded) but rebuilding the query engine:
- projection tables ≈ 292 ms
- readable event batches ≈ 124 ms
Both are rebuilt from scratch on every refresh, and both are cacheable.
What to do
In crates/attemptdb-query (cache.rs, tables.rs):
- Cache readable Arrow batches per segment the way
ScanCache already caches decoded segments, so a refresh concatenates cached batches instead of re-encoding every event.
- Build the projection tables from typed column builders over the projection that the incremental projector already produced, instead of going through the generic
TableBuilder row-by-row path.
Done when
cargo run -p attemptdb-bench -- step refresh shows a materially lower reload time at 200 k events (today: 0.5 s against 5.6 s from scratch), the numbers land in docs/benchmarks.md, and cargo test -p attemptdb-query -p attemptdb-ui -p attemptdb-mcp -p attemptdb-server stays green — the cache is shared by all four.
Performance work with a benchmark that already exists to prove it. Not a first issue, but well bounded.
Reloading a database after new events land costs about 0.45 s even with the incremental path in place. The expensive part is no longer decoding (segments are cached, and only newly listed ones are decoded) but rebuilding the query engine:
Both are rebuilt from scratch on every refresh, and both are cacheable.
What to do
In
crates/attemptdb-query(cache.rs,tables.rs):ScanCachealready caches decoded segments, so a refresh concatenates cached batches instead of re-encoding every event.TableBuilderrow-by-row path.Done when
cargo run -p attemptdb-bench -- step refreshshows a materially lower reload time at 200 k events (today: 0.5 s against 5.6 s from scratch), the numbers land indocs/benchmarks.md, andcargo test -p attemptdb-query -p attemptdb-ui -p attemptdb-mcp -p attemptdb-serverstays green — the cache is shared by all four.Performance work with a benchmark that already exists to prove it. Not a first issue, but well bounded.