A hash trie based pebble differ

[mj-palanker]

adds a new index which groups grants by their "bucket" (a hash of the existing index's keys) so they are easy to read in order, stores the hash of the content of the grant on that index, builds a trie over said hashes.

This lets us do neat stuff like diff grants within an entitlement cheaply.
we compare just the root hashes first, and can dive lower when they don't match.
Even the leaf nodes have hashes and we can find either missing or differing hashes to find the diffing grants within the bucket.

[github-actions]

🟡 Suggestion: The err variable captured by the callback closure is silently overwritten when IterateGrantsBySync returns nil (which it does when the callback returns false). If batch.Set fails, err = setErr fires, then err = e.IterateGrantsBySync(...) overwrites it with nil, losing the error. Use a separate var innerErr error like diffGrants does.

Suggested change

	err = e.IterateGrantsBySync(ctx, syncID, func(r *v3.GrantRecord) bool {
	hk := grantHashIndexKey(idBytes, r)
	if hk == nil {
	return true
	}
	if setErr := batch.Set(hk, grantContentHash(r), nil); setErr != nil {
	err = setErr
	return false
	}
	return true
	})
	var innerErr error
	err = e.IterateGrantsBySync(ctx, syncID, func(r *v3.GrantRecord) bool {
	hk := grantHashIndexKey(idBytes, r)
	if hk == nil {
	return true
	}
	if setErr := batch.Set(hk, grantContentHash(r), nil); setErr != nil {
	innerErr = setErr
	return false
	}
	return true
	})
	if err == nil {
	err = innerErr
	}

[github-actions]

General PR Review: A hash trie based pebble differ

Blocking Issues: 0 | Suggestions: 1 | Threads Resolved: 0
Review mode: full
View review run

Review Summary

This PR adds a per-entitlement grant digest (XOR set hash over a flat hash table) that enables the diff driver to skip unchanged entitlements entirely and localize changes to specific principal-hash buckets, replacing the O(all applied grants) full scan with a targeted comparison. The implementation spans new digest core logic, grant-specific wiring, incremental maintenance on all post-seal mutation paths, an index migration backfill, and comprehensive tests including cross-width comparison and incremental-equals-rebuild invariant checks. One minor error-handling issue found in the migration backfill.

Security Issues

None found.

Correctness Issues

None found.

Suggestions

• pkg/dotc1z/engine/pebble/index_migrations.go:120-130: Callback closure captures the same err variable that receives the IterateGrantsBySync return value, silently losing a batch.Set error when the iterator returns nil.

Prompt for AI agents

Verify each finding against the current code and only fix it if needed.

## Suggestions

In `pkg/dotc1z/engine/pebble/index_migrations.go`:
- Around line 120-130: The `err` variable is shared between the `IterateGrantsBySync` return
  assignment and the callback closure's `err = setErr`. When `batch.Set` fails inside the
  callback, `err = setErr` fires and the callback returns false, but then
  `err = e.IterateGrantsBySync(...)` overwrites it with nil (since IterateGrantsBySync returns
  nil when the callback returns false). Fix: declare a separate `var innerErr error` before
  the call, use `innerErr = setErr` inside the callback, and after the call add
  `if err == nil { err = innerErr }`.

[github-actions]

No blocking issues found.