Repository navigation
perf(perm-compile): read only the profile's catalogs' grants and rules - #845
Merged
Merged
Conversation
`ir_load::load_ep` read every grant and rule in the installation on each profile's compile and OPA rebuild, though the lowerings skip every grant and path rule on a catalog the profile does not reach. With N profiles moving together that cost O(N^2): measured, `load_ep` was 56 of 61 ms of `snapshot_parts_via` at N = 10 and grew with the grants read. It now reads the grants and path rules on the profile's reachable catalogs plus every tag rule (`list_grants_for_catalogs_via`, `list_rules_for_catalogs_via`). One rebuild is flat at about 47 ms; the last of 10 profiles sharing a tag service goes current in 493 ms p50 (954 ms before), 20 in 971 ms (3113 ms). Principals are expanded for those rows only, so a compile mints a username only for someone its own policies name. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This is the first Task 26 follow-up: a profile's compile and its OPA rebuild now read only the grants and rules that profile can lower.
The cost (measured first).
ir_load::load_epread every grant and every rule in the installation, on every compile and every OPA rebuild. The lowerings then skipped every grant and path rule on a catalog the profile does not reach.load_epwas 55.9 ms of the 61.2 mssnapshot_parts_viaat N = 10, and it grew with the grants read (820 × N).Changes
egret-permgains two reads:list_grants_for_catalogs_via: the grants on the given catalogs;list_rules_for_catalogs_via: every tag-selected rule, plus the path rules on the given catalogs.They share row parsing with
list_grants_viaandlist_rules_via, which are unchanged and still serve the API listings.load_epreads through them, using the catalogs the profile reaches (catalogs_for_service_via, already read just before).Compiled output is unchanged.
lower_grantsandlower_rulesalready skip every row on another catalog.Side effect: fewer usernames minted. Principals are now expanded for these rows only, so a compile mints a username only for someone its own policies name.
premint_model_usernamesstill sweeps the whole model, and its doc comment now says it covers a superset of what each compile reads.Harness:
opa_shared_tag_service_refresh_latencynow timesload_epalone and prints how many grants it read;harness_support::load_ep_model_ids_for_test.Report:
docs/en/reports/2026-11-06-gov-sp2b-opa-latency.mdgains the section "Follow-up 2026-11-09: the loader reads only the profile's catalogs".Results
Shared-tag refresh, release build, 10 runs per N, p50 in ms:
Test plan
the_loader_reads_only_the_grants_and_rules_its_catalogs_reach(tests/metadata_tables.rs). It sets up two Trino profiles, each on its own catalog, with a grant and a path rule on each catalog and one tag rule. It checks that each profile reads its own grant and path rule and the tag rule, and neither of the other profile's rows.load_epback on the installation-wide reads: fails at "not the other catalog's grant";cargo test -p egret-perm: 69 passed;cargo test -p egret-perm-compile --features harness: 1019 passed, 9 ignored, includinggolden_compileon Ranger;golden_compilewithEGRET_GOLDEN_MECHANISM=opa: passed.cargo fmt --checkand clippy-D warningson both crates (all targets, withharness) are clean.Documentation
egret-data/docs#72 updates the "Shared tag service" note on the Trino on OPA page.
Checklist
🤖 Generated with Claude Code