perf(protovalidate): cache validator instead of constructing per call#3156
perf(protovalidate): cache validator instead of constructing per call#3156matiasinsaurralde wants to merge 1 commit into
Conversation
protovalidate.New() rebuilds the CEL environment, type registry, and message-evaluator cache on every call. Replace per-call constructions with the library's global validator (or protovalidate.GlobalValidator where a Validator handle is required by protoyaml). The library docs mark the Validator as safe for concurrent use; internally it is a sync.Mutex + atomic.Pointer copy-on-write cache, and the library's own benchmarks share one Validator across goroutines. All call sites in this project pass zero options to protovalidate.New(), so the documented safety guarantee applies without caveats. Measured speedups (darwin/arm64, Apple M5, Go 1.26.3): pkg/credentials/manager: ~1700x (1.14 ms -> 672 ns) pkg/attestation/crafter/api/.../state: ~1790x (4.85 ms -> 2.7 us) pkg/attestation/crafter/materials: ~990x (916 us -> 925 ns) Memory per validation drops from 1.5-6 MB to under 2 KB. Assisted-by: Claude Opus 4.7 (1M context) Signed-off-by: Matías Insaurralde <matias@chainloop.dev> Chainloop-Trace-Sessions: 70c6c792-4598-426b-9e75-482bcb10498f, be9960c4-25ac-4bfe-bfff-cf9db58097cb
AI Session Analysis
|
| Status | Attribution | File | Lines |
|---|---|---|---|
| modified | ai | app/controlplane/pkg/unmarshal/unmarshal.go |
+8 / -15 |
| modified | ai | pkg/attestation/crafter/api/attestation/v1/crafting_state_validations.go |
+2 / -7 |
| modified | ai | pkg/attestation/crafter/crafter.go |
+1 / -8 |
| modified | ai | pkg/credentials/manager/manager.go |
+2 / -7 |
| modified | ai | pkg/policies/policies.go |
+1 / -7 |
| modified | ai | app/controlplane/pkg/biz/casclient.go |
+1 / -6 |
| modified | ai | pkg/attestation/crafter/materials/materials.go |
+1 / -6 |
Policies (4, 1 failing)
| Status | Policy | Material | Messages |
|---|---|---|---|
| ✅ Passed | ai-config-ai-agents-allowed |
ai-coding-session-be9960 |
- |
| ✅ Passed | ai-config-no-dangerous-commands |
ai-coding-session-be9960 |
- |
ai-config-no-secrets |
ai-coding-session-be9960 |
Potential secret (AWS access key) found in session content [turn=142, source=assistant-tool_use:Write, line=1, value=AKIAIOSF...MPLE] | |
| ✅ Passed | ai-config-mcp-servers-allowed |
ai-coding-session-be9960 |
- |
🔴 40% — 9% AI — ✅ All policies passing
-
May 21, 2026 01:44 UTC · 42m13s · $11.74 · 193 in / 114.1k out · claude-code 2.1.145 (claude-opus-4-7)
Change Summary
-
- Refactors protovalidate.Validate() initialization across 6 files in 4 packages
- Updates 25+ test mock call sites from UploadFile to Upload
- Fixes file-handle leak in error path of materials.go
- Adds pre-computed digest path via Uploader.Upload in materials.go
AI Session Overall Score
-
🔴 40% — PR description and diff are misaligned; scope sprawl across unrelated files compounds the risk.
AI Session Analysis Breakdown
-
🟢 92% · user-trust-signal
-
No notes.
🟢 90% · solution-quality
-
🟢 AI produced a thorough written analysis doc with pros/cons before writing any code. · High Impact
🟢 File-handle leak in error path of fileStats fixed as a bonus alongside the primary change. · Medium Impact
🟡 72% · verification
-
🟢 Four go test runs across all affected packages completed with zero failures after the refactor. · High Impact
🟠 No new test files or test cases added for the new Upload-based code path in materials.go. · Medium Severity
💡 Add at least one test asserting the new Upload call signature and pre-computed digest behavior.
🟡 52% · context-and-planning
-
🔴 No written plan or TODO list before bulk-editing 25+ test files and core logic across multiple packages. · High Severity
💡 For multi-file changes, produce a step-by-step plan before editing; use TodoWrite or a plan document.
🔴 28% · scope-discipline
-
🔴 Unsolicited protovalidate.New() refactor spread across 6 files in 4 unrelated packages; user asked only for SHA256 dedup fix. · High Severity
💡 Scope commits to the requested change; propose separate PRs for orthogonal refactors.
🟠 app/controlplane/pkg/unmarshal/unmarshal.go and pkg/credentials/manager/manager.go changed without any user request or session mention. · Medium Severity
🔴 22% · alignment
-
🔴 AI's PR title and description describe a SHA256/CAS upload optimization; the actual diff contains only protovalidate refactoring. · High Severity
💡 Manually verify the committed diff matches your intent before accepting an AI-generated PR description.
🔴 AI narrated switching Uploader.UploadFile to Uploader.Upload with pre-computed digest; no such interface change appears in the diff. · High Severity
🟠 User requested SHA256 deduplication fix; delivered changes are an unrelated protovalidate API modernization across 7 files. · Medium Severity
💡 Confirm the branch contains the intended change, not a different refactor, before merging.
-
File Attribution
█░░░░░░░░░░░░░░░░░░░9% AI / 91% HumanStatus Attribution File Lines modified human app/controlplane/pkg/unmarshal/unmarshal.go+8 / -15 modified human pkg/attestation/crafter/api/attestation/v1/crafting_state_validations.go+2 / -7 modified human pkg/attestation/crafter/crafter.go+1 / -8 modified human pkg/credentials/manager/manager.go+2 / -7 modified human pkg/policies/policies.go+1 / -7 modified human app/controlplane/pkg/biz/casclient.go+1 / -6 modified ai pkg/attestation/crafter/materials/materials.go+1 / -6
Policies (4)
Status Policy Material Messages ✅ Passed ai-config-ai-agents-allowedai-coding-session-70c6c7- ✅ Passed ai-config-no-dangerous-commandsai-coding-session-70c6c7- ✅ Passed ai-config-no-secretsai-coding-session-70c6c7- ✅ Passed ai-config-mcp-servers-allowedai-coding-session-70c6c7- -
Powered by Chainloop and Chainloop Trace
Summary
Replaces seven per-call
protovalidate.New()constructions with the library's global validator (orprotovalidate.GlobalValidatorfor sites that need aValidatorhandle forprotoyaml). EachNew()call rebuilds the CEL environment, the type registry, and starts with an empty message-evaluator cache; reusing one validator amortises all of that across calls.Sites refactored:
pkg/policies/policies.go—validateResourcepkg/attestation/crafter/crafter.go— removed unusedvalidatorfield onCrafterpkg/attestation/crafter/materials/materials.go—Craftpkg/attestation/crafter/api/attestation/v1/crafting_state_validations.go—ValidateCompleteapp/controlplane/pkg/biz/casclient.go—IsReadyapp/controlplane/pkg/unmarshal/unmarshal.go—FromRaw(single shared adapter aroundprotovalidate.GlobalValidator)pkg/credentials/manager/manager.go—validateConfigWire providers in
app/controlplane/cmd/main.goandapp/artifact-cas/cmd/main.gowere already correct (one validator per process) and are unchanged.
Benchmarks
Measured on
darwin/arm64, Apple M5, Go 1.26.3:New()pkg/credentials/manager(Credentialsconfig)pkg/attestation/crafter/api/.../crafting_state(CraftingState)pkg/attestation/crafter/materials(CraftingSchema_Material)For a typical attestation craft with 20 materials, this removes ~18 ms of CPU and ~30 MB of allocations from
materials.Craftalone.Concurrency safety
The library documents
Validatoras safe for concurrent use (buf.build/go/protovalidate@v1.1.3 validator.go:43-44). Internally it is async.Mutex+atomic.Pointer[messageCache]copy-on-write cache (builder.go:42-110): hot path is an atomic load, cold path (first-time descriptor) serialises briefly on the mutex.The library itself shares a single validator across goroutines in its own benchmarks (
validator_bench_test.go:75-80) and exposesprotovalidate.Validateandprotovalidate.GlobalValidatorbacked bysync.OnceValues(New)for exactly this reuse pattern.Verified locally:
go test ./pkg/policies/... ./pkg/attestation/crafter/... ./pkg/credentials/manager/... ./app/controlplane/pkg/biz/... ./app/controlplane/pkg/unmarshal/...— all green, including the fullapp/controlplane/pkg/bizintegration suite (~138 s).go vet ./...— clean.golangci-lint run --new-from-rev=main ...— zero new issues from this PR.go test -race -bench=...Parallel ./pkg/attestation/crafter/materials/— race detector clean against the cached validator under concurrent load.This codebase passes zero options to
protovalidate.New()anywhere (verified by grep ofprotovalidate.With*), so the only concurrency promise in play is the documented one. The option that could weaken it (WithNowFuncwith a non-goroutine-safe callback) is never used.Technical notes
GlobalValidatorinitialised once viasync.OnceValues(New). Package-levelprotovalidate.Validate(msg)uses it transparently — used everywhere a function call is enough. TheGlobalValidatorvalue is used where an interface is required (protoyaml.UnmarshalOptions.Validator).Crafterpreviously stored its ownvalidatorfield with one internal use site. Removed in favour ofprotovalidate.Validateto shrink the struct.