Skip to content

perf(crafter): avoid redundant SHA256 pass on every CAS upload#3142

Merged
matiasinsaurralde merged 1 commit into
mainfrom
fix/cas-sha256-dup-computation
May 25, 2026
Merged

perf(crafter): avoid redundant SHA256 pass on every CAS upload#3142
matiasinsaurralde merged 1 commit into
mainfrom
fix/cas-sha256-dup-computation

Conversation

@matiasinsaurralde
Copy link
Copy Markdown
Contributor

Summary

When uploading a material to a CAS backend, the file's SHA256 was being computed twice and the file opened three times:

  1. fileStats opens the file, computes SHA256, rewinds — producing an *os.File that the caller holds.
  2. uploadAndCraft then called Uploader.UploadFile, which re-opened the file, recomputed SHA256, rewound, then streamed it.

The reader from step 1 was already in the right state to be streamed. This change calls Uploader.Upload with that reader and the digest already produced by fileStats, eliminating the redundant open + hash pass while leaving the CAS protocol and on-the-wire bytes unchanged.

Per material, local work drops from 3 reads + 2 SHA256 passes to 2 reads + 1 SHA256 pass. Savings scale linearly with material size.

Also closes a file-handle leak in fileStats when SHA256 or Seek returns an error.

Uploader.UploadFile is retained — the CLI's artifact upload command still uses it.

@chainloop-platform
Copy link
Copy Markdown
Contributor

chainloop-platform Bot commented May 21, 2026

AI Session Analysis

Avg score Sessions Failing policies Attribution Files Lines Total Duration
🟡 80% 1 ✅ 0 4% AI / 96% Human 26 +76 / -49 42m13s

🟡 80% — 4% AI — ✅ All policies passing

May 21, 2026 01:44 UTC · 42m13s · $11.74 · 193 in / 114.1k out · claude-code 2.1.145 (claude-opus-4-7)

View session details ↗

Change Summary

  • Fixes SHA256 double-computation bug in artifact upload by reusing an already-open, already-hashed reader.
  • Fixes a file-handle leak on SHA256/Seek error paths in fileStats.
  • Updates 25 test files to match the new Upload interface signature (license-header and parameter changes).

AI Session Overall Score

🟡 80% — Solid root-cause fix with good scope discipline, but missing a targeted deduplication test.

AI Session Analysis Breakdown

🟢 90% · alignment

🟢 AI precisely implemented the requested SHA256 deduplication fix without any scope drift. · High Impact

🟢 90% · scope-discipline

No notes.

🟢 90% · solution-quality

🟢 Fix targets the exact root layer, reusing an already-hashed reader instead of masking the symptom. · High Impact

🟢 90% · user-trust-signal

No notes.

🟡 62% · context-and-planning

🟢 Thorough written analysis with risk assessment and speedup estimate produced before any code edits. · Medium Impact

🟠 Initial prompt was one sentence referencing a prior session artifact with no explicit goals or constraints. · Medium Severity

💡 State optimization goal, scope, and risk tolerance up front rather than relying on prior session context.

🟡 No structured plan produced before multi-file edits; written analysis doc served as informal substitute. · Low Severity

🟡 62% · verification

🟠 No new test added asserting SHA256 deduplication; 25 test changes are license-header-only updates. · Medium Severity

💡 Add a test asserting the Uploader is called once per artifact upload, not twice.

🟠 User never confirmed the fixed behavior worked correctly end-to-end; final message only requested a PR title. · Medium Severity


File Attribution

░░░░░░░░░░░░░░░░░░░░ 4% AI / 96% Human

Status Attribution File Lines
modified human pkg/attestation/crafter/materials/ghas_test.go +5 / -4
modified human pkg/attestation/crafter/materials/cyclonedxjson_test.go +4 / -3
modified human pkg/attestation/crafter/materials/openapi_test.go +4 / -3
modified human pkg/attestation/crafter/materials/sarif_test.go +4 / -3
modified ai pkg/attestation/crafter/materials/materials.go +5 / -1
modified human pkg/attestation/crafter/crafter_test.go +3 / -2
modified human pkg/attestation/crafter/materials/asyncapi_test.go +3 / -2
modified human pkg/attestation/crafter/materials/blackduck_test.go +3 / -2
modified human pkg/attestation/crafter/materials/evidence_test.go +3 / -2
modified human pkg/attestation/crafter/materials/gitlab_test.go +3 / -2
modified human pkg/attestation/crafter/materials/helmchart_test.go +3 / -2
modified human pkg/attestation/crafter/materials/jacoco_test.go +3 / -2
modified human pkg/attestation/crafter/materials/junit_xml_test.go +3 / -2
modified human pkg/attestation/crafter/materials/openvex_test.go +3 / -2
modified human pkg/attestation/crafter/materials/runnercontext_test.go +3 / -2
modified human pkg/attestation/crafter/materials/twistcli_scan_test.go +3 / -2
modified human pkg/attestation/crafter/materials/zap_test.go +3 / -2
modified human pkg/attestation/crafter/materials/attestation_test.go +2 / -2
modified human pkg/attestation/crafter/materials/slsaprovenance_test.go +2 / -2
modified human pkg/attestation/crafter/materials/artifact_test.go +2 / -1
modified human pkg/attestation/crafter/materials/chainloop_ai_agent_config_test.go +2 / -1
modified human pkg/attestation/crafter/materials/chainloop_ai_coding_session_test.go +2 / -1
modified human pkg/attestation/crafter/materials/csaf_test.go +2 / -1
modified human pkg/attestation/crafter/materials/gitleaks_test.go +2 / -1
modified human pkg/attestation/crafter/materials/graphql_test.go +2 / -1

…and 1 more file(s).


Policies (4)

Status Policy Material Messages
✅ Passed ai-config-ai-agents-allowed ai-coding-session-70c6c7 -
✅ Passed ai-config-no-dangerous-commands ai-coding-session-70c6c7 -
✅ Passed ai-config-no-secrets ai-coding-session-70c6c7 -
✅ Passed ai-config-mcp-servers-allowed ai-coding-session-70c6c7 -

Powered by Chainloop and Chainloop Trace

Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 26 files

Re-trigger cubic

@matiasinsaurralde matiasinsaurralde requested a review from a team May 21, 2026 02:34
Copy link
Copy Markdown
Member

@migmartri migmartri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Of the curiosity how did you find this?

…o CAS

uploadAndCraft already computes the artifact's SHA256 via fileStats, then
called Uploader.UploadFile which re-opened the file and re-hashed it before
streaming. Switch to Uploader.Upload with the reader and digest from
fileStats so the file is opened once and hashed once per material.

Also fix a file-handle leak in fileStats when SHA256 or Seek fails.

Chainloop-Trace-Sessions: 70c6c792-4598-426b-9e75-482bcb10498f

Signed-off-by: Matías Insaurralde <matias@chainloop.dev>
@matiasinsaurralde matiasinsaurralde force-pushed the fix/cas-sha256-dup-computation branch from c8703f8 to c86adab Compare May 25, 2026 05:35
@matiasinsaurralde
Copy link
Copy Markdown
Contributor Author

Of the curiosity how did you find this?

Mostly digging into casclient code and uploader logic plus some Claude insights.

Upload and UploadFile have been around for a long time. UploadFile just computes the hash and ends up calling Upload, so in this change we can call it directly from materials.

@matiasinsaurralde matiasinsaurralde merged commit a221b6c into main May 25, 2026
15 checks passed
@matiasinsaurralde matiasinsaurralde deleted the fix/cas-sha256-dup-computation branch May 25, 2026 05:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants