Skip to content

chore(sdk): drop dead TDFConfig fields and deprecate the TDFFormat enum - #3947

Merged
dmihalcik-virtru merged 1 commit into
mainfrom
dspx-2604-18-dead-config
Sep 25, 2026
Merged

dmihalcik-virtru merged 1 commit into
mainfrom
dspx-2604-18-dead-config

Conversation

@dmihalcik-virtru

@dmihalcik-virtru dmihalcik-virtru commented Sep 1, 2026 •

Copy link
Copy Markdown
Member

Part 18 of 20 in the DSPX-2604 re-cut. Base branch: dspx-2604-17-createtdf-delegates.

This stack replaces #3782 / #3865 / #3921, which stay open and untouched
until it lands. Nothing here is a rebase of those branches — the work was
re-cut from the ticket so each PR stands on its own.

Proposed Changes

TDFConfig.enableEncryption was set to true at construction and never read
again; nothing could turn it off and nothing branched on it. tdfFormat was
likewise fixed at JSONFormat forever. readActionName was a leftover constant
with no references. All three are gone.

rootIntegrityAlg and segmentIntegrityAlg go with them. #3940 (revised) fixed
the chunked writer's root to HS256 and its segments to GMAC, and #3946 made
CreateTDF delegate to it, so nothing reads either field any more. No exported
option ever set one, and the defaults they held were exactly those two
algorithms, so the manifest is byte-identical. They are unexported, so they
are deleted outright rather than deprecated.

TestIntegrityAlgDefaults asserted on those fields and could not survive
them. The invariant it guarded is now asserted on a manifest CreateTDF
actually produced — TDFSuite.testEncrypt checks manifest.Algorithm is
HS256 and manifest.SegmentHashAlgorithm is GMAC — so every encrypt case in
the suite carries it rather than one test reading back a struct field.

TDFFormat, JSONFormat, and XMLFormat are exported, so they are deprecated
rather than deleted. XML manifests were never implemented and the enum has no
remaining consumer inside the SDK.

Checklist

  • I have added or updated unit tests
  • I have added or updated integration tests (if appropriate)
  • I have added or updated documentation

Testing Instructions

make build && make test

Nothing reads any of the removed fields; the check is that the tree still
builds across sdk, service, otdfctl, examples and tests-bdd, and
that TDFSuite still emits HS256/GMAC on every encrypt case.

The full DSPX-2604 stack — 20 PRs
# PR Based on
01 #3930 chore: bump go.work toolchain to go1.25.12 and simplify an rt_test condition main
02 #3931 feat(sdk): make the zipstream clock injectable for deterministic ZIP output main
03 #3932 fix(sdk): reject a zipstream write set that omits segment 0 #3931
04 #3933 fix(sdk): map ReadAt plaintext offsets from cumulative segment sizes main
05 #3934 chore(sdk): extract integrityAlgorithmString, createPolicyBinding, signAssertions main
06 #3935 chore(sdk): add direct tests for createKeyAccess, encryptMetadata and tdfSalt main
07 #3936 fix(sdk): fill each segment with io.ReadFull and size the buffer to the input main
08 #3937 chore(cli): move streaming IO helpers into pkg main
09 #3938 fix(cli): stream encrypt instead of buffering the whole payload #3937
10 #3939 fix(cli): stream decrypt and inspect instead of buffering #3938
11 #3940 feat(sdk): add a chunked segment writer (experimental) dspx-2604-base-11 = #3932 + #3934 + #3935
12 #3941 fix(sdk): stop GetManifest from splitting the key under the lock #3940
13 #3942 fix(sdk): reject a chunked split naming a KAS with no resolved public key #3941
14 #3943 chore(sdk): alias experimental/tdf manifest and assertion types #3942
15 #3944 fix(sdk): emit spec-compliant key access in experimental/tdf and delegate Writer #3943
16 #3945 feat(sdk): accept io.Reader in CreateTDF and drop the 64 GB payload cap #3936
17 #3946 chore(sdk): rewrite CreateTDF on top of the chunked writer dspx-2604-base-17 = #3944 + #3945
18 #3947 chore(sdk): drop dead TDFConfig fields and deprecate the TDFFormat enum #3946
19 #3948 fix(cli): drop the encrypt-side stdin spool dspx-2604-base-19 = #3947 + #3939
20 #3949 feat(sdk): graduate the chunked writer to stable API #3948

Reviewable in parallel right now, since they sit directly on main and depend on
nothing else: 01, 02, 04, 05, 06, 07, 08.

Why three PRs have a dspx-2604-base-* base. A GitHub PR takes one base branch,
but 11, 17 and 19 each build on more than one parent. The base-* branches are empty
merge commits that exist only to join those parents so the PR diff shows exactly its
own change and nothing else. They contain no code, have no PR of their own, and go
away once their parents land — retarget the child onto main at that point.

Wants a cross-SDK xtest run before merge: 15, 17 (and therefore 20). They touch
the KAS wire format.

Red checks you may see are network flakes, not this stack. Four distinct ones hit
this batch and all clear on re-run: golangci-lint config verify timing out on
https://golangci-lint.run/.../golangci.v2.8.jsonschema.json (fails the whole go (<module>) job and fail-fast cancels its siblings), the bats installer getting a 403,
Docker Hub timing out on keycloak/keycloak:26.4, and buf reporting "the server
hosted at that remote is unavailable" while the Java SDK generates sources. The
govulncheck step also emits ##[error] annotations against the go1.25.11 stdlib, but
it is continue-on-error: true and never fails a job — 01 bumps the toolchain and
clears those annotations.

Summary by CodeRabbit

  • Bug Fixes

    • Policy creation now returns a validation error when supplied with an empty fully qualified name, rather than proceeding with invalid input.
  • Configuration

    • TDF configuration no longer exposes settings for enabling encryption, choosing a format, or selecting root and segment integrity algorithms. The format constants are marked deprecated; XML format is documented as never implemented. Existing applications that set these options may need to update their configuration.

@dmihalcik-virtru
dmihalcik-virtru requested review from a team as code owners September 1, 2026 02:57
@coderabbitai

coderabbitai Bot commented Sep 1, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Advanced

Run ID: a3366734-e2f1-43b7-ae86-091a465f46cd

📥 Commits

Reviewing files that changed from the base of the PR and between 5002041 and 6a92b71.

📒 Files selected for processing (1)
  • sdk/tdf.go
💤 Files with no reviewable changes (1)
  • sdk/tdf.go

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.


📝 Walkthrough

Walkthrough

TDF creation now resolves key access before writing payload segments and delegates segment writing and manifest finalization to a chunked writer. The SDK also removes unused configuration fields and updates policy-binding and encryption tests.

Changes

TDF SDK Updates

Layer / File(s) Summary
DEK splitting and key-access resolution
sdk/tdf.go, sdk/tdf_helpers_test.go
The SDK splits the DEK into shares, resolves template key access, builds policy objects from FQNs, and accepts a base64 policy string for policy binding. Related policy-binding tests now pass strings.
Chunked TDF creation and finalization
sdk/tdf.go, sdk/tdf_config.go, sdk/tdf_helpers_test.go, sdk/tdf_test.go
CreateTDF validates segment bounds and uses a chunked writer for segment output and manifest finalization. The SDK removes unused configuration fields, marks format declarations deprecated, and checks manifest integrity algorithms in encryption tests.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant SDK.CreateTDF
  participant newTDFChunkedWriter
  participant resolveKeyAccess
  participant chunkedWriter
  participant outputWriter
  SDK.CreateTDF->>newTDFChunkedWriter: Create writer before payload writing
  newTDFChunkedWriter->>resolveKeyAccess: Resolve policy and key access
  newTDFChunkedWriter-->>SDK.CreateTDF: Return chunked writer
  SDK.CreateTDF->>chunkedWriter: WriteSegment for each input segment
  chunkedWriter-->>SDK.CreateTDF: Return segment TDF data
  SDK.CreateTDF->>outputWriter: Copy segment TDF data
  SDK.CreateTDF->>chunkedWriter: Finalize with options
  chunkedWriter-->>SDK.CreateTDF: Return manifest and written byte count
Loading

Suggested reviewers: biscoe916

Merge Risk: ⚪ Minimal · up to 6a92b

The change has no identified merge-blocking issue in the supplied evidence; complete the stated build and test checks before merging.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the primary changes: removing unused TDFConfig fields and deprecating the TDFFormat enum and related symbols.
Docstring Coverage ✅ Passed Docstring coverage is 87.50% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 16 functions across 7 files.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

A rabbit watched the DEK divide,
As shares were tucked and keys replied.
The chunks went neatly through the stream,
A manifest closed the work complete.
Then carrots marked the test’s green gleam.

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added comp:sdk A software development kit, including library, for client applications and inter-service communicati size/s labels Sep 1, 2026
@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor
Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric Value
Approved Decision Requests 1000
Denied Decision Requests 0
Total Time 263.429245ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric Value
Approved Decision Requests 1000
Denied Decision Requests 0
Total Time 141.528355ms

Benchmark Statistics

Name № Requests Avg Duration Min Duration Max Duration

Bulk Benchmark Results

Metric Value
Total Decrypts 100
Successful Decrypts 100
Failed Decrypts 0
Total Time 479.038113ms
Throughput 208.75 requests/second

TDF3 Benchmark Results:

Metric Value
Total Requests 5000
Successful Requests 5000
Failed Requests 0
Concurrent Requests 50
Total Time 46.438200834s
Average Latency 463.360788ms
Throughput 107.67 requests/second

@dmihalcik-virtru
dmihalcik-virtru force-pushed the dspx-2604-17-createtdf-delegates branch from 02e414b to f6a3cf2 Compare September 1, 2026 03:30
@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor
Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric Value
Approved Decision Requests 1000
Denied Decision Requests 0
Total Time 245.704906ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric Value
Approved Decision Requests 1000
Denied Decision Requests 0
Total Time 126.430398ms

Benchmark Statistics

Name № Requests Avg Duration Min Duration Max Duration

Bulk Benchmark Results

Metric Value
Total Decrypts 100
Successful Decrypts 100
Failed Decrypts 0
Total Time 424.100084ms
Throughput 235.79 requests/second

TDF3 Benchmark Results:

Metric Value
Total Requests 5000
Successful Requests 5000
Failed Requests 0
Concurrent Requests 50
Total Time 46.477013608s
Average Latency 463.812651ms
Throughput 107.58 requests/second

@github-actions

Copy link
Copy Markdown
Contributor

X-Test Failure Report

@github-actions

Copy link
Copy Markdown
Contributor
Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric Value
Approved Decision Requests 1000
Denied Decision Requests 0
Total Time 246.644205ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric Value
Approved Decision Requests 1000
Denied Decision Requests 0
Total Time 144.086702ms

Benchmark Statistics

Name № Requests Avg Duration Min Duration Max Duration

Bulk Benchmark Results

Metric Value
Total Decrypts 100
Successful Decrypts 100
Failed Decrypts 0
Total Time 427.005681ms
Throughput 234.19 requests/second

TDF3 Benchmark Results:

Metric Value
Total Requests 5000
Successful Requests 5000
Failed Requests 0
Concurrent Requests 50
Total Time 59.360866932s
Average Latency 592.216807ms
Throughput 84.23 requests/second

@github-actions

Copy link
Copy Markdown
Contributor
Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric Value
Approved Decision Requests 1000
Denied Decision Requests 0
Total Time 221.267527ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric Value
Approved Decision Requests 1000
Denied Decision Requests 0
Total Time 109.053781ms

Benchmark Statistics

Name № Requests Avg Duration Min Duration Max Duration

Bulk Benchmark Results

Metric Value
Total Decrypts 100
Successful Decrypts 100
Failed Decrypts 0
Total Time 385.529297ms
Throughput 259.38 requests/second

TDF3 Benchmark Results:

Metric Value
Total Requests 5000
Successful Requests 5000
Failed Requests 0
Concurrent Requests 50
Total Time 1m0.043488789s
Average Latency 598.99969ms
Throughput 83.27 requests/second

@dmihalcik-virtru
dmihalcik-virtru force-pushed the dspx-2604-17-createtdf-delegates branch from 79e745d to a5d2340 Compare September 24, 2026 14:50
@github-actions

Copy link
Copy Markdown
Contributor

X-Test Failure Report

@github-actions

Copy link
Copy Markdown
Contributor
Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric Value
Approved Decision Requests 1000
Denied Decision Requests 0
Total Time 244.280559ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric Value
Approved Decision Requests 1000
Denied Decision Requests 0
Total Time 132.868158ms

Benchmark Statistics

Name № Requests Avg Duration Min Duration Max Duration

Bulk Benchmark Results

Metric Value
Total Decrypts 100
Successful Decrypts 100
Failed Decrypts 0
Total Time 418.542129ms
Throughput 238.92 requests/second

TDF3 Benchmark Results:

Metric Value
Total Requests 5000
Successful Requests 5000
Failed Requests 0
Concurrent Requests 50
Total Time 59.925342296s
Average Latency 597.863832ms
Throughput 83.44 requests/second

@github-actions

Copy link
Copy Markdown
Contributor
Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric Value
Approved Decision Requests 1000
Denied Decision Requests 0
Total Time 239.513501ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric Value
Approved Decision Requests 1000
Denied Decision Requests 0
Total Time 137.057595ms

Benchmark Statistics

Name № Requests Avg Duration Min Duration Max Duration

Bulk Benchmark Results

Metric Value
Total Decrypts 100
Successful Decrypts 100
Failed Decrypts 0
Total Time 423.570496ms
Throughput 236.09 requests/second

TDF3 Benchmark Results:

Metric Value
Total Requests 5000
Successful Requests 5000
Failed Requests 0
Concurrent Requests 50
Total Time 58.64490122s
Average Latency 585.194126ms
Throughput 85.26 requests/second

@dmihalcik-virtru
dmihalcik-virtru force-pushed the dspx-2604-17-createtdf-delegates branch from a5d2340 to 9318a77 Compare September 24, 2026 15:10
@github-actions

Copy link
Copy Markdown
Contributor

X-Test Failure Report

@github-actions

Copy link
Copy Markdown
Contributor
Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric Value
Approved Decision Requests 1000
Denied Decision Requests 0
Total Time 194.170891ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric Value
Approved Decision Requests 1000
Denied Decision Requests 0
Total Time 104.174862ms

Benchmark Statistics

Name № Requests Avg Duration Min Duration Max Duration

Bulk Benchmark Results

Metric Value
Total Decrypts 100
Successful Decrypts 100
Failed Decrypts 0
Total Time 361.765771ms
Throughput 276.42 requests/second

TDF3 Benchmark Results:

Metric Value
Total Requests 5000
Successful Requests 5000
Failed Requests 0
Concurrent Requests 50
Total Time 47.27345428s
Average Latency 471.779473ms
Throughput 105.77 requests/second

@github-actions

Copy link
Copy Markdown
Contributor
Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric Value
Approved Decision Requests 1000
Denied Decision Requests 0
Total Time 251.183141ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric Value
Approved Decision Requests 1000
Denied Decision Requests 0
Total Time 135.596231ms

Benchmark Statistics

Name № Requests Avg Duration Min Duration Max Duration

Bulk Benchmark Results

Metric Value
Total Decrypts 100
Successful Decrypts 100
Failed Decrypts 0
Total Time 430.009451ms
Throughput 232.55 requests/second

TDF3 Benchmark Results:

Metric Value
Total Requests 5000
Successful Requests 5000
Failed Requests 0
Concurrent Requests 50
Total Time 1m2.568801542s
Average Latency 624.282855ms
Throughput 79.91 requests/second

Base automatically changed from dspx-2604-17-createtdf-delegates to main September 24, 2026 16:29
TDFConfig.enableEncryption was set to true at construction and never read
again; nothing could turn it off and nothing branched on it. tdfFormat was
likewise fixed at JSONFormat forever. readActionName was a leftover constant
with no references. All three are gone.

rootIntegrityAlg and segmentIntegrityAlg go with them: once CreateTDF
delegates to the chunked writer, which emits an HS256 root and GMAC segments
unconditionally, nothing reads either field and no exported option ever set
one. The defaults they carried were exactly those two algorithms, so the
manifest is unchanged. TestIntegrityAlgDefaults asserted on the fields; the
same invariant is now asserted on a manifest CreateTDF actually produced, in
TDFSuite.testEncrypt, so every encrypt case in the suite checks it.

TDFFormat, JSONFormat, and XMLFormat are exported, so they are deprecated
rather than deleted. XML manifests were never implemented and the enum has no
remaining consumer inside the SDK.

Signed-off-by: Dave Mihalcik <dmihalcik@virtru.com>

wip: resolve conflict 18
@github-actions

Copy link
Copy Markdown
Contributor

@github-actions

Copy link
Copy Markdown
Contributor
Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric Value
Approved Decision Requests 1000
Denied Decision Requests 0
Total Time 253.759464ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric Value
Approved Decision Requests 1000
Denied Decision Requests 0
Total Time 135.979019ms

Benchmark Statistics

Name № Requests Avg Duration Min Duration Max Duration

Bulk Benchmark Results

Metric Value
Total Decrypts 100
Successful Decrypts 100
Failed Decrypts 0
Total Time 444.318028ms
Throughput 225.06 requests/second

TDF3 Benchmark Results:

Metric Value
Total Requests 5000
Successful Requests 5000
Failed Requests 0
Concurrent Requests 50
Total Time 1m0.523929977s
Average Latency 603.900463ms
Throughput 82.61 requests/second

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Govulncheck found vulnerabilities ⚠️

The following modules have known vulnerabilities:

  • otdfctl
  • service
  • tests-bdd

See the workflow run for details.

@github-actions

Copy link
Copy Markdown
Contributor
Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric Value
Approved Decision Requests 1000
Denied Decision Requests 0
Total Time 254.900168ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric Value
Approved Decision Requests 1000
Denied Decision Requests 0
Total Time 133.562517ms

Benchmark Statistics

Name № Requests Avg Duration Min Duration Max Duration

Bulk Benchmark Results

Metric Value
Total Decrypts 100
Successful Decrypts 100
Failed Decrypts 0
Total Time 423.478133ms
Throughput 236.14 requests/second

TDF3 Benchmark Results:

Metric Value
Total Requests 5000
Successful Requests 5000
Failed Requests 0
Concurrent Requests 50
Total Time 59.406465515s
Average Latency 593.135591ms
Throughput 84.17 requests/second

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Govulncheck found vulnerabilities ⚠️

The following modules have known vulnerabilities:

  • otdfctl
  • service
  • tests-bdd

See the workflow run for details.

@dmihalcik-virtru
dmihalcik-virtru added this pull request to the merge queue Sep 25, 2026
Merged via the queue into main with commit 1f9c799 Sep 25, 2026
48 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp:sdk A software development kit, including library, for client applications and inter-service communicati size/s

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants