Skip to content

pkc%test(bls): add the programs that write the BLS corpora - #112

Merged
kwvg merged 4 commits into
developfrom
pasta/bls-corpus-gen
Nov 3, 2026
Merged

kwvg merged 4 commits into
developfrom
pasta/bls-corpus-gen

Conversation

@PastaPastaPasta

Copy link
Copy Markdown
Member

Motivation

The BLS corpora in pkgs/pkc/corpus are written from the reference implementation, but the programs that wrote them are not in the repository. This pull request adds them, with regen.py to write every corpus again from a pinned release, so a corpus can be checked or extended later. It is split out of #101, which adds one more program on top of it.

Additional Information

Description Notes
Add the reference implementation as a submodule pkgs/pkc/contrib/dash, pinned at v24.0.0-rc.5 (8163290aa0), shallow and with update = none, so a clone, git submodule update and CI checkouts skip it. Only regen.py checks it out. Bazel, CodeQL and the crate package skip it, as its tree carries build files of its own.
Add programs that write the BLS corpora Each gen_*.cpp links the release's BLS library (dashbls, with relic), plus a few of the release's own sources where a corpus needs them, and prints one corpus. regen.py extracts the release's src, builds the library, compiles and runs every program, and writes the corpora only when all of them succeed. The nine corpora come out byte-identical, so this changes no corpus. pkgs/pkc/contrib/README.md has the recipe.
Add autotools and GMP to the dev shell contrib/nix/shell/dev.nix gains autoconf, automake, libtool, pkg-config and GMP for regen.py. The CI shell is unchanged.

Bazel has no build for the library, and the release's copy of the library builds with autotools only, as the reference implementation stopped tracking relic's CMake configuration template. So regen.py runs that build, which needs a POSIX shell, and GMP is required.

When these programs were added, the corpora were also checked equal at v23.1.8 (728f505583).

The C++ here is as it was in #101 and predates the clang-format style in #111. As discussed in #101, this branch is open for the C++ lint and static analysis commits. Whichever of this and #111 lands second will adapt.

Breaking Changes

None. No library code changes.

How Has This Been Tested?

  • regen.py writes the nine corpora byte-identical to the committed ones, with GCC 15 and with clang 21 building both the library and the programs, from a fresh clone that fetches the submodule and from an existing build.
  • A failed library build, a submodule checked out at another commit and an invalid option each stop the run with nothing written.
  • Only Linux x86_64 was run. macOS and Windows are untested, and the dev shell change has not been run in a Nix shell.
  • cargo test -p dash-pkc --all-features passes, and the lint suite is clean with the submodule checked out.
  • CI (just test lint fuzz::test, CodeQL included) is green on each commit.

Checklist

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated relevant unit/integration/functional tests
  • I have made corresponding changes to the documentation
  • I did not use agentic assistance (i.e. LLMs, AI) except as disclosed and can answer questions,
    comments and concerns without said assistance. I assume authorship of the code submitted.
  • I have assigned this pull request to a milestone (for repository code-owners and collaborators only)

Agent disclosure

Code, commit messages and this description were generated with Claude Code using Claude Opus 5.5, running autonomously as an agent on behalf of the author. The agent also built the pinned release's BLS library and wrote the corpora with it. The change was reviewed by a Claude Code agent using Opus 5.5.

🤖 Generated with Claude Code

@PastaPastaPasta PastaPastaPasta added this to the 0.3 milestone Nov 2, 2026
@PastaPastaPasta
PastaPastaPasta requested a review from kwvg November 2, 2026 18:38
@coderabbitai

coderabbitai Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 32 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration
Commits

Reviewing files that changed from the base of the PR and between b59f374 and 5e0c21a.

Files ignored due to path filters: 2
  • .github/workflows/actions.lock is excluded by !**/*.lock, !**/*.lock
  • MODULE.bazel.lock is excluded by !**/*.lock, !**/*.lock
Files selected for processing: 30
  • .bazelignore
  • .github/paths.yml
  • .github/workflows/build_corpora.yml
  • .gitmodules
  • contrib/nix/mods/corpora.nix
  • contrib/nix/shell/ci.nix
  • contrib/nix/shell/common.nix
  • maint/.clang-tidy
  • maint/codeql/cpp/suppressions.toml
  • maint/codeql/rust/codeql-config.yml
  • maint/common.py
  • maint/lint/lint_codeql.py
  • pkgs/pkc/Cargo.toml
  • pkgs/pkc/contrib/CMakeLists.txt
  • pkgs/pkc/contrib/README.md
  • pkgs/pkc/contrib/dash
  • pkgs/pkc/contrib/gen_aggregate.cpp
  • pkgs/pkc/contrib/gen_hash.cpp
  • pkgs/pkc/contrib/gen_ies.cpp
  • pkgs/pkc/contrib/gen_keygen.cpp
  • pkgs/pkc/contrib/gen_llmq_100.cpp
  • pkgs/pkc/contrib/gen_secure_aggregate.cpp
  • pkgs/pkc/contrib/gen_serialize.cpp
  • pkgs/pkc/contrib/gen_sign.cpp
  • pkgs/pkc/contrib/gen_threshold.cpp
  • pkgs/pkc/contrib/regen.py
  • pkgs/pkc/contrib/util/common.hpp
  • pkgs/pkc/contrib/util/core.hpp
  • pkgs/pkc/contrib/util/no_random.cpp
  • pkgs/pkc/contrib/util/relic_conf.patch

No actionable comments were generated in the recent review.

Summary

Summary by CodeRabbit

  • Documentation

    • Added instructions for generating cryptographic test data, including setup requirements, output options, and release selection.
  • Chores

    • Added test-data generators covering BLS signing, aggregation, key generation, serialization, hashing, encryption, and threshold operations.
    • Added automated checks to regenerate and validate test data, with safeguards against incomplete updates.
    • Updated development and CI configuration to support these tools and checks.

This change adds BLS and LLMQ corpus generators that build against a pinned Dash source. Shared helpers and CMake configuration support the generators. The regeneration script checks the staged pin, builds and runs selected programs, and replaces corpus files only after all selected programs succeed. A new CI workflow checks generated files and runs lint and CodeQL analysis when configured corpus-related changes occur, or on manual and scheduled runs.

Sequence Diagram(s)

sequenceDiagram
  participant Developer
  participant regen.py
  participant DashSubmodule
  participant CMake
  participant CorpusGenerators
  participant CorpusFiles
  Developer->>regen.py: Select programs and output paths
  regen.py->>DashSubmodule: Check staged release pin
  regen.py->>CMake: Prepare source tree and build selected targets
  regen.py->>CorpusGenerators: Run selected programs
  CorpusGenerators->>regen.py: Return generated data
  regen.py->>CorpusFiles: Replace corpus files after all runs succeed
Loading

Suggested reviewers: thepastaclaw, kwvg

Merge Risk: Minimal · previous review b59f3

The previously identified corpus-write and build-credential risks are addressed. No actionable merge-blocking risk remains beyond normal checks.

Pre-merge checks: Passed 4
Check name Status Explanation
Title check Passed The title clearly identifies the main change: adding programs that write the BLS corpora.
Description check Passed The description directly explains the added corpus generators, regeneration workflow, pinned reference implementation, testing, and related development-shell changes.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

CodeRabbit Verify

  • Run CodeRabbit Verify to test the changes in this pull request.
Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between b9ab99a and b59f374.

Files ignored due to path filters: 2
  • .github/workflows/actions.lock is excluded by !**/*.lock, !**/*.lock
  • MODULE.bazel.lock is excluded by !**/*.lock, !**/*.lock
Files selected for processing: 14
  • .github/paths.yml
  • .github/workflows/build_corpora.yml
  • maint/codeql/cpp/suppressions.toml
  • maint/lint/lint_codeql.py
  • pkgs/pkc/contrib/gen_aggregate.cpp
  • pkgs/pkc/contrib/gen_hash.cpp
  • pkgs/pkc/contrib/gen_ies.cpp
  • pkgs/pkc/contrib/gen_keygen.cpp
  • pkgs/pkc/contrib/gen_llmq_100.cpp
  • pkgs/pkc/contrib/gen_secure_aggregate.cpp
  • pkgs/pkc/contrib/gen_serialize.cpp
  • pkgs/pkc/contrib/gen_sign.cpp
  • pkgs/pkc/contrib/gen_threshold.cpp
  • pkgs/pkc/contrib/util/common.hpp

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown

Note

This pull request has no conflicts! 🎊 🎉 🎊

@coderabbitai coderabbitai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Fix comments: Fix CodeRabbit comments on this PR

Comment thread pkgs/pkc/contrib/regen.py Resolved
@thepastaclaw

thepastaclaw commented Nov 2, 2026 •

Copy link
Copy Markdown

✅ Final review complete — no blockers (commit 5e0c21a)

@codecov

codecov Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ All tests successful. No failed tests found.

📢 Thoughts on this report? Let us know!

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@thepastaclaw thepastaclaw left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final validation — Phase 1 + Phase 2

Static verification of the exact base-to-head range, pinned reference sources, regeneration driver, and nine generators found no actionable defects; no Rust library sources or committed corpora change. Current lint, stable builds, cross-builds, and dash-pkc MSRV checks pass, while additional MSRV/platform jobs remain pending; the displayed failures in “Tag pull requests” and “Comment on pull requests” are cancelled administrative jobs with no steps, not code failures. No builds or tests were run in this static-only lane, so byte-identical corpus regeneration and Nix-shell execution were not independently confirmed.

Review provenance

Source: reviewer 1: muse-spark-1.3-contributor (agent: phase1-reviewer, role: general); reviewer 2: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 3: gpt-6.1-sol (agent: phase2-reviewer, role: security-auditor); final verifier: gpt-6.1-sol (agent: sol-verifier, role: final-verifier)

  • Triage: normal by gpt-6.1-sol (effort low) — This substantial change adds corpus-generation programs and build tooling rather than changing production cryptography or another critical runtime surface.
  • Phase 1 reviewers: muse-spark-1.3-contributor — general (completed, effort xhigh); agent phase1-reviewer
  • Phase 1 model: muse-spark-1.3-contributor — not quota-gated; passed over gemini-3.8-flash-high (antigravity quota lookup failed), glm-5.3-flash (not used above high effort; tier asks max)
  • Fresh verifier: gpt-6.1-sol — final-verifier; agent sol-verifier
  • Phase 2 reviewers: gpt-6.1-sol — general (completed, effort high); agent phase2-reviewer, gpt-6.1-sol — security-auditor (completed, effort high); agent phase2-reviewer

@thepastaclaw thepastaclaw left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review — Final validation — Phase 1 + Phase 2

Static verification of the complete diff at fa6c595 found no actionable in-scope defects; the changes add reference-backed corpus tooling without modifying Rust library code or committed corpora. The pinned reference sources support the build approach and generator semantics, and the current output-writing code addresses the historical truncation concern. Current CI shows lint, documentation, stable-platform checks, and two MSRV build/test jobs passing, with additional MSRV and cross-target checks pending; corpus regeneration and the Nix shell were not independently executed.

🔴 0 blocking | 🟡 0 suggestion(s) | 💬 0 nitpick(s)

Review provenance

Source: reviewer 1: muse-spark-1.3-contributor (agent: phase1-reviewer, role: general); reviewer 2: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 3: gpt-6.1-sol (agent: phase2-reviewer, role: security-auditor); reviewer 4: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 5: gpt-6.1-sol (agent: phase2-reviewer, role: security-auditor); final verifier: gpt-6.1-sol (agent: sol-verifier, role: final-verifier)

  • Triage: normal by gpt-6.1-sol (effort low) — This substantial change adds corpus-generation programs and build orchestration, but does not change production cryptography, consensus behavior, or existing corpora.
  • Phase 1 reviewers: muse-spark-1.3-contributor — general (completed, effort xhigh); agent phase1-reviewer
  • Phase 1 model: muse-spark-1.3-contributor — not quota-gated; passed over gemini-3.8-flash-high (antigravity quota lookup failed), glm-5.3-flash (not used above high effort; tier asks max)
  • Fresh final gate: an independent Phase-2 review ran after iterative findings were reconciled
  • Fresh verifier: gpt-6.1-sol — final-verifier; agent sol-verifier
  • Phase 2 reviewers: gpt-6.1-sol — general (completed, effort high); agent phase2-reviewer, gpt-6.1-sol — security-auditor (completed, effort high); agent phase2-reviewer, gpt-6.1-sol — general (completed, effort high); agent phase2-reviewer, gpt-6.1-sol — security-auditor (completed, effort high); agent phase2-reviewer
🤖 Prompt for all review comments with AI agents
These findings are from an automated code review. Verify the current code and confirm that no unresolved issues remain.

No unresolved findings remain from the prior review on this head.

@kwvg kwvg left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebase on latest develop to incorporate base-sdk#111

Comment thread maint/codeql/rust/codeql-config.yml Resolved
Comment thread contrib/nix/shell/dev.nix Resolved
Comment thread pkgs/pkc/contrib/util/common.hpp Resolved
Comment thread pkgs/pkc/contrib/README.md Resolved
Comment thread pkgs/pkc/contrib/README.md Resolved
Comment thread pkgs/pkc/contrib/gen_hash.cpp Resolved
Comment thread pkgs/pkc/contrib/gen_hash.cpp Resolved
Comment thread pkgs/pkc/contrib/gen_hash.cpp Resolved
Comment thread pkgs/pkc/contrib/gen_hash.cpp Resolved
Comment thread pkgs/pkc/contrib/core.hpp Resolved
@PastaPastaPasta

Copy link
Copy Markdown
Member Author

Thanks for the review. I'll take all of it, including switching the generator build to CMake with the template patched back in, so Autotools stays out of the dev shell. I'm rebasing on develop for #111 first and will reply on each thread with the commit that addresses it.


🤖 Posted autonomously by Claude on behalf of pasta.

@github-actions github-actions Bot added the status:behind This pull request can be rebased on the base branch. label Nov 3, 2026
Pinned at v24.0.0-rc.5 for the programs that write the BLS corpora.
It is only checked out on request, and Bazel, CodeQL and the crate
package skip it, as its tree carries build files of its own.
The C++ programs that write test corpora build with CMake against
Boost's headers and GMP. The CI shell includes the module, and the dev
shell with it.
@PastaPastaPasta

Copy link
Copy Markdown
Member Author

Rebased on develop at 02ba9c2, which has #111, and the programs pass its C/C++ lints. lint_clang reads the release's headers from the last regen.py build and skips the programs when there is none, as it does for any unit whose include directories are absent, so the corpus job runs it after its build. The traced CodeQL build leaves pkgs/pkc/contrib out: every program has its own entry point, and a database joins all units into one program, where those clash.


🤖 Posted autonomously by Claude on behalf of pasta.

@PastaPastaPasta
PastaPastaPasta requested a review from kwvg November 3, 2026 11:04

@coderabbitai coderabbitai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Fix comments: Fix CodeRabbit comments on this PR

Comment thread .github/workflows/build_corpora.yml Resolved
@github-actions github-actions Bot removed the status:behind This pull request can be rebased on the base branch. label Nov 3, 2026
Comment thread pkgs/pkc/contrib/gen_aggregate.cpp Resolved
Comment thread pkgs/pkc/contrib/gen_hash.cpp Resolved
Comment thread pkgs/pkc/contrib/gen_llmq_100.cpp Resolved
Comment thread pkgs/pkc/contrib/gen_llmq_100.cpp Resolved
Comment thread pkgs/pkc/contrib/gen_llmq_100.cpp Resolved
Comment thread pkgs/pkc/contrib/gen_llmq_100.cpp Resolved
Comment thread pkgs/pkc/contrib/gen_sign.cpp Resolved
@PastaPastaPasta

Copy link
Copy Markdown
Member Author

Thanks, all taken. In every program, the fixed values move to the top of the file as constexpr constants, apart from the logic, following the gen_llmq_100.cpp pattern. Arguments get inline /*name=*/ comments, the quorum struct takes its key array with /*pk_array=*/ comments, and kType gets its doc comment. The separate file for the values the programs can't compute is left for the follow-up, as you suggest. I'll reply in each thread with the commit.


🤖 Posted autonomously by Claude on behalf of pasta.

@thepastaclaw thepastaclaw left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review — Final validation — Phase 1 + Phase 2

The exact-head corpus regeneration, unchanged-output check, and clang lint job passed; this PR changes no production library code or committed corpus data. Static verification confirmed that the new exclusion bypasses CodeQL analysis for all first-party corpus generators, which should be addressed before merging. Lint, MSRV, and several other checks remain pending; the cancelled tagging/commenting jobs are unrelated to code validation.

🔴 1 blocking | 💬 1 nitpick(s)

Review provenance

Source: reviewer 1: muse-spark-1.3-contributor (agent: phase1-reviewer, role: general); reviewer 2: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 3: gpt-6.1-sol (agent: phase2-reviewer, role: security-auditor); final verifier: gpt-6.1-sol (agent: sol-verifier, role: final-verifier)

  • Triage: normal by gpt-6.1-sol (effort low) — This is a substantial, cross-cutting corpus-generation and build-tooling change, but the diff does not change production cryptography or another critical runtime surface.
  • Phase 1 reviewers: muse-spark-1.3-contributor — general (completed, effort xhigh); agent phase1-reviewer
  • Phase 1 model: muse-spark-1.3-contributor — not quota-gated; passed over gemini-3.8-flash-high (antigravity quota lookup failed), glm-5.3-flash (not used above high effort; tier asks max)
  • Fresh verifier: gpt-6.1-sol — final-verifier; agent sol-verifier
  • Phase 2 reviewers: gpt-6.1-sol — general (completed, effort high); agent phase2-reviewer, gpt-6.1-sol — security-auditor (completed, effort high); agent phase2-reviewer
🤖 Prompt for all review comments with AI agents
These findings are from an automated code review. Verify each finding against the current code and only fix it if needed.

In `maint/lint/lint_codeql.py`:
- [BLOCKING] maint/lint/lint_codeql.py:276-280: Preserve CodeQL coverage for the corpus generators
  The new `_UNANALYSED` pathspec excludes every generator and `util/no_random.cpp` from CodeQL extraction, even after regeneration supplies their headers. This defeats the C/C++ analysis introduced by #111 specifically in preparation for corpus generators: it removes all CERT checks, not just diagnostics caused by combining independent executable entry points. The corpus workflow runs clang-tidy after regeneration but never runs CodeQL, so that job does not restore the missing coverage. Keep these first-party units in extraction and address cross-executable diagnostics through target-separated analysis or narrowly matched suppressions using the existing mechanism. Run the analysis after regeneration so its include directories are available.

In `pkgs/pkc/contrib/gen_serialize.cpp`:
- [NITPICK] pkgs/pkc/contrib/gen_serialize.cpp:22: Explain why both serialization arms use legacy signing
  The hardcoded `true` is correct: both arms must encode the same G2 point, rather than independently sign with their respective schemes. `ser_pairs()` pairs entries by index, and `signature_serialization_matches_vectors()` checks that the legacy point compresses to the paired IETF entry. Add a short local comment explaining why the signing flag deliberately differs from the serialization flag, so this does not look like an accidental hardcoded scheme.

Comment thread maint/lint/lint_codeql.py Resolved
Comment thread pkgs/pkc/contrib/gen_serialize.cpp Resolved

@kwvg kwvg left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK b9ab99a, conditional on green CI and PastaClaw concerns being addressed

@coderabbitai coderabbitai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Fix comments: Fix CodeRabbit comments on this PR

Comment thread pkgs/pkc/contrib/util/common.hpp Resolved

@thepastaclaw thepastaclaw left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review — Final validation — Phase 1 + Phase 2

Verified the findings against b9ab99a. The serialization comment addresses the earlier documentation finding, but the blanket CodeQL exclusion still bypasses the existing CERT analysis for all ten new first-party C++ translation units. This was a static review; current-head lint, build, and corpus-workflow checks remain pending, with no completed code-check failure shown.

🔴 1 blocking

1 carried-forward finding(s) already raised on this PR; not re-posting as new inline comments.

Review provenance

Source: reviewer 1: muse-spark-1.3-contributor (agent: phase1-reviewer, role: general); reviewer 2: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 3: gpt-6.1-sol (agent: phase2-reviewer, role: security-auditor); final verifier: gpt-6.1-sol (agent: sol-verifier, role: final-verifier)

  • Triage: normal by gpt-6.1-sol (effort low) — This is a substantial cross-cutting addition of corpus generators, regeneration tooling, and build/CI integration, but it does not change production cryptography or another critical runtime surface.
  • Phase 1 reviewers: muse-spark-1.3-contributor — general (completed, effort xhigh); agent phase1-reviewer
  • Phase 1 model: muse-spark-1.3-contributor — not quota-gated; passed over gemini-3.8-flash-high (antigravity quota lookup failed), glm-5.3-flash (not used above high effort; tier asks max)
  • Fresh verifier: gpt-6.1-sol — final-verifier; agent sol-verifier
  • Phase 2 reviewers: gpt-6.1-sol — general (completed, effort high); agent phase2-reviewer, gpt-6.1-sol — security-auditor (completed, effort high); agent phase2-reviewer
🤖 Prompt for all review comments with AI agents
These findings are from an automated code review. Verify each finding against the current code and only fix it if needed.

In `maint/lint/lint_codeql.py`:
- [BLOCKING] maint/lint/lint_codeql.py:276-280: Preserve CodeQL coverage for the corpus generators
  (existing thread: https://github.com/dashpay/base-sdk/pull/112#discussion_r4454491805)
  The planned restoration is not present at this head. `_UNANALYSED` excludes `pkgs/pkc/contrib` before extraction, removing all nine generators and `util/no_random.cpp` even after their generated headers exist. The corpus workflow runs regeneration and clang-tidy but no CodeQL analysis, while the ordinary lint workflow does not regenerate the required headers. Consequently, none of these new first-party translation units receives the existing CERT C++ checks. This blanket bypass weakens the project's static-analysis gate; it is not evidence of an exploitable vulnerability in the generators. The extraction commands compile each unit separately with `-fsyntax-only`, so multiple entry points do not require a linker failure or directory-wide exclusion. Restore these units, run their analysis after regeneration, and address any actual cross-executable diagnostics through target separation or the existing exact path/message suppressions.

Each program links the pinned release's BLS library and writes one
corpus. regen.py builds them with CMake and writes the nine corpora,
and the C/C++ linters read its last build when there is one.

@thepastaclaw thepastaclaw left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review — Final validation — Phase 1 + Phase 2

Reviewed the complete diff at b59f374; it changes corpus-generation tooling and configuration, not runtime library code or committed corpora. Both prior findings are addressed, but the new workflow lacks the host configuration required to launch its Nix-wrapped CodeQL. Validation was static only: build and lint checks remain pending, while the two metadata checks shown as failing were cancelled without executing steps.

🔴 1 blocking

Review provenance

Source: reviewer 1: muse-spark-1.3-contributor (agent: phase1-reviewer, role: general); reviewer 2: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 3: gpt-6.1-sol (agent: phase2-reviewer, role: security-auditor); reviewer 4: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 5: gpt-6.1-sol (agent: phase2-reviewer, role: security-auditor); final verifier: gpt-6.1-sol (agent: sol-verifier, role: final-verifier)

  • Triage: normal by gpt-6.1-sol (effort low) — This is a substantial cross-cutting change to corpus-generation programs, build tooling, CI and static analysis, but it does not change production cryptography or another critical surface.
  • Phase 1 reviewers: muse-spark-1.3-contributor — general (completed, effort xhigh); agent phase1-reviewer
  • Phase 1 model: muse-spark-1.3-contributor — not quota-gated; passed over gemini-3.8-flash-high (antigravity quota lookup failed), glm-5.3-flash (not used above high effort; tier asks max)
  • Fresh final gate: an independent Phase-2 review ran after iterative findings were reconciled
  • Fresh verifier: gpt-6.1-sol — final-verifier; agent sol-verifier
  • Phase 2 reviewers: gpt-6.1-sol — general (completed, effort high); agent phase2-reviewer, gpt-6.1-sol — security-auditor (completed, effort high); agent phase2-reviewer, gpt-6.1-sol — general (completed, effort high); agent phase2-reviewer, gpt-6.1-sol — security-auditor (completed, effort high); agent phase2-reviewer
🤖 Prompt for all review comments with AI agents
These findings are from an automated code review. Verify each finding against the current code and only fix it if needed.

In `.github/workflows/build_corpora.yml`:
- [BLOCKING] .github/workflows/build_corpora.yml:121-123: Enable unprivileged user namespaces before running CodeQL
  This job runs on ubuntu-24.04 but omits the host setup needed by the Nix-provided CodeQL. `contrib/nix/mods/codeql.nix` wraps Linux invocations in `fhs-run`, whose `buildFHSEnv` implementation launches bubblewrap. Ubuntu's AppArmor restriction on unprivileged user namespaces prevents that Nix-store bubblewrap from starting, so the analysis fails before extracting the generators. The existing lint job handles this requirement in `.github/workflows/build_stable.yml:54-56`. Add an equivalent step using `shell: bash` and `sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0` before invoking CodeQL; the setting must be applied in this job because the other workflow's runner does not share its host configuration.

Comment thread .github/workflows/build_corpora.yml Resolved
The job writes the corpora with regen.py, checks them unchanged and
lints and analyses the programs against that build, weekly and on
changes to the programs, the corpora, their lint setup or Nix shells.

@thepastaclaw thepastaclaw left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review — Final validation — Phase 1 + Phase 2

The complete base-to-head static review at 5e0c21a found no actionable in-scope defects: both prior blockers are fixed, and the serialization-documentation nitpick is withdrawn after checking the paired-point consumers. No builds or tests were run locally; current build, lint, and corpus-workflow checks remain pending. The failing “Comment on pull requests” and “Tag pull requests” checks are policy/automation checks, not code-validation failures.

🔴 0 blocking | 🟡 0 suggestion(s) | 💬 0 nitpick(s)

Review provenance

Source: reviewer 1: muse-spark-1.3-contributor (agent: phase1-reviewer, role: general); reviewer 2: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 3: gpt-6.1-sol (agent: phase2-reviewer, role: security-auditor); reviewer 4: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 5: gpt-6.1-sol (agent: phase2-reviewer, role: security-auditor); final verifier: gpt-6.1-sol (agent: sol-verifier, role: final-verifier)

  • Triage: normal by gpt-6.1-sol (effort low) — This is a substantial tooling, corpus-generation, and CI change, but the diff does not change production cryptography or another critical runtime surface.
  • Phase 1 reviewers: muse-spark-1.3-contributor — general (completed, effort xhigh); agent phase1-reviewer
  • Phase 1 model: muse-spark-1.3-contributor — not quota-gated; passed over gemini-3.8-flash-high (antigravity quota lookup failed), glm-5.3-flash (not used above high effort; tier asks max)
  • Fresh final gate: an independent Phase-2 review ran after iterative findings were reconciled
  • Fresh verifier: gpt-6.1-sol — final-verifier; agent sol-verifier
  • Phase 2 reviewers: gpt-6.1-sol — general (completed, effort high); agent phase2-reviewer, gpt-6.1-sol — security-auditor (completed, effort high); agent phase2-reviewer, gpt-6.1-sol — general (completed, effort high); agent phase2-reviewer, gpt-6.1-sol — security-auditor (completed, effort high); agent phase2-reviewer
🤖 Prompt for all review comments with AI agents
These findings are from an automated code review. Verify the current code and confirm that no unresolved issues remain.

No unresolved findings remain from the prior review on this head.

@kwvg kwvg left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK 5e0c21a, conditional on green CI

@kwvg
kwvg merged commit 836fd69 into develop Nov 3, 2026
52 of 57 checks passed
PastaPastaPasta added a commit that referenced this pull request Nov 3, 2026
…plementation does (#101)

* [03eefd7] pkc%test(bls): add a program that writes the BLS read corpus (pasta)
* [6bff16d] pkc%fix(bls): read legacy G1 `x = 0` with the sign bit set (pasta)
* [ac2355f] pkc%fix(bls): accept legacy encodings off the curve in `is_canonical` (pasta)
* [137d329] pkc%test(bls): check BLS reads against the reference outcomes (pasta)
* [1ebd2c2] pkc%doc: update changelog for the legacy BLS reads (pasta)

Description:

`````
## Motivation

`dash-pkc` reads BLS bytes at two levels. `to::<S>()` decodes an element, and `is_canonical::<S>()` says whether the scheme reads the bytes as an element that encodes back to them. Under the legacy scheme, the reference implementation accepts three kinds of input that `dash-pkc` refuses:
- a public key with `x = 0` with the sign bit set, (0, -2). blst's decompression writes this point out but reports an error;
- a public key encoding an `x` below the field prime with no point on the curve, with bits 5-6 clear, under either sign;
- a signature encoding an `x` with no point on the curve, with both halves below the field prime and the sign bit and the top three bits of `x.c1` clear.

This pull request makes `to::<BlsScChia>()` decode the first kind and `is_canonical::<BlsScChia>()` accept all three. `to` still refuses the off-curve encodings, since there is no point to return for them, and its rustdoc says so. The basic scheme is unchanged, and so is the stray-bit masking on the legacy G1 path.

## Additional Information

This pull request is based on `pasta/bls-corpus-gen` (#112), which adds the reference implementation as a submodule and the programs that write the other BLS corpora.

| Description | Notes |
| ----------- | ----- |
| Add a program that writes the BLS read corpus | `gen_read.cpp` builds seeded inputs with relic's field and curve operations and writes `pkgs/pkc/corpus/bls_read.json5`: 79 public-key and 130 signature inputs, each with the library's outcome under the legacy and basic schemes, `null`, `valid` or `invalid`. For each input the legacy scheme reads as valid, relic also gives whether its `x` has a point on the curve. `regen.py` writes it with the other corpora. |
| Read legacy G1 `x = 0` with the sign bit set | `to` and `is_canonical` accept it, and it round-trips. With the sign bit clear, stray bits 5-6 mask `x = 0` down to the null encoding, so those bytes still do not decode. Adds `BlsScheme` methods `g1_bytes`, a point in the scheme's 48-byte layout, and `pk_ietf_bytes`, the IETF compressed form of an encoding in that layout; `BlsScChia` moves its legacy encoder and its legacy-to-IETF conversion into them. |
| Accept legacy encodings off the curve in `is_canonical` | New `BlsScheme` methods `is_canonical_pk` and `is_canonical_sig` hold each scheme's rule. `BlsScIetf` keeps the previous rule, decode plus byte-exact re-encode. `BlsScChia` also accepts the off-curve encodings. `g2_bytes` is the G2 counterpart of `g1_bytes`, and `sig_ietf_x_bytes` gives the `x` of a G2 encoding in the IETF compressed layout, with the sign bit clear. |
| Check BLS reads against the reference outcomes | Every corpus input is checked under both schemes, against `is_null`, `is_canonical` and the scheme's own predicate. `to` must fail for each legacy input the corpus marks off the curve, and decode and re-encode to itself for every other valid input. |
| Update the changelog | Under "Changed" for the two read changes, and "Added" for the `BlsScheme` methods. |

Each outcome in `bls_read.json5` comes from the release's read of the input:
- the all-zero input is `null`;
- otherwise the input is `valid` when the library's element reader, under the scheme flag, returns an element other than the identity that encodes back to the input;
- anything else is `invalid`.

The program is C++ because the library's element readers are its C++ API over relic. `pkgs/pkc/contrib/README.md` has the regeneration recipe.

## Breaking Changes

None. `is_canonical::<BlsScChia>()` accepts more inputs and `to::<BlsScChia>()` decodes one more. `BlsScheme` gains six required methods, but the trait is sealed, so no implementation outside `dash-pkc` exists.

CI (`just test lint fuzz::test`, CodeQL included) is green on each commit.

## Checklist

- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have added or updated relevant unit/integration/functional tests
- [x] I have made corresponding changes to the documentation
- [ ] I did not use agentic assistance (i.e. LLMs, AI) except as [disclosed](#agent-disclosure) and can answer questions,
      comments and concerns without said assistance. I assume authorship of the code submitted.
- [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

### Agent disclosure

Code, commit messages and this description were generated with Claude Code using Claude Opus 5.5, running autonomously as an agent on behalf of the author. The agent also built the pinned release's BLS library and wrote the corpora with it. The change was reviewed by a Claude Code agent using Opus 5.5 and a Codex agent using GPT-6.1.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
`````
PastaPastaPasta added a commit to PastaPastaPasta/base-sdk that referenced this pull request Nov 4, 2026
Takes dashpay#100 dashpay#88 dashpay#108 dashpay#107 dashpay#111 dashpay#110 dashpay#112 dashpay#115 dashpay#114 dashpay#101. ChainSetup keeps
the DEC-1043 policy fields; the BS11 BLS wrappers, i64 TxOut values and
MAX_SER_SIZE payload bound stay for Core parity.
kwvg added a commit to kwvg/base-sdk that referenced this pull request Nov 4, 2026
…plementation does (dashpay#101)

* [03eefd7] pkc%test(bls): add a program that writes the BLS read corpus (pasta)
* [6bff16d] pkc%fix(bls): read legacy G1 `x = 0` with the sign bit set (pasta)
* [ac2355f] pkc%fix(bls): accept legacy encodings off the curve in `is_canonical` (pasta)
* [137d329] pkc%test(bls): check BLS reads against the reference outcomes (pasta)
* [1ebd2c2] pkc%doc: update changelog for the legacy BLS reads (pasta)

Description:

`````
## Motivation

`dash-pkc` reads BLS bytes at two levels. `to::<S>()` decodes an element, and `is_canonical::<S>()` says whether the scheme reads the bytes as an element that encodes back to them. Under the legacy scheme, the reference implementation accepts three kinds of input that `dash-pkc` refuses:
- a public key with `x = 0` with the sign bit set, (0, -2). blst's decompression writes this point out but reports an error;
- a public key encoding an `x` below the field prime with no point on the curve, with bits 5-6 clear, under either sign;
- a signature encoding an `x` with no point on the curve, with both halves below the field prime and the sign bit and the top three bits of `x.c1` clear.

This pull request makes `to::<BlsScChia>()` decode the first kind and `is_canonical::<BlsScChia>()` accept all three. `to` still refuses the off-curve encodings, since there is no point to return for them, and its rustdoc says so. The basic scheme is unchanged, and so is the stray-bit masking on the legacy G1 path.

## Additional Information

This pull request is based on `pasta/bls-corpus-gen` (dashpay#112), which adds the reference implementation as a submodule and the programs that write the other BLS corpora.

| Description | Notes |
| ----------- | ----- |
| Add a program that writes the BLS read corpus | `gen_read.cpp` builds seeded inputs with relic's field and curve operations and writes `pkgs/pkc/corpus/bls_read.json5`: 79 public-key and 130 signature inputs, each with the library's outcome under the legacy and basic schemes, `null`, `valid` or `invalid`. For each input the legacy scheme reads as valid, relic also gives whether its `x` has a point on the curve. `regen.py` writes it with the other corpora. |
| Read legacy G1 `x = 0` with the sign bit set | `to` and `is_canonical` accept it, and it round-trips. With the sign bit clear, stray bits 5-6 mask `x = 0` down to the null encoding, so those bytes still do not decode. Adds `BlsScheme` methods `g1_bytes`, a point in the scheme's 48-byte layout, and `pk_ietf_bytes`, the IETF compressed form of an encoding in that layout; `BlsScChia` moves its legacy encoder and its legacy-to-IETF conversion into them. |
| Accept legacy encodings off the curve in `is_canonical` | New `BlsScheme` methods `is_canonical_pk` and `is_canonical_sig` hold each scheme's rule. `BlsScIetf` keeps the previous rule, decode plus byte-exact re-encode. `BlsScChia` also accepts the off-curve encodings. `g2_bytes` is the G2 counterpart of `g1_bytes`, and `sig_ietf_x_bytes` gives the `x` of a G2 encoding in the IETF compressed layout, with the sign bit clear. |
| Check BLS reads against the reference outcomes | Every corpus input is checked under both schemes, against `is_null`, `is_canonical` and the scheme's own predicate. `to` must fail for each legacy input the corpus marks off the curve, and decode and re-encode to itself for every other valid input. |
| Update the changelog | Under "Changed" for the two read changes, and "Added" for the `BlsScheme` methods. |

Each outcome in `bls_read.json5` comes from the release's read of the input:
- the all-zero input is `null`;
- otherwise the input is `valid` when the library's element reader, under the scheme flag, returns an element other than the identity that encodes back to the input;
- anything else is `invalid`.

The program is C++ because the library's element readers are its C++ API over relic. `pkgs/pkc/contrib/README.md` has the regeneration recipe.

## Breaking Changes

None. `is_canonical::<BlsScChia>()` accepts more inputs and `to::<BlsScChia>()` decodes one more. `BlsScheme` gains six required methods, but the trait is sealed, so no implementation outside `dash-pkc` exists.

CI (`just test lint fuzz::test`, CodeQL included) is green on each commit.

## Checklist

- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have added or updated relevant unit/integration/functional tests
- [x] I have made corresponding changes to the documentation
- [ ] I did not use agentic assistance (i.e. LLMs, AI) except as [disclosed](#agent-disclosure) and can answer questions,
      comments and concerns without said assistance. I assume authorship of the code submitted.
- [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

### Agent disclosure

Code, commit messages and this description were generated with Claude Code using Claude Opus 5.5, running autonomously as an agent on behalf of the author. The agent also built the pinned release's BLS library and wrote the corpora with it. The change was reviewed by a Claude Code agent using Opus 5.5 and a Codex agent using GPT-6.1.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
`````
kwvg added a commit that referenced this pull request Nov 5, 2026
* [f3ac1b6] pkc%test(bls): run the legacy map's own steps in `gen_hash.cpp` (pasta)

Description:

`````
## Motivation

In #101 the program that writes `bls_read.json5` was reworked so that nothing in it stands in for the library. `gen_hash.cpp`, from #112, still repeats the legacy hash to G2's tag hashing and reduction steps with relic and takes only the final point from the library, so nothing showed that the steps it records are the ones the library runs. This pull request is the follow-up agreed in [the #101 review](#101 (comment)).

## Additional Information

| Description | Notes |
| ----------- | ----- |
| Give two steps of the legacy map external linkage | `pkgs/pkc/contrib/util/legacy_steps.patch` gives the release's legacy map's encoding of a field element and its cofactor multiplication external linkage. Their code is unchanged. `regen.py` applies it after the relic template patch, and the build directory's name now covers both patches, so a build from before is not reused. |
| Check the recorded steps against the library | For each message, `gen_hash.cpp` runs those two steps on the four field elements it writes, as the map does: the encodings of the two pairs, their sum and its cofactor multiple. The program stops, writing nothing, unless the result is the library's own map of the message. The hashing and reduction are still repeated with relic, as the map computes them inline. |

The check ties every `_hash` and `_fp` value to the library except the order of the two pairs of tags, since the sum of their encodings is the same either way. `pkgs/pkc/contrib/README.md` says so.

## Breaking Changes

None. No library code changes, and `bls_hash.json5` is unchanged.

## How Has This Been Tested?

- `regen.py --program hash` extracts and builds the patched release afresh and writes `bls_hash.json5` byte-identical to the committed one, with GCC 15. Only Linux x86_64 was run.
- With a wrong byte planted in the tag hashing, the program stops and nothing is written. Swapping the two elements of a pair, negating an element or a pair, and leaving out the cofactor step each stop it too.
- The lint suite is clean with the submodule checked out.
- CI (`just test lint fuzz::test`) is green on the commit.

## Checklist

- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional tests
- [x] I have made corresponding changes to the documentation
- [ ] I did not use agentic assistance (i.e. LLMs, AI) except as [disclosed](#agent-disclosure) and can answer questions,
      comments and concerns without said assistance. I assume authorship of the code submitted.
- [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

### Agent disclosure

Code, commit messages and this description were generated with Claude Code using Claude Opus 5.5, running autonomously as an agent on behalf of the author. The agent also built the pinned release's BLS library and wrote the corpus with it. The change was reviewed by a Claude Code agent using Opus 5.5.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
`````
@kwvg
kwvg deleted the pasta/bls-corpus-gen branch November 5, 2026 15:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants