Skip to content

test: segregate wallet-mode coverage and deduplicate functional test runs - #7497

Open
PastaPastaPasta wants to merge 5 commits into
dashpay:developfrom
PastaPastaPasta:claude/functional-test-ci-optimization-d76b73
Open

test: segregate wallet-mode coverage and deduplicate functional test runs#7497
PastaPastaPasta wants to merge 5 commits into
dashpay:developfrom
PastaPastaPasta:claude/functional-test-ci-optimization-d76b73

Conversation

@PastaPastaPasta

@PastaPastaPasta PastaPastaPasta commented Jul 30, 2026

Copy link
Copy Markdown
Member

Issue being fixed or feature implemented

Functional tests account for roughly 79 minutes of serial runtime per CI job, and several whole tests are run twice with configurations whose divergent surface is only a small fraction of the test. The duplicated runs cost roughly 450 seconds of serial time per job:

  • feature_governance.py, feature_governance_cl.py and feature_dip3_deterministicmns.py run in both --legacy-wallet and --descriptors modes. This dates back to the descriptor-wallet support series (feat: support rpc protx-register for descriptor wallets - part VI #6003, feat: support descriptor wallets for RPC governance votemany, votealias #6094): these tests were the regression tests for descriptor vote signing and protx funding, so both modes were wired up (c72ec70 added feature_governance.py --descriptors in the same commit that implemented descriptor vote signing). The tests themselves are overwhelmingly consensus-side (superblock budgets, trigger creation, DMN list updates, reorgs, payment enforcement) and wallet-mode-agnostic; only the RPC entry points that fund, look up keys and sign through the wallet differ per wallet mode.
  • feature_llmq_signing.py runs twice (default and --spork21), duplicating the entire signing-session flow which does not depend on the spork.
  • feature_llmq_simplepose.py --disable-spork23 re-runs the contribution-miss banning section even though that ban path is not gated on spork23.

What was done?

Wallet coverage is segregated from consensus coverage, and per-test duplication is removed where the second run added no coverage:

  1. New wallet_dash_rpcs.py, run in both --legacy-wallet and --descriptors modes (~11-15s per mode). It covers exactly the wallet-mode-divergent Dash RPC surface: gobject prepare (wallet collateral funding) -> list-prepared -> submit; gobject vote-many/vote-alias (CheckWalletOwnsKey/IsMine lookup and CWallet::SignGovernanceVote -> SignMessage SPKM dispatch); protx register_fund; the external-collateral path protx register_prepare + signmessage + register_submit; protx update_service; protx update_registrar. Small 2-masternode topology, no quorums, no superblock cycles.
  2. feature_governance.py and feature_governance_cl.py run in --descriptors mode only. The legacy-wallet coverage of the governance wallet surface is carried by wallet_dash_rpcs.py.
  3. feature_dip3_deterministicmns.py runs in --descriptors mode only, same justification for the protx wallet surface.
  4. feature_llmq_signing.py merged into a single run. The shared flow runs with spork21 off; SPORK_21_QUORUM_ALL_CONNECTED is then enabled mid-test, one more quorum is mined, and the spork21-only sections run against it: all-connected topology plus symmetric QSENDRECSIGS checks, the submit=false RPC parameter with QSIGSHARE P2P share submission to the recovery member, and the recovery-member isolation scenario. mine_quorum() reads spork state at call time, so expected connection counts adjust automatically.
  5. feature_llmq_simplepose.py --disable-spork23 skips the contribution-miss banning section (MarkBadMember -> PoSePunish is not spork23-gated; spork23 only gates VerifyConnectionAndMinProtoVersions and probes, so that section was identical in both variants). A single quorum is mined in normal conditions instead so the spork23-specific sections start from the same state as in the spork23-enabled run.

Deliberately lost permutations, called out explicitly:

  • spork21 active from the very first DKG (previously feature_llmq_signing.py --spork21): covered by feature_llmq_data_recovery.py, which enables SPORK_21_QUORUM_ALL_CONNECTED at the top of run_test on a fresh chain, before the first DKG, and then forms both llmq_test and llmq_test_v17 quorums every cycle under it. (feature_llmq_connections.py additionally covers the chain's first rotation (dip0024) DKGs under spork21, though it mines three non-rotation quorums before enabling the spork.) The one delta is quorum size: the removed variant used 5-member quorums while feature_llmq_data_recovery.py uses 4- and 3-member ones; the spork21 connection path has no first-DKG-specific branching that depends on member count.
  • legacy-wallet x governance/DIP3 consensus scenarios: the consensus logic in those tests does not depend on the wallet type; the wallet surface they exercised is now covered in both wallet modes by wallet_dash_rpcs.py.

Changed entries were re-slotted in BASE_SCRIPTS according to their new runtimes.

How Has This Been Tested?

Local machine (Apple Silicon, 14 cores), all timings from test/functional/test_runner.py -j3 runs before and after the changes:

Runner entry Before After
feature_governance.py --legacy-wallet 77 s removed
feature_governance.py --descriptors 73 s 69 s
feature_governance_cl.py --legacy-wallet 36 s removed
feature_governance_cl.py --descriptors 22 s 25 s
feature_dip3_deterministicmns.py --legacy-wallet 99 s removed
feature_dip3_deterministicmns.py --descriptors 92 s 84 s
feature_llmq_signing.py 58 s 76 s (merged)
feature_llmq_signing.py --spork21 66 s removed
feature_llmq_simplepose.py --disable-spork23 102 s 77 s
wallet_dash_rpcs.py --legacy-wallet - 14 s
wallet_dash_rpcs.py --descriptors - 11 s
Total (affected entries, serial) 625 s 356 s

That is a ~270 s serial saving per CI job on this machine; CI runners are slower, so the absolute saving there is expected to be larger (~380 s estimated from recent CI run timings).

Additional verification:

  • wallet_dash_rpcs.py passes in both wallet modes.
  • The merged feature_llmq_signing.py passes standalone and under parallel load.
  • Both feature_llmq_simplepose.py variants pass; the --disable-spork23 variant was additionally stress-tested with multiple concurrent instances.
  • test/lint/all-lint.py passes for the touched files (the only failure is pre-existing lint-cppcheck-dash warnings in unrelated C++ files untouched by this PR).

Breaking Changes

None. Test-only changes.

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have made corresponding changes to the documentation (not applicable)
  • I have assigned this pull request to a milestone (for repository code-owners and collaborators only)

🤖 Generated with Claude Code

PastaPastaPasta and others added 5 commits July 29, 2026 22:31
… paths

Dash-specific RPCs touch the wallet in a small number of places: gobject prepare funds the proposal collateral from the wallet, gobject vote-many/vote-alias look up voting keys via CheckWalletOwnsKey/IsMine and sign via CWallet::SignGovernanceVote -> SignMessage (SPKM dispatch), and the protx register/update family funds, signs and submits special transactions through the wallet, including the external-collateral register_prepare + signmessage + register_submit path.

Cover exactly this surface in a dedicated test that runs in both --legacy-wallet and --descriptors modes using a small 2-masternode topology with no quorums and no superblock cycles (~13s per mode locally). This will allow the consensus-heavy tests that currently run twice only to keep a single wallet mode without losing wallet coverage.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
feature_governance.py and feature_governance_cl.py were run in both --legacy-wallet and --descriptors modes because they were the regression tests for descriptor-wallet governance vote signing (dash#6003, dash#6094). Their coverage is almost entirely consensus-side and wallet-agnostic (superblock budgets, trigger creation and voting, ChainLock interaction); the wallet-mode-divergent surface (gobject prepare collateral funding, vote-many/vote-alias key lookup and vote signing) is now covered in both wallet modes by wallet_dash_rpcs.py, so run the governance tests in descriptors mode only.

Saves roughly 110 seconds of serial functional-test time per CI job.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
feature_dip3_deterministicmns.py was run in both wallet modes for the same historical reason as the governance tests: it was part of the descriptor-wallet support regression suite (dash#6003, dash#6094). The test exercises consensus-side deterministic masternode list behavior (registration heights, collateral spends, reorgs, payment enforcement) which is wallet-mode-agnostic. The wallet-facing protx paths it goes through (register_fund, register, update_service, update_registrar and external-collateral signing) are covered in both wallet modes by wallet_dash_rpcs.py, so run it in descriptors mode only.

Saves roughly 100 seconds of serial functional-test time per CI job.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
feature_llmq_signing.py ran twice, once with spork21 off and once with --spork21, duplicating the whole signing-session flow (quorum mining, sign/verify RPCs, recovered-sig expiration) which does not depend on the spork. Run the shared flow once with spork21 off, then enable SPORK_21_QUORUM_ALL_CONNECTED mid-test, mine one more quorum and run the spork21-only sections on it: all-connected topology and symmetric QSENDRECSIGS checks, the submit=false RPC parameter with QSIGSHARE P2P share submission to the recovery member, and the recovery-member isolation scenario. mine_quorum() reads spork state at call time, so expected connection counts adjust automatically.

The only permutation lost is spork21 being active from the very first DKG, which is already covered by feature_llmq_connections.py (it mines quorums right after activating spork21 on a fresh quorum-less chain and verifies member connections and probes).

Saves roughly 60 seconds of serial functional-test time per CI job.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…riant

feature_llmq_simplepose.py runs twice, with spork23 enabled and disabled. The first section (test_banning via isolate_mn) exercises the DKG contribution-miss ban path (MarkBadMember -> PoSePunish), which is not gated on spork23: spork23 only gates VerifyConnectionAndMinProtoVersions and connection probes. Running that section in the --disable-spork23 variant repeated identical coverage, so skip it there together with its repair_masternodes call. Instead, mine a single quorum in normal conditions so that the spork23-specific sections below (no banning for unreachable or outdated nodes when PoSe checks are off) start from the same state as in the spork23-enabled run: an existing quorum and all masternodes healthy. Those sections are otherwise unchanged and the skipped section does not feed any state into them: masternodes are unbanned either way once repair_masternodes has run and expected contributor counts are derived from mninfo at call time.

Also move the --disable-spork23 entry further down the runner list to match its reduced runtime. Saves roughly 20-30 seconds of serial functional-test time per CI job.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@thepastaclaw

thepastaclaw commented Jul 30, 2026

Copy link
Copy Markdown

✅ Final review complete — no blockers (commit 95996ba)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 95996baf4e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +35 to +36
# mined after that. spork21 being active from the very first DKG is
# covered by feature_llmq_connections.py.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Restore first-DKG spork21 coverage

Keep a test where SPORK_21 is active before the first DKG: the cited replacement does not cover that scenario, because feature_llmq_connections.py mines three llmq_test quorums at lines 36, 58, and 70 before enabling SPORK_21 at lines 74-76, then proceeds with a rotating quorum. Consequently, regressions specific to initializing or forming the first quorum with the spork already active are no longer detected after removing the --spork21 variant.

AGENTS.md reference: AGENTS.md:L164-L164

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right that feature_llmq_connections.py was the wrong citation — it mines three llmq_test quorums (lines 36, 58, 70) before enabling SPORK_21 at line 75, so it does not cover "spork21 active before the first DKG" for that quorum type. The PR description has been corrected.

However, the permutation itself is still covered in every CI job: feature_llmq_data_recovery.py enables SPORK_21_QUORUM_ALL_CONNECTED at the very top of run_test (line 164), on a fresh chain before its first mine_quorum() (line 173), and then forms both llmq_test and llmq_test_v17 quorums every DKG cycle with the spork active from before the first DKG ever. That exercises first-quorum formation under all-connected topology at least as thoroughly as the removed --spork21 variant did (two quorum types concurrently vs one). Additionally, feature_llmq_connections.py does cover the chain's first rotation (dip0024) DKGs under spork21 via mine_cycle_quorum() after the spork is enabled.

The only delta vs the removed variant is quorum size (5 members there vs 4/3 in data_recovery); the spork21 path (EnsureQuorumConnections all-connected + probes) has no first-DKG-specific branching that depends on member count, so we are not restoring the variant.

@coderabbitai

coderabbitai Bot commented Jul 30, 2026

Copy link
Copy Markdown

Review Change Stack

Walkthrough

The PR updates functional coverage for LLMQ signing and recovery, Simple PoSe behavior under SPORK23 modes, and wallet-dependent governance and protx RPC paths. It adds wallet_dash_rpcs.py, runs it in legacy-wallet and descriptor modes, and adjusts the base functional test selection for descriptor, governance, deterministic masternode, and LLMQ variants.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant WalletDashRPCsTest
  participant WalletRPC
  participant Blockchain
  participant ProtxState
  WalletDashRPCsTest->>WalletRPC: prepare and sign governance or protx operation
  WalletRPC->>Blockchain: submit transaction
  Blockchain->>WalletDashRPCsTest: confirm mined transaction
  WalletDashRPCsTest->>ProtxState: query resulting state
  ProtxState->>WalletDashRPCsTest: return updated state
Loading

Possibly related PRs

Suggested reviewers: knst

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the main change: splitting wallet-mode coverage and removing duplicated functional-test runs.
Description check ✅ Passed The description is detailed and directly matches the test-only changes in the pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@PastaPastaPasta
PastaPastaPasta requested review from UdjinM6 and knst July 30, 2026 04:06

@thepastaclaw thepastaclaw left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final validation — Codex + Sonnet

The test-only refactor preserves the intended wallet-mode, SPORK21, and SPORK23 coverage; in particular, first-DKG SPORK21 coverage remains in feature_llmq_data_recovery.py. One documentation-only issue remains: the signing test comment and corresponding commit message cite feature_llmq_connections.py instead of the test that actually provides that coverage.

Source: reviewers codex/general=gpt-5.6-sol(completed); codex/dash-core-commit-history=gpt-5.6-sol(completed); claude/general=claude-sonnet-5(failed); claude/dash-core-commit-history=claude-sonnet-5(completed); claude/general=claude-sonnet-5(completed); verifier=codex/final-verifier=gpt-5.6-sol(completed) fallback_for_sonnet_verifier=true; coordinator=openclaw-agent/cliproxy/gpt-5.6-sol(orchestration-only).

Review provenance

  • Codex reviewers: gpt-5.6-sol — general (completed), gpt-5.6-sol — dash-core-commit-history (completed)
  • Verifier: gpt-5.6-sol — final-verifier (fallback)
  • Sonnet reviewers: claude-sonnet-5 — general (failed), claude-sonnet-5 — dash-core-commit-history (completed), claude-sonnet-5 — general (completed)

💬 1 nitpick(s)

Comment on lines +35 to +36
# mined after that. spork21 being active from the very first DKG is
# covered by feature_llmq_connections.py.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💬 Nitpick: Correct the stale first-DKG coverage citation

feature_llmq_connections.py mines three llmq_test quorums at lines 36, 58, and 70 before enabling SPORK_21_QUORUM_ALL_CONNECTED at lines 74-76, so it does not cover SPORK21 being active for the first DKG. The actual replacement coverage is in feature_llmq_data_recovery.py, which enables SPORK21 at line 164 before its first mine_quorum() at line 173. Update this comment and amend the matching rationale in commit 436bcfaf339 so the source and commit history accurately explain why removing the original permutation is safe; the functional coverage itself remains intact.

Suggested change
# mined after that. spork21 being active from the very first DKG is
# covered by feature_llmq_connections.py.
# mined after that. spork21 being active from the very first DKG is
# covered by feature_llmq_data_recovery.py.

source: ['codex']

@thepastaclaw

Copy link
Copy Markdown

Functional test CI timing comparison

Develop baseline: d41edd90 (successful develop CI push run 30402917810; exact base 2e52d33f run 30483914944 failed)
PR run: 95996baf (run 30512523696, conclusion: failure)

mode develop baseline PR delta change
linux64 23m 44s 20m 48s -2m 56s -12.4%
linux64_ubsan 22m 16s 19m 02s -3m 14s -14.5%
linux64_tsan 38m 30s 25m 41s (failed) -12m 49s -33.3%
linux64_sqlite 19m 25s 24m 05s 4m 40s 24.0%
linux64_nowallet 6m 52s 6m 56s 0m 04s 1.0%
linux64_multiprocess 34m 35s 33m 00s -1m 35s -4.6%
Total 2h 25m 22s 2h 9m 32s -15m 50s -10.9%

Across the six comparable modes, the PR spent 15m 50s less total time in Run functional tests (-10.9% overall).

Values above are GitHub Actions Run functional tests step durations, not whole-job runtimes, and they can still move around a bit from runner noise.

Caveat: the PR CI run finished with failure because linux64_tsan failed in its Run functional tests step, so that row reflects a completed failed step rather than a clean pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants