Skip to content

fix(sei-cosmos): harden paginated RPC queries against DoS via limit, offset, and count_total caps (PLT-361)#3494

Open
amir-deris wants to merge 35 commits into
mainfrom
amir/plt-361-lower-sei-cosmos-pagination-query-limit
Open

fix(sei-cosmos): harden paginated RPC queries against DoS via limit, offset, and count_total caps (PLT-361)#3494
amir-deris wants to merge 35 commits into
mainfrom
amir/plt-361-lower-sei-cosmos-pagination-query-limit

Conversation

@amir-deris

@amir-deris amir-deris commented May 21, 2026

Copy link
Copy Markdown
Contributor

Problem

Three separate vectors allow a single RPC call to trigger unbounded KV store iteration:

  1. Limit too largeMaxLimit was math.MaxUint64; callers could request billions of items in one call.
  2. count_total=true unbounded scan — after serving the requested page, the paginator continued iterating the entire remaining store just to populate pagination.total. Implicit limit=0 also silently enabled this behaviour.
  3. Offset too large — no cap on pagination.offset; a caller with offset=1_000_000_000 forces the iterator to skip a billion entries before serving a single result.
  4. GetBlockWithTxs allocation — user-supplied limit was passed directly into make([]*txtypes.Tx, 0, limit) before any validation.

Changes

sei-cosmos/types/query/pagination.go

  • Lowers MaxLimit to 1_000
  • Adds MaxOffset = 10_000 and VerifyPaginationOffset(); enforced in ParsePagination and paginate()
  • Adds MaxScanLimit = 10_000 — fires when count_total=true and the iterator travels more than MaxScanLimit entries past the end of the requested page (count > end + MaxScanLimit), preventing full-store counts while still allowing count_total on reasonably-sized stores
  • Removes the implicit countTotal = true side-effect when limit == 0; callers must opt in explicitly

sei-cosmos/types/query/filtered_pagination.go

  • Same MaxOffset and MaxScanLimit guards applied to FilteredPaginate and GenericFilteredPaginate
  • Fixes a bug in the original scan cap where totalIter (raw store iterations) was compared against end (a filtered-hit count), causing the limit to fire mid-page for selective filters — a query with a 1% pass rate and limit=100 could be rejected before accumulating its first result
  • Replaces the single mixed-space guard with a two-phase approach:
    • Phase 1 (numHits < end): caps raw iterations at end + MaxScanLimit — prevents full-store walks when the filter produces too few hits to fill the page (no-hits DoS path); cannot fire once the page starts completing
    • Phase 2 (numHits >= end): tracks iterations after page completion via pageCompleteIter and caps at MaxScanLimit — limits post-page count_total scanning without any risk of mid-page interference

sei-cosmos/x/auth/tx/service.go

  • Calls pagination.VerifyPaginationLimit(limit) before make([]*txtypes.Tx, 0, limit) in GetBlockWithTxs

Behaviour summary

Request Before After
limit > 1,000 accepted, full scan InvalidArgument
limit = 0 default 100 items + implicit count_total=true default 100 items, no implicit count
count_total=true, store has > end + 10,000 entries full store scan InvalidArgument
count_total=true, selective filter, numHits < end rejected mid-page assembly completes page; errors only if filter is too sparse to fill page within end + MaxScanLimit raw iterations
offset > 10,000 accepted, skips N entries InvalidArgument
GetBlockWithTxs with limit=1_000_000_000 allocates ~1 GB slice InvalidArgument

Regression risk

Breaking change for clients sending limit > 1,000, offset > 10,000, or relying on limit=0 to return a total count. Clients should use limit ≤ 1,000, follow next_key for subsequent pages, and set count_total=true explicitly only when the store is known to be small.

ExportGenesis and the TotalSupply invariant were migrated to collectAllTotalSupply, which internally paginates at MaxLimit per page to collect all denominations without being blocked by the new limit.

🤖 Generated with Claude Code

@amir-deris amir-deris self-assigned this May 21, 2026
@amir-deris amir-deris changed the title Updated max limit for sei_cosmos query pagination fix: lower sei-cosmos pagination query MaxLimit to 10,000 May 21, 2026
@cursor

cursor Bot commented May 21, 2026

Copy link
Copy Markdown

PR Summary

High Risk
Breaking API behavior for clients using large limits/offsets or relying on limit=0 for totals; changes touch core query pagination used widely by gRPC modules.

Overview
Hardens paginated RPC queries against DoS by capping limit, offset, and how far iterators may scan when building pages or count_total.

sei-cosmos/types/query: MaxLimit drops from math.MaxUint64 to 1,000; new MaxOffset (10,000) and MaxScanLimit (10,000) with VerifyPaginationLimit / VerifyPaginationOffset on Paginate, ParsePagination, FilteredPaginate, and GenericFilteredPaginate. limit == 0 no longer forces count_total=true—clients must set count_total explicitly. Offset/count_total paths return InvalidArgument when scans exceed the caps; filtered pagination uses a two-phase scan limit so sparse filters can still fill a page without mid-page false rejects.

Call sites: GetBlockWithTxs validates pagination limit before make(..., limit). Bank ExportGenesis and the TotalSupply invariant use new CollectAllTotalSupply (key-based paging at MaxLimit) instead of a single huge page request.

Tests expect pagination.total == 0 unless count_total=true; some integration tests now pass count_total=true where totals are asserted.

Reviewed by Cursor Bugbot for commit 55b905b. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions

github-actions Bot commented May 21, 2026

Copy link
Copy Markdown

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedJun 12, 2026, 6:14 PM

Comment thread sei-cosmos/types/query/pagination.go Outdated
@codecov

codecov Bot commented May 21, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 68.08511% with 30 lines in your changes missing coverage. Please review.
✅ Project coverage is 58.19%. Comparing base (0a2c388) to head (55b905b).

Files with missing lines Patch % Lines
sei-cosmos/types/query/filtered_pagination.go 53.84% 18 Missing and 6 partials ⚠️
sei-cosmos/x/auth/tx/service.go 0.00% 3 Missing ⚠️
sei-cosmos/x/bank/keeper/keeper.go 84.61% 1 Missing and 1 partial ⚠️
sei-cosmos/x/bank/keeper/genesis.go 50.00% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #3494      +/-   ##
==========================================
- Coverage   59.22%   58.19%   -1.03%     
==========================================
  Files        2214     2130      -84     
  Lines      183389   173272   -10117     
==========================================
- Hits       108604   100832    -7772     
+ Misses      64994    63479    -1515     
+ Partials     9791     8961     -830     
Flag Coverage Δ
sei-chain-pr 74.10% <68.08%> (?)
sei-db 70.41% <ø> (ø)
sei-db-state-db ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
sei-cosmos/types/query/pagination.go 86.59% <100.00%> (+3.05%) ⬆️
sei-cosmos/x/bank/keeper/invariants.go 51.35% <100.00%> (ø)
sei-cosmos/x/bank/keeper/genesis.go 76.59% <50.00%> (ø)
sei-cosmos/x/bank/keeper/keeper.go 80.70% <84.61%> (+0.76%) ⬆️
sei-cosmos/x/auth/tx/service.go 7.51% <0.00%> (-0.06%) ⬇️
sei-cosmos/types/query/filtered_pagination.go 62.35% <53.84%> (-4.84%) ⬇️

... and 288 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment thread sei-cosmos/types/query/pagination.go
@amir-deris amir-deris changed the title fix: lower sei-cosmos pagination query MaxLimit to 10,000 fix(sei-cosmos): enforce 10,000-item cap on all paginated RPC query entry points May 22, 2026
@amir-deris amir-deris changed the title fix(sei-cosmos): enforce 10,000-item cap on all paginated RPC query entry points fix(sei-cosmos): cap paginated RPC queries at 10,000 items per page (PLT-361) May 22, 2026
Comment thread sei-cosmos/x/bank/keeper/keeper.go
@amir-deris amir-deris requested review from bdchatham and masih May 22, 2026 22:48
Comment thread sei-cosmos/types/query/pagination.go
@amir-deris amir-deris changed the title fix(sei-cosmos): cap paginated RPC queries at 10,000 items per page (PLT-361) fix(sei-cosmos): cap paginated RPC queries at 1,000 items per page (PLT-361) May 26, 2026
@amir-deris amir-deris changed the title fix(sei-cosmos): cap paginated RPC queries at 1,000 items per page (PLT-361) fix(sei-cosmos): harden paginated RPC queries against DoS via limit, offset, and count_total caps (PLT-361) May 26, 2026
Comment thread sei-cosmos/types/query/filtered_pagination.go
Comment thread sei-cosmos/types/query/pagination.go Outdated
Comment thread sei-cosmos/types/query/pagination_test.go Outdated
Comment thread sei-cosmos/types/query/filtered_pagination.go Outdated
Comment thread sei-cosmos/types/query/pagination.go
Comment thread sei-cosmos/x/auth/tx/service.go

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 6b0b9fc. Configure here.

Comment thread sei-cosmos/types/query/filtered_pagination.go
return true, nil
})
s.Require().Error(err)
s.Require().Contains(err.Error(), "scanned more than")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we add a selective-filter case here? The scan-limit tests are all all-pass / all-fail, but the bug this PR actually fixes is the mid-page one — a ~1-in-N filter that needs a few thousand raw scans to fill its page should still succeed, not get rejected. That's the regression I'd want pinned so we don't quietly bring back the old totalIter-vs-end comparison.


func (suite *IntegrationTestSuite) TestGetPaginatedTotalSupplyMaxLimitExceeded() {
_, _, err := suite.app.BankKeeper.GetPaginatedTotalSupply(suite.ctx, &query.PageRequest{Limit: query.MaxLimit + 1})
suite.Require().Error(err)

@bdchatham bdchatham Jun 9, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a multi-page case here for 1,000 denoms asserting collectAllTotalSupply returns the full set? Everything's single-page today, so the cross-page merge is untested, and that's the path feeding the in-consensus TotalSupply invariant. Given the fork stakes I'd want it covered.

Comment thread sei-cosmos/types/query/pagination.go
Comment on lines +21 to +28
// Scan limits: to prevent unbounded store walks, this function caps iteration at MaxScanLimit
// entries both before the page is filled (Phase 1) and after (Phase 2). If the cap is hit:
// - Phase 1 (page not yet full): returns an error — the page could not be assembled.
// - Phase 2, CountTotal=true: returns an error — the total count could not be determined.
// - Phase 2, CountTotal=false: returns the assembled page with NextKey=nil. This does NOT
// guarantee there are no further results; it means no next filtered hit was found within
// MaxScanLimit entries past the page boundary. Callers that require reliable full traversal
// of sparse datasets should use key-based pagination instead.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Slight concern with the growing size of these comments. A bit of a smell to me suggesting we could make the code more expressive so that it effectively explains itself. It does already include comments in the implementation

@masih

masih commented Jun 10, 2026

Copy link
Copy Markdown
Collaborator

THere is some overlap between this PR and #3524 that's worth distilling before review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants