Skip to content

Cursor-based pagination APIs - #18

Open
ArtyomSavchenko wants to merge 6 commits into
developfrom
find-cursor
Open

Cursor-based pagination APIs#18
ArtyomSavchenko wants to merge 6 commits into
developfrom
find-cursor

Conversation

@ArtyomSavchenko

@ArtyomSavchenko ArtyomSavchenko commented Jul 29, 2026

Copy link
Copy Markdown
Member

Summary

Adds cursor-based pagination APIs for processing large result sets without loading all documents through findAll in a single request.

New APIs:

  • findAllPage — fetches one page.
  • iterateAll — asynchronously iterates over the complete result set.
  • Both methods are also available through TxOperations.

Implementation

  • Stateless signed keyset cursors.
  • Cursors are bound to the account, workspace, class, query, and sorting configuration.
  • Automatic _id tie-breaker for deterministic ordering.
  • Scalar sorting with null value support.
  • Maximum page size of 1,000 documents.
  • Adapter-level PostgreSQL pagination.
  • Minimal keyset implementation for MongoDB.
  • In-memory pagination for the model domain.
  • total represents the complete accessible result set without applying the current cursor.
  • Pagination uses the existing security pipeline, including private-space access checks.

The API is exposed through:

  • Core Client
  • TxOperations
  • WebSocket/RPC connections
  • REST API
  • PlatformClient
  • Presentation and media client adapters

Usage

Fetch individual pages:

const page = await client.findAllPage(
  chunter.class.Channel,
  { archived: false },
  {
    limit: 100,
    sort: { name: SortingOrder.Ascending }
  }
)

const nextPage = await client.findAllPage(
  chunter.class.Channel,
  { archived: false },
  {
    limit: 100,
    sort: { name: SortingOrder.Ascending },
    cursor: page.nextCursor
  }
)

Iterate over the complete result set:

for await (const channel of client.iterateAll(
  chunter.class.Channel,
  { archived: false },
  {
    limit: 100,
    sort: { name: SortingOrder.Ascending }
  }
)) {
  // Process channel
}

Testing

Added API tests covering:

  • Large result sets without gaps or duplicates
  • Result sets containing 0, 1, 3, 10, and 11 documents
  • Stable ordering with duplicate and null sort values
  • Automatic _id tie-breaking
  • Private-space access restrictions
  • Cursor ownership validation
  • Rejection of cursors reused with a different query
  • total calculation
  • iterateAll through TxOperations

Signed-off-by: Artem Savchenko <armisav@gmail.com>
Signed-off-by: Artem Savchenko <armisav@gmail.com>
@ArtyomSavchenko ArtyomSavchenko changed the title Find cursor Cursor-based pagination APIs Jul 29, 2026
Signed-off-by: Artem Savchenko <armisav@gmail.com>
Signed-off-by: Artem Savchenko <armisav@gmail.com>
Signed-off-by: Artem Savchenko <armisav@gmail.com>
@ArtyomSavchenko
ArtyomSavchenko marked this pull request as ready for review July 30, 2026 09:18
@ArtyomSavchenko
ArtyomSavchenko marked this pull request as draft July 30, 2026 09:19
Signed-off-by: Artem Savchenko <armisav@gmail.com>
@ArtyomSavchenko
ArtyomSavchenko marked this pull request as ready for review July 30, 2026 09:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant