Summary
Every paginated endpoint in the SDK requires callers to write a manual while loop to collect all pages. As soon as a second paginated endpoint was added (get_entity_budget_flows, which now returns PaginatedResponse[dict[str, Any]]), the same boilerplate appeared again. We should ship a first-class pagination helper before more callers bake the loop into their code.
Boilerplate this would replace:
all_flows: list[dict] = []
page = 1
while True:
resp = client.get_entity_budget_flows(uei, page=page, limit=100, fiscal_year=2024)
all_flows.extend(resp.results)
if resp.next is None:
break
page += 1
Options for discussion
Three shapes are on the table. The issue should land a decision before implementation begins.
Option 1 — Per-endpoint generator methods
Add an iter_* method alongside each list_* / sub-resource method:
for flow in client.iter_entity_budget_flows(uei, fiscal_year=2024):
process(flow)
- Discoverable via IDE autocomplete.
- Fully typed — return type is
Iterator[T] where T matches the endpoint's item type.
- Downside: doubles the public surface area (one
get_* + one iter_* per paginated endpoint).
Option 2 — Generic paginate() helper
A single helper (as a method or free function) that accepts any paginated callable:
for flow in client.paginate(client.get_entity_budget_flows, uei, fiscal_year=2024):
process(flow)
# or as a free function:
from tango import paginate
for flow in paginate(client.get_entity_budget_flows, uei, fiscal_year=2024):
process(flow)
- Minimal surface area — one new symbol covers every current and future paginated endpoint.
- Call-site is slightly awkward (passing the method as a callable).
- Generic typing is harder to express precisely (
ParamSpec / TypeVar gymnastics).
- Works for both page-based and cursor-based conventions if implemented carefully.
Option 3 — Method on PaginatedResponse
Attach iteration behavior to the response object itself:
resp = client.get_entity_budget_flows(uei, fiscal_year=2024)
for flow in resp.iter_remaining(client):
process(flow)
- No new top-level surface; discovery happens on the object you already hold.
- Mixes data and behavior:
PaginatedResponse becomes aware of the client and of how to fetch more pages, which doesn't match the current SDK design.
- Partial-page scenarios (user already consumed some results from
resp) are awkward.
Decision criteria
The decision should cover:
- Which shape (1, 2, 3, or a blend — e.g. Option 2 as the implementation primitive + Option 1 as thin wrappers built on top of it).
- Scope of the first pass: page-based pagination only, or cover cursor-based endpoints in the same PR?
Out of scope
- Auto-pagination by default: changing
list_* / get_* methods to return an iterator would be a breaking change to existing callers.
- Async iteration: no async client today; that's a separate concern.
- Bulk/concurrent page fetching utilities.
Acceptance signal
This issue is resolved when the team has agreed on a shape and scope. Implementation is a follow-on PR.
Summary
Every paginated endpoint in the SDK requires callers to write a manual
whileloop to collect all pages. As soon as a second paginated endpoint was added (get_entity_budget_flows, which now returnsPaginatedResponse[dict[str, Any]]), the same boilerplate appeared again. We should ship a first-class pagination helper before more callers bake the loop into their code.Boilerplate this would replace:
Options for discussion
Three shapes are on the table. The issue should land a decision before implementation begins.
Option 1 — Per-endpoint generator methods
Add an
iter_*method alongside eachlist_*/ sub-resource method:Iterator[T]whereTmatches the endpoint's item type.get_*+ oneiter_*per paginated endpoint).Option 2 — Generic
paginate()helperA single helper (as a method or free function) that accepts any paginated callable:
ParamSpec/TypeVargymnastics).Option 3 — Method on
PaginatedResponseAttach iteration behavior to the response object itself:
PaginatedResponsebecomes aware of the client and of how to fetch more pages, which doesn't match the current SDK design.resp) are awkward.Decision criteria
The decision should cover:
Out of scope
list_*/get_*methods to return an iterator would be a breaking change to existing callers.Acceptance signal
This issue is resolved when the team has agreed on a shape and scope. Implementation is a follow-on PR.