Reliable REST API pagination is less about adding a page parameter and more about defining stable ordering, clear continuation rules, and safe client behavior. This guide compares offset, page-number, cursor, keyset, and link-based pagination, then provides checklists and reusable request loops for retrieving records without unnecessary duplicates, skips, or runaway requests.
Overview
Pagination divides a potentially large collection into smaller responses. A client requests one portion, processes it, and follows the API's instructions to request the next portion. The server may expose pagination through query parameters, such as limit and offset, or through a cursor, a continuation token, or links in the response body.
The right pattern depends on the data and the operation. Offset pagination is easy to understand and often convenient for small, mostly stable collections. Cursor and keyset pagination are usually better suited to large or frequently changing datasets because they avoid repeatedly scanning from the beginning of a result set. Link-based navigation can hide implementation details from clients and is useful when the server wants to control how traversal works.
Before implementing a client, answer four questions:
- What ordering does the endpoint guarantee?
- What happens when records are inserted, deleted, or updated during traversal?
- How does the response indicate that more data exists?
- Can the client safely retry the same request?
Pagination is also an API contract concern. Document the accepted limit range, default behavior, sorting rules, cursor lifetime if relevant, and the meaning of an empty page. Validate these details alongside the rest of the API contract; an OpenAPI validation workflow can help catch mismatches before they reach client code.
Pagination patterns compared
| Pattern | Typical request | Strengths | Risks and trade-offs |
|---|---|---|---|
| Offset | ?limit=50&offset=100 | Simple, familiar, and convenient for controlled navigation. | Large offsets can become expensive; changing data can cause skips or duplicates. |
| Page number | ?page=3&page_size=50 | Easy to expose in user interfaces and documentation. | Has the same consistency problems as offset pagination and can obscure the actual starting position. |
| Cursor | ?limit=50&after=encoded_cursor | Efficient for sequential traversal and commonly works well with changing collections. | Cursors are opaque, may expire, and are less convenient for jumping to an arbitrary page. |
| Keyset | ?limit=50&after_id=12345 | Uses indexed sort keys and supports efficient continuation over large tables. | Requires a stable, carefully defined sort order and is not ideal for every filter or sort option. |
| Link-based | Links such as next in the response | Keeps navigation rules on the server and makes clients less dependent on parameter names. | Clients must parse and follow links correctly; available navigation may vary by endpoint. |
Checklist by scenario
When building a new endpoint
- Choose a default sort order and document it. A result set without deterministic ordering is difficult to paginate reliably.
- Add a unique tie-breaker to the sort. For example, sort by
created_atand then by a unique identifier rather than by a timestamp alone. - Set a sensible default limit and enforce a maximum. Treat the requested limit as a preference within the server's allowed range.
- Define the continuation contract:
nextlink, cursor, offset, or an explicit boolean such ashas_more. - Return enough metadata for the client to know whether it should continue, but avoid requiring an expensive total count unless the product genuinely needs it.
- Specify how filters and sort parameters are preserved across pages. A continuation link or cursor should not silently lose the original query.
- Test the endpoint with zero records, fewer records than the limit, exactly one full page, and multiple full pages.
When consuming offset or page-number pagination
- Send an explicit, bounded page size instead of relying on an undocumented default.
- Use a stable sort whenever the API supports one. Without it, the same record can move between pages.
- Stop when the response says there are no more results, when a next link is absent, or when the returned item count is less than the requested limit, according to the documented contract.
- Set a maximum number of pages or records for operational jobs. This protects against a faulty termination condition.
- For exports requiring an exact snapshot, ask whether the API supports a snapshot, version, or time-bound filter. Ordinary offset pagination does not automatically provide snapshot consistency.
When consuming cursor or keyset pagination
- Treat the cursor as opaque. Store and resend it exactly as returned; do not parse or construct it unless the API explicitly defines it as a public keyset.
- Preserve the original filters and sort settings for every subsequent request.
- Advance only after successfully processing the current page. Persisting the next cursor too early can make a restart skip unprocessed records.
- Detect a repeated cursor. A server or client bug that returns the same continuation token can otherwise create an infinite loop.
- Expect the final page to contain fewer items, no items, or an explicit end marker depending on the API contract.
When following response links
- Prefer the server-provided next link over rebuilding its query string.
- Handle a missing next link as the normal end of traversal.
- Apply authentication, timeout, and host allow-list rules to each followed URL.
- Do not assume that links use the same parameter names or URL shape across endpoints.
When writing a reusable client loop
A safe loop should separate fetching, processing, retrying, and continuation. A generic cursor-based shape looks like this:
cursor = null
seen_cursors = set()
while true:
response = get_items(limit=50, cursor=cursor)
process(response.items)
next_cursor = response.next_cursor
if next_cursor is null:
break
if next_cursor in seen_cursors:
raise PaginationError("cursor did not advance")
seen_cursors.add(next_cursor)
cursor = next_cursor
In production, add request timeouts, bounded retries for appropriate transient failures, rate-limit handling, structured logging, and a cancellation mechanism. If processing is not idempotent, record a durable item identifier or checkpoint so a retry does not create an unintended duplicate side effect.
What to double-check
- Limit semantics: Confirm whether the server treats the limit as a maximum, whether it silently caps it, and where it reports the effective value.
- Empty pages: An empty response may mean the end, a filter mismatch, or a temporary condition. Follow the endpoint's documented rule rather than guessing.
- Changing datasets: Inserts before an offset can shift later records. Deletions can do the same. Cursor or keyset pagination can reduce this problem, but it does not guarantee an immutable snapshot.
- Sort stability: A non-unique sort column needs a unique secondary key. Check ascending and descending behavior separately.
- Retries: Retrying a GET is commonly simpler than retrying a write, but the client should still respect rate limits and server guidance. Confirm whether a cursor remains valid after a delay.
- Authentication and permissions: Permissions or tenant filters must remain consistent across pages. A changed authorization context can produce an incomplete or inconsistent traversal.
- Response size: A high limit may reduce request count but increase memory use, latency, and failure cost. Measure a practical limit for the workload.
- Observability: Log endpoint, page or cursor position, item count, elapsed time, retry count, and termination reason. Avoid logging sensitive cursor contents if they may contain encoded data.
When debugging a pagination issue, inspect the actual request sequence rather than only the final error. The browser Network tab workflow described in this request debugging guide is useful for checking query parameters, response links, status codes, and timing. For database-backed endpoints, also review the query plan and indexes; the SQL debugging checklist covers related checks for duplicates and slow joins.
Common mistakes
- Using page numbers as an implicit consistency guarantee: Page three is not a permanent slice when records can change between requests.
- Sorting only by a timestamp: Equal timestamps can produce unstable boundaries. Add a unique tie-breaker.
- Stopping on a short page without confirming the contract: This is often reasonable, but some APIs apply filtering or authorization after selecting a page. Use the documented continuation signal.
- Following a cursor forever: Keep a maximum-work safeguard and detect repeated cursors or unchanged links.
- Persisting the cursor at the wrong time: Save progress after successful processing, not merely after receiving a response.
- Assuming retries are harmless: Re-fetching a page can be safe for reads but may duplicate downstream actions. Design processing to be idempotent where possible.
- Counting everything for every request: Exact totals can add database work and may become stale immediately. Include them only when the user experience or contract requires them.
- Changing pagination behavior without compatibility planning: Existing clients may depend on defaults, parameter names, or response fields. Introduce changes deliberately and test old clients.
When to revisit
Review pagination before a seasonal planning cycle, a major data import, a reporting or export feature, or a change in traffic volume. Revisit it whenever the underlying workflow or toolchain changes: a database migration, new index strategy, API version, SDK update, authentication model, filtering requirement, or client runtime can alter assumptions that once seemed safe.
Use this final pre-release checklist:
- Document the pagination pattern, default limit, maximum limit, ordering, and end-of-results signal.
- Test empty, partial, exact-boundary, multi-page, changing-data, retry, and repeated-cursor cases.
- Verify that filters, sorting, tenant scope, and authorization remain consistent across every request.
- Measure database latency and response size at realistic limits; adjust indexes or limits when evidence supports it.
- Confirm that client processing is idempotent or has a durable checkpoint strategy.
- Set safeguards for timeouts, rate limits, maximum pages, cancellation, and infinite-loop detection.
- Update API documentation, contract tests, SDK examples, and operational dashboards together.
Pagination should be treated as a repeatable integration contract, not a loop added at the edge of an application. If you make the ordering, continuation, consistency, and failure behavior explicit, the same checklist can guide a small administrative endpoint, a high-volume export, or a new client integration.