Performance

Bulk scans

Read every matching row in bounded batches, for exports and background jobs, with one PostgreSQL cursor.

resource.scan(options, consume) reads every row that matches a request, in batches. Use it for exports and background jobs. Normal pagination selects a new page each time; on PostgreSQL a scan runs the selection once and fetches from a cursor.

Read the batches

const 
exportedRows
= await
ordersResource
.
scan
(
{
request
,
context
: {
orgId
: "acme" },
batchSize
: 1000,
count
: "exact",
load
: "detail",
}, async (
batches
) => {
let
processed
= 0;
for await (const {
rows
,
totalRows
} of
batches
) {
processed
+=
rows
.
length
;
console
.
log
({
processed
,
totalRows
});
} return
processed
;
}, );

The callback receives an async iterable of { rows, totalRows? }. Whatever it returns is what scan returns. Rows are typed like query's, with the default load profile, the load you pass, or a view.

OptionDefaultNotes
requestnoneFilters, sorting and search. Pagination and facets are ignored.
batchSize1000A positive integer, set by your server. Independent of the public page-size limit.
count"none""exact" adds totalRows to every batch, but may process the whole selection first.
signalnoneAn AbortSignal, checked between database operations.
dbthe engine'sAn existing repeatable-read or serializable transaction.
  • The scope and validation run once.
  • A unique id is added to the sort, so the order is stable.
  • Await your writer inside the loop. The engine doesn't fetch ahead or keep rows in memory.
  • An empty selection yields no batches, and scans don't compute summaries.

Snapshot and connection

PostgreSQL scans run DECLARE … NO SCROLL CURSOR, FETCH and CLOSE in a repeatable-read, read-only transaction. Rows and their relations come from one snapshot, so concurrent writes don't shift rows between batches.

That connection and snapshot stay open for the whole callback, so:

  • bound the number of concurrent exports;
  • don't upload files or wait on other services inside the callback;
  • expect sorting and exact counts to use temporary storage in the database.

A scan can't resume from a checkpoint.

The cursor is closed when the callback returns, on break, on errors and on cancellation. Cancellation doesn't interrupt a statement that is already running; use statement timeouts for that.

To share a snapshot with other work, pass your own transaction as db. The engine doesn't commit or roll it back. A read-committed transaction is rejected.

import { 
drizzle
} from "drizzle-orm/node-postgres";
const
db
=
drizzle
({
connection
: {
database
: "app" },
relations
});
await
db
.
transaction
(
async (
transaction
) => {
await
ordersResource
.
scan
(
{
request
,
db
:
transaction
,
context
: {
orgId
: "acme" } },
async (
batches
) => {
for await (const {
rows
} of
batches
) {
console
.
log
(
rows
.
length
);
} }, ); }, {
isolationLevel
: "repeatable read",
accessMode
: "read only" },
);

Other databases and custom steps

The cursor path needs:

  • an asynchronous PostgreSQL driver with interactive transactions (not HTTP-only drivers);
  • a unique, non-null root id, and unique relation keys for the sort columns. A sort through an ambiguous relation is rejected rather than exporting duplicates.

Other databases, and resources with a custom strategy.query or strategy.ids, scan by bounded pagination instead. The scope resolves once, facets are skipped and an exact count runs on the first page only, but there is no single snapshot. A custom strategy.rows works with both paths.

Copyright © 2026