[cata-on]

Problem

Two bugs of the same area, in @electric-sql/pglite 0.5.8 to 0.5.15, with a filesystem that has something to sync (idb:// is the one that matters).

1. PGliteWorker drops the options of execProtocol*, so every protocol message syncs the filesystem.
BasePGlite sends each protocol message of a query with syncToFs: false and syncs once at the end of the query. PGliteWorker.execProtocolStream(message) sends only the message to the worker, and the worker calls db.execProtocolStream(message) without options, so the default syncToFs: true applies. Each Parse, Describe, Bind, Execute and Sync runs a full syncToFs(), which is an Emscripten FS.syncfs(false) with idb://.

Measured in Chromium with idb://:

one protocol message, with the sync	15.7 ms
one protocol message, without the sync	0.06 ms
read transaction of 7 parameterised queries, direct PGlite	about 18 ms
the same through PGliteWorker (before)	742 ms
the same through a worker that runs each message with syncToFs: false and syncs once after a Sync outside a transaction (the behaviour this PR gives)	50 ms

A parameterised query is six messages. The tests in this PR count the calls that reach the filesystem: a parameterised query() through the worker made 7 syncToFs() calls (6 messages + the one at the end of the query) against 1 on a direct instance, and the 7 queries of a transaction made 42 between the first and the last one.

The same dropped options affect onNotice and throwOnError, which are silently ignored through the worker, and execProtocolRawStream(message, { onRawData }) cannot work at all: the callback is part of the RPC message and postMessage throws a DataCloneError.

2. A resolved transaction() has not synced (already reported in #1065).
transaction() runs COMMIT through #runExec while #inTransaction is still true, so the if (!this.#inTransaction) await this.syncToFs() at the end of #runExec is skipped, and the flag is cleared only afterwards. Nothing syncs after the commit. The committed transaction reaches the filesystem at the next query outside a transaction, and is lost if the page reloads or the worker stops before that. This is also true of a direct (non-worker) instance, and of ROLLBACK (no sync after it either).

Root cause (at bac163e)

1. packages/pglite/src/base.ts L165-L175: #execProtocolNoSync calls this.execProtocolStream(message, { ...options, syncToFs: false }), and #runQuery/#runExec call syncToFs() once at the end (L325-L327, L374-L376).
2. packages/pglite/src/worker/index.ts L335-L355, L368-L373: the tab-side execProtocolRaw, execProtocol and execProtocolStream accept only message; execProtocolRawStream sends its options, functions included. On the worker side, L658-L689 call db.execProtocol*(message) without options.
3. packages/pglite/src/pglite.ts L1033-L1041: execProtocolRaw defaults to syncToFs = true, which is what the worker then gets.
4. packages/pglite/src/base.ts L514-L538: the COMMIT at L518 runs with the flag set, the flag is cleared at L528, and transaction() returns with no sync.

Through the worker, bug 1 hides bug 2, because every message syncs. Fixing only bug 1 turns bug 2 into data loss (see below).

Fix

Commit 1, fix(pglite): sync to the filesystem when a transaction ends (base.ts): after the COMMIT, the flag is cleared and transaction() awaits one syncToFs() before it resolves. A transaction that ends with a rollback (throwing callback, tx.rollback(), a COMMIT that reports a rollback) also ends with one sync, which does not replace the error that ended the transaction. The flag is also cleared if the ROLLBACK itself fails. A transaction costs one sync for its end, plus the sync after its BEGIN that already existed; the statements in between do not sync.

Commit 2, fix(pglite): forward the exec protocol options of PGliteWorker to the worker (worker/index.ts):

• execProtocolRaw, execProtocol and execProtocolStream send syncToFs and throwOnError to the worker, which passes them on. Only these two are sent, as the others are functions and postMessage cannot clone them.
• onNotice runs on the tab, with the notices among the messages the worker returns. It runs when the call returns, not while the statement runs.
• execProtocolRawStream takes ExecProtocolOptionsStream like the interface says. The worker collects the chunks (copies them, they are views of the WASM heap) and returns them, and the tab calls onRawData with each one. It was a DataCloneError before, so nothing that worked changes.

Not changed: a DatabaseError still reaches the tab as a plain Error, and with throwOnError: false it is in the returned messages with its text only, as the structured clone drops the class.

Tests

packages/pglite/tests/transaction-sync.test.ts (direct instance, MemoryFS subclass that counts syncToFs() calls and keeps a snapshot of PGDATA taken at each sync, as IDBFS would persist it):

• a parameterised query and an exec of several statements sync once;
• no sync inside a transaction and exactly one after the COMMIT;
• a rolled back transaction (explicit and by exception) ends with a sync;
• a transaction that has resolved is in the snapshot: reopen from it with loadDataDir and read the row.

packages/pglite/tests/worker-sync.test.ts with tests/fixtures/sync-worker-thread.js: a real leader worker (worker() from dist/worker/index.js) in a Node.js worker_threads thread, PGliteWorker on the main thread, the RPC over BroadcastChannel and navigator.locks as in a browser. The same checks through the worker, plus:

• the syncToFs option of execProtocol, execProtocolStream and execProtocolRaw reaches the filesystem layer (false gives no call, the default gives one);
• throwOnError, onNotice (for execProtocol, execProtocolStream and exec) and execProtocolRawStream work through the worker;
• durability through the worker: after await db.transaction(...) and db.close() (which terminates the worker), the snapshot holds the row.

They fail without the fix and pass with it (pnpm vitest run <file>, Node 24, results at bac163e):

	without the commit	with it
transaction-sync.test.ts (commit 1, base.ts from bac163e)	3 of 5 fail: no sync after the COMMIT (0 instead of 1), no sync after a rollback (1 instead of 2), the committed row is not in the snapshot	5 of 5 pass
worker-sync.test.ts (commit 2, worker/index.ts from before it)	7 of 8 fail: 7 syncs for a parameterised query instead of 1, 3 for an exec instead of 1, 42 inside the 7-query transaction, syncToFs: false ignored (3 calls), throwOnError ignored, onNotice never called, DataCloneError for onRawData	8 of 8 pass

pnpm test of the package (test:basic and test:node): 68 files, 360 tests passed, 1 skipped, type errors none. pnpm lint, prettier --check ./src ./tests and pnpm typecheck are clean. A changeset (patch) is added for each commit.

Durability

Commit 2 alone would turn bug 2 into data loss through the worker: with the options forwarded, nothing syncs after the COMMIT, and the worker test "has persisted a transaction when it resolves, and the worker is gone" fails on that combination (the committed row is not in the snapshot taken when the worker is terminated right after transaction() resolved). With commit 1 as well it passes. This is why the two commits belong together. The same test on a direct instance fails on bac163e, so it is not only a worker problem.

Related

• [BUG]: PGlite suppresses synchronization while inside a transaction #1065 (the report of the transaction half) and fix: sync to the filesystem when a transaction ends #1066 (a fix for it that was closed without merging). This PR takes the same view of the terminal sync; the commit keeps the COMMIT under the in-transaction flag and syncs explicitly after it, so there is exactly one sync after the commit.
• [BUG]: syncToFs() resolves immediately, without syncing, when another sync is already queued #1191 and fix(pglite): make syncToFs() wait for a queued sync #1193: syncToFs() can resolve before a queued sync has started. Independent of this change, and complementary, as the sync that ends a transaction relies on it.
• fix(pglite): apply worker parsers and serializers on the main thread #1114 also edits worker/index.ts (the constructor and the init message, not the methods touched here).

Not covered

The tests use a filesystem double in Node.js, not IDBFS in a browser. The 15.7 ms / 742 ms / 50 ms figures are from Chromium with idb:// and are not reproduced by these tests, which count calls. The browser suites (pnpm test:web) were not run.

🤖 Generated with Claude Code