Skip to content

fix(archiver): speed up log-by-tag queries - #25254

Open
spalladino wants to merge 1 commit into
spl/batched-kv-getsfrom
spl/faster-get-private-logs-by-tags
Open

fix(archiver): speed up log-by-tag queries#25254
spalladino wants to merge 1 commit into
spl/batched-kv-getsfrom
spl/faster-get-private-logs-by-tags

Conversation

@spalladino

@spalladino spalladino commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Stack

Part of A-1817. Bottom to top, each PR targets the branch below it:

Context

getPrivateLogsByTags was reported at 100-200ms per call in production, ~10x slower than other node RPC calls. The existing node_rpc_perf bench couldn't see it (~0.2ms) because it measured a single random tag; measured PXE traffic at the node boundary is very different: ~100 tags per call (median 84, p95 100), includeEffects on, no fromBlock, and a near-total miss rate (0.45% of tags match; 86% of calls return nothing). Profiling with representative benches attributed the cost to: per-tag scans running sequentially, each paying 2-4 native msgpack cursor round trips (10-entry pages vs a 20-log limit); queries queueing behind writes on lmdb-v2's single writer queue; and, for log-heavy responses, re-hydrating thousands of Fr fields through layered Zod schemas and a Buffer-to-hex-to-BigInt round trip.

Approach

Queries stay transactional: they still run in db.transactionAsync with exactly the same snapshot-consistency semantics as before, and therefore still serialize with writes. Taking them off the writer queue without losing that guarantee needs the read-only snapshots from #25280 and is done in #25315, on top of this PR.

Within that unchanged envelope:

  • Archiver log store: per-tag scans run 8-wide via asyncPool over the transaction's snapshot instead of one at a time, preserving per-tag result order.
  • kv-store lmdb-v2: range scans with limit <= 128 are fetched as a single one-page cursor request (one native round trip, no cursor slot held). A narrow fast path in WriteTransaction.iterate delegates to that one-page read when the pending batch is empty — safe because nothing pending can shadow or suppress a committed entry — so it also applies to reads inside a transaction.
  • Foundation: hexSchemaFor collapses its refine/refine/transform chain into a single transform, and fromHexString builds field elements via BigInt('0x…') directly instead of round-tripping through a Buffer (~5x faster per field, benefits all hex deserialization).

Measured impact (uncontended, vs base):

  • Production-shaped call (100 tags, ~all misses, includeEffects), in-process: 5.3ms → 3.2ms (~40%).
  • 100-tag all-hits call: in-process 7.8ms → ~5.5ms; over HTTP 19.2ms → ~15ms.
  • Queued behind 5 concurrent 100-tag queries: 41ms → ~33ms (reads still serialize; each item in the queue got cheaper).
  • Client-side parse of a 100-log response: 2.24ms → 1.9ms.

Benches and tests added:

  • node_rpc_perf bench now mints to private so blocks contain real private logs, and adds getPrivateLogsByTags_100tags shaped after the measured recipient-sync traffic (100 tags, 1 hit + 99 misses, includeEffects, referenceBlock; ~3.4ms avg) plus getPrivateLogsByTags_100tags_allhits and getPublicLogsByTags_100tags stress variants, alongside the existing single-tag series.
  • node_rpc_perf also gains getPrivateLogsByTags_100tags_queued, which measures how much a 100-tag query queues behind five concurrent siblings.
  • New log_store_write_contention test: asserts a tag query racing queued write transactions returns results identical to an uncontended query.

@github-actions github-actions Bot added the port-to-next Forward-port this merged PR into next label Aug 18, 2026
@spalladino spalladino changed the title fix(archiver): unblock and speed up log-by-tag queries fix(archiver): speed up log-by-tag queries Aug 20, 2026
@AztecBot

Copy link
Copy Markdown
Collaborator

Flakey Tests

🤖 says: This CI run detected 1 tests that failed, but were tolerated due to a .test_patterns.yml entry.

\033FLAKED\033 (8;;http://ci.aztec-labs.com/0c4e03ece8955af1�0c4e03ece8955af18;;�):  yarn-project/end-to-end/scripts/run_test.sh simple src/multi-node/slashing/slash_veto_demo.test.ts (246s) (code: 0) group:e2e-p2p-epoch-flakes

@spalladino
spalladino force-pushed the spl/faster-get-private-logs-by-tags branch from 7832313 to 55bc733 Compare August 25, 2026 19:34
@spalladino
spalladino changed the base branch from merge-train/spartan-v5 to spl/batched-kv-gets August 25, 2026 19:34
@spalladino
spalladino force-pushed the spl/faster-get-private-logs-by-tags branch from 55bc733 to 654e2b4 Compare August 25, 2026 19:44
@spalladino spalladino removed the port-to-next Forward-port this merged PR into next label Aug 25, 2026
@spalladino
spalladino force-pushed the spl/faster-get-private-logs-by-tags branch from 654e2b4 to 337936d Compare August 25, 2026 21:13
Tag log queries fan the per-tag scans out 8 wide instead of walking tags one at
a time, and bounded scans now read a single cursor page instead of paging in
tens of entries at a time. `SINGLE_PAGE_LIMIT` caps how large a bounded scan may
be before it goes back to paging.

Supporting changes on the read path: an empty write batch short circuits to a
plain read, `Fr.fromHexString` rejects over-long input on byte length rather
than after parsing, and `hexSchemaFor` runs a single transform instead of
chaining two.

The queries stay inside `db.transactionAsync`, so they serialize with queued
writes by design and cannot return a torn result.

`node_rpc_perf` gains the production query shape (100 tags per call, real tags,
a reference block anchor, `includeEffects` on) plus a probe that measures how
much a query queues behind five concurrent siblings.
@spalladino
spalladino force-pushed the spl/faster-get-private-logs-by-tags branch from 337936d to 226b1fc Compare August 25, 2026 21:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants