Skip to content

chore: publish new package versions#4694

Merged
balegas merged 1 commit into
mainfrom
changeset-release/main
Jul 22, 2026
Merged

chore: publish new package versions#4694
balegas merged 1 commit into
mainfrom
changeset-release/main

Conversation

@github-actions

@github-actions github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

This PR was opened by the Changesets release GitHub action. When you're ready to do a release, you can merge this and the packages will be published to npm automatically. If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated.

Releases

@electric-ax/durable-streams-server-rust@0.1.5

Patch Changes

  • 9acd04f: Memory-mode CPU fix: batch meta sidecar flushes into a periodic sweep (durable-streams-rust: memory mode uses ~5x more CPU than wal at identical low load (per-append debounced meta sidecar flush) #4691).

    --durability memory appends no longer schedule a per-stream debounced sidecar flush (a timer task + full sidecar rewrite per stream per 100 ms — ~5x wal-mode CPU at high stream cardinality under low per-stream rates). Appends and TTL read touches now only mark the stream dirty in a store-level set; a single 1 s sweeper flushes all dirty sidecars in one pass, mirroring the batched checkpoint treatment wal mode got in the write-path overhaul. The sidecar's producer/access state remains a non-durable lagging flush; its lag bound moves from 100 ms to the 1 s sweep cadence. Durable flush-on-close/delete paths are unchanged, and a pending flush can no longer resurrect the sidecar of a hard-deleted stream.

  • 012dc4a: Recovery hardening: durability barriers (committer fdatasync, checkpoint syncfs, segment seal) are fail-stop instead of retryable-in-place (a retried fsync can falsely succeed on Linux and ack/recycle lost bytes); failed checkpoints re-register their dirty set (previously a transient error + restart truncated acked bytes); torn sidecars are quarantined instead of deleting the stream's data file; missing stream-lane mounts refuse to boot instead of letting the WAL reset destroy the lane's records; append stage failures roll back the data write and producer state (500'd bytes no longer resurrect; retries no longer swallowed as duplicates); sealing cuts at the durable frontier; unreadable sealed chunks fail the read instead of serving a response with missing interior bytes; dir fsyncs added across the WAL metadata lifecycle.

  • db4977d: Eliminate the WAL write cardinality cliff (10.4k → 383k appends/s @100k streams; 212k @1m).

    • Checkpoint durability now uses one syncfs barrier per stream lane on Linux (was O(touched-streams) per-file fdatasync — the barrier storm that collapsed throughput at high stream counts).
    • New --wal-checkpoint-interval-ms (per-shard time trigger, default 3000) and --wal-checkpoint-wal-bytes (retained-WAL size budget, 0 = off): checkpoint cadence is an explicit crash-replay budget and shards self-stagger instead of storming together.
    • New --stream-lanes N (default 1 = unchanged layout): hash stream data files across streams/<0..N>/ dirs, one per device, spreading checkpoint writeback over N devices with N parallel barriers. The lane count is persisted and validated on open.
    • New --server-stats N telemetry (SRV_STATS: cpu / inflight / service / lock / durability-wait per interval) — the dependency-free bottleneck diagnostics used to find all of the above. Memory-mode plain appends no longer queue redundant sidecar flushes.
    • Removed dead/diagnostic flags: --wal-fsync-parallel, --wal-meta-gate, --mem-meta-gate, --meta-sweep-disable, --meta-sweep-stats, --tier local / --tier-local-dir (tier is off|s3). --durability memory combined with --tier is now rejected at startup. WAL records without PAYLOAD_CHECKSUMMED decode as torn (no released writer ever emitted them).

    Deployment guidance (device layout, CPU pinning, checkpoint budgets): WAL_TUNING.md.

@core/sync-service@1.7.8

Patch Changes

  • 5a298c6: Bound the memory pinned by a shape response served to a slow or stalled client. Two compounding issues let every stalled connection pin its entire in-flight log chunk (~10 MB by default) for as long as the serve lived: the log file was read eagerly as one whole-range binary whose entries were served as sub-binary slices (pinning the full chunk plus the full entry list), and the JSON encoder batched response elements by item count only, so a batch of large rows grew into a multi-megabyte unit held in full by the request process and the socket's driver queue. Accumulated stalled serves could exhaust node memory (observed in production: ~400 stalled serves pinning ~3.9 GB, immune to GC since the references are live). The log is now read lazily in 64 KiB blocks and encoder batches are additionally capped at 256 KiB, bounding the pinned memory per stalled connection to well under 1 MB regardless of chunk size.
  • 7b8fecc: Fix a cold nested subquery shape returning an initial HTTP 500 while its dependency snapshots were still in progress. When an outer shape's consumer initialized, it subscribed to each dependency materializer using GenServer.call with the default 5s timeout. A dependency materializer stays blocked in start-up until its own snapshot starts, so if that snapshot took longer than 5s the subscribe timed out, the outer shape was removed, and the client's first request 500'd (a retry succeeded once the snapshot finished). The subscribe now waits with :infinity, consistent with the other materializer calls, so a cold nested shape waits for its dependency materializers and completes without an externally visible 500. Liveness is unaffected: the caller already monitors the materializer, so a dead dependency surfaces as a call exit rather than being masked by a short timeout.
  • a1026bd: fix: make call-home telemetry opt-in
  • 4c2498a: Stop subquery shapes from being spuriously removed during a server restart. When
    a dependency consumer's inline call to its materializer raced the materializer's
    shutdown, the resulting :noproc exit crashed the consumer and removed the shape
    from disk, causing a 409 must-refetch after the restart. The consumer now
    absorbs that exit and lets the monitored :DOWN drive a clean stop.
  • b2cf71d: Add ELECTRIC_TCP_READ_TIMEOUT to configure the socket read / HTTP keep-alive
    idle timeout (ThousandIsland's read_timeout, default 60s). When Electric runs
    behind a connection-pooling proxy such as an AWS ALB, this must be set above
    the proxy's idle timeout — otherwise the proxy races Electric's unannounced
    idle close when reusing a pooled connection and clients see intermittent 502s.

@github-actions
github-actions Bot force-pushed the changeset-release/main branch 9 times, most recently from 1969a1d to 3fa8978 Compare July 20, 2026 09:37
@github-actions
github-actions Bot force-pushed the changeset-release/main branch from 3fa8978 to 5109f71 Compare July 21, 2026 13:43
@github-actions
github-actions Bot force-pushed the changeset-release/main branch from 5109f71 to 6f28983 Compare July 21, 2026 15:00
@netlify

netlify Bot commented Jul 21, 2026

Copy link
Copy Markdown

Deploy Preview for electric-next ready!

Name Link
🔨 Latest commit 6f28983
🔍 Latest deploy log https://app.netlify.com/projects/electric-next/deploys/6a5f8973dc600100083b10e1
😎 Deploy Preview https://deploy-preview-4694--electric-next.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@balegas
balegas merged commit 88793e7 into main Jul 22, 2026
5 checks passed
@balegas
balegas deleted the changeset-release/main branch July 22, 2026 14:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant