Skip to content

Cleanup/hardening: fix five open correctness/security defects, adopt ADR-341 invariants - #933

Merged
ruvnet merged 3 commits into
mainfrom
claude/ruvector-cleanup-hardening-m2ax54
Sep 5, 2026
Merged

Cleanup/hardening: fix five open correctness/security defects, adopt ADR-341 invariants#933
ruvnet merged 3 commits into
mainfrom
claude/ruvector-cleanup-hardening-m2ax54

Conversation

@ruvnet

@ruvnet ruvnet commented Aug 26, 2026

Copy link
Copy Markdown
Owner

Summary

One hardening pass over the open defect backlog: five issues fixed with regression tests, the underlying invariants recorded as ADR-341, and three new issues filed for defects found during the work (#930, #931, #932).

Numbering note: this ADR was authored as ADR-340, but main allocated ADR-340 to the signed retrieval-receipt anchoring ADR (#949) first, so the merge commit renumbers it to ADR-341 and fixes the two cypher_exec.rs comments that cited "ADR-340 invariant 1" for this ADR's NaN rule.

Closes #825, closes #901, closes #903, closes #907, closes #908.

Fixes

#825ruvector-delta-index: DeltaHnsw insert hang (root-caused: self-deadlock)

The hang was not an unterminated traversal: connect_node's prune branch held a write lock on a neighbor node while prune_neighbors re-locked the same node through self.distance(). parking_lot locks are not reentrant, so the inserting thread deadlocked — but only when a neighbor list overflowed m0, which the unseeded test RNG made nondeterministic. Reproduced deterministically (seeded, 400 inserts, dim 8 — hung pre-fix, <1s post-fix).

  • Neighbor's vector now comes from the held guard via a split borrow; prune_neighbors drops self-loops before measuring distances.
  • greedy_search carries an explicit progress bound (strictly-decreasing distance ⇒ ≤ nodes.len() moves) so a malformed graph terminates loudly.
  • Tests seeded; prune-heavy regression test added; ruvector-delta-index rejoins the core-platform CI shard (checklist item from ruvector-delta-index test_insert_and_search hangs indefinitely (DeltaHnsw insert/search) #825) — and passed there in 7 minutes on this PR's runs.

#901ruvector-tiny-dancer-core: NaN confidence panicking route()

Non-finite score/uncertainty now rejected at a single choke point covering both the gated and ungated paths, tripping the circuit breaker (matching the VoI path's contract); the decision sort uses total_cmp as defense-in-depth. Regression test poisons the model weights with NaN (the issue's corrupted-model scenario) and asserts error-not-panic.

#907ruvector-graph: strong-Arc pool (unlinked-database reuse + unbounded growth)

Ported ruvector-core's post-#902 reference shape wholesale: Weak per-path slots, Drop-under-the-slot-guard eviction, slot reaping. Regression tests: erased-path recreation observes an empty database; reopen-after-last-drop preserves data and reaps the pool entry; 40-iteration barrier-synchronized concurrent drop-vs-open probe, zero tolerated failures. The non-canonicalized-key tail note from #907 is split out as #932 so it survives this close.

#908ruvector-context: require_private_root owner check

The root must now be owned by the process euid in addition to being 0700, via rustix::process::geteuid() (no unsafe, already in Cargo.lock through fs4 — one line of lockfile churn, no new transitive dependency). As the issue argued, this rejects only roots that already failed later with a bare EACCES.

#903 — harness: locale-dependent benchmark cache key

canonical() in benchmark.ts now sorts keys by code unit (RFC 8785), with the Intl.Collator-stub regression test copied from the #890 fix shape. The same defect in research.ts (feeding sha256/embeddingSpaceId digests) is fixed in the same way.

⚠ One-time cache invalidation (called out per the issue): changing the ordering changes the benchmark cache key, so existing .metaharness/cache entries no longer match and benchmarks recompute once. This is the accepted cost of a locale-independent key and the reason this change ships in its own commit path rather than riding a feature PR.

ADR-341 — Correctness-Hardening Invariants for Hot-Path Primitives

Records the six invariants these fixes instantiate so the defect classes stop recurring: total float orderings on untrusted values, no lock re-entry (read through the held guard), the Weak+Drop pool shape as the only sanctioned pool pattern, identity-not-just-mode filesystem trust checks, code-unit canonical encodings for anything hashed, and seeded RNGs in tests of nondeterministically-triggered behavior. node scripts/adr-index.mjs --check passes post-merge (372 ADR files, next available 342).

Follow-up issues filed from this pass

CI status (see PR comments for detail)

Green everywhere the diff reaches, including core-platform with delta-index restored. Two pre-existing, tracked reds inherited from main, both commented on this PR: Tests (core-and-rest) caps out at 240 min compiling (#928, maintainer CI-policy decision), and Build Tiny Dancer linux-arm64-musl (#900, toolchain drift — fix ready in #934).

Validation

  • cargo test -p ruvector-tiny-dancer-core — 49 passed
  • cargo test -p ruvector-context — 44 passed
  • cargo test -p ruvector-delta-index — 15 passed (pre-fix: hung; suite now <1s)
  • cargo test -p ruvector-graph — 264 passed post-merge (both sides' changes together)
  • cargo clippy over the four touched crates — clean; cargo fmt --all --check — clean
  • harness: npm test — 167 pass / 0 fail
  • node scripts/adr-index.mjs --check — OK

🤖 Generated with claude-flow

https://claude.ai/code/session_01L5ffi8NiKAQNabK3D5t2gK

claude and others added 2 commits August 26, 2026 02:45
…nts (ADR-340)

Fixes five open defects as one hardening pass, and records the underlying
invariants in ADR-340 so the defect classes stop recurring:

- #901 ruvector-tiny-dancer-core: route() now rejects non-finite model
  output at a single choke point on BOTH the gated and ungated paths
  (tripping the circuit breaker, matching the VoI path's contract), and the
  decision sort uses total_cmp as defense-in-depth. Regression test poisons
  the model weights with NaN and asserts an error, not a panic.

- #825 ruvector-delta-index: DeltaHnsw::connect_node's prune branch held a
  write lock on a neighbor while re-locking the same node through
  self.distance() — parking_lot locks are not reentrant, so inserts
  deadlocked whenever a neighbor list overflowed m0 (nondeterministic under
  the unseeded test RNG; burned a 3h52m CI job). The neighbor's vector now
  comes from the held guard via a split borrow, prune_neighbors drops
  self-loops before measuring distances, greedy_search carries an explicit
  progress bound, and the tests are seeded. A prune-heavy regression test
  (400 inserts, dim 8) completes in <1s. ruvector-delta-index rejoins the
  core-platform CI shard.

- #907 ruvector-graph: the redb pool now holds Weak per-path slots with
  Drop-under-guard eviction and slot reaping, ported from ruvector-core's
  post-#902 reference shape. Regression tests: erased-path recreation never
  reuses the unlinked database; reopening after last drop preserves data
  and reaps the pool entry; 40-iteration barrier-synchronized concurrent
  drop-vs-open probe with zero tolerated failures.

- #908 ruvector-context: require_private_root now requires the root to be
  owned by the process euid (rustix::process::geteuid — no unsafe, no new
  transitive dependency) in addition to mode 0700. Rejects only roots that
  already failed later with bare EACCES.

- #903 harness: canonical() in benchmark.ts and research.ts sorts keys by
  code unit (RFC 8785) instead of localeCompare, making cache keys and
  digests locale-independent. NOTE: this changes the benchmark cache key,
  invalidating existing .metaharness/cache entries once (documented in the
  issue as the accepted cost). Regression test stubs Intl.Collator with a
  reversed comparator and asserts the encoding is unchanged.

ADR-340 records the six invariants (total float orderings, no lock
re-entry, Weak+Drop pool shape, identity-not-just-mode trust checks,
code-unit canonical encodings, seeded nondeterministic tests).

Closes #825, #901, #903, #907, #908.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01L5ffi8NiKAQNabK3D5t2gK

ruvnet commented Aug 26, 2026

Copy link
Copy Markdown
Owner Author

CI note: Build Tiny Dancer linux-arm64-musl is red here, and the failure is not this PR's.

It is the known toolchain drift from #900: the link of third-party redb fails with unsupported linker arg: --fix-cortex-a53-843419 before any first-party code is reached. This PR surfaces it only because it touches crates/ruvector-tiny-dancer-core/, which re-triggers the path-filtered workflow that hadn't run since its last green pass on 2026-08-03.

Root cause is now pinned down empirically (details in the fix PR): stable 1.98.0 (2026-08-18) began emitting the Cortex-A53 erratum flag for aarch64-unknown-linux-musl, and no zig release accepts it — verified against zig 0.13.0, 0.14.1, 0.15.2, and 0.16.0. The failure reproduces with a minimal cdylib link on 1.98.0 + zig 0.13.0 and disappears on 1.97.0 with the same zig.

The fix is #934 (pin toolchain: "1.97.0" beside the existing zig 0.13.0 pin), kept as the separate CI PR that #900 explicitly asked for rather than folded into this diff. Once #934 merges, this check goes green here on re-run; every other failure signal on this PR (the initial Rustfmt red) is already fixed on the current head.


Generated by Claude Code

ruvnet commented Aug 26, 2026

Copy link
Copy Markdown
Owner Author

CI note: Tests (core-and-rest) was cancelled at the 240-minute cap, and this failure is not this PR's.

It is exactly the pre-existing #928 compile-size problem: the job ran 02:53→06:53 UTC and was killed at the cap during compilation, the same outcome #928 documents on PR #926's run 32702332738 (cancelled mid-compile of ruvector-temporal-tensor-wasm, 16/17 other jobs green) — filed the day before this PR existed. This PR's diff adds only test code and small fixes to crates in that shard, nothing that moves a four-hour compile wall.

Not re-running: the cap-out already reproduced on an independent PR (#926, per #928), so a re-run would spend another four runner-hours to confirm what is already confirmed. Not fixing here: #928 explicitly reserves the shard split/caching change for maintainer review as a CI-policy decision, so porting a workflow change into this PR would preempt that.

Signal coverage note: everything this PR actually changes is tested by green shards — core-platform (which now includes ruvector-delta-index again, passing in 7 minutes), core-and-rest-heavy, core-and-rest-wasm, both Clippy jobs, Rustfmt, and the harness job all pass on this head. The two remaining reds are both pre-existing and tracked: this one (#928) and Tiny Dancer arm64-musl (#900, fix in #934).


Generated by Claude Code

Main allocated ADR-340 to the signed retrieval-receipt anchoring ADR
(PR #949) while this branch held the number for the correctness-hardening
invariants ADR. Main merged first, so its allocation stands: this branch's
ADR renumbers to ADR-341, INDEX.md keeps main's ADR-340 row and gains the
ADR-341 row (next available: 342; the numbering guard passes).

Also corrects two comments main's graph-node Cypher fix added citing
'ADR-340 invariant 1' for the NaN-comparison rule — that content lives in
the hardening ADR, which is now ADR-341.

No semantic conflicts: main's ruvector-graph changes touch
cypher/parser.rs and graph.rs only, not the storage pool this branch
reworked; full ruvector-graph suite passes on the merge (264 tests).

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01L5ffi8NiKAQNabK3D5t2gK
@ruvnet ruvnet changed the title Cleanup/hardening: fix five open correctness/security defects, adopt ADR-340 invariants Cleanup/hardening: fix five open correctness/security defects, adopt ADR-341 invariants Sep 2, 2026
@ruvnet
ruvnet marked this pull request as ready for review September 5, 2026 14:01
@ruvnet
ruvnet merged commit 0974938 into main Sep 5, 2026
77 of 79 checks passed
@ruvnet
ruvnet deleted the claude/ruvector-cleanup-hardening-m2ax54 branch September 5, 2026 14:01
ruvnet added a commit that referenced this pull request Sep 5, 2026
ruvnet added a commit that referenced this pull request Sep 5, 2026
#959)

* feat(retrieval-receipt): add periodic index_state_root anchoring

Implements ADR-341: independent, query-decoupled signing of
index_state_root via a new AnchorPurpose::StateAnchor, reusing ADR-340's
Ed25519 signing machinery unchanged. Lets an auditor authenticate a
checkpoint of the index's state in O(1) without holding any specific
query receipt or replaying the full write history.

StateAnchorPolicy/StateAnchorLog/verify_state_anchor implement a
write-count-based anchoring interval with an exact, disclosed staleness
bound (interval_writes - 1), operating directly over WriteGate roots
rather than any RetrievalIndex.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01KV63T53qpYkXYkAKXvfPXT

* bench(retrieval-receipt): add state-anchor interval sweep and replay-cost table

Sweeps interval_writes in {1, 8, 32, 128, 512} over 5,000 writes, measuring
anchor count vs. the closed-form N/W prediction, exact staleness bound,
O(1) anchor-verify cost flatness, tamper detection, and amortized signing
cost. Adds a separate, explicitly non-gated verify_integrity O(n) scaling
table so the O(1) anchor-verify numbers are never read as a substitute for
full write-history integrity checking.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01KV63T53qpYkXYkAKXvfPXT

* docs: add ADR-341 for periodic index_state_root anchoring

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01KV63T53qpYkXYkAKXvfPXT

* docs: add 2026-09-03 nightly research report and gist

Research README, standalone gist, and raw 3-run benchmark output for the
periodic index_state_root anchoring experiment (ADR-341), continuing the
2026-08-31 nightly run's named Next Research item.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01KV63T53qpYkXYkAKXvfPXT

* docs(adr): renumber ADR-341 → ADR-342 (collision with #933) and rebase

Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_019xHM4rAH4aaShb4DTr1n6s

---------

Co-authored-by: Claude <noreply@anthropic.com>
ruvnet added a commit that referenced this pull request Sep 5, 2026
ruvnet added a commit that referenced this pull request Sep 5, 2026
… scheduling (ADR-343) (#952)

* feat(retrieval-receipt): add bounded batch-fill scheduling (ADR-341)

Adds BatchFillPolicy/BatchScheduler for deciding when a signed-receipt
batch closes (fixed-size-only or size-or-timeout hybrid), with unit
tests, extending ADR-340's signed anchoring without modifying it.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01DYBHRF7WVzMyTRpe9Fo4qd

* docs(adr): add ADR-341 for signed-receipt batch-fill latency

Records the design decision, threat model, evidence summary, and
rejection criteria for the bounded batch-fill scheduling policy added
in the prior commit. Regenerates the ADR index via scripts/adr-index.mjs.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01DYBHRF7WVzMyTRpe9Fo4qd

* docs(research): nightly report on signed-receipt batch-fill latency

Real, reproduced-3x discrete-event simulation evidence turning ADR-340's
CPU-only signing-amortization result into an end-to-end receipt-
availability latency claim: fixed-size-only batching's p99 latency is
unbounded under light load (756ms measured) while a bounded size-or-
timeout hybrid policy stays within its configured bound (50ms) at every
tested regime, at negligible amortization cost under sufficient load.
Includes raw per-run output and a standalone gist write-up.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01DYBHRF7WVzMyTRpe9Fo4qd

* docs(adr): renumber ADR-341 → ADR-343 (collision with #933) and rebase

Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_019xHM4rAH4aaShb4DTr1n6s

---------

Co-authored-by: Claude <noreply@anthropic.com>
ruvnet added a commit that referenced this pull request Sep 5, 2026
ruvnet added a commit that referenced this pull request Sep 5, 2026
…DR-344) (#955)

* feat(memory-admission): add global-min-cut gated streaming memory admission

Implements MincutGatedAdmission and AdaptiveMincutAdmission alongside a
NearestCentroidThreshold baseline behind a shared AdmissionPolicy trait.
Uses a self-contained Stoer-Wagner global min-cut (terminal-free, unlike
ruvector-namespace-merge's S-T max-flow) to gate write-time cluster
admission for streaming agent memory.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01MUQvcKaga5t9Q8XPZxByLv

* test(memory-admission): add integration coverage for admission policies

Covers no-lost-vectors across all three policies, bounded cluster count
under both mincut policies, decide() read-only invariance, and centroid
unit-norm preservation across merges.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01MUQvcKaga5t9Q8XPZxByLv

* bench(memory-admission): add matched-cluster-budget benchmark

Compares all three policies at the same final cluster count: binary-
searches the baseline threshold to match candidate A's natural cluster
count under a fixed tau, so purity/recall comparisons aren't skewed by
independently hand-picked operating points (an earlier unmatched run
showed purity alone is gameable by over-fragmentation).

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01MUQvcKaga5t9Q8XPZxByLv

* docs(memory-admission): add nightly research report and public gist

Documents the matched-budget hypothesis, methodology, and measured
result (candidate A ACCEPT: +4.50pp purity, +7.83pp recall@10 at matched
17-cluster budget; candidate B self-calibration REJECT: drifts to the
safety-valve cap and loses 12.3pp recall). Preserves the original
uncalibrated run and threshold/tau sweeps as raw evidence.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01MUQvcKaga5t9Q8XPZxByLv

* docs(adr): add ADR-341 for mincut-gated streaming memory admission

Regenerated docs/adr/INDEX.md via node scripts/adr-index.mjs to register
ADR-341 and advance the allocation counter to 342.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01MUQvcKaga5t9Q8XPZxByLv

* docs(adr): renumber ADR-341 → ADR-344 (collision with #933) and rebase

Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_019xHM4rAH4aaShb4DTr1n6s

---------

Co-authored-by: Claude <noreply@anthropic.com>
ruvnet added a commit that referenced this pull request Sep 5, 2026
Also fixes the nightly README's ADR link, which pointed at ../../adr/
(docs/research/adr/, a non-existent path) instead of ../../../adr/.
The RNG seed 341 in examples/mincut_gated_forgetting_bench.rs is a
seed, not an ADR reference, and is left unchanged.

Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_019xHM4rAH4aaShb4DTr1n6s
ruvnet added a commit that referenced this pull request Sep 5, 2026
…ence (ADR-345) (#961)

* feat(agent-memory): add mincut-gated forgetting and eviction witnesses

Add graph_forget::MincutGatedForgetting (feature-gated: mincut-forget), a
CompactionPolicy that layers a ruvector-mincut boundary signal on top of
the existing CoherencePolicy scalar score, to test whether structural
"bridge" memories can be protected from eviction. Factor CoherencePolicy's
scoring into weighted_importance() for reuse.

Add witnessed_compaction::compact_witnessed + EvictionWitnessChain: an
always-on eviction path that emits a chained ADR-134 witness record per
evicted entry before mutating the store, closing the gap where admission
and retrieval are witnessed but deletion is not. Adds one action_kind
constant (LEDGER_COMPACT_EVICT = 0xA7).

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01TKszxZVDnLu1fCH2BAwFi5

* bench(agent-memory): add mincut-gated forgetting benchmark and probes

- mincut_gated_forgetting_bench: the fixed, falsifiable acceptance test
  (baseline CoherencePolicy vs. MincutGatedForgetting Soft/Hard) with real
  timing, bridge-survival, recall, and witness tamper-detection measurement.
- mincut_scaling_probe: RuVectorGraphAnalyzer::partition() latency vs. graph
  size (50-11400ms across n=50..400).
- mincut_determinism_probe: reproduces partition() returning an
  empty/unusable result in ~50% of repeated calls on an identical graph.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01TKszxZVDnLu1fCH2BAwFi5

* docs: add ADR-341 for mincut-gated forgetting

Records the rejected hypothesis, measured evidence (performance and
non-determinism findings against ruvector-mincut's RuVectorGraphAnalyzer),
and the decision to keep the module feature-gated and unpromoted while
retaining the eviction-witness half as a default-on capability.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01TKszxZVDnLu1fCH2BAwFi5

* docs: add nightly research report and gist for mincut-gated forgetting

Full methodology, raw benchmark output, scaling/determinism evidence,
rejected alternatives, and next-research directions for the 2026-09-05
nightly run (ADR-341).

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01TKszxZVDnLu1fCH2BAwFi5

* docs(adr): renumber ADR-341 → ADR-345 (collision with #933) and rebase

Also fixes the nightly README's ADR link, which pointed at ../../adr/
(docs/research/adr/, a non-existent path) instead of ../../../adr/.
The RNG seed 341 in examples/mincut_gated_forgetting_bench.rs is a
seed, not an ADR reference, and is left unchanged.

Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_019xHM4rAH4aaShb4DTr1n6s

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment