feat(core): define mapping readers with ordered compaction support - #9106
Open
LuQQiu wants to merge 6 commits into
Open
feat(core): define mapping readers with ordered compaction support#9106LuQQiu wants to merge 6 commits into
LuQQiu wants to merge 6 commits into
Conversation
This was referenced Sep 9, 2026
LuQQiu
added this pull request to stack #9108
September 9, 2026 19:16
LuQQiu
removed this pull request from stack #9108
September 10, 2026 03:15
LuQQiu
added this pull request to stack #9117
September 10, 2026 03:15
LuQQiu
removed this pull request from stack #9117
September 10, 2026 18:17
LuQQiu
added this pull request to stack #9137
September 10, 2026 18:17
LuQQiu
marked this pull request as ready for review
September 10, 2026 18:21
Contributor
There was a problem hiding this comment.
✅ Gate recommendation: approve.
The mapping seam is appropriately scoped: ordered compaction keeps the existing bitmap/rank behavior, while the async contract lets external row-map implementations override batch reads for efficient I/O. The tests cover deletion, ordering, duplicate, invalid-address, and single/batch-equivalence semantics.
The current stack order is appropriate: land #9136 before this PR.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack reordered: the standalone format proposal is now #9136 (replacing #9065), followed by #9106 → #9064 → #9067 → #9068 → #9107 in native stack #9137. This PR’s diff is byte-identical to its pre-reorder diff. All branches include main
31d78d170; no history was rewritten. Formatting checks pass; Clippy and tests remain blocked by the unavailableobject_store_opendal 0.60.1dependency.Define a
MappingReadercontract for single and batch physical-row-ID remapping (u64toOption<u64>). Single-row remapping is required; batch remapping defaults to calling it in order, and implementations can override the batch path. The ordered-compaction adapter wraps the existingRowAddrRemap; it does not duplicate source/destination metadata. Coverage belongs to the common lineage reader. The legacy FRI reader and writer are unchanged.This is the base of the six-PR FRI reader stack. Stable partitions implement the same contract in #9064; graph decoding remains in #9067, and graph composition/planner integration in #9068. Scalar and vector consumers follow together in #9107.
Validation: mapping contract tests cover default-batch delegation/error propagation, deleted rows, duplicates, out-of-order and empty batches, single/batch equivalence, invalid fragments, and offsets outside the source. Workspace Clippy (
--all --tests --benches -- -D warnings) andcargo fmt --allpassed.No format documentation changes. Draft pending stack review.
Latest main synchronization (2026-09-10): merged
d7b031f9cthrough the stack without rewriting history.cargo fmt --alland whitespace checks pass. Workspace Clippy could not run: main now requiresobject_store_opendal 0.60.1, but the crates.io index currently resolves only up to 0.60.0. Tests were not rerun after this synchronization because dependency resolution is blocked. Earlier validation above predates this merge.