Skip to content

feat(core): define mapping readers with ordered compaction support - #9106

Open
LuQQiu wants to merge 6 commits into
lu/fri-spec-firstfrom
lu/fri-mapping-reader
Open

feat(core): define mapping readers with ordered compaction support#9106
LuQQiu wants to merge 6 commits into
lu/fri-spec-firstfrom
lu/fri-mapping-reader

Conversation

@LuQQiu

@LuQQiu LuQQiu commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Stack reordered: the standalone format proposal is now #9136 (replacing #9065), followed by #9106#9064#9067#9068#9107 in native stack #9137. This PR’s diff is byte-identical to its pre-reorder diff. All branches include main 31d78d170; no history was rewritten. Formatting checks pass; Clippy and tests remain blocked by the unavailable object_store_opendal 0.60.1 dependency.

Define a MappingReader contract for single and batch physical-row-ID remapping (u64 to Option<u64>). Single-row remapping is required; batch remapping defaults to calling it in order, and implementations can override the batch path. The ordered-compaction adapter wraps the existing RowAddrRemap; it does not duplicate source/destination metadata. Coverage belongs to the common lineage reader. The legacy FRI reader and writer are unchanged.

This is the base of the six-PR FRI reader stack. Stable partitions implement the same contract in #9064; graph decoding remains in #9067, and graph composition/planner integration in #9068. Scalar and vector consumers follow together in #9107.

Validation: mapping contract tests cover default-batch delegation/error propagation, deleted rows, duplicates, out-of-order and empty batches, single/batch equivalence, invalid fragments, and offsets outside the source. Workspace Clippy (--all --tests --benches -- -D warnings) and cargo fmt --all passed.

No format documentation changes. Draft pending stack review.

Latest main synchronization (2026-09-10): merged d7b031f9c through the stack without rewriting history. cargo fmt --all and whitespace checks pass. Workspace Clippy could not run: main now requires object_store_opendal 0.60.1, but the crates.io index currently resolves only up to 0.60.0. Tests were not rerun after this synchronization because dependency resolution is blocked. Earlier validation above predates this merge.

@LuQQiu
LuQQiu removed this pull request from stack #9108 September 10, 2026 03:15
@LuQQiu
LuQQiu added this pull request to stack #9117 September 10, 2026 03:15

@lance-gatekeeper lance-gatekeeper Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gate recommendation: approve.

The mapping seam is appropriately scoped: ordered compaction keeps the existing bitmap/rank behavior, while the async contract lets external row-map implementations override batch reads for efficient I/O. The tests cover deletion, ordering, duplicate, invalid-address, and single/batch-equivalence semantics.

The current stack order is appropriate: land #9136 before this PR.

@lance-gatekeeper lance-gatekeeper Bot added the K-approved Latest Gatekeeper recommendation permits acceptance. label Sep 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request K-approved Latest Gatekeeper recommendation permits acceptance.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant