Skip to content

perf(variant): build BinaryView arrays directly - #10640

Open
cakeni wants to merge 1 commit into
apache:mainfrom
cakeni:perf/variant-builder-finalization
Open

perf(variant): build BinaryView arrays directly#10640
cakeni wants to merge 1 commit into
apache:mainfrom
cakeni:perf/variant-builder-finalization

Conversation

@cakeni

@cakeni cakeni commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

Finalizing VariantArrayBuilder replayed every offset through BinaryViewBuilder::try_append_view, repeating bounds and value validation for buffers and offsets produced internally by the Variant builders.

What changes are included in this PR?

  • Add a focused Criterion benchmark that times only build() for 262,144 small values.
  • Construct views directly from the recorded offset slices.
  • Build the final BinaryViewArray from those validated views, with the safety invariants documented at the unchecked constructor.

Are these changes tested?

  • cargo +stable-x86_64-pc-windows-gnu test -p parquet-variant-compute --lib (347 passed)
  • cargo +stable-x86_64-pc-windows-gnu bench -p parquet-variant-compute --bench variant_kernels -- variant_array_builder_build_262k_small_values --noplot
    • Before: [2.9213 ms, 2.9390 ms, 2.9577 ms]
    • After: [2.0393 ms, 2.0538 ms, 2.0689 ms]
    • Criterion change: [-30.816%, -30.118%, -29.428%], p = 0.00
  • cargo fmt --all -- --check
  • git diff --check

Are there any user-facing changes?

No API or behavior changes. VariantArrayBuilder::build is approximately 30% faster in the focused many-small-values benchmark.

@github-actions github-actions Bot added the parquet-variant parquet-variant* crates label Aug 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

parquet-variant parquet-variant* crates

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Variant] Benchmark and optimize BinaryView construction in VariantArrayBuilder

1 participant