[fix](variant) Keep JSON bool distinct from 0/1 when a numeric value widens the path

[eldenmoon]

What problem does this PR solve?

Issue Number: None

Related PR: #67675

Problem Summary:
Within a single Variant V2 segment, when a path's first value is a JSON bool and a plain number
(TINYINT/SMALLINT/INT/BIGINT/LARGEINT/FLOAT/DOUBLE) arrives afterwards, the path silently turned the
stored true/false into the integer 1/0. Canonical Variant/JSON semantics require true to
never compare or group equal to 1, so this changed GROUP BY / equality / display results depending
on row order.

Root cause: path_least_common_type() (be/src/storage/segment/variant/v2/variant_path_builder.cpp)
delegates ordinary scalar promotion to get_least_supertype_jsonb() /
get_numeric_type() (be/src/core/data_type/get_least_supertype.cpp:62-63), which counts
TYPE_BOOLEAN as an 8-bit unsigned integer so it can share width promotion with the real integer
types. That rule is correct for get_numeric_type()'s other callers (data_type_array_serde.cpp,
nested_group_streaming_write_plan.cpp, variant_util.cpp), but for a Variant path it merges BOOL and
a number into one numeric column instead of falling back to JSONB. promote() then casts the
already-written true value to that numeric type, turning it into 1 in storage. The opposite
order (a number first, then a bool) already fell back to JSONB and kept the bool correct, because
append_integer() throws when a BOOL value hits an already-numeric typed-path column and the
catch block in VariantPathBuilder::append() promotes to JSONB; only bool-first was affected.

Reproduction (before the fix):

• BE UT (new): VariantPathBuilderTest.BoolFirstThenNumericFallsBackToJsonbPreservingBooleanValue
  failed with the path staying a numeric type and to_string() returning "1" instead of "true".
  BoolIntDoubleFalseSequenceKeepsBooleansAndNumbersDistinct and
  BoolAndIntArraysFallBackToJsonbElementPreservingBooleanValue (ARRAY vs ARRAY) failed
  the same way.
• SQL: a 1-bucket DUPLICATE table with one INSERT putting {"k": true} before {"k": 1} read back
  var['k'] as 1 for the true row, and GROUP BY var['k'] collapsed the two logically distinct
  rows into one group of size 2 instead of two groups of size 1.

Fix: in path_least_common_type(), when exactly one side is TYPE_BOOLEAN and the other is one of
TINYINT/SMALLINT/INT/BIGINT/LARGEINT/FLOAT/DOUBLE, return the JSONB type directly instead of
delegating to the shared numeric-tower helper (mirroring the existing DECIMAL/ARRAY special-casing
in the same function). The shared get_least_supertype.cpp numeric rule is left untouched since it
has other callers that legitimately want BOOLEAN folded into the numeric tower; only Variant path
type inference is changed. Arrays are covered automatically because ARRAY-vs-ARRAY merges already
recurse into this function for the element type.

Compaction: separate loads still lost the booleans when compaction merged segment types. With one
INSERT storing {"k": true} and another storing {"k": 1}, each segment stores its own physical
type (BOOLEAN and BIGINT). Full compaction merged them with the generic
get_least_supertype_jsonb(), so the compacted subcolumn became BIGINT and the stored booleans were
cast: before compaction v['k'] returned true, false, 1, 0 (4 GROUP BY groups), after compaction
1, 0, 1, 0 (2 groups), and DESC with describe_extend_variant_column showed v.k bigint. Plain,
array (array<bigint>) and object paths were affected, for a limited variant_max_subcolumns_count
(get_compaction_subcolumns_from_subpaths()) and for 0 (get_compaction_subcolumns_from_data_types());
get_compaction_nested_columns() and update_least_schema_internal() merge the same way.

The BOOLEAN/number rule now lives in variant_util (is_variant_boolean_numeric_mix() and
get_least_common_variant_path_type()) and is used wherever Variant path types are merged: the path
builder and the four compaction schema-merge helpers above. A BOOLEAN mixed with a non-boolean number
at the same array depth becomes JSONB; every other combination keeps the existing promotion.

After the fix: all reproduction cases above pass; true/false read back unchanged within a segment
and after compaction, and GROUP BY produces the correct distinct groups.

Release note

Fixed a Variant V2 storage bug where a JSON boolean value could be stored as the integer 1/0 when the
same path held numbers, either later in the same segment or in other rowsets merged by compaction.

Check List (For Author)

• Test:
  • Unit Test: added VariantPathBuilderTest.BoolFirstThenNumericFallsBackToJsonbPreservingBooleanValue,
    BoolIntDoubleFalseSequenceKeepsBooleansAndNumbersDistinct, and
    BoolAndIntArraysFallBackToJsonbElementPreservingBooleanValue in
    be/test/storage/variant/variant_column_writer_reader_test.cpp. Ran the full test file
    (VariantPathBuilderTest, VariantShredderTest, VariantColumnWriterReaderTest,
    VariantWriterCompatibilityTest, VariantSpecializedWriterCompatibilityTest): before the fix the
    3 new cases failed as described above and all other cases passed; after the fix all 91 run
    cases passed (2 pre-existing skips: "NestedGroup write path is not available in this build",
    unrelated to this change).
  • Unit Test (compaction): added SchemaUtilTest.UpdateLeastSchemaKeepsBooleanDistinctFromNumbers and
    SchemaUtilTest.CompactionSubcolumnsKeepBooleanDistinctFromNumbers in
    be/test/exec/common/schema_util_test.cpp. Before the fix both failed (the merged columns were
    BIGINT/DOUBLE instead of JSONB). After the fix, SchemaUtilTest, VariantPathBuilderTest,
    VariantShredderTest, VariantColumnWriterReaderTest and the Variant writer compatibility tests:
    126 passed, 2 skipped (NestedGroup write path not available in this build, unrelated).
  • Regression test: added regression-test/suites/variant_p0/test_variant_bool_numeric_widening.groovy,
    generated its .out with -genOut and verified every row by reasoning. Also ran the full
    variant_p0 directory (173 suites) on the first fixed build: 9 unrelated pre-existing failures (1
    missing S3/OSS credential for an outfile export test; 8 "debug_point/remove ... HTTP 500"
    failures confirmed via direct API probe to be caused by config::enable_debug_points being
    disabled in this cluster, affecting only inverted-index debug-point suites), none touching
    Variant type inference; the other 164 suites passed.
  • Regression test (compaction): the suite now loads booleans and numbers in separate rowsets for
    variant_max_subcolumns_count 10 and 0 and checks plain, array and object paths plus GROUP BY
    before and after full compaction. Before the fix the compacted values read back as 1/0 with 2
    GROUP BY groups; after the fix the suite passes with its .out regenerated by the test.
• Behavior changed: Yes - a Variant path that holds both JSON booleans and plain numbers, within one
  segment or across compacted segments, is stored as JSONB instead of a numeric type, so
  true/false are preserved instead of becoming 1/0.
• Does this need documentation: No

🤖 Generated with Claude Code

[eldenmoon]

run buildall

[hello-stephen]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?

[eldenmoon]

/review

[github-actions]

Requesting changes for one data-correctness gap and one mandatory regression-test convention issue.

Checkpoint conclusions:

• Goal and proof: the new builder-local rule statically fixes BOOL-plus-number kind preservation when both values meet in one segment, and the direct unit/regression oracles cover that case. The goal is not complete end to end because routine Variant subcolumn compaction can still merge separately typed BOOL/BIGINT segments to BIGINT and persist 0/1.
• Scope, parallel paths, and condition: the new primitive check is small, symmetric, constant-time, and clearly commented. The production local/cloud compaction schema-union helpers were not given the same Variant-specific rule; this is the P1 inline finding.
• Tests and results: the scalar/ARRAY unit expectations and generated SQL output are internally consistent and deterministic, and the negative comparison cases use the required form. Coverage needs separate segments plus pre/post forced-compaction display and GROUP BY. The single fixed-table suite also violates the hardcoded-table-name rule (P3). The review contract prohibited builds/tests, so these conclusions are static and author/CI test claims were not independently executed.
• Concurrency and lifecycle: the helper and builder state are synchronous and segment-writer-local; no shared mutable state, locks, atomics, ownership edge, callback, or cross-TU static-initialization dependency is introduced.
• Configuration: no setting is added. Existing Variant subcolumn vertical compaction is enabled by default, making the P1 path production-reachable.
• Compatibility and persistence: no function symbol, wire protocol, or storage-format change is introduced. The P1 operates on valid existing BOOL/BIGINT physical leaves and persistently rewrites the compacted output, including mixed old/new segment populations.
• Transactions, writes, and FE/BE variables: transaction/version/delete-bitmap semantics and FE-BE variable passing are unchanged; the outstanding issue is deterministic data rewriting during compaction.
• Observability: existing schema/cast VLOGs are sufficient; no new metric or INFO log is needed.
• Performance and other correctness: the constant-size switch has negligible cost. No additional memory, nullable/const-shape, ignored-Status, segment-ID, crash-safety, or security issue was found.

User focus: no additional user-provided review focus was supplied.

Review completion: complete static review of all four changed files and relevant upstream/downstream paths. Two bounded rounds converged with no new valuable findings; every candidate was accepted, dismissed with evidence, or deduplicated. No existing or live inline review duplicated these comments. Reviewed exact head 2eeffcc against base 28577df.

[github-actions]

[P1] Apply this rule when compaction merges Variant segment schemas

This special case only runs when both values meet in the same VariantPathBuilder. With separate inserts, one segment can publish k as BOOLEAN and another as BIGINT. Default-enabled Variant subcolumn compaction gathers both physical types, and get_compaction_subcolumns_from_data_types() still resolves them with the generic get_least_supertype_jsonb() to BIGINT; the merger's SegmentIterator then casts the BOOL leaf to the target BIGINT, rewriting true/false as 1/0 in the compacted output. Please share the Variant-specific BOOL/number rule with the production compaction schema-union helpers (including nested and nested-group regular subcolumns) and add a separate-segment regression that checks display/GROUP BY before and after forced compaction. The single-INSERT test never crosses this boundary.

[github-actions]

[P3] Hardcode this single test table name

The repository test contract requires ordinary single-table regression cases to hardcode the table name instead of introducing a def tableName-style variable. This suite has only one fixed table, so please remove table_name and use test_variant_bool_numeric_widening directly in the DROP, CREATE, INSERT, and query statements.

[hello-stephen]

BE UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	63.03% (29321/46522)
Line Coverage	47.96% (305893/637810)
Region Coverage	43.62% (246977/566244)
Branch Coverage	45.14% (114721/254146)

[eldenmoon]

run buildall

[hello-stephen]

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	76.20% (34326/45050)
Line Coverage	61.09% (385202/630554)
Region Coverage	57.46% (324108/564010)
Branch Coverage	58.21% (147487/253375)

[hello-stephen]

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	75.87% (34186/45057)
Line Coverage	60.71% (382844/630632)
Region Coverage	57.10% (322063/564047)
Branch Coverage	57.89% (146684/253403)