HIVE-30035: Upgrade to Parquet 1.18.0

[abstractdog]

What changes were proposed in this pull request?

Bump parquet.version from 1.16.0 → 1.18.0 in the root pom.xml, and pin parquet-avro to the same version in iceberg/pom.xml's <dependencyManagement> via a new iceberg.parquet.version property (defaulting to ${parquet.version}).

Also update TestHiveSchemaConverter:

• Drop (MAP_KEY_VALUE) from the 5 expected schema strings.
• Expect a null annotation instead of MapKeyValueTypeAnnotation.getInstance() in testMapOriginalType.

Why are the changes needed?

iceberg-data brings parquet-avro transitively, and iceberg-shading bundles org.apache.parquet:* into the shaded Iceberg jar without relocating it:

<!-- iceberg/iceberg-shading/pom.xml -->
<relocations>
  <relocation><pattern>org.apache.avro</pattern>...</relocation>
  <relocation><pattern>org.apache.orc</pattern>...</relocation>
  <relocation><pattern>shaded.parquet</pattern>...</relocation>   <!-- parquet-hadoop internals only -->
  <relocation><pattern>com.google</pattern>...</relocation>
  <relocation><pattern>com.fasterxml</pattern>...</relocation>
  <!-- no entry for org.apache.parquet -->
</relocations>
<artifactSet>
  <includes>
    ...
    <include>org.apache.parquet:*</include>
    ...

Confirmed on the built shaded jar:

Package	Unrelocated	Under org/apache/hive/iceberg/… prefix
org/apache/parquet/…	2018	0
org/apache/avro/… (control)	0	481

Left alone, iceberg-data's transitive Parquet 1.17.1 classes would sit at org.apache.parquet.* next to Hive's 1.18.0 classes at the same coordinates. Pinning parquet-avro (the root of the subtree the bundle embeds) in the Iceberg pom forces the two onto one version. Managing it at the root would spread Parquet artifact overrides across the whole build for what is really an Iceberg-bundle concern.

The TestHiveSchemaConverter changes follow from a Parquet 1.18 behavior change: ConversionPatterns.mapType no longer stamps the deprecated MAP_KEY_VALUE annotation on a MAP's inner key_value group — the Parquet MAP standard treats it as redundant.

Does this PR introduce any user-facing change?

• Newer Parquet library version on the classpath.
• Parquet files Hive writes for a MAP<K,V> column no longer stamp the deprecated MAP_KEY_VALUE annotation on the inner key_value group — following Parquet 1.18's ConversionPatterns.mapType, which the Parquet MAP standard treats as redundant. Old files remain readable; standards-compliant readers rely on the outer MAP annotation and are unaffected. Tools that inspect the raw schema (e.g. parquet-tools) will show repeated group key_value { … } instead of repeated group key_value (MAP_KEY_VALUE) { … }.

How was this patch tested?

mvn -pl ql test -Dtest=TestHiveSchemaConverter:

Tests run: 30, Failures: 0, Errors: 0, Skipped: 0

[deniskuzZ]

@abstractdog, we need this in iceberg pom.xml

    <iceberg.avro.version>${avro.version}</iceberg.avro.version>
    <iceberg.parquet.version>${parquet.version}</iceberg.parquet.version>
.....
      <!-- iceberg-data pulls parquet-avro, the root of the Parquet subtree the bundle embeds. -->
      <dependency>
        <groupId>org.apache.parquet</groupId>
        <artifactId>parquet-avro</artifactId>
        <version>${iceberg.parquet.version}</version>
      </dependency>

or drop the iceberg.avro.version & iceberg.parquet.version and use root pom vars

[abstractdog]

@deniskuzZ : this is now ready for review

[deniskuzZ]

+1

[deniskuzZ]

restarted CI

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud