Repository navigation
Antalya 26.8: Added support for iceberg v3 unknown data type - #2537
Conversation
…next commit) --- Original cherry-pick message follows: Merge pull request #2363 from Altinity/iceberg_unknown_data_type Antalya 26.6: Added support for iceberg v3 unknown data type # Conflicts: # src/Storages/ObjectStorage/DataLakes/Iceberg/IcebergWrites.cpp # src/Storages/ObjectStorage/DataLakes/Iceberg/MultipleFileWriter.cpp # src/Storages/ObjectStorage/DataLakes/Iceberg/tests/gtest_iceberg_metadata_generator.cpp # src/Storages/ObjectStorage/StorageObjectStorageSource.cpp # tests/integration/test_storage_iceberg_with_spark/test_writes_field_ids_spark_read.py
Kept antalya-26.8 versions of extendSchemaForPartitions and test_writes_field_ids_spark_read.py, which already contain the source PR's content. Dropped bundled-in tests from other PRs in gtest_iceberg_metadata_generator.cpp. Dropped the setStorageColumnEncoding context line in MultipleFileWriter: on antalya-26.8 the column mapper is already initialized via createColumnMapperFromFields. Adapted the tuple-subcolumn extraction in StorageObjectStorageSource to iterate requested_columns_copy, since antalya-26.8 has no row-lineage columns_to_extract list.
…ya-26.8/pr-2363 # Conflicts: # src/Storages/ObjectStorage/DataLakes/Iceberg/tests/gtest_iceberg_metadata_generator.cpp
CI triageVerdict: 0 of the red checks are caused by this PR. Everything failing is infra, a missing CI baseline, or known-environmental swarm flakiness. This PR is a self-contained Iceberg data-lake read-path change, and none of the failing areas (container CVE scans, parser-memory baseline, swarm cluster/resilience tests) are reachable from the code it touches. Head SHA checked:
Why the diff can't be responsibleThe change only affects the data-lake / object-storage read path:
None of this touches cluster discovery, node-failure resilience, the SQL parser, or Docker image contents. Per-failure detailGrype Scan (keeper + server-alpine) — these scan the built container images for OS/dependency CVEs (keeper: 4 high/critical; server-alpine: 1). They flag packages in the base image, not ClickHouse source, and fail on essentially every PR on this branch. Not actionable from this PR; needs a base-image/dependency bump at the branch level (separate from this feature). Parser memory check — reported Regression swarms (aarch64 + release) — both arches report
All three fail on Bottom lineCI is functionally green for this PR — the real test gates (builds, stateless, integration, stress, AST fuzzer, BuzzHouse, compat/install) are all OK. The remaining red is infra/flaky noise that exists independently of this change. No fix is needed on the PR itself; a re-run of the parser-memory and swarms jobs should clear those, and the Grype/CVE findings belong to the branch's base image rather than this feature. 🤖 automated triage by @blau-ai · evidence: praktika |
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Support for iceberg v3
unknowndatatype which is maps toNullable(Nothing)in Clickhouse. Read and write path(Parquet) (#2363 by @subkanthi).CI/CD Options
Exclude tests:
Regression jobs to run:
Cherry-picked from #2363.