Skip to content

[SS-366] update iceberg-rust dep to v0.10.1 - #38084

Open
ublubu wants to merge 1 commit into
MaterializeInc:mainfrom
ublubu:iceberg-rust
Open

[SS-366] update iceberg-rust dep to v0.10.1#38084
ublubu wants to merge 1 commit into
MaterializeInc:mainfrom
ublubu:iceberg-rust

Conversation

@ublubu

@ublubu ublubu commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

I've rebased our iceberg-rust fork's changes onto upstream's v0.10.1:
https://github.com/MaterializeInc/iceberg-rust/compare/04ae06bdb15a6fd7c7927d29d4e0f6a33de0f1f9..a51ff2c8dce700a1c158c372f8ce94906b5631b1

iceberg-rust$ make test passes.

(For comparison: our changes prior to the rebase)

cc @DAlperin: The rebase included this change to row_delta.rs::RowDeltaOperation::existing_manifest.

        // Include all existing manifests with added or existing files, plus delete-only
        // manifests — dropping those would let removed files reappear as live data (#2148).
        Ok(manifest_list
            .entries()
            .iter()
            .filter(|entry| {
                entry.has_added_files() || entry.has_existing_files() || entry.has_deleted_files()
            })

previously:

        // Include all existing manifests with added or existing files
        Ok(manifest_list
            .entries()
            .iter()
            .filter(|entry| entry.has_added_files() || entry.has_existing_files())

This PR upgrades arrow and parquet from v57 to v58.

reqsign also has a small API change, which means some changes to the AwsSdkCredentialLoader used for Iceberg FileIO.

@ublubu
ublubu marked this pull request as ready for review August 6, 2026 17:14
@ublubu
ublubu requested review from a team as code owners August 6, 2026 17:14
@ublubu
ublubu requested a review from patrickwwbutler August 6, 2026 17:22
Comment thread Cargo.toml
iceberg-catalog-rest = { git = "https://github.com/MaterializeInc/iceberg-rust.git", rev = "dedd9231ee88ee979b648e14792878b40e74c20a" }
iceberg-storage-opendal = { git = "https://github.com/MaterializeInc/iceberg-rust.git", rev = "dedd9231ee88ee979b648e14792878b40e74c20a" }
# All changes should go to the `mz_v0.10.x` branch.
iceberg = { git = "https://github.com/MaterializeInc/iceberg-rust.git", rev = "a51ff2c8dce700a1c158c372f8ce94906b5631b1" }

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i feel like we should change this to a branch name instead of rev number, I'm pretty sure this syntax works as well and is considerably more readable:

Suggested change
iceberg = { git = "https://github.com/MaterializeInc/iceberg-rust.git", rev = "a51ff2c8dce700a1c158c372f8ce94906b5631b1" }
iceberg = { git = "https://github.com/MaterializeInc/iceberg-rust.git", branch = "mz_v0.10.x" }

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm now realizing that this does slightly change the way we would need to update the branch - ie we'd need to be quite careful about what gets pulled into the mz_v0.10.x branch as that would then affect MZ in prod, but it's probably a good thing to do some facsimile of a release process for our iceberg fork

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants