Is your feature request related to a problem or challenge?
The eager DataFusion introduced by #2671 scan stores planned tasks in shared Arc<[FileScanTask]> groups. However, ArrowReader::read() only accepts a stream of owned FileScanTask, so every task must be cloned before it can be read.
This clone copies the task's owned data and is repeated for every file in the scan.
Related discussion: #2298 (comment)
Describe the solution you'd like
Allow ArrowReader to read shared FileScanTask without cloning their contents.
The shared input must remain alive for the lifetime of the returned stream. The existing owned-task API should remain available because TableScan::plan_files() can already transfer its tasks directly without cloning.
Willingness to contribute
I would be willing to contribute to this feature with guidance from the Iceberg Rust community
Is your feature request related to a problem or challenge?
The eager DataFusion introduced by #2671 scan stores planned tasks in shared
Arc<[FileScanTask]>groups. However,ArrowReader::read()only accepts a stream of ownedFileScanTask, so every task must be cloned before it can be read.This clone copies the task's owned data and is repeated for every file in the scan.
Related discussion: #2298 (comment)
Describe the solution you'd like
Allow
ArrowReaderto read sharedFileScanTaskwithout cloning their contents.The shared input must remain alive for the lifetime of the returned stream. The existing owned-task API should remain available because
TableScan::plan_files()can already transfer its tasks directly without cloning.Willingness to contribute
I would be willing to contribute to this feature with guidance from the Iceberg Rust community