Description
Two bookkeeping paths perform quadratic work when a tiering round contains many splits:
TieringSourceEnumerator.assignSplits() repeatedly calls remove(0) on an ArrayList, shifting the remaining entries on every assignment.TieringCommitOperator.collectTableAllBucketWriteResult() scans all results collected so far and rebuilds a set and a list for every new result. Collecting N results therefore takes O(N²) work.
Use a deque for pending splits and an incremental per-table result accumulator with an expected count and bucket deduplication. Preserve assignment order, failover cleanup, duplicate-bucket rejection, and inconsistent-result-count validation. Commit only after all expected results arrive.
Verify equivalent behavior for normal completion, duplicate results, inconsistent counts, and failure cleanup. Queue removal and result accumulation should each take O(N) total work per round. Split prefetching and changes to reader resource lifecycles are outside this task.
Willingness to contribute
Description
Two bookkeeping paths perform quadratic work when a tiering round contains many splits:
TieringSourceEnumerator.assignSplits()repeatedly callsremove(0)on anArrayList, shifting the remaining entries on every assignment.TieringCommitOperator.collectTableAllBucketWriteResult()scans all results collected so far and rebuilds a set and a list for every new result. Collecting N results therefore takes O(N²) work.Use a deque for pending splits and an incremental per-table result accumulator with an expected count and bucket deduplication. Preserve assignment order, failover cleanup, duplicate-bucket rejection, and inconsistent-result-count validation. Commit only after all expected results arrive.
Verify equivalent behavior for normal completion, duplicate results, inconsistent counts, and failure cleanup. Queue removal and result accumulation should each take O(N) total work per round. Split prefetching and changes to reader resource lifecycles are outside this task.
Willingness to contribute