[python] Support newer PyArrow versions

[XiaoHongbo-Hope]

Purpose

Pypaimon currently declares pyarrow<20, which prevents it from sharing an environment with libraries that require newer PyArrow releases. This change widens the supported range to pyarrow<24 while retaining the 19.0.0 exclusion. The recommended range for new environments remains pyarrow>=16,!=19.0.0,<20. Regular Python 3.12 and 3.13 CI pins PyArrow 23.0.0 after installing Vortex, and a focused compatibility job covers the affected filesystem paths.

PyArrow 22 and 23 use a newer AWS SDK whose default checksum requests and batch-delete behavior are incompatible with some S3-compatible services. The compatibility path requests checksums only when required and falls back to bounded concurrent object deletion when the SDK rejects multi-object deletion. Directory creation remains delegated to PyArrow.

PyArrow 24 and 25 remain excluded because ARROW-50188 can deadlock processes that retain an S3FileSystem during interpreter shutdown. Its proposed fix, ARROW-50193, has not been merged.

Tests

• Python 3.12 regular CI with PyArrow 23.0.0: license and flake8 checks passed; 4861 passed, 122 skipped; Java-Python interoperability passed
• Python 3.13 regular CI with PyArrow 23.0.0: flake8 passed; 4860 passed, 123 skipped; Java-Python interoperability passed
• PyArrow 19.0.1, 20.0.0, 21.0.0, 22.0.0, and 23.0.0: 50 passed per version in oss_legacy_mode_test.py and file_io_test.py
• PyArrow 23.0.0 real DLF REST catalog canary: worker-deserialized FileIO write/read/delete, create database/table, commit and scan 3 rows, recursively delete table files, and drop table/database