Worked on the ray-project/ray repository to address a critical limitation in PyArrow when reading large Parquet files with nested column types. Developed a fallback reading strategy in Python that detects when a Parquet row group exceeds the 2GB threshold and automatically switches to processing smaller, metadata-driven batches. This approach leverages PyArrow and data engineering techniques to ensure compatibility with complex schemas, introducing schema and metadata checks to trigger the fallback only when necessary. The solution included a safe batch sizing algorithm and comprehensive regression tests, maintaining existing behavior for flat schemas while improving reliability for large, nested data processing workflows.