Worked on foundational improvements to Spark SQL’s Sort-Partitioned Join (SPJ) architecture within the apache/spark repository, focusing on modularity and correctness. Refactored SPJ logic from BatchScanExec into a new KeyGroupedPartitionedScan base class, enabling connectors to reuse SPJ functionality across different scan types. Addressed a critical correctness issue by fixing partial clustering behavior to respect required key-grouped distribution when executing joins, ensuring accurate query results. The work leveraged Scala, SQL, and Spark, and emphasized robust data engineering and distributed systems practices. These changes laid the groundwork for broader SPJ deployment and improved the maintainability and extensibility of Spark’s query execution.