Worked on enhancing data engineering workflows in the snowflakedb/snowpark-python and xupefei/spark repositories, focusing on API development and robust data processing. Delivered features such as Spark API compatibility for apply_in_pandas and multi-aggregate pivot support, enabling seamless data transformation and improved compatibility with Spark-like outputs. Addressed schema inference robustness by refining error handling and metadata integrity, including user-provided schema support for Parquet reads and fallback mechanisms for schema inference failures. Improved test reliability by correcting teardown logic in xupefei/spark. Leveraged Python, SQL, and Snowpark, emphasizing thorough testing, error handling, and logging to support resilient, flexible data pipelines and developer experience.