Worked on optimizing memory efficiency in the deepspeedai/DeepSpeed repository, focusing on ZeRO-Offload stages 1 and 2. Addressed a GPU memory usage issue by correcting the Host-to-Device data type and enabling 16-bit pinned memory buffers for H2D transfers, which reduced memory consumption from approximately three times to one time that of params_FP16. This fix, implemented in Python, improved resource utilization and allowed for larger model training and more predictable multi-GPU scaling. The work demonstrated strong skills in deep learning, memory management, and performance optimization, contributing to enhanced cost efficiency and stability in distributed training environments.