Worked on the NVIDIA-NeMo/Automodel repository to enhance the reliability and scalability of distributed training for large machine learning models. Addressed a critical issue in saving PEFT adapters during pipeline-parallel training by implementing logic to gather adapter configurations and weights from all pipeline stages, ensuring complete and reproducible checkpoints. Developed comprehensive unit tests to validate the gathering, union, and deduplication of adapter tensors, reducing the risk of partial saves and deployment inconsistencies. Utilized Python and PyTorch to refine the adapter saving process, including improvements to warning logic for degenerate gathers, and focused on robust support for PEFT and LoRA techniques.