Worked on stabilizing Mixture of Experts (MoE) training workflows in the NVIDIA-NeMo/Automodel repository by addressing a critical edge-case in gradient checkpointing for single-GPU setups. Implemented a Python-based guard that enables gradient checkpointing only when the epoch size exceeds one, preventing training errors that previously disrupted MoE experiments. This targeted fix required a strong understanding of PyTorch internals, deep learning model optimization, and the nuances of distributed training. By resolving this bug, the work improved the reliability of single-GPU MoE workflows and reduced edge-case failures, supporting more robust production experiments in machine learning and deep learning environments.