During June 2025, Akhattar contributed to the NVIDIA/Megatron-LM repository by addressing a critical issue in Mixture-of-Experts (MoE) model training. He refactored the compute_routing_scores_for_aux_loss function to return both routing scores and a top-k experts mask, which enabled correct load balancing for token-level and sequence-level auxiliary losses. This change improved the stability and scalability of MoE models by reducing the risk of routing misbalance during training. Akhattar’s work, implemented in Python using PyTorch and deep learning techniques, also enhanced codebase maintainability by isolating routing-score computation and auxiliary loss logic, facilitating easier future enhancements and debugging.