Worked on NVIDIA/Megatron-LM to address a critical issue in Mixture of Experts (MoE) model training by fixing the auxiliary loss routing logic. Refactored the compute_routing_scores_for_aux_loss function to return both routing scores and a top-k experts mask, ensuring correct load balancing for token-level and sequence-level auxiliary losses. This change improved training stability and scalability by reducing the risk of routing misbalance. The solution was implemented using Python and PyTorch, leveraging deep learning expertise and knowledge of transformer models. The codebase was made more maintainable by isolating routing-score computation, facilitating easier future enhancements and ongoing model development.