Worked on NVIDIA/Megatron-LM, focusing on enhancing the transformer engine and improving model stability. Developed support for with_kwargs in transformer engine hooks, enabling more flexible forward and backward hook signatures and facilitating advanced experimentation. Addressed a bug in Megatron-LM by refining the unpacking of outputs from fused RMSNorm residuals, ensuring correct handling when outputs are tuples and improving the reliability of fine-grained callables. Employed Python and deep learning techniques throughout the development process, demonstrating careful attention to commit clarity and targeted bug resolution. The work contributed to more robust training pipelines and reduced edge-case failures in complex model configurations.