Worked on optimizing transformer model performance in the NVIDIA/Megatron-LM repository by implementing a fused multi-latent attention (MLA) down-projection within the attention mechanism. This approach reduced the number of general matrix multiplication (GEMM) operations and lowered memory bandwidth requirements during attention calculations, directly improving throughput and resource utilization for large-scale deep learning models. Leveraged PyTorch and Python to integrate the optimization, ensuring compatibility with existing Megatron-LM tests and workflows. The work enabled more efficient training and inference, supporting scalability for larger transformer architectures and maintaining stability across deployment scenarios without introducing regressions or compromising integration reliability.