Worked on NVIDIA/TransformerEngine to address stability and correctness issues in distributed training with MCore DDP. Focused on refining backward-pass tensor handling and correcting gradient accumulation logic for fused operations, which improved numerical reliability during large-scale deep learning workloads. Implemented safe CPU offloading of tensor data to prevent misalignment and instability in mixed CPU/GPU environments. The work involved low-level manipulation of tensors and maintenance of distributed systems, leveraging expertise in PyTorch, C++, and GPU computing. These changes enhanced the robustness of the framework, reducing debugging time for model developers and supporting more consistent performance in production training pipelines.