May 2026 monthly summary for yhyang201/sglang focusing on MLA Attention Performance Optimization on ROCm MXFP4. The work delivered improved data movement and throughput, addressing memory-copy inefficiencies and enhancing buffer allocation for MLA attention calculations on ROCm MXFP4. Impact: reduced memory bandwidth pressure, faster attention computations, enabling smoother model scaling on ROCm hardware.

April 2026 — jeejeelee/vllm: Delivered targeted performance and stability improvements in the MLA attention path. Implemented a MLA dual RMS norm fusion pass for Q and KV to optimize kernel launches and boost throughput in ROCm/AITER environments, followed by a compatibility hotfix that gates the feature behind AITer version support to prevent errors with older stacks. This work enhances model inference speed while maintaining stability across deployments, and positions the project for broader hardware support.

In 2026-03, delivered a critical bug fix and supporting improvements for the ROCm/aiter MOE path, focusing on the ck_moe_stage1 split-K forward pass. The changes address an undersized temporary output buffer and activation slice handling to prevent double-zeroing, improving forward-pass correctness and performance. Implemented memory and dtype handling refinements, aligned buffers with the CK kernel, and updated fused_moe.py to reflect changes. These updates reduce risk of incorrect zeros, stabilize MOE forward passes, and lay groundwork for improved throughput in production workloads.