March 2026 monthly summary for pytorch/pytorch. Focused on stabilizing the ROCm backend for CK SDPA dropout and delivering a concise, business-value driven improvement across the codebase. Implemented a targeted memory access fix to GPU memory handling for dropout, while maintaining Dynamo compatibility in output handling. Result is increased training stability and reliability on ROCm GPUs, reducing runtime errors and enabling broader hardware coverage for production workloads.

January 2026: Delivered a critical stability improvement in the PyTorch SDPA dropout path, fixing a device-side memory access fault and aligning tensor lifecycles and RNG handling. This results in more reliable attention computations on GPUs (ROCm) and reduces crashes during training and inference. Change tracked in PR #154864, with code contributions that enhance ROCm compatibility and overall GPU performance.

Month: 2025-09 — Summary of key features delivered, major improvements, and value realized in graphcore/pytorch-fork. Focused on ROCm optimization and kernel enhancements to boost stability and performance on ROCm-enabled platforms. Delivered build-time optimizations for CK SDPA, updated CK integration, and integrated AITER Fav3 forward kernels to accelerate tensor operations. No explicit bugs fixed this month; emphasis on performance, compatibility, and build reliability improvements.

August 2025 — ROCm/pytorch: Key features delivered and bugs fixed focused on performance, stability, and backend reliability. Highlights include the Composable Kernel (CK) kernel generation optimization to reduce kernel proliferation and the device-side memory access fix for SDPA with dropout on ROCm, improving attention stability and backend reliability.

July 2025 ROCm/pytorch: Delivered initial AITER-based optimization for ROCm backward assembly kernels in multi-head attention, enabling improved throughput for transformer workloads on ROCm devices. Key commit: b5ce77c1f5964293299eb1366f341872a4e47fa6. No major user-facing features beyond kernel optimization; no documented bug fixes this month. Foundations laid for further kernel-level performance gains and future work on mha_bwd optimizations.