July 2026 monthly summary for ROCm/aiter focusing on hardware compatibility, memory budgeting, and reliability improvements for DeepSeek-V4 deployments on AMD MI300X. What was delivered: - AMD MI300X JIT/LDS compatibility fix: dynamically adjusts Triton kernel tile sizes to fit the 64 KB LDS limit on gfx942, enabling DeepSeek-V4 indexing workloads to run on MI300X. - Occupancy-based memory budgeting: introduced an estimator to shrink BLOCK_KV and num_stages when memory constraints are hit, preventing JIT resource overflows and out-of-resources errors. - Code gating and cross-arch fixes: conditional tile sizing for gfx942, preserving default tiles on other architectures; cleaned up budget comments and simplified calculations to drive maintainability. Impact and accomplishments: - Enabled end-to-end DSv4/DeepSeek-V4 serving on MI300X (gfx942) with stable throughput and accuracy across tested configurations; dramatically reduced prior JIT aborts due to LDS overflows. - Demonstrated stability across diverse tests: GSM8K 5-shot, n=200 exact_match 0.955 (eager) / 0.955 (graphs); thorough throughput sweeps (up to ~2.6K total tok/s at 8k/1k) with no engine errors or preemption. - Added regression coverage for NUM_HEADS=64, HEAD_SIZE=128 on gfx942 to ensure future changes do not reintroduce regressions. Technologies and skills demonstrated: - Triton kernel tiling and dynamic scheduling, LDS budgeting, and occupancy-based optimization. - Conditional architecture-aware gating and robust memory budgeting in a high-performance ML inference stack. - ROCm 6.x integration, DSv4 indexing workflow, and end-to-end validation pipelines. Business value: - Expands hardware support and reliability for production ML inference workloads on MI300X, enabling larger indexers and faster throughput without sacrificing stability, ultimately reducing operational risk and enabling customers to deploy DeepSeek-V4 powered services on new AMD hardware.