Senyu Tong developed a block-sparse paged attention kernel for sliding window attention in the apple/axlearn repository, targeting performance optimization for long-context models on TPU. Using JAX and Python, Senyu enhanced the kernel’s logit bias handling and mask functions to improve both accuracy and robustness. The implementation focused on increasing memory efficiency and compute throughput, addressing the challenges of scaling attention mechanisms to longer sequences. Senyu also contributed comprehensive unit tests and benchmarks to validate the kernel’s correctness and performance on TPU hardware. The work demonstrated depth in machine learning and TPU programming, delivering a robust, production-ready feature within one month.