In July 2025, Grisha Sizov developed localized local attention masking for padded keys in the facebookresearch/xformers repository. He introduced the make_local_attention method for BlockDiagonalPaddedKeysMask, creating BlockDiagonalLocalAttentionPaddedKeysMask to enable local attention within padded key masks. This approach improved scalability for long-sequence transformer models by allowing more efficient attention computation on padded inputs. Grisha’s work leveraged deep learning and attention mechanisms, using Python to implement the feature and integrate it with existing transformer infrastructure. The solution addressed a core challenge in handling long inputs, laying the foundation for future performance improvements in machine learning workloads involving transformers.