Nicolas Grande developed a distributed weight synchronization optimization for the google/tunix repository, focusing on enhancing support for repeating key-value head tensors in large-scale machine learning models. Using Python, JAX, and TensorFlow, he improved the synchronization process by introducing mechanisms to clear the key-value cache, ensuring updates operate on fresh data and preventing stale state issues. He also optimized destination pytree structures to reduce memory overhead and improve performance during weight updates. This work addressed memory management and synchronization latency, enabling more efficient distributed training workflows. All changes were integrated with the existing pipeline and thoroughly documented for maintainability and future development.