During February 2025, Marco Pagliarini developed and integrated the AdEMAMix optimizer into the swiss-ai/Megatron-LM repository, focusing on deep learning and distributed systems. He implemented the AdEMAMix optimizer class in Python and C++, ensuring seamless integration with Megatron-LM’s optimizer selection flow. Marco extended the configuration and argument parsing logic to expose AdEMAMix-specific parameters, enabling researchers to experiment with this new optimization approach. His work positioned the optimizer for downstream benchmarking and validation, reflecting a strong understanding of optimizer implementation and PyTorch. The project demonstrated depth in both system integration and enabling flexible experimentation within large-scale model training workflows.