Worked on enhancing memory efficiency and training scalability for transformer-based models in the apple/axlearn repository. Focused on implementing rematerialization patterns for neuron configurations, enabling more efficient use of memory during deep learning model training. Updated regular expressions to support selective saving and offloading of transformer layers, which helps reduce peak memory usage and supports larger-scale experiments. Expanded test coverage to ensure the new rematerialization specifications integrated smoothly into the training loop. Utilized Python and deep learning frameworks, applying expertise in machine learning and transformers to lay the groundwork for improved throughput and resource management in large-scale model training.