Apoorv Gupta enhanced the apple/axlearn repository by developing rematerialization patterns aimed at improving memory efficiency and training scalability for transformer-based deep learning models. Using Python and leveraging expertise in machine learning and transformers, Apoorv implemented specialized remat configurations for neuron layers, enabling selective offloading and reducing peak memory usage during model training. The work included updating regex patterns to support targeted saving and offloading of transformer components, as well as expanding test coverage to ensure robust integration of these features within the training loop. This contribution addressed memory constraints and laid a foundation for more scalable large-scale model training.