During March 2025, Xueh enhanced the ROCm/Megatron-LM repository by implementing MultimodalRotaryEmbedding (mrope) support for GPT models. This work introduced a new 'mrope' positional embedding type, expanding the model architecture to better support multimodal tasks. Xueh integrated the feature using C++ and Python, ensuring robust argument validation and comprehensive unit tests to maintain code reliability. The addition of mrope enables GPT models within Megatron-LM to handle a broader range of input modalities, positioning the repository for future multimodal deployments. The engineering approach demonstrated depth in deep learning and transformer model architecture, with a focus on maintainability and extensibility.