Developed and integrated MultimodalRotaryEmbedding (mrope) support for GPT within the ROCm/Megatron-LM repository, expanding the model’s multimodal capabilities. This work involved introducing a new 'mrope' positional embedding type and seamlessly incorporating it into the existing GPT model architecture using C++ and Python. The implementation included comprehensive argument validation and the addition of unit tests to ensure correctness and maintainability. By focusing on deep learning and transformer model architecture, the changes positioned ROCm/Megatron-LM for future multimodal deployments, while also improving testing coverage and architectural readiness. No bug fixes were recorded during this period, with efforts concentrated on feature development.