Worked on the modelscope/ms-swift repository to address compatibility issues between the Qwen2 Transformer model and TorchAcc when using transformers version 4.45 or higher. Focused on deep learning and model optimization, the work involved updating the attention mechanism to correctly apply rotary positional embeddings and refining the causal mask logic. This ensured that positional embeddings remained properly aligned and that autoregressive behavior was preserved across library updates. The solution, implemented in Python, maintained stable production usage by validating compatibility with the latest transformers releases, allowing the repository to function reliably with evolving dependencies and preventing embedding misalignment or causality errors.