Developed and integrated BD-LoRA, a block-diagonal variant of LoRA, into the huggingface/peft repository to enhance distributed inference for large language models. This work focused on reducing communication overhead in tensor parallelism, thereby accelerating model serving and improving scalability. The implementation included new configuration options, example scripts, and seamless integration with existing serving tools, enabling straightforward adoption in distributed environments. Experiments were initiated and documented to validate BD-LoRA’s performance benefits, including integration with vLLM. The project leveraged deep learning, model optimization, and Python programming skills to deliver higher throughput and lower latency in production distributed inference settings.