Developed and integrated Expert Parallelism with Load Balancing (EPLB) support into the jeejeelee/vllm repository, enabling scalable distributed inference using NVFP4 FusedMoE. Focused on enhancing the model optimizer, the work introduced EPLB within ModelOptNvFp4FusedMoE to improve model scalability and performance across multi-GPU deployments. Implemented comprehensive end-to-end tests to validate EPLB’s interaction with the FusedMoE layer in distributed settings, ensuring correctness and stability. Leveraged Python and expertise in distributed systems, machine learning, and model optimization to reduce inference latency and increase throughput, directly supporting more efficient production workloads and better resource utilization in large-scale environments.