Developed FP8 quantization and Gaudi inference support for the bytedance-iaas/vllm repository, focusing on enhancing model serving performance and efficiency on Intel Gaudi hardware. Leveraged Python and PyTorch to integrate Intel Neural Compressor, enabling end-to-end deployment workflows that utilize hardware-specific optimizations. The work introduced quantization techniques that reduce inference costs and improve throughput, while establishing a foundation for future benchmarking and further model optimization. No major bugs were reported during this period, reflecting a stable implementation. This contribution advanced the repository’s capabilities in machine learning model optimization, particularly for environments requiring efficient, hardware-accelerated inference using quantization methods.