Worked on the kvcache-ai/sglang repository to enhance tensor-parallel attention and mixture of experts (MoE) inference efficiency. Addressed a bug by implementing a local non-padded token count function, ensuring correct computation of num_token_non_padded across tensor-parallel ranks during prefill, which improved reliability and correctness. Developed a feature to skip SiLU and GELU activations for masked experts in MoE models, reducing redundant computation and increasing inference throughput. The work involved deep learning and machine learning concepts, leveraging Python and CUDA, and included well-documented, collaborative commits that improved both system performance and maintainability for large-scale inference workloads.