Worked on enhancing the kvcache-ai/ktransformers repository by integrating Qwen3MoE models, focusing on scalable Mixture-of-Experts (MoE) improvements within the KTransformers framework. Leveraged deep learning and model optimization techniques using PyTorch and Python to refine the attention mechanism and optimize sparse MoE block performance. Addressed critical edge cases by fixing bugs in the KQwen3MoeSparseMoeBlock, which improved model stability and inference reliability. Additionally, resolved adapter compatibility issues for llamafactory, streamlining integration workflows and reducing setup complexity. Validated these enhancements through end-to-end flows, resulting in improved throughput and usability for production workloads and laying groundwork for future model expansion.