Joshua Hong developed a per-layer sliding window enhancement for the KV Cache in the apache/tvm repository, introducing a new attention type called MHA_SLIDING. Using C++ and Python, he updated data structures to support per-layer offset calculations, enabling accurate and efficient cross-layer attention in transformer models. His work specifically addressed the needs of models like Gemma3 that use customized rope parameters, improving both correctness and performance. By focusing on KV Cache management and LLM optimization, Joshua delivered a robust foundation for more dynamic attention patterns and scalable deployment, demonstrating depth in engineering and a clear understanding of advanced transformer workloads.