Developed a per-layer sliding window enhancement for the KV Cache in apache/tvm, introducing the new MHA_SLIDING attention type to support advanced transformer workloads. Leveraging C++ and Python, the work involved updating data structures to enable per-layer offset calculations, ensuring accurate and efficient cross-layer attention. This technical approach improved both correctness and performance for models such as Gemma3, particularly those utilizing customized rope parameters. The feature lays groundwork for more dynamic attention patterns and scalable deployment in large language models. No major bugs were reported during the development period, reflecting a focused and robust engineering effort in LLM optimization and cache management.