Worked on the rasbt/llms-from-scratch repository, focusing on improving the correctness and stability of transformer internals. Addressed a critical bug in the MultiHeadAttention module by correcting the masking logic for Key-Value caching, ensuring that the number of tokens in queries and keys was accurately reflected during attention score calculations. This fix was particularly important for cached decoding scenarios, where incorrect masking could lead to unreliable model outputs. Leveraged deep learning and machine learning expertise with PyTorch and Python to maintain codebase reliability, prioritizing robust engineering practices and clear commit traceability while resolving a nuanced edge-case in transformer attention mechanisms.