Worked on the ROCm/flash-attention repository to develop a fake tensor mode aimed at accelerating compile-time tests and reducing GPU memory usage. Leveraging Python and PyTorch, implemented compile-only test passes using PyTorch’s FakeTensorMode, introducing decorators and helper functions to guard kernel execution and data-dependent operations for correctness in fake mode. Added an environment flag to enable this mode and refined the test infrastructure to support parallelization with pytest-xdist. Refactored tests to minimize reliance on fake tensor predicates and replaced certain randomization functions, resulting in faster continuous integration cycles and a more scalable, maintainable testing environment for machine learning workflows.