June 2026 monthly highlights for flashinfer-ai/flashinfer: - Key feature delivered: cuTile backend support for three GEMM operations (mm_bf16, bmm_bf16, gemm_fp8_nt_groupwise) implemented using the public CUDA Tile API, eliminating TileGym runtime dependencies and enabling faster, more portable GEMM computations. - Notable implementation details: ported kernels from NVIDIA TileGym to the public cuda.tile API; build system notes included to avoid CUDA runtime conflicts in CI environments. - Top achievements and performance: mm_bf16 shows strong speedups (median latency improvements around 1.69x vs cuDNN and 1.52x vs tinygemm); gemm_fp8_nt_groupwise with cuTile is within ~5% of Cutlass for small M and shows competitive results across other shapes, with larger M shapes highlighting areas for autotune. - Build/CI impact: resolved dependency conflicts by isolating the build chain and adopting a warn-and-continue approach when building in clean environments; CI now pre-installs the CUDA tile compile chain to ensure reliable imports and quicker iteration. - Business impact: expanded FlashInfer capabilities on modern GPUs, reduced external dependencies, improved inference throughput for BF16/FP8 GEMM workloads, and laid groundwork for broader adoption across deployment environments. - Technologies and skills demonstrated: CUDA, cuTile, GEMM optimization, BF16/FP8 workflows, performance benchmarking and analysis, kernel porting from TileGym, build tooling (PEP 517 isolation), CI reliability, and cross-hardware validation.