Month: 2025-10. Delivered two high-impact improvements across LiteRT-LM and TensorFlow Lite, focusing on initialization reliability, delegate identification robustness, and cross-repo business value. Key work includes reintroducing NPU warm-up inference for Gemma3 in LiteRT-LM with a new buffer Fill function and a prefill/decode sequence to ensure correct model initialization, plus a refactor in TensorFlow Lite that strengthens opaque delegate checks by introducing TfLiteDelegateIsOpaque and validating the opaque_delegate_builder.

In Sep 2025, LiteRT-LM delivered a key feature: NPU Latency Benchmarking and Reporting, enabling optional latency breakdowns for the NPU executor and adjusting executor creation to support benchmarking. This provides actionable latency insights for prefill and decode operations, improving performance visibility and guiding optimization. No major bugs fixed this month in LiteRT-LM. The work strengthens confidence in deployment readiness and enables data-driven improvements.

2025-08 monthly summary for google-ai-edge/LiteRT-LM. Delivered key features enabling multi-signature embedding models and cross-modality NPU processing, while cleaning up build/configuration to streamline deployment. Fixed critical memory propagation issues to Gemma3n and removed obsolete dynamic linking dependencies, improving stability and release readiness. Summary of impact: improved model compatibility, stability, and deployment efficiency across Gemma3n/Gemma3 embeddings and multi-signature architectures; enhanced cross-modality support on NPU and cleaner build pipelines.

July 2025 monthly summary for google-ai-edge/LiteRT-LM: Implemented NPU backend integration and CPU variant support for Gemma3n, expanding hardware compatibility and performance options for edge deployments. Updated session creation to include NPU and configured the NPU executor to run AOT-compiled Gemma3 models; ensured test scripts can execute the .litertlm file on NPU. Refactored the executor to support the CPU variant of Gemma3n models packaged in the .litertlm format, including new embedder contexts, per-layer embedding computations, and adjustments to buffer sharing and sampling logic.

June 2025 performance summary for google-ai-edge/LiteRT-LM: Delivered unified model format and strengthened NPU execution path, driving deployment reliability, cross-hardware consistency, and maintainability. Standardized on the .litertlm format across models, loaders, and resource loading; enhanced NPU initialization, AOT mask support, reset capability, and logit processing; and fixed a critical typo to prevent misconfiguration. The work reduces integration risk, accelerates deployment, and improves observability across CPU/GPU/NPU.

May 2025 Monthly Summary for google-ai-edge/LiteRT-LM focusing on performance and maintainability. Delivered NPU decode speedups, flexible quantization loading, benchmarking capabilities, and a substantial internal refactor to strengthen the executor architecture and quantization ecosystem. These changes reduce latency, improve throughput, and provide instrumentation for production readiness.

April 2025: Enhanced NPU executor test workflow for google-ai-edge/LiteRT-LM by introducing flexible CLI-based configuration for model and component paths. This change decouples test inputs from a single binary path, enabling dynamic testing of Gemma3, embedder, auxiliary, tokenizer models, the LiteRT dispatch library, and the input prompt. The update reduces test friction, expands validation coverage for new components, and accelerates integration testing across configurations.