In April 2026, shipped a configurable preemptive CUDA memory OOM handling improvement for PyTorch inference serving in pytorch/pytorch. The change introduces a per_process_memory_fraction guard and a new throw_on_cudamalloc_oom boolean flag on the CUDA caching allocator, enabling preemptive rejection of allocations that would exceed the configured limit. If the budget would be exceeded, an OutOfMemoryError is thrown immediately rather than allowing a driver allocation that could crash the process, improving stability and reliability for inference workloads. Configuration is exposed via PYTORCH_CUDA_ALLOC_CONF (e.g., per_process_memory_fraction:0.95,throw_on_cudamalloc_oom:true). Observers are notified for monitoring and metrics, and the server process remains alive to allow graceful error handling by the serving framework. This work directly supports higher uptime, safer multi-tenant inference deployments, and easier client error handling under memory pressure.

March 2026 highlights for pytorch/pytorch focusing on GPU memory management resilience. Implemented a preemptive GPU memory guard in the CUDA allocator by introducing a throw_on_cudamalloc_oom flag in combination with per_process_memory_fraction. When the configured memory limit would be exceeded, allocations are rejected with an OutOfMemoryError instead of triggering a fatal GPU runtime abort, enabling graceful error handling in serving frameworks and reducing downtime under memory pressure. Configurability via PYTORCH_CUDA_ALLOC_CONF (e.g., PYTORCH_CUDA_ALLOC_CONF=per_process_memory_fraction:0.95,throw_on_cudamalloc_oom:true). Impactful for inference-serving reliability and client experience."

October 2025 monthly summary for the pytorch/FBGEMM repo focused on stabilization of prediction outputs through a targeted rollback. Restored correct tensor scaling and reliable inference across affected models by reverting a prior EmbeddingSpMDM8Bit_Sve change. Commit: 5beb3e6e0ef5ec830461ce163c012864677647a9 (Back out "Add EmbeddingSpMDM8Bit_Sve" (#4961)).

Monthly summary for 2025-08 (pytorch/pytorch): Restored stability in CUDA memory allocation configuration by reverting deprecated changes to CUDAAllocatorConfig, ensuring reliable behavior and compatibility with AcceleratorAllocatorConfig across CUDA builds and training workflows.

June 2025 (2025-06) – PyTorch: Delivered a safety-focused bug fix to ScriptModule hook registration, improving stability and developer experience. Implemented a type check to prevent forward hook registration on ScriptModule instances via register_forward_pre_hook, addressing an error encountered during hook setup. The change was implemented in pytorch/pytorch with commit 977abe786d907c1ff76528a550e3d53c9f3b1044. This fixes the error 'register_foward_pre_hook not supported on ScriptModule' (#156904). Benefits include reduced runtime failures during model construction and tooling, better API safety, and smoother user workflows.

April 2025 (2025-04) monthly summary for repository pytorch/torchrec focused on robustness and compatibility in embedding collection. Delivered a bug fix to the DecoupleEmbeddingCollection Forward method: the method now returns the correct data structure, eliminating compatibility issues with subsequent transform passes. The change reduces downstream failures and stabilizes the embedding data flow across the training and inference pipeline.

March 2025: Implemented QuantEBC Feature Order Caching for Inference to optimize the forward path by caching feature order and avoiding unnecessary indexing. Added robust edge-case handling for empty EmbeddingCollections/EmbeddingBagCollections, improving inference reliability. These changes reduce latency and prevent failures in edge cases, aligning with performance and robustness goals for pytorch/torchrec. Commits included: c5a4ff15a235c90c7df628764b549c91e4c1f03a; 055119ec2ebd53dbe38a98c7b2203bb75667660d.