Developed backend support for the Rowwise RMSprop with Adaptive Regularization (RMSpropAR) optimizer in the pytorch/FBGEMM repository, focusing on scalable training of sparse embedding tables. The work established end-to-end kernel and backend infrastructure, including CUDA and CPU kernel templates, code generation registration, and BUCK build integration. Leveraging C++ and CUDA, the implementation introduced a per-row state design with EMA-based momentum and a staleness-aware, lazy weight decay mechanism to stabilize sparse feature training. This foundation enables efficient iteration on sparse embeddings and prepares for seamless frontend integration, addressing the challenges of sparsity-aware optimization in large-scale machine learning systems.