Developed and integrated an On-Demand Profiling feature for the NVIDIA/Megatron-LM repository, enabling dynamic inspection of training workloads through a new command-line interface flag and startup logic. This addition allows users to activate a profiling server during training runs without modifying code, improving observability and facilitating faster performance debugging for large-scale distributed systems. The implementation focused on system configuration and CLI design, leveraging Python to seamlessly embed profiling capabilities into the existing training script. By enabling real-time workload inspection, the work laid the foundation for future profiling-driven optimizations and enhanced the maintainability of performance tuning workflows in distributed environments.