Concise monthly summary for 2026-04 focused on the Apache Spark DAGScheduler work: Key features delivered - DAGScheduler Internal Refactor for Maintainability and Consistency: Introduced explicit RDD graph traversal helpers to replace duplicated BFS boilerplate; refactored six methods to use the new helpers, improving readability and reducing maintenance risk without changing functionality. Commits include 5365b3f0e821d34d18ee79ba3d3e93dc63212d71. - Unify stage retry limit checks into canRetryStage helper: Centralized retry logic to enforce maxConsecutiveStageAttempts and maxStageAttempts in one place; updated abort paths and messaging to clearly indicate which limit was exceeded. Added unit test covering total-attempts enforcement. Commit 54294aef818992918e3f536cb0d94a2452ea9281. - Extract scheduleResubmit() helper to remove identical code blocks: Refactored scheduling of ResubmitFailedStages into a single helper to reduce duplication across rollback, fetch failure, and barrier-stage failure paths. Commit c9e2d65176e4ce4045d7ca80d971ab6d02c5a9f4. Major bugs fixed - Fixed inconsistent retry behavior between fetch-failure and non-fetch-failure paths by unifying retry limits and clarifying abort reasons; enforced total-attempts limit with clear messaging. Corresponding changes stabilized DAGScheduler behavior under failure conditions. Commits above. Overall impact and accomplishments - Substantive maintainability improvements for DAGScheduler through reduced boilerplate, improved readability, and centralized logic, enabling faster future changes with lower risk. - Strengthened reliability of scheduling paths under failure conditions due to consolidated retry logic and clearer abort messaging; unit tests provide coverage for corner cases (e.g., total attempts vs consecutive attempts). - No user-facing behavior changes; all changes are internal refactors with tests passing across the existing Spark test suite. Technologies/skills demonstrated - Scala and Spark internals: DAGScheduler refactor and scheduling logic - Algorithmic refactoring: BFS/graph traversal helpers and code deduplication - Test-driven development: added unit tests for retry-limit behavior; leveraged DAGSchedulerSuite - Software quality practices: clear commit messages, risk-reducing refactors, and maintainability focus