During January 2026, Serhii Petrenko developed a Sitemap Discovery System for the apify/crawlee repository, focusing on improving web crawler coverage and reliability. He introduced an asynchronous discoverValidSitemaps utility in TypeScript, leveraging Node.js and asynchronous programming to efficiently identify valid sitemap URLs by parsing robots.txt and probing common sitemap paths, including /sitemap_index.xml. This approach reduced crawler gaps and enabled more complete indexing for users. Serhii also implemented lightweight runtime checks using inlined urlExists and integrated got-scraping, ensuring compatibility with existing systems. The work demonstrated thoughtful feature design and refactoring, laying a foundation for future enhancements without addressing critical bugs.