Fetch, parse, and validate robots.txt files for any domain, instantly extracting crawl rules, sitemaps, and bot directives with high accuracy.