Robots.txt Checker
See whether your site has a robots.txt file and what it tells crawlers.
Want the full picture?
This tool checks one thing. A full SiteScore audit checks SEO, technical SEO, performance, mobile, accessibility, content, local SEO and conversion — with evidence for every finding.
Run a Full SiteScore AuditHow this check works
robots.txt sits at the root of your domain and tells crawlers which parts of the site they may request. We fetch it directly and show you what is actually served.
A missing robots.txt is not automatically a problem — most crawlers will simply crawl everything they can find. Equally, having one is not automatically good.
What matters is whether the rules match your intent. A site-wide Disallow: / blocks compliant crawlers from the whole site, which is usually only correct for staging environments.