Check your robots.txt for AI policy contradictions
We fetch your robots.txt once and read two things: which named AI crawlers are blocked, and any Cloudflare Content-Signal line stating what AI systems may do with your content. When the two of them together refuse the AI use your site presumably wants, we say so plainly, with the lines as evidence.
Data source: read directly This tool reads the URL you give it directly. No third-party data provider is involved and nobody else sees your address.
Free, no sign-up, no email. We fetch your homepage once and read it. The full result shows here.
Questions
- What is a Content-Signal line?
- A robots.txt directive (Cloudflare Content Signals Policy) declaring what AI systems may do with your content, like search=yes, ai-train=no. It is a stated policy on top of the access rules, and the two can disagree.
- Is blocking AI training crawlers a contradiction?
- No, and this check will not flag it. Refusing training is not refusing citations. The contradiction is blocking the crawlers that feed AI answers, or an ai-input=no signal, while wanting to appear in those answers.
- Is this free?
- Yes. One fetch of one public file, full result on the page.
Check something else
- Generate a robots.txt that lets AI engines inWrites the permission file. The crawler testers tell you whether you are blocked; this is the file that unblocks you.
- Does your brand name resolve to YOU?The name-collision check: it probes sibling TLDs for live same-named sites and reads your page for the exact defenses (description, disambiguation, the X-is-a-Y sentence) engines resolve an entity from.
- Is your llms.txt valid?Reads and parses the llms.txt file itself, rather than testing a crawler rule.
- See every tracker and tag on your siteFacts with evidence, not a tracking health score: each detection shows the exact snippet that matched, and the popular gated version of this report is exactly why this one is not gated.