Free tool
Is your robots.txt quietly blocking AI engines?
An engine that cannot read your pages can never cite them. Paste a domain and see exactly which AI crawlers your robots.txt lets in, which it blocks, and which of your rules were written for user agents that no longer exist.
The check nobody else runs
Blocked crawlers
The decisive failure. If GPTBot or ClaudeBot is disallowed, that engine cannot read a single page, and no amount of content work will change it.
Retired user agents
A rule for anthropic-ai or Claude-Web looks like a considered policy and matches nothing today. We name the replacement you are missing.
Written vs absent
No robots.txt and a robots.txt that allows everything both leave you reachable. Only one is a decision. We do not report them as the same thing.
What we check against
11 live AI crawlers, grouped by what each one is for, plus 3 retired user agents we flag if you have written rules for them. This is the same catalog the product measures against.
Live (11)
- GPTBotChatGPT (indexing)
- OAI-SearchBotChatGPT Search
- ChatGPT-UserChatGPT (live browsing)
- ClaudeBotClaude
- Claude-UserClaude (live browsing)
- Claude-SearchBotClaude Search
- PerplexityBotPerplexity
- Perplexity-UserPerplexity (live browsing)
- Google-ExtendedGemini & AI Overviews
- Applebot-ExtendedApple Intelligence
- CCBotCommon Crawl (feeds many models)
Retired (3)
- anthropic-aiuse ClaudeBot, Claude-User, Claude-SearchBot
- Claude-Webuse Claude-User
- cohere-aiuse cohere-training-data-crawler
Reachable is not the same as cited
This tool measures whether an engine can read you. Whether it actually cites you is a different measurement, from a different source, and the gap between the two is the most useful thing on your dashboard. The full audit measures both.