Is Claude-Web blocked on your site?
Claude-Web is a RETIRED user-agent. Superseded by Claude-User. A rule for Claude-Web matches nothing today. Blocking it does not stop Claude today, and this tester tells you that instead of counting it as a meaningful block.
Free, no sign-up. We fetch one public file and read it. Nothing is stored.
Questions
- What does Claude-Web do?
- Claude-Web is used by Claude. It fetches a page live when a user asks about it.
- Should I block Claude-Web?
- There is little point: this token is retired, so blocking it protects nothing. Superseded by Claude-User. A rule for Claude-Web matches nothing today.
- Is this check free?
- Yes, and there is no signup. We fetch one public file, robots.txt, and read the rules for this user-agent. You see the full result, including every failure we find.
Check something else
- Is Claude-User blocked on your site?Tests the Claude-User user-agent specifically, which fetches a page live when a user asks about it.
- Is ClaudeBot blocked on your site?Tests the ClaudeBot user-agent specifically, which builds the index the model answers from.
- Is anthropic-ai blocked on your site?Tests the retired anthropic-ai token, so you learn whether you are keeping a robots.txt rule that no longer matches any live crawler.
- Is your llms.txt valid?Reads and parses the llms.txt file itself, rather than testing a crawler rule.