The free checkers

What it is

A cluster of single-purpose free tools. One robots.txt tester for each of the fourteen crawlers in our named catalog, including the retired user-agent tokens people still write rules for, plus an llms.txt validator, a sitemap.xml checker, and a server-side rendering checker.

When to use it

To answer one narrow question fast, on any domain, with no account. Also for checking a prospect’s site before a call.

Step by step

1. 1Go to /tools and pick the check you need.
2. 2Enter a domain. Each tool fetches one public file and reads it.
3. 3Read the whole result, including everything that failed.
4. 4For the full picture across crawler access, readability, schema, and a scored action plan, run the free audit at /audit.

What you will see

Per crawler: whether your robots.txt blocks that specific user-agent, what the crawler is for (building the index, searching, fetching live for a user, or collecting training data), and, for a retired token, that the rule is dead and blocking it protects nothing today. The llms.txt and sitemap checkers also catch the failure a status-code checker passes: a site that answers any unmatched path with its HTML app shell, so the file returns 200 while every parser reading it gets markup.