Did an AI crawler actually read your published files?
Join the llms.txt and robots.txt you published to the named AI crawlers we observed fetching them.
What it is
The other half of a generator. Every tool in this category writes an llms.txt and tells you it will help. This panel reports which named AI crawler actually requested the file, on what date, and what HTTP status it got back.
When to use it
After you publish any of the Build files, and whenever you want to check that a change to robots.txt did not lock a crawler out.
Step by step
- 1Publish the generated files on your own site.
- 2Connect server logs under Settings, Tracking, so we can observe crawler requests.
- 3Open Build and read the panel below the file cards.
What you will see
Per published file, a line per named crawler: read on a date, requested and got an error, not yet observed, or not being watched.
Measured, never manufactured
Four rules keep this from becoming a fake metric. Silence is not absence: with no log ingestion we say we are not watching rather than claiming a crawler skipped you. Requests made before you published are excluded, because a fetch of the old file says nothing about the new one. A non-200 is reported as an error rather than counted as a read. And a fetch is never described as a citation, because it is not one.
Check it yourself, free
Related guides
Next
How accurate is the detector itself?The published precision and recall of our own brand-mention detector, including the cases it gets wrong.