Should I use public/robots.txt or the robots metadata route?

Either serves the same purpose; the failure mode is defining both and forgetting which one wins in your setup. Pick one, delete the other, and verify the deployed URL.

My robots.txt allows AI crawlers but my content is not being cited. Why?

On Next.js the usual suspect is rendering, not permission: content assembled in client components after mount is invisible to a crawler that does not execute your JavaScript. Run our server-side rendering checker to see exactly what a crawler receives.

Where does llms.txt go in a Next.js project?

public/llms.txt, which serves at yoursite.com/llms.txt. No route or configuration needed.

Next.js robots.txt generator for AI crawlers

Next.js gives you the whole file, two ways: a static robots.txt in public/, or a robots metadata route that generates it in code. You are the platform here, which moves the interesting question from "may I edit the file" to "does my rendering strategy leave anything for an allowed crawler to read".

Your domain

Runs in your browser. Your domain is never sent to us, and nothing is stored.

Type your domain and the file appears here.

Applying this on Next.js

Generate the AI-crawler rules with the tool above.
Serve them either as public/robots.txt or from the robots metadata route. One source, not both: two definitions is how a deploy ships the stale one.
Deploy, then re-read yoursite.com/robots.txt on production rather than localhost.
Run the server-side rendering checker on a real content page. On Next.js the robots file is the easy half; whether your content is in the server HTML is the half that decides citations.

llms.txt on Next.js

Drop the generated llms.txt into public/ and it serves at your domain root. Next.js is the one platform on this list where both files are fully yours with no workaround. llms.txt Validator · llms.txt Generator

Can AI crawlers read your Next.js content at all?

This is the platform where the SSR question is sharpest. Server components and static rendering put your content in the HTML; client components that fetch after mount leave an allowed crawler reading an empty shell. An AI crawler you carefully allowed in robots.txt gains nothing from a page whose content arrives via JavaScript it will not run. Server-Side Rendering Checker

Questions

Should I use public/robots.txt or the robots metadata route?: Either serves the same purpose; the failure mode is defining both and forgetting which one wins in your setup. Pick one, delete the other, and verify the deployed URL.
My robots.txt allows AI crawlers but my content is not being cited. Why?: On Next.js the usual suspect is rendering, not permission: content assembled in client components after mount is invisible to a crawler that does not execute your JavaScript. Run our server-side rendering checker to see exactly what a crawler receives.
Where does llms.txt go in a Next.js project?: public/llms.txt, which serves at yoursite.com/llms.txt. No route or configuration needed.

By platform

The file is half the story. Whether AI engines then read and cite your Next.js site is the half you cannot see from your own dashboard. Run the free AI visibility audit