Check any website’s llms.txt file and robots.txt directives to understand how it controls AI crawler and LLM access.
We’ll check for llms.txt, analyze robots.txt, and generate a full AI access report.