Free robots.txt tester

    Paste your robots.txt, give it a URL and a crawler — see whether it is allowed or blocked, and exactly which line decides it.

    LemAudit Editorial
    By LemAudit Editorial · Research teamLast reviewed

    Paste the full contents of the file served at https://yourdomain.com/robots.txt.

    A full URL or just the path. Query strings are included in matching, as Google does.

    The user-agent token the crawler identifies itself with.

    How we made this page

    Last tested: .

    This tool runs entirely in your browser — no server call, no logging, no data retention. We compute the output from your inputs the moment you submit, and validate the result against the relevant public spec before showing it.

    Free robots.txt generator
    Answer a few questions and get a robots.txt that says exactly what you meant — with the site-killing mistakes flagged before you upload it.
    What CMS is this website running?
    Paste any URL and we will tell you which CMS it runs on, along with the framework, analytics, AI-bot policy and agent-readiness signals live on the site right now.
    Free llms.txt generator
    Fill in your site name, summary and key resources — get a spec-compliant llms.txt you can drop straight into your site root.

    Check the rest of the site while you're here

    This tool fixes one thing. A free LemAudit scan checks the whole site — technical health, on-page, performance, security and AI search readiness — and scores it in about 30 seconds.

    Already ran one? Create a free account to save reports, re-run them on a schedule and track scores over time.