Free robots.txt generator

    Answer a few questions and get a robots.txt that says exactly what you meant — with the site-killing mistakes flagged before you upload it.

    LemAudit Editorial
    By LemAudit Editorial · Research teamLast reviewed

    Each line must start with a forward slash. Wildcards (*) and end-of-URL anchors ($) are supported.

    Useful when you disallow a folder but want one path inside it crawled.

    Absolute URL. Add your sitemap index here rather than every shard.

    Bing and Yandex honour this. Google ignores it — use Search Console instead.

    How we made this page

    Last tested: .

    This tool runs entirely in your browser — no server call, no logging, no data retention. We compute the output from your inputs the moment you submit, and validate the result against the relevant public spec before showing it.

    Free robots.txt tester
    Paste your robots.txt, give it a URL and a crawler — see whether it is allowed or blocked, and exactly which line decides it.
    Free sitemap index generator
    List your sitemap shards, one per line — get a sitemapindex file you can submit to Search Console as a single entry.
    Free llms.txt generator
    Fill in your site name, summary and key resources — get a spec-compliant llms.txt you can drop straight into your site root.

    Check the rest of the site while you're here

    This tool fixes one thing. A free LemAudit scan checks the whole site — technical health, on-page, performance, security and AI search readiness — and scores it in about 30 seconds.

    Already ran one? Create a free account to save reports, re-run them on a schedule and track scores over time.