Robots.txt Generator

Create a properly formatted robots.txt file to control how search engine crawlers access your website. Block AI bots, protect admin areas, and manage crawl behavior.

Warnings

-No sitemap URL specified. Adding a sitemap helps search engines discover your pages.

Rule Set

User-Agent

Crawl Delay (seconds)

Disallow Paths

Allow Paths

Additional Directives

Sitemap URL

Host (optional)

Generated robots.txt

User-agent: *
Disallow: /admin/
Disallow: /private/
Disallow: /api/
Allow: /

Save this content as robots.txt in the root directory of your website.