Robots.txt Generator

Control how search engines crawl your site. Configure the rules, then copy or download a valid robots.txt for your site root.

Preset

User-agentDisallow paths (one per line)Allow paths (one per line)

Crawl delay (sec)Sitemap URL

robots.txt

User-agent: *
Disallow: /admin
Disallow: /dashboard
Disallow: /api

Sitemap: https://www.example.com/sitemap.xml

Place this file at the root of your domain: https://yourdomain.com/robots.txt