Robots.txt Generator

Generate a valid robots.txt file to control search engine crawler access to your website.

100% Private

Instant

Any Device

Free Forever

Rule Set 1

User-Agent

Allow Paths

Disallow Paths

Crawl Delay (seconds, optional)

Global Settings

Sitemap and host directives

Sitemap URL

Host (optional)

Generated robots.txt

Place this file at your domain root

User-agent: *
Allow: /
Disallow: /admin/
Disallow: /private/

Frequently Asked Questions

Create XML sitemaps.

Generate SEO meta tags.

Generate OpenGraph and Twitter Card meta tags.

Format and beautify HTML code with proper indentation.