What is a robots.txt file?

It's a text file at your site's root that tells search engine crawlers which pages they can or cannot explore.

Are presets customizable?

Yes, you can start from a preset (Allow All, Block All, WordPress…) and add/modify rules.

Yes, a dedicated field lets you add your sitemap.xml URL to the file.

🇬🇧WebFileTools/robots.txt Generator

Create a complete robots.txt file with presets and visual rules.

Presets:

User-agent

AllowDisallowCrawl-delay (optional)

Sitemap URL (optional)

robots.txt

User-agent: *
Allow: /
Disallow: /admin
Disallow: /api

The robots.txt generator creates a complete robots.txt file with a visual interface. Presets for common CMS and customizable rules.

Choose a preset or start from scratch, add Allow/Disallow rules per user-agent, add your sitemap and download the file.

What is a robots.txt file?: It's a text file at your site's root that tells search engine crawlers which pages they can or cannot explore.
Are presets customizable?: Yes, you can start from a preset (Allow All, Block All, WordPress…) and add/modify rules.
Can I add a sitemap?: Yes, a dedicated field lets you add your sitemap.xml URL to the file.

Generate SEO meta tags and Open Graph for your site. Real-time preview. Free.

Create .env files visually with random secret generation. Export and copy in one click.

Create cron expressions visually with human-readable description. Common presets and next run times.