Free Tool
robots.txt Generator
Pick your platform, add your domain, and get a robots.txt pre-loaded with the disallow rules that platform actually needs, plus an optional block for AI crawlers (GPTBot, CCBot, Google-Extended, ClaudeBot).
How It Works
Three Steps, Done
01
Pick your platform
WordPress or Shopify: each gets a different default rule set.
02
Add your domain
Used to build the correct Sitemap: line at the bottom.
03
Download or copy
Grab the file and upload it, or paste the rules into your platform's robots.txt editor.
Why It Matters
More Than Just Convenient
- A wrong robots.txt can accidentally block Google from crawling your whole site, so this uses tested, platform-correct defaults.
- WordPress and Shopify need different disallow paths; using a generic template for either is a common mistake.
- One click to also block AI training crawlers, if that's a policy you want.
FAQ
Common Questions
Most SEO plugins (Yoast, Rank Math) let you edit robots.txt directly from their settings, no file upload needed.
Yes, modern Shopify themes support a robots.txt.liquid template you can edit from Online Store > Themes > Edit code.
No. Googlebot (used for Search) is separate from Google-Extended (used for AI training). Blocking AI crawlers does not affect normal search indexing.
Need This Done For You?
Let's Talk About Your Project
If this tool solved a small piece of a bigger problem, I can help with the rest: WordPress and Shopify development, speed optimization, and ongoing support.