If you check your server logs, there is a good chance you will find a visitor called GPTBot. It is OpenAI’s crawler, and it has been quietly reading public websites since August 2023. That leaves every site owner with a simple question: should you let GPTBot in, or shut it out? The answer matters more than it looks, because blocking the wrong bot can quietly cut you out of ChatGPT while doing nothing you actually intended.
This guide explains what GPTBot is, how it differs from the other OpenAI bots people confuse it with, whether you should allow or block it, and how to control it on WordPress without hand-editing files on every deploy.
What Is GPTBot?
GPTBot is OpenAI’s official web crawler. In OpenAI’s own words, it “is used to crawl content that may be used in training our generative AI foundation models.” So its job is training data collection, not live answers. It identifies itself with the user agent GPTBot/1.3 (the full string is Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.3; +https://openai.com/gptbot), and it honors your robots.txt rules. It cannot reach password-protected pages or paywalled content, and it skips pages that collect personal information.
If you have already looked at a full list of web crawlers hitting your site, GPTBot is usually one of the busier AI names in the log.
GPTBot vs OAI-SearchBot vs ChatGPT-User
This is the part that trips people up. OpenAI runs several crawlers, including an ads crawler called OAI-AdsBot, and the three below do different jobs. Blocking one does not block the others, and treating them as one bot is how site owners accidentally cut themselves out of ChatGPT search while only meaning to opt out of training.
| OpenAI bot | What it does |
|---|---|
| GPTBot | Crawls content that may be used to train OpenAI’s foundation models |
| OAI-SearchBot | Surfaces websites in ChatGPT’s search features |
| ChatGPT-User | Visits a page when a user action in ChatGPT or a custom GPT triggers a live fetch |
The takeaway: if you block GPTBot, you are opting out of training, not out of being cited. ChatGPT can still surface and reference your pages through OAI-SearchBot and ChatGPT-User. If your goal is visibility in AI answers, blocking GPTBot does not help or hurt that directly. The same logic applies to other AI bots, which is why the ChatGPT-User and OpenAI crawler roles are worth understanding before you block anything.
Should You Allow or Block GPTBot?
There is no single right answer, but the trade-off is clear. Allowing GPTBot means your public content may be used to train future OpenAI models. Blocking it, in OpenAI’s words, “indicates a site’s content should not be used in training generative AI foundation models.” One point removes a lot of confusion: allowing or blocking GPTBot does not change your standing in Google’s regular search results.
Allow GPTBot if you want your expertise represented in how these models understand your topic, and you are comfortable with public content being part of training. Block it if you publish original research, paid or proprietary material, or you simply prefer not to contribute to model training. Either way, remember that robots.txt is an honor-system request. Well-behaved crawlers like GPTBot follow it, but the standard cannot force a badly behaved bot to comply.
How to Allow or Block GPTBot on WordPress
GPTBot reads your robots.txt file, so that is where the decision lives. To block it, add this rule:
User-agent: GPTBot
Disallow: /To allow it, you simply do not disallow it, or you can be explicit:
User-agent: GPTBot
Allow: /The catch on WordPress is that many sites serve a virtual robots.txt that WordPress or another plugin generates on the fly, so there is no physical file to edit, and SEO plugins can overwrite your changes. If you would rather generate and manage the file properly, a robots.txt generator for WordPress is the cleaner route, and the broader approach to WordPress robots.txt for AI crawlers is worth reading if you plan to manage more than one bot.
Manage GPTBot and Every AI Crawler Without Editing robots.txt
Hand-editing robots.txt works for one bot. It gets tedious once you are trying to make deliberate decisions about GPTBot, ClaudeBot, PerplexityBot, Google-Extended, and the rest, especially when a host or SEO plugin keeps intercepting the file. This is where RankReady helps. It gives you 31 AI crawler controls, so you can allow or block GPTBot, ClaudeBot, PerplexityBot, Google-Extended, Bytespider, and 26 more individually, from one screen. Those choices are auto-synced to a real robots.txt, even if a host plugin tries to intercept it.
It also shows you what is actually happening. RankReady keeps a training-bot log of which AI crawlers indexed which pages, and a separate view of citation-bot candidates, the fetches from ChatGPT-User, OAI-SearchBot, PerplexityBot, and others, each one timestamped. On top of crawler control, it generates an llms.txt file and supports Content Signals directives like ai-train, search, and ai-input. RankReady is free, GPL licensed, with no credit card and zero telemetry. If AI visibility is the wider goal, pair this with a proper approach to AI search engine optimization for WordPress.
The Short Version
GPTBot is OpenAI’s training crawler, and it is separate from the bots that put you in ChatGPT’s answers. Decide whether you want your content used for training, set the robots.txt rule to match, and do not assume blocking GPTBot changes your Google rankings or your ChatGPT citations, because it does not. If you manage more than one AI bot, control them from a single place and keep a log of who visited, so the decision stays yours instead of the default.
