# ClaudeBot Is Hitting Your WordPress Site: Should You Allow or Block? (2026) ## Key Takeaways - ClaudeBot makes 819,256 requests in 15 days, averaging 55,000 hits daily from a single AI crawler. - ClaudeBot is the user-agent used by Anthropic to fetch web content for training and product use. - Blocking ClaudeBot stops training data collection but does not stop Claude-User or Claude-SearchBot. - On a typical mid-traffic WordPress site, ClaudeBot hits roughly one request every four seconds. - The robots.txt rules for blocking ClaudeBot include disallowing paths like /wp-admin/ and /checkout/. Last month a developer on r/webdev posted real Vercel logs from one site: **Claude alone made 819,256 requests in 15 days**. That is roughly 55,000 hits a day, every day, from a single AI crawler — and the Vercel bill went from $30 in November to $1,933.93 the next month. The same thread mentions Perplexity at 2.5M and Meta at 11.1M, but Claude was the new entry that surprised people, because most WordPress site owners did not even know its crawler had a name yet. That crawler has a name. It is **ClaudeBot**, the user-agent Anthropic uses to fetch web content for training and product use. If you run WordPress and you have not opened your access log in a while, there is a very good chance ClaudeBot is already in there. The question every site owner asks me now is the same: do I allow it, do I block it, or is there a middle option? **TLDR:** ClaudeBot is real, it hits WordPress sites hard, and the choice is not binary. You can allow it everywhere (full AI visibility, full server cost), block it everywhere in `robots.txt` (zero AI visibility, lower cost), or — the answer I push clients toward — allow it on the pages you want cited and block it on the pages that drain your server. What you will learn: how to identify ClaudeBot in your access log, how to allow or block it per-page in WordPress, how this differs from Claude-User and Claude-SearchBot, and how [RankReady](https://store.posimyth.com/plugins/rankready/) tracks every ClaudeBot hit on your site in real time so you can make that decision with data instead of guesswork. Table Of Contents ## What ClaudeBot actually is (and the other two Anthropic bots almost nobody mentions) ClaudeBot is one of three crawlers Anthropic operates. The other two are Claude-User and Claude-SearchBot. They look similar in your access log, but they have completely different jobs, and that matters because the allow-or-block decision changes for each one. | Bot | What it does | If you block it, you lose | | --- | ------------ | ------------------------- | | **ClaudeBot** | Crawls the open web to collect training data for Claude models | Future model training on your content | | **Claude-User** | Fetches a URL in real time when a Claude user pastes a link or asks Claude to read a page | The chance to be cited inside a live Claude conversation | | **Claude-SearchBot** | Indexes pages to power Claude's search-result quality (the Sonnet/Opus search feature) | Visibility in Claude's search-grounded answers | Most WordPress posts you will find online conflate all three under "ClaudeBot." They are not the same. Blocking ClaudeBot stops training data collection but does **not** stop Claude-User or Claude-SearchBot. That distinction is the difference between "I do not want my content used to train future models" and "I do not want to appear in Claude answers at all." Pick the right bot for the right outcome. The user-agent strings (per Anthropic's official documentation) all start with `Mozilla/5.0 (compatible;` and then differ at the end. ClaudeBot ends with `ClaudeBot/1.0; +claudebot@anthropic.com`. Claude-User and Claude-SearchBot follow the same pattern with their own names. The IP range list is published and refreshed at `https://claude.com/crawling/bots.json` — at time of writing it is a mix of `216.73.216.0/22` and around twenty `34.x.x.x` Google Cloud /32 addresses. If a request claims to be ClaudeBot but the source IP is not in that JSON, it is a spoofer. Treat it as such. ![Anthropic ClaudeBot official documentation page on support.claude.com showing the three bots — ClaudeBot, Claude-User, Claude-SearchBot — and how to block each](https://theplusaddons.com/wp-content/uploads/2026/05/anthropic-claudebot-docs-v2.png)Anthropic's official ClaudeBot documentation on support.claude.com — three separate bots, three separate jobs, three separate decisions.   ## How to spot ClaudeBot in your WordPress access log right now You do not need a plugin to confirm ClaudeBot is hitting you. SSH into your host and run this one line against your nginx or Apache access log: `tail -10000 /var/log/nginx/access.log | grep -i ClaudeBot` Or on Apache: `tail -10000 /var/log/apache2/access.log | grep -i ClaudeBot` If you see output, ClaudeBot is already crawling you. On a typical mid-traffic WordPress site I checked last week, ClaudeBot was hitting roughly one request every four seconds — not enough to crash the site, but more than enough to fill PHP workers if you are on a cheap shared host with no page cache for bot traffic. Here is what a clean log slice from a real WordPress install looks like: ![Example WordPress server access log filtered to ClaudeBot hits, showing user-agent ClaudeBot/1.0 across six Anthropic IPs in four seconds](https://theplusaddons.com/wp-content/uploads/2026/05/wordpress-access-log-claudebot.png)Example access log slice. Ten ClaudeBot requests in four seconds across six unique /32 IPs from the published Anthropic range. User-agent matches the docs exactly.   If you are on managed WordPress hosting (Kinsta, WP Engine, Cloudways, Pantheon, Pressable) and you do not have shell access, the access log is still there — every panel has a "view raw access log" option. Use the host search and filter for `ClaudeBot`. Same answer in fifteen seconds. ## What other WordPress site owners are actually doing about it I pulled the last twelve months of r/Wordpress and r/webdev threads via the Reddit JSON API to see what people who run real sites are saying. Two things stood out. First, the Vercel post hit 3,200 upvotes in a few weeks — this is not a quiet topic. Second, the WordPress-specific complaint is consistent: AI crawlers hit pages that are **not in the page cache**, so they hit PHP every time, and on WordPress that means database queries, theme rendering, plugin filters — the works. ![Reddit r/webdev thread showing real Vercel crawler stats — Claude made 819,256 requests in 15 days with 3,200 upvotes](https://theplusaddons.com/wp-content/uploads/2026/05/reddit-claudebot-traffic.png)Real Reddit thread from r/webdev (3,200 upvotes, 727 comments) plus r/Wordpress comments. Verified via Reddit JSON API, numbers verbatim from the posts.   A WPMU DEV support agent said it plainly in the r/Wordpress "Is ClaudeBot ignoring crawl limits?" thread: "We see it happening daily and it will cause spikes in the CPU usage especially when hitting a search query such as product, native /?s= or event calendars." Another user in the same thread, thedragonturtle, made the cache point cleanly: "The AI crawlers don't use your site like humans, so whatever pages they hit are rarely in the cache so they create MORE server load than real users." That is the WordPress-specific bit nobody explains well. On a static site, a crawler hit costs you almost nothing. On WordPress, every uncached ClaudeBot hit can trigger a full PHP execution, twenty plugin filters, and a handful of database queries. Multiply that by 55,000 requests a day and you have a real bill — even if it is just the host pushing you into the next plan tier. ## The honest allow-or-block framework I have seen people argue both extremes online. Block everything because the deal is broken (no referral traffic back). Allow everything because the future of search is here. Both are too clean. Here is the framework I actually run with for POSIMYTH and for client sites. **Allow ClaudeBot if:** - Your business benefits from being cited by AI tools (most B2B SaaS, agencies, publishers, course creators)- You have working page caching that absorbs bot hits (Cloudflare APO, WP Rocket, FastCGI cache, LiteSpeed)- Your content is informational, evergreen, and not paywalled- You have a way to track which pages get cited (see RankReady below) **Block ClaudeBot if:** - Your content is your monetised product (premium courses, paid research, gated PDFs)- You are on a shared host where PHP workers cap out at 4-8 and bot traffic measurably slows real users- You have measured no AI referral traffic over the last 90 days- You explicitly do not want your work used as training data **Do both (the middle path):** - Allow ClaudeBot on the cacheable, evergreen pages you want cited- Block ClaudeBot on `/checkout/`, `/my-account/`, `/?s=`, `/cart/`, `/wp-admin/`, `/wp-json/` and any high-cost dynamic URL- Allow Claude-User and Claude-SearchBot everywhere unless you have a specific reason not to (these are the bots that produce the citation, not the training crawl) That last option is the one I push clients toward. It costs you almost nothing in citation visibility and saves the server from the worst of the burn. ## The robots.txt rules that actually work (copy-paste) This is the block I drop into the `robots.txt` for sites taking the middle path. It allows ClaudeBot on content but blocks the expensive dynamic stuff: `# Allow ClaudeBot on content, block on dynamic/admin paths User-agent: ClaudeBot Crawl-delay: 2 Disallow: /wp-admin/ Disallow: /wp-json/ Disallow: /checkout/ Disallow: /cart/ Disallow: /my-account/ Disallow: /?s= Disallow: /*?add-to-cart= Disallow: /*?orderby= # Same rules for Claude-SearchBot User-agent: Claude-SearchBot Crawl-delay: 2 Disallow: /wp-admin/ Disallow: /wp-json/ Disallow: /checkout/ Disallow: /cart/ Disallow: /my-account/ Disallow: /?s= # Always allow Claude-User (the live citation bot) User-agent: Claude-User Allow: /` If you want to **block ClaudeBot entirely** (no training, but still let citation bots through): `User-agent: ClaudeBot Disallow: / User-agent: Claude-User Allow: / User-agent: Claude-SearchBot Allow: /` If you want to **block everything Anthropic**: `User-agent: ClaudeBot Disallow: / User-agent: Claude-User Disallow: / User-agent: Claude-SearchBot Disallow: /` Anthropic states in their official docs that all three bots respect `robots.txt` and that `Crawl-delay` is honoured "where appropriate." The r/Wordpress thread I quoted earlier had one user (Dr_Beatdown) reporting sustained high request rates from ClaudeBot until they blocked it at the `.htaccess` level — so if you do block it and the traffic does not drop within 24 hours, escalate to host-level rules. That is rare in my experience, but it does happen. A sister POSIMYTH post — the [WordPress robots.txt for AI crawlers](https://theplusaddons.com/blog/wordpress-robots-txt-ai-crawlers/) deep dive — walks through editing `robots.txt` on every major host (WP Engine, Kinsta, Cloudways) including the Yoast/Rank Math virtual robots.txt override. Read that one alongside this if you have never edited robots.txt on a managed host before. ## What llms.txt adds on top of robots.txt There is a newer file format being adopted by AI labs: `llms.txt`. It does not block anything — it tells AI crawlers which of your pages you want them to use, and in what order. ClaudeBot does not yet officially read `llms.txt`, but Anthropic's own documentation publishes one for their site, which is a strong signal. I cover the WordPress setup in detail in the [llms.txt for WordPress](https://theplusaddons.com/blog/llms-txt-wordpress/) post. Treat it as the carrot and `robots.txt` as the stick. Two newer pieces of the AI-WordPress stack worth knowing about while you are here: WebMCP (the protocol that lets AI agents read your site's structured data live) and MCP servers in general. The [WebMCP for WordPress](https://theplusaddons.com/blog/webmcp-wordpress/) and [What is an MCP Server on WordPress](https://theplusaddons.com/blog/what-is-mcp-server-wordpress/) posts dig into those. ClaudeBot is the crawl side. WebMCP is the live-query side. Both matter. ## The Elementor angle — block ClaudeBot from your builder previews If you use Elementor, there is a specific path you almost certainly want to disallow: `/?elementor-preview=`. Every page on your site has a builder-preview URL that renders the full builder UI. ClaudeBot does not need it, it triggers heavy backend execution, and it can leak draft content into training data. Add this to the block above: `User-agent: ClaudeBot Disallow: /?elementor-preview= Disallow: /?elementor_library=` Same logic for Gutenberg's `/?wp-block-editor=` preview routes if you use them. If you build custom headers, footers, or page templates in Elementor, [The Plus Addons Header Builder](https://theplusaddons.com/elementor-builder/header-builder/) makes one more decision easy: which header variants you serve to logged-out visitors (the bot path) versus logged-in visitors. Most sites do not need to differentiate, but if you do, it is a one-click setting. ## Where ClaudeBot fits in the bigger AI-WordPress picture The trap to avoid: thinking ClaudeBot is the whole AI traffic story. It is one bot. There are dozens. GPTBot, ChatGPT-User, OAI-SearchBot, PerplexityBot, Perplexity-User, Google-Extended, Applebot-Extended, Bytespider, Amazonbot, Meta-ExternalAgent, and a long tail of less-known crawlers. Each has its own user-agent, its own IP range, its own robots.txt opinion. The Vercel post that kicked off this article showed Meta sending **11 million requests in 30 days** to a single site — almost half of real user traffic. Claude was just the new entrant. Managing all of that by hand in `robots.txt` is doable. Watching all of that in your access log every week, finding the pages they over-crawl, deciding which to allow, which to block, which to rate-limit, which produced an actual AI citation last month — that is the bit that breaks down at scale. ### RankReady's AI Crawler Log — the dashboard view of every ClaudeBot hit This is the part of the post where I tell you what we built and why it matters for the ClaudeBot decision specifically. [RankReady](https://store.posimyth.com/plugins/rankready/) is a WordPress plugin we built at POSIMYTH for AI-first SEO. One of its core modules is the **AI Crawler Log**, and it solves the exact problem this post describes. Four ways the log changes how you decide on ClaudeBot: ### 1. Real-time hit feed per bot RankReady sits at the WordPress request layer (before page cache) and logs every AI crawler hit with bot name, IP, URL, response code, response size, and timestamp. You do not need to SSH or grep. Open the dashboard, filter to ClaudeBot, see the last 24 hours. ### 2. Per-page hit count and citation tracking The log groups hits by URL. You see exactly which posts ClaudeBot is hammering — and which ones it ignores. Pair that with the citation tracker module and you see which pages produced an actual Claude citation in the last 30 days, so you can stop guessing what to allow. ### 3. One-click block per bot, per URL pattern Instead of editing `robots.txt` by hand, you toggle ClaudeBot off for `/checkout/`, `/cart/`, `/?s=` and the like inside the plugin. RankReady writes the rule to the WordPress virtual robots.txt (or appends to a real one if you have it), so it survives theme swaps and host migrations. ### 4. Spoof detection — verify the bot is real RankReady checks every claimed ClaudeBot request against the published `claude.com/crawling/bots.json` IP list. Spoofers — and there are plenty — get flagged automatically. You stop wasting block rules on traffic that was never really ClaudeBot. [![RankReady WordPress plugin — AI SEO toolkit for ClaudeBot, GPTBot, Perplexity, and Google AI Overview tracking](https://theplusaddons.com/wp-content/uploads/2026/05/RankReady-WP.org-.png)](https://store.posimyth.com/plugins/rankready/)RankReady plugin — AI Crawler Log, citation tracker, schema generator, and seven other AI-SEO modules in one install.   Full module list in the current RankReady build: - AI Crawler Log (the one this post is about)- AI Citation Tracker (Claude / ChatGPT / Perplexity / Google AI Overview)- AI-Optimised Schema Generator- llms.txt Builder- robots.txt Manager (per-bot, per-URL rules)- FAQ Schema for AI Search- Internal Linking for AI Discovery- Content Freshness Signals- WebMCP Endpoint [Install RankReady free →](https://store.posimyth.com/plugins/rankready/)   [![RankReady — the AI-first WordPress SEO plugin from POSIMYTH](https://theplusaddons.com/wp-content/uploads/2026/05/Featured-Image.jpg)](https://store.posimyth.com/plugins/rankready/)RankReady — built by POSIMYTH for the AI-SEO era. One click install, no account required.   ## Your 10-minute ClaudeBot action checklist - SSH or open your host's raw access log. Grep for `ClaudeBot`. Confirm it is hitting you.- Decide which of the three paths you want: allow everywhere, block everywhere, middle path.- Copy the relevant robots.txt block from the section above and paste it into your site's `robots.txt` (or the Yoast/Rank Math virtual `robots.txt` editor).- Add the Elementor preview disallow if you use Elementor.- Wait 24 hours, re-check the access log. ClaudeBot should be hitting only the URLs you allowed.- Cross-check the requesting IPs against `https://claude.com/crawling/bots.json` to filter spoofers.- Install [RankReady](https://store.posimyth.com/plugins/rankready/) if you want this without doing it by hand and across every other AI bot at the same time. ## Wrapping up ClaudeBot is not the enemy and it is not free traffic. It is a crawler with a clear identity, a public IP list, and `robots.txt` compliance — which puts it ahead of most of the AI bots hitting your site right now. The right move depends on your business: allow it where citations help you, block it where dynamic pages drain your server, and verify every hit is genuine before you waste a rule on a spoofer. Whatever you choose, **see the data before you decide**. A grep of last week's access log is the floor. RankReady's AI Crawler Log is the version that actually scales when you have ten of these bots to manage at once. And if you build with Elementor, [The Plus Addons](https://theplusaddons.com/elementor-widgets/) keeps the front-end speed up so the bot hits that do come through cost you less PHP per request. ## Frequently Asked Questions **Q: What should I do if ClaudeBot is causing high server usage?** A: If ClaudeBot is consuming too many resources, consider blocking it on high-cost dynamic pages like /checkout/ or /cart/ while allowing it on evergreen content. This approach helps maintain citation visibility without overwhelming your server. Implementing the right rules in your robots.txt can effectively manage this traffic. **Q: Is it beneficial to allow ClaudeBot on my site?** A: Allowing ClaudeBot can be advantageous if your content is informational and you want it cited in AI tools. This is particularly useful for B2B SaaS companies and publishers. However, ensure you have effective caching in place to handle the bot traffic without negatively impacting user experience. **Q: What are the differences between ClaudeBot, Claude-User, and Claude-SearchBot?** A: ClaudeBot collects training data for future models, while Claude-User fetches URLs in real-time for live citations during conversations. Claude-SearchBot indexes pages to enhance search result quality. Understanding these differences helps you make informed decisions about which bots to allow or block based on your content strategy. **Q: How does using llms.txt benefit my WordPress site compared to robots.txt?** A: While robots.txt restricts bot access, llms.txt specifies which pages AI crawlers can use and in what order. Although ClaudeBot does not currently read llms.txt, adopting this format signals your preferences to other AI crawlers and can optimize how your content is utilized by them.