Blocking AI Platforms
How to evaluate and implement AI platform blocking decisions using SeeLLM's what-if analysis.
Before You Block
Blocking an AI platform is a significant decision. Before acting, use SeeLLM's what-if analysis to understand the impact.
Using the What-If Tool
Via MCP, ask your AI assistant:
"What would happen if I blocked GPTBot from my site?"
Or via the API:
curl "https://api.seellm.com/api/analytics/whatif-block?domain=example.com&platform=gptbot&range=30d" \
-H "Authorization: Bearer sk_live_..."The analysis shows:
- How many visits you'd lose
- Estimated value impact
- Whether the platform drives referral traffic (real users) or just crawls
When to Consider Blocking
| Reason | Recommended Action |
|---|---|
| Platform only crawls, never refers users | Consider blocking — low value, high resource cost |
| Platform drives referral traffic | Keep it — those are real users discovering your content |
| Excessive crawl rate causing performance issues | Rate-limit rather than fully block |
| Legal / licensing concerns | Block with robots.txt and verify compliance |
How to Block
Via robots.txt
The standard way to block AI crawlers:
# Block OpenAI's training crawler
User-agent: GPTBot
Disallow: /
# Block Anthropic's crawler
User-agent: ClaudeBot
Disallow: /
# Allow ChatGPT to fetch pages for users (referrals)
User-agent: ChatGPT-User
Allow: /Note the distinction: GPTBot crawls for training, while ChatGPT-User fetches pages to answer user questions. You may want to block one but allow the other.
Via Cloudflare Rules
For more granular control, use Cloudflare WAF rules to block by user agent, IP range, or ASN.
Monitoring After Blocking
After implementing a block, monitor your SeeLLM dashboard to verify:
- The blocked platform's visits drop to zero
- No unexpected impact on other traffic
- Referral traffic from that platform's users still works (if you only blocked the crawler)