ThapaAI Search
Your site can be perfect and still be invisible to ChatGPT
AI assistants read your website by sending automated software to fetch your pages. Many hosts and security products turn that software away by default, or serve it a "prove you're human" challenge instead of your content. Your site looks completely normal to you. The assistant received nothing, so it can never mention you.
This is the most expensive problem we find, and it is almost always invisible. There's no error message, no dip in your analytics, no broken page. The work simply never arrives.
Why would my own website block anything?
Usually nobody chose to. Every site has a small instruction file at its front door — robots.txt — that tells automated visitors what they're allowed to read. Plenty of hosting platforms and security services ship a default that names the AI crawlers and disallows them, on the reasonable-sounding theory that AI companies shouldn't train on your content for free.
The catch is that the same crawlers do two different jobs. Blocking them stops the training, and it also stops you being recommended. You opted out of the answer, not just the training set.
The second cause is security. A firewall or bot-protection product can't easily tell an AI crawler from a scraper, so it challenges both. The challenge page returns successfully — so nothing is logged as an error — and the assistant stores a page that says "checking your browser".
How do I check whether it's happening to me?
Looking at your website tells you nothing, because you're not the visitor being blocked. The only real test is to request your pages while identifying as each crawler, and compare what comes back against what a browser gets. If they differ, you've found it.
The crawlers worth testing by name:
- GPTBot and OAI-SearchBot — OpenAI, which is what ChatGPT uses
- PerplexityBot — Perplexity runs its own index entirely
- ClaudeBot — Anthropic
- Google-Extended and Bingbot — Google's AI answers and, through Bing, ChatGPT and Copilot
- Applebot — Siri and Apple Maps
Does being in Google mean being in ChatGPT?
No, and this is the single most common misunderstanding in the whole category. The assistants don't share one index.
| Assistant | Reads | Can you submit? |
|---|---|---|
| Google AI Overviews, Gemini | Google's index | Yes — Search Console |
| ChatGPT, Microsoft Copilot | Bing's index | Yes — Bing Webmaster Tools |
| Claude | Brave Search | No console exists |
| Perplexity | Its own crawler | No submission tool |
Two of the four can be filed with. Bing is the one almost everyone skips, and it's the one carrying ChatGPT. The other two are earned — by being readable, and by being cited on sites they already read. Anyone offering to "submit your site to ChatGPT" is describing something that does not exist.
What about sites that build themselves in the browser?
Related problem, same symptom. Many AI crawlers don't run JavaScript. If your page arrives as an empty shell and fills itself in once scripts load — which describes a lot of page-builder and single-page sites — then what the crawler stores is the empty shell.
The same trap catches scroll animations. The usual way to build them ships every section invisible and switches it on with code, so a crawler that doesn't run the code sees a blank page. Ships visible, animate afterwards — that's the fix, and it costs nothing if you do it from the start.
Common questions
If I unblock AI crawlers, am I giving away my content?
Partly, and it's a real trade rather than a trick question. You're exchanging some training use for the chance to be named when someone asks for a business like yours. For most local and service businesses that trade is obviously worth it. For a publisher whose product is the words, it may not be.
How long until an assistant picks up the change?
Days to weeks, depending on the engine and how often it crawls you. There's no submit button for most of them, so it isn't instant.
Will this get me recommended?
No. It removes the reason you'd be impossible to recommend. Being readable is the floor. Being quotable, and being cited elsewhere, is what gets you named — and neither can help while the door is shut.
Can I check this myself?
Yes. If you're comfortable at a command line, request your homepage with each crawler's user-agent and read what comes back. If you'd rather not, it's part of our $149 Website Diagnostic, and all we need to start is the URL.