Skip to content

Can Google and ChatGPT read your website?

Your robots.txt file decides which crawlers may read your site. Enter your address to see, crawler by crawler, whether search engines and AI tools such as ChatGPT, Claude, and Perplexity are let in.

Your domain is enough. The check reads its robots.txt file.

How the check works

It fetches your robots.txt and applies the rules the way crawlers do under the robots.txt standard: a crawler follows the group that names it, falls back to the rules for every bot only when none does, and the most specific path rule wins. It then reports, for each crawler, whether it may read your homepage and whether any paths are blocked.

It does not check firewalls or server settings that block bots before robots.txt is read. If a crawler shows as allowed here but still cannot reach you, that is the next place to look.

Questions

Can ChatGPT see my website?

ChatGPT search uses a crawler called OAI-SearchBot, and fetches pages for users with ChatGPT-User. If your robots.txt blocks them, ChatGPT cannot read your pages to cite them. This tool shows whether it does.

Does blocking GPTBot remove me from ChatGPT search?

No. GPTBot collects content for training OpenAI's models. ChatGPT search uses OAI-SearchBot, which is controlled separately. You can block training and still allow search.

Does blocking Google-Extended affect Google Search or AI Overviews?

No. Google says Google-Extended controls use in Gemini and its AI models, not Search. AI Overviews and AI Mode use the normal Googlebot crawl.

Why are crawlers blocked if I never set that up?

Security plugins, hosting firewalls, and some site builders add robots.txt rules or block AI crawlers by default. A staging rule that blocks everything can also survive a launch.

Does robots.txt stop all bots?

It is a request, not a lock. Reputable crawlers from Google, Microsoft, OpenAI, Anthropic, Apple, and Perplexity follow it. Bots that ignore it need to be blocked at the server or firewall.

For what to do once crawlers can reach you, read how to get your business recommended by AI search. More free checks are on the tools page.