Do I need to know how to code?

No. Every problem is explained in plain English with the exact fix. If your site's built with an AI tool, you also get a file to paste into it and say "repair it".

Which website builders does it work with?

Any public URL. We detect Lovable, Bolt, v0, Replit, Framer, Webflow, Wix, WordPress, Shopify and more, and tailor the fixes to your builder.

Is this a security penetration test?

No — and we never claim to be. Scanvify checks what's publicly visible from outside. It doesn't test your server, database or logged-in areas. It's a launch-readiness and public-health check.

How much does it cost?

A free scan of your homepage with your top problems. A full-depth report with fixes is a one-time ₹299 in India ($12 internationally). If you scan often, India also has monthly plans from ₹499/mo.

What is a website launch-readiness check?

It's a scan of everything that decides whether a new site succeeds with real visitors: does it work on phones, load fast, show up on Google and AI search, pass accessibility basics, and avoid public security mistakes. Scanvify runs 40+ of these checks across every page and scores the result.

Can ChatGPT, Perplexity and AI search engines see my website?

Not always. Many AI-built sites render their content with client-side JavaScript, so AI crawlers — which mostly read the raw HTML your server returns — see a nearly empty page. Scanvify checks exactly what search engines and AI crawlers receive from your site and tells you how to fix it.

How is Scanvify different from free SEO audit tools?

Free SEO tools check one page for one thing and report in jargon. Scanvify scans every page across six areas — mobile, speed, SEO, AI-search visibility, accessibility and security — and gives you the exact fix for every issue: a ready-to-paste prompt if your site is AI-built (Lovable, Bolt, Framer and more), plain instructions otherwise.

What is… · updated 2026-08-17

What is an AI crawler?

An AI crawler is an automated bot — like GPTBot, ClaudeBot or PerplexityBot — that reads public web pages so an AI assistant can describe or recommend them later. It is separate from Googlebot: allowing Google does not allow AI crawlers, and many websites block them without realising it.

Which AI crawlers actually matter?

A handful do most of the work. Each belongs to a different company, and each has to be allowed separately — there is no single switch that covers them all.

GPTBot (OpenAI) — reads pages so ChatGPT knows about them.
OAI-SearchBot (OpenAI) — powers ChatGPT's search results specifically.
ClaudeBot (Anthropic) — Claude's crawler.
PerplexityBot (Perplexity) — Perplexity cites sources heavily, so this one carries unusual weight.
Google-Extended (Google) — controls whether your content feeds Gemini and Google's AI answers. Note this is a permission token, not a separate crawler: Googlebot does the actual fetching.
CCBot (Common Crawl) — a public dataset that many AI models are trained from.

How is an AI crawler different from Googlebot?

Googlebot exists to build a list of links. An AI crawler exists to build an answer. That difference has two practical consequences.

First, permissions are separate. A site can be perfectly visible in Google search and completely invisible to ChatGPT, because the two are governed by different rules.

Second, JavaScript matters more. Googlebot will usually render your page and wait for JavaScript to run. Most AI crawlers largely read the raw HTML your server returns. If your content is assembled in the browser — common on sites built with Lovable, Bolt, v0 and similar tools — a human sees a full page and an AI crawler sees an almost empty one.

Why would my site be blocking AI crawlers?

Usually because someone else decided for you. Since 2025 most CDNs block AI crawlers by default on new setups — Cloudflare in particular — and that block happens in front of your site, before any request reaches your own configuration.

This is the part that catches people out: your robots.txt can explicitly welcome AI crawlers while your firewall turns every one of them away. Nothing on your website tells you it is happening. The only way to know is to make a request as each crawler and look at what comes back.

Should I allow AI crawlers?

It depends on how you make money, and it is a genuine choice rather than an obvious one.

If you sell a product or a service, allowing them is usually right — being the business an AI assistant names is quickly becoming as valuable as ranking on Google.

If your content *is* the product — journalism, research, paid courses — blocking or charging is a legitimate position, and many large publishers have taken it. What you want to avoid is the third case: blocking by accident, because a default was set for you and nobody checked.

Common questions

Does allowing Googlebot also allow AI crawlers?

No. They are governed separately. A site can rank perfectly well in Google search and still be completely invisible to ChatGPT, Claude and Perplexity.

How do I check if AI crawlers can read my site?

Request your homepage while identifying as each crawler and compare the response to a normal browser request. If the browser gets a page and the crawler gets a 403, something is blocking it. Scanvify's full report runs this test for you against every major AI crawler and tells you which setting to change.

Will blocking AI crawlers hurt my Google ranking?

Blocking AI-specific crawlers like GPTBot or ClaudeBot does not affect classic Google rankings, which depend on Googlebot. But a blanket block that also catches Googlebot absolutely will — which is a common and expensive mistake.

Is blocking AI crawlers the same as blocking AI training?

Not quite. Some crawlers gather training data, others fetch pages live to answer a question someone is asking right now. Cloudflare now separates these by purpose — Search, Agent and Training — so you can allow the ones that send you customers while blocking the ones that only take content.

Check this on your own site

Get a free Launch Score for your site in under a minute — no signup. The full report tests whether each AI crawler can actually reach you, and tells you exactly what to change if it can't.

Scan my site free →