16 AI crawlers tracked by name
Corvane Legal: AI content engine
A German law firm’s publishing system with 1,100+ pages live on its own website. It counts each AI crawler’s visits and sends an alert if they stop.
- Content engine
- Crawler tracking
Found · The audit with my time in it
When a buyer asks ChatGPT, Claude, Perplexity and Gemini for a firm like yours, do they name you or a competitor? Within 7 business days you’ll know, you’ll know why, and you’ll have a fixed price for every fix.
Fixed price. What you pay in cash is credited in full to any build within 60 days.
Proof on file
My own previous site0
words of page content visible to non-JavaScript crawlers
The first site I audited this way was my own. It looked finished to people. AI tools received an empty page. This site is the rebuild.
observed 2026-10-05 · raw HTML fetch
03 Fit
Built for you if
Not for you if
04 The problem
A buyer asks an AI assistant for the best firms for their problem. It answers in seconds, with names. If yours isn’t one of them, you never hear about it: no visit, nothing in your statistics. G2 found that 51% of B2B software buyers start research in an AI chatbot. 6sense reports that 85–95% of B2B buyers buy from their day-one shortlist.
The causes are usually ordinary and fixable: a security setting that turns AI tools away, pages whose text is loaded later by a script so AI sees nothing, a company description that differs on every profile. None of it shows from the inside. In your own browser, the site looks fine. The audit looks from the outside, the way an AI assistant does.
Signs you need this
05 What you get
For one website in one language. Every measurement has a date and says how it was made, so you can repeat it next quarter.
Questions a buyer in your market would ask, plus a few about your name, put to ChatGPT, Claude, Perplexity and Gemini. You see who they name and quote instead of you, and how often you appear beside them.
AI companies send programs called crawlers to read websites. I check whether your site lets each one in, or a security setting turns it away.
What an AI tool actually receives, page by page, with a fix for every page and ready-to-paste text where the page gives enough facts.
Your key pages as a person sees them, next to what an AI tool receives, with screenshots and the speed on a phone and a computer.
The hidden labels that tell AI tools who you are, what you offer and where, checked page by page. Missing ones are written out for you.
I take you through the findings on a call, recorded so partners who weren’t there can watch it later.
Every fix ranked by effort and impact, with a fixed price for anything I would do. Do it yourself, with your agency, or with me.
06 How it runs
My software runs the same tests the same way every time. I go through the results with you and write the plan.
Agents at work Human gate: nothing moves on until a person approves
Step 1: Day 1
It starts when your payment arrives. When it’s done, the report and a PDF are on your report page.
Step 2: Days 1–2
My first email asks for three questions your best buyers ask and two competitors you lose work to. Reply with what you have; I suggest the rest.
Human gate You tell me who you lose to.
Step 3: Days 2–5
Every finding, every answer the assistants gave, and your questions and competitors. Anything I can’t repeat stays out of the plan.
Human gate I review every finding before the walkthrough.
Step 4: Within 7 business days
The recorded 45-minute walkthrough, the 90-day plan and a fixed quote for every fix I would do.
Human gate You decide what happens next. There is no obligation.
07 Proof
A law firm’s system I built checks AI crawler visits every day. The audit applies the same checks to your site, once and in depth.
16 AI crawlers tracked by name
A German law firm’s publishing system with 1,100+ pages live on its own website. It counts each AI crawler’s visits and sends an alert if they stop.
15 would-be dead links caught
Every old address, internal link and archived page checked before launch, so no link the clinic has earned breaks.
Want to see the format first? Run the free audit on your site, and read what the Premium AI Audit report covers: the Deep Audit is that report with 30 questions per assistant, plus my time.
08 Price
The audit costs a fixed €1,500$1,650. What you pay in cash is credited in full to any build within 60 days. The scope only changes if you ask for more, and then I quote it first.
Quoted separately
Always included
Paid by you, directly to each provider, in your own accounts. I add no markup and no hosting fee.
| What | How it works | Paid to |
|---|---|---|
| During the audit | Nothing. I use my own tools and accounts. You only give read-only access if you want your security settings checked directly. | Nobody |
| After the audit | Each fix in the 90-day plan states its own running cost, if it has one. Most have none. | Stated per fix |
What you pay in cash is credited in full to any build within 60 days. If you build elsewhere, the plan and quotes are yours to take with you.
Fixed price · excl. VAT · 7 business days
Bought the Premium AI Audit in the last 60 days? What you paid for it comes off: €49 for a site without a work address, so you pay €1,451$54 for a site without a work address, so you pay $1,596. What you pay here is what counts toward a build later. Use the address you bought it with: I email that address the link that takes the credit off.
WIMOCO · Vegard Wikeby · Monterrey Building for the web since 2006. Personal reply within one business day. Secure payment by Stripe. I never see your card.
Vegard has been a great help in all things online related to my hotel and restaurant business. Truly professional and result orientated, Vegard has consistently improved our online presence.
I highly recommend Vegard. He did a great job on our now rebranded website at very short notice. Not once but twice! He's calm, creative, takes initiative where needed but defers to clear requirements and is very professional. What more could you ask...?
09 Handover
No licence fees to me, no lock-in, no proprietary platform. Everything is dated and repeatable: ask the same questions yourself next quarter, or hand the report to any agency.
10 Under the hood
Hand this to whoever looks after your website. It lists exactly what the audit tests.
robots.txt rules for each, plus a request with each crawler’s user agent, so a CDN or firewall block shows up. Google-Extended and Applebot-Extended are robots.txt names only; for those two the rules are the test.11 Questions

Not sure the audit is the right first step?
20 minutes with me, not a sales rep. You leave with a clear next step — whether or not we work together.
Text extracted from the HTML this server sends — no JavaScript executed, the way GPTBot, ClaudeBot and PerplexityBot fetch pages. It is also the Markdown twin I publish for AI agents.
Loading…
<!doctype html> <title>Custom Web Tools & Cookie-Free Websites | AIWebRevolution</title> … <body> <div id="root"></div> <script type="module" src="/assets/index.js"></script> </body> → 0 words of page content. Beautiful to people. Blank to ChatGPT.
Two minutes · personal reply