A crawler asks for a page
GPTBot, ClaudeBot, PerplexityBot or a fetcher acting for a person requests /pricing.
How it works·Process//Scan, connect, measure
Scan. Connect. Serve. Measure.
Setup takes minutes and needs no changes to your pages.
STEP 01
Scan
The scan fetches your pages as each major AI crawler would and records what it receives, what it is refused and what is missing.
0/ 100
Three problems to fix first. Two take under ten minutes.
Fix
llms.txt
No file found at /llms.txt.
LLMReady writes it from your sitemap and top pages.
Review
Crawler access
GPTBot is allowed. ClaudeBot and PerplexityBot are blocked by a firewall rule.
Pass
Sitemap health
412 URLs listed, all return 200.
STEP 02
Connect
Add LLMReady at your CDN or edge, or send logs. It works with any platform. Every AI request is classified the same day.
| Connect with | You get |
|---|---|
| Free scan | The report and the fixes. No connection needed. |
| Server logs | AI traffic analytics, alerts and attribution. |
| CDN or edge | Everything above, plus bot policy, identity checks and agent pages. |
STEP 03
Serve and measure
Crawlers get agent pages that stay current. You get a weekly report on crawl success, refused requests and the visits and sales AI answers send back.
curl -A GPTBot https://acme-supply.example/pricingHTTP/2 200 content-type: text/markdown# Pricing> Acme Supply sells reusable packaging to small food businesses. ## Plans- Starter, $19 a month, one location- Growth, $59 a month, up to five locations llms.txt found Markdown found JSON-LD found
Flow·One request//What happens in a few milliseconds
What happens when a crawler arrives.
01
02
LLMReady identifies the bot
The user agent is checked against the vendor's published IP ranges, so a fake name is caught.
03
Policy decides
Your per-bot rule says allow, limit or block, with a reason that is logged.
04
The right page is served
Allowed bots get the agent page: clean markdown and structured data. People get your normal site.
05
The request is recorded
Bot, page, status, bytes and any AI referral are stored for the report.
FAQ
Setup questions.
01Do I have to change my pages?
No. LLMReady sits at your CDN or edge, or reads your logs. Your templates stay as they are.
02Does my normal site change for visitors?
No. Only crawlers that your policy allows receive the agent version.
03What if I only want the analytics?
Send logs and skip agent pages. Analytics, alerts and the readiness scan work on their own.
04How fast is setup?
The scan needs only an address. Connecting the edge takes minutes, and classification starts the same day.
Beta·Early access
Join the beta and see what AI does with your site.
Leave your email and you will hear when your spot is ready.