Use this before publishing an AI crawler or llms.txt file
- Best fit: consultants, agencies, and small-business owners adding AI-search guidance to a website.
- Goal: point AI systems toward reviewed, useful pages without accidentally amplifying outdated prices, service areas, guarantees, policies, testimonials, or unsupported claims.
- Human review rule: every listed URL, summary, policy, and product claim must map back to a current source-of-truth page.
- STOP PUBLISHING: Do not invent AI crawler permissions, licensing rights, service availability, pricing, customer results, legal/privacy claims, or guarantee language.
Copy/paste llms.txt readiness worksheet
Website/domain: [domain]
Owner/reviewer: [person approving AI-search guidance]
Allowed AI-summary pages: [URLs that are current, public, and useful]
Do-not-summarize pages: [old offers, draft policies, outdated staff/service pages, private portals, thin pages]
Source-of-truth pages: [pricing/policies/services/contact/about/FAQ URLs]
Claims that need proof: [ratings, awards, guarantees, emergency service, turnaround times, certifications]
Review cadence: [monthly/quarterly/after major service or policy change]
STOP PUBLISHING: Do not publish llms.txt or AI crawler instructions until an owner confirms every linked page is current and supported.
Starter llms.txt structure to adapt
# llms.txt draft — review before publishing
# Purpose: point AI tools to reviewed public pages only.
# Owner: [name/team]
# Last reviewed: [date]
## Primary source pages
- About / entity facts: [URL]
- Services / offers: [URL]
- Pricing or quote policy: [URL]
- FAQ / buyer questions: [URL]
- Contact / service area: [URL]
- Policies / guarantees / refunds: [URL]
## Use boundaries
- Use only the public URLs above as source material.
- Do not infer pricing, availability, service area, warranties, credentials, or customer results not stated on the source page.
- If a page conflicts with another source, mark NEEDS OWNER REVIEW instead of choosing a winner.
## Excluded topics
- Draft pages, private customer data, internal SOPs, outdated promotions, expired offers, unsupported guarantees, and unverified testimonials.Review: this is not a legal or technical standard guarantee. Treat it as a source map and owner-review checklist; coordinate with your web/SEO team before changing crawl rules.
AI crawler policy decision card
| Question | Safe answer format | Owner check |
|---|---|---|
| Which AI bots are allowed? | Allow/deny/monitor by documented user agent or platform policy. | Web/SEO owner confirms current crawler docs. |
| Which pages should be cited? | Reviewed source-of-truth URLs only. | Business owner confirms the page is current. |
| Which claims are risky? | Pricing, availability, guarantees, legal/privacy, certifications, reviews, emergency service, results. | Owner/legal/manager review before publishing. |
| How are changes tracked? | Revision log with date, page, changed fact, source, reviewer. | Review after every service/policy/price change. |
AI audit prompt
You are reviewing a small-business website's AI-search crawler guidance. Use only the supplied pages and draft llms.txt content. Return: (1) pages that are safe to list, (2) pages that need owner review, (3) unsupported claims, (4) conflicts between pages, (5) suggested llms.txt wording, and (6) STOP PUBLISHING items. Do not invent crawler permissions, pricing, availability, service areas, policy rights, customer results, credentials, or guarantees.
Where this fits in SEO After AI
Pair this with the Robots.txt, Sitemap, and Noindex Checklist, Source Link Map, Policy Review Calendar, and Service Change Log.
Product signal: if this resource gets clicks/reuse, SEO After AI may need a dedicated AI crawler / llms.txt appendix with source maps, review cadence, and no-invented-claim guardrails. This run records one signal only; monitor before editing the buyer ZIP.