What your website needs to appear in AI answers

Can AI crawlers reach your pages and read them?

AI bots must be allowed in your robots.txt, and the content must be present in the raw HTML the server returns. Both halves matter: an allowed crawler that receives an empty JavaScript shell is as blind as a blocked one. In the December 2024 Vercel and MERJ measurement, GPTBot, ClaudeBot and PerplexityBot did not execute JavaScript. Googlebot, whose infrastructure Gemini also uses, and Applebot rendered pages; Bing says Bingbot can render JavaScript with limits.

This is why our audits score every page on two bases: what a browser renders, and what a raw fetch returns. The two scores can be far apart on the same page, and the raw-fetch score is the one most AI systems act on. Content that only appears after client-side rendering, such as product data loaded by a script or FAQ text injected on scroll, does not exist for those crawlers.

• →robots.txt: allow the AI crawlers you want to be read by (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, OAI-SearchBot). A blanket block here removes you from the sources AI answers are built from.
• →Raw HTML: view the page source (not the developer-tools DOM) and search for the sentences you want quoted. If they are missing there, most AI fetchers never see them.

Does your structured data describe what is actually on the page?

Three blocks of JSON-LD do the work: an Organization with a real entity graph, a FAQPage that mirrors the questions visible on the page 1:1, and a page-level type that matches the page's role. Schema does not buy citations on its own; what it does is make your pages and entities unambiguous to the crawlers and search indexes AI answers are built from.

• →Organization with an entity graph: define it once, give it an @id, and point sameAs at profiles that confirm the entity, such as Wikidata and LinkedIn. Every other page references that @id instead of redefining the company.
• →FAQPage that mirrors visible FAQs: same questions, same wording as the page shows. Schema-only FAQs that appear nowhere on the visible page add markup without adding quotable content.
• →Types that match page roles: Article for guides, Service for service pages, Product for products. A page marked up as something it is not reads as noise.

Do your pages answer the question in the first sentences?

A direct answer makes a passage easier to extract, but does not determine whether an AI system will cite it. Put the definition or the answer in the first one or two sentences under each heading, and phrase headings as the questions buyers actually ask. A heading that asks "How long does an audit take?" followed by "Two weeks from kickoff to delivery" is quotable; three paragraphs of context before the number is not.

• →Questions as headings: real buyer questions, in their words, one per section.
• →Answer first, detail second: the direct answer opens the section, the supporting explanation follows it.
• →Specifics over adjectives: supported numbers, dates and named things make an answer more precise and easier to check.

Is it unambiguous who is behind the site?

Your site is unambiguous when it uses one brand name and one description across the site, points its schema's sameAs at real external profiles such as Wikidata and LinkedIn, and has an about page that plainly says who is behind the site, what the company does and where it operates. AI systems assemble brands from many sources, and inconsistency splits your identity into fragments.

• →One name, one description: the same entity wording on the homepage, the about page, and in the schema.
• →sameAs to real profiles: Wikidata and LinkedIn entries that exist and match, so external systems can confirm the entity.
• →An honest about page: named people, real credentials, a real address. Named people and verifiable company details help readers and systems understand who is responsible for the content.

Do your date signals agree with each other, and with reality?

A page carries its age in three places: the visible date on the page, the article:modified_time meta tag, and the lastmod entry in the sitemap. They should show the same date, and that date should be true. Matching date signals describe when a page was updated; they do not establish accuracy or guarantee a citation.

• →Visible dates: a published or updated line readers can see, on every content page.
• →article:modified_time: present on article pages and equal to the visible date.
• →Sitemap lastmod: bumped when the content actually changes, and only then. Mass-bumping every page to today is a false signal, and it is detectable.

Do you need llms.txt?

No major AI engine reads llms.txt for search or citation today, and Google has confirmed it does not use the file. We ship one on our own site because it costs minutes and does no harm, and that is the whole case for it: a cheap optional extra, never a lever. If a vendor's pitch centers on llms.txt, the pitch is built on a file the engines ignore.

Its plausible future is AI-agent browsing rather than search. Worth a text file, never worth a budget line. Spend the effort on the five requirements above, which are read today.

Common questions

Keep reading: GEO vs SEO explains how these requirements relate to the SEO work you already run, and how it works covers the measurement method behind our audits.