Help Centre in AI answer engines: how to get cited

Key takeaways

• If you want citations in AI search engines, you have to allow crawlers such as OAI-SearchBot and PerplexityBot explicitly in your robots.txt.
• FAQPage schema gives AI models structured facts they can cite directly, without lengthy interpretation.
• Visibility in generative AI needs continuous monitoring, because probabilistic answers fluctuate from day to day.

Mechanics: how AI search engines retrieve support knowledge

Classic search engines and generative answer engines process support content in fundamentally different ways. Where a conventional web index weights pages by relevance and lists blue links, systems such as Perplexity, ChatGPT Search or Google AI Overviews use a two-stage procedure: retrieval-augmented generation (RAG). When someone asks a question, they search the web in real time for matching passages, extract the relevant facts and formulate a coherent answer from them.

Two completely separate data paths exist in modern language models, and support teams need to keep them apart:

• Training data: these static datasets are fed in while the model is trained. They age quickly, hold no up-to-date software releases and produce answers without verifiable source citations.
• Live retrieval: for concrete support or product questions, AI search engines fetch pages through a live web search. OpenAI separates the two paths explicitly: GPTBot collects training data, while OAI-SearchBot records pages for ChatGPT search, and only a site that allows this bot appears in the search answers¹.

If you want to be cited in AI answers, you do not have to wait for the next model update. What decides it is technical accessibility for web crawlers and an architecture that makes facts immediately readable for algorithms.

Opening the crawler gates: robots.txt for AI bots

Access to your Help Centre is controlled in the central robots.txt file. Many organisations block AI crawlers across the board out of concern about model training. The effect, though, is that search bots are locked out as well and the brand stays completely invisible in generative answers.

OpenAI and other providers separate their user agents by purpose. A look at the server configuration decides between reach and exclusion:

One technical reality belongs in the picture as well: according to OpenAI's crawler documentation, the robots.txt rules may not apply to the ChatGPT-User agent, because the fetch is triggered directly by a person in the chat². Inclusion in ChatGPT search, by contrast, is decided by OAI-SearchBot, which is why allowing it (and PerplexityBot) in robots.txt remains mandatory.

llms.txt: the table of contents for AI agents

Alongside robots.txt, a new standard is establishing itself: the llms.txt file in a domain's root directory. Where robots.txt defines who may access a site, llms.txt tells autonomous systems where the most important information is stored in structured form. The goal is a lean Markdown version of the documentation, without CSS, JavaScript or nested HTML markup.

A clean llms.txt saves tokens in the model's context window and speeds up processing. Even so, a sober look at its current benefit is worthwhile:

• Primary use: at present it is mainly developer tools, IDE agents (such as Cursor or Windsurf) and specialised RAG pipelines that process llms.txt.
• Large search engines: crawlers such as Googlebot or OAI-SearchBot still use the regular HTML DOM and structured markup for citations, not llms.txt.
• Effort and return: an llms.txt can be generated automatically in a short time. For technical SaaS products it is solid groundwork, but it does not replace a semantic web structure.

If you run technical documentation or interface references, provide an llms.txt as a signpost. For general visibility in ChatGPT or Google, the HTML document itself remains the primary source.

Structured data: FAQPage schema for citations

AI crawlers do not scan web pages the way human readers do. They break HTML into chunks and calculate statistical relevance. Pages with valid JSON-LD markup offer algorithms a semantic shortcut, because relations between entities and answers are recognisable without guesswork.

The empirical GEO-16 study on the citation practice of AI answer engines bears this out: 1,702 citations from three answer engines were analysed, and signals for metadata and recency, semantic HTML and structured data showed the strongest associations with a citation³. Validated markup such as FAQPage or TechArticle raises the retrieval algorithm's confidence in the accuracy of the content.

Many support teams make the mistake of bundling dozens of questions into endless accordions on a single overview page. For language models, that structure dilutes the context:

• One focus per URL: one article per specific question delivers the highest confidence. The algorithm assigns the answer to exactly one problem.
• A clear FAQPage schema: the JSON-LD in the HTML header has to match the visible text exactly. Contradictions between schema and content lead to devaluation.
• A clean H2 hierarchy: headings belong in the form of concrete questions, not as abstract feature labels.

Structured data reduces ambiguity. It is the foundation on which extraction models build reliable AI answers.

Front-loading: the answer in the first 30 per cent

Classic marketing copywriting rewards long dwell time and narrative arcs. Introductions often open with a description of the problem, and the solution follows several paragraphs later. For AI search engines this structure is dysfunctional. Extraction algorithms work through sections of text linearly and often stop evaluating early.

Research shows a clear pattern in how citations are awarded: 44.2% of ChatGPT citations and 55% of citations in Google AI Overviews come from the first 30% of a document⁴. After the first third, the curve falls steeply. Hide your key statement deep in the text and you lose more than half of your chances of being cited.

From this follows a binding structure for help articles:

• The first 150 to 200 words: the precise definition or the direct answer goes right at the top. No preamble, no greeting.
• Structure after the answer: step-by-step instructions, exceptions and further detail follow only below the core solution.
• Self-contained paragraphs: every subsection delivers a statement that stands on its own and can be extracted as a citation snippet.

Measuring visibility: share of voice instead of traffic

Traditional web analytics reach their limits with generative answer engines. AI systems often deliver answers directly in the interface, without users clicking the source link. Traffic alone no longer reliably measures how a Help Centre performs in AI search.

On top of that comes the probabilistic nature of language models: a single manual prompt is not a statistical statement. Answers vary with phrasing, time of day and personalisation. Professional monitoring relies on standardised measurement methods:

• Fixed prompt sets: define a catalogue of 50 to 100 typical customer questions for your product.
• Regular queries: test the same prompts automatically at fixed intervals across ChatGPT Search, Perplexity and Google AI Overviews.
• Share of voice: measure the percentage of answers in which your Help Centre is linked as a source, in direct comparison with competitors.
• Context check: verify whether your brand is cited as the correct solution or whether outdated instructions are being reproduced.

Systematic monitoring uncovers knowledge gaps and shows where you need to build your Help Centre to be named as a primary source.

Your knowledge for customers and AI alike

A modern Help Centre does two jobs at once today: it gives customers an understandable point of contact and supplies AI crawlers with machine-readable knowledge. Maintaining content twice, once for people and once for machines, creates redundancy and stale data.

ComLayer resolves that split through an integrated Help Centre on its own subdomain (such as help.yourshop.com). The same structured articles that external answer engines crawl serve at the same time as the exclusive knowledge base for the internal AI support agent. Maintained once, in sync at every level.

Three transparent plans cover different team sizes:

• Free: EUR 0 per month. Support widget, shared inbox and Help Centre under the ComLayer domain for two seats, as a free way in.
• Pro: EUR 49 per month plus EUR 12 per seat. Your own Help Centre domain, unlimited knowledge sources and 500 AI answers included.
• Scale: EUR 199 per month plus EUR 10 per seat. For high ticket volumes with 5,000 AI answers included and all Pro features.

Clean structure. Fast access. No compromises on data security.

Frequently asked questions