Knowledge base for AI support: structure and mistakes
A guide to building and maintaining a knowledge base for AI support, to avoid the typical data errors and secure precise answers.
Martin Semmele
Key insights
- Without checked data, AI systems hallucinate in up to 52 per cent of cases.
- Structured chunks and clean metadata raise match accuracy by 5 to 10 percentage points.
- Fragmented data structures in businesses often make AI projects fail.
- Help articles maintained once serve as a fixed knowledge base for customers and AI alike.
The bottleneck: why AI projects fail on knowledge
An AI in support is only as good as the documentation it accesses. When the AI agent hallucinates wrong answers or finds no solution to simple questions, that is rarely down to the language model chosen. The cause is almost always an unmaintained or unstructured body of data.
In a support context, large language models mostly work via retrieval-augmented generation (RAG). On a customer request, the system first searches your knowledge base for matching blocks of text and passes them to the model as context. If clean facts are missing there, or if the documents hold outdated details, the AI inevitably produces errors. Unchecked company data is therefore one of the main causes of faulty answers in day-to-day operation.
- Incomplete documentation: routine questions have to be answered by human agents because articles are missing.
- Outdated information: the AI quotes old prices or superseded delivery times that no longer apply.
- Unstructured running text: important details get lost in long paragraphs without clear headings.
So before you embed an AI widget on your website, you have to tidy up your knowledge. An AI agent does not replace missing documents, it makes gaps in your knowledge immediately visible to your customers.
The structure: how content is cut for an AI
Throwing a hundred-page PDF guide or unsorted notes into the knowledge base does not work. When indexing, AI systems break texts into smaller sections, so-called chunks. If that text is confusing or too long, match accuracy suffers.
Optimise your help articles by one clear rule: one topic per article, short paragraphs and unambiguous headings. A Snowflake study on the performance of RAG architectures shows that structured, heading-based chunking methods (markdown-aware chunking) raise the AI's match accuracy by 5 to 10 percentage points compared with unstructured running text1.
| Criterion | Unsuitable for AI | Optimal for AI |
|---|---|---|
| Length | Long omnibus articles covering many topics | Focused articles on exactly one concern |
| Formatting | Continuous running text without any bold | Clear paragraphs, bullet points and H3 headings |
| Metadata | No details on validity or category | Unambiguous titles, tagging and update date |
Good articles are not only a data source for the AI, they also help human users in the Help Centre. A cleanly formatted help article resolves questions on two routes at once.
Resolving contradictions: the single source of truth
In many businesses, knowledge exists in fragments in different places. The e-commerce team keeps notes in Notion, support works with internal macros, and the website carries outdated FAQs. When two sources deliver different answers, the AI decides at random or combines contradictory details.
For reliable AI support you need one single binding source (single source of truth). The AI may only access approved documents that are maintained centrally. An example: if the Help Centre states a return period of 14 days but the internal handbook says 30 days, the agent will confuse customers.
- 01Step 1: list all existing sources and identify the core information free of duplicates.
- 02Step 2: clean up contradictions between marketing, support and product management.
- 03Step 3: delete or archive outdated documents instead of indexing them along with the rest.
- 04Step 4: connect the AI agent exclusively to the checked Help Centre.
The AI reads your documents literally. When different terms are used for the same process, the recognition rate drops. Consistent terms prevent misunderstandings.
Maintenance rhythm: when and how updating happens
A knowledge base is not a one-off project. As soon as your team introduces new products, adjusts prices or changes processes, the documentation has to follow. Without fixed maintenance rhythms, the knowledge goes out of date within a few months.
Assign a fixed team member as owner to every help article. Define fixed review intervals for critical topics such as billing, shipping conditions and returns. In practice, experience shows that documents without a responsible person are not maintained.
The update process should be coupled directly to your product release cycle. As soon as a feature changes, the corresponding page in the knowledge base has to be updated before the rollout.
Typical mistakes at the start
Many support teams start with expectations that are too high and upload hundreds of old files unchecked. The result is imprecise answers, high error rates and disappointed customers.
The RAND Corporation reports that by some estimates more than 80 per cent of all AI projects fail, twice as often as IT projects without AI2. As one of the five main reasons, the authors name missing or insufficient data that a model could work with sensibly in the first place.
- Unchecked importing: making hundreds of outdated PDFs or internal documents available to the AI unseen.
- Missing source citations: allowing answers that hold no precise reference to the original article.
- No fallback to people: when the AI does not know the answer, the conversation breaks off instead of handing over to the team.
- Ignoring special cases: trying to automate complex individual cases with AI instead of only catching standard questions.
Better to begin with the 20 most frequent support questions and extend the knowledge step by step. A small, perfect body of data beats a huge, unmaintained collection of material.
Spotting gaps in your knowledge and closing them
An AI agent does not only answer customer questions, it also exposes weak points in your documentation. When customers ask questions for which the Help Centre holds no answer, the system should record those gaps transparently.
Instead of making imprecise guesses, the AI has to hand unclear cases over to your team straight away. In that case, modern inboxes automatically generate an AI draft reply with source citations that your staff can check, add to and send.
- Evaluate unanswered questions: check regularly which topics the AI found no matching source for.
- Write new articles: create new help articles straight from the unresolved tickets.
- Measure answer quality: evaluate user feedback (Helpful / Not helpful) directly on the articles.
That creates a closed loop: every unresolved ticket leads to a new help article, so the AI answers correctly by itself next time.
From theory into practice
With a cleanly structured knowledge base and clearly defined handover rules, AI support turns from an experiment into a reliable channel. Well-maintained systems catch recurring routine questions and keep your support agents free for complex cases.
This is exactly where ComLayer comes in. The platform connects Help Centre, widget and shared inbox on one shared body of data. The AI agent answers strictly from the knowledge sources you have added and supplies the exact source with every answer.
- Free: base version for small projects with support widget, Help Centre and 5 knowledge sources.
- Pro: includes the AI agent, your own Help Centre domain and unlimited knowledge sources.
- Scale: for high ticket volumes with extended included allowances for AI answers.
Find out more about transparent ComLayer plans or read the details on privacy and security with EU hosting in Hamburg.
Frequently asked questions
Why do AI agents hallucinate in customer support?
AI models hallucinate when they access outdated, contradictory or unchecked documents. A study on RAG systems showed that unchecked base data led to invented answers in 52 per cent of cases. The solution is a strictly maintained knowledge base with a clear structure.
How do articles have to be structured for the AI?
Content should be precise, unambiguous and well enriched with metadata. Long running text without a clear structure makes matching harder. Structured, heading-based chunks raise match accuracy by 5 to 10 percentage points over unstructured running text. Each article should cover only one specific topic.
What happens when the database holds contradictory information?
When terms are defined differently in different departments, the AI combines them into a faulty answer. Because knowledge in many businesses lies scattered across notes, macros and old FAQs, a uniform definition is strictly necessary. A term may carry only one meaning.
How often does the knowledge base have to be maintained?
Maintenance is an ongoing process. Documents need a set expiry date or clear responsibilities. Outdated knowledge means the AI answers fast, but with the wrong content. Unanswered questions in the support inbox expose gaps and show where new knowledge is missing.
Why do so many AI projects in support fail?
Many projects start successfully as a pilot but fail in productive use. That is often down to poor data quality and missing maintenance processes once the system is scaled. The RAND Corporation reports that by some estimates more than 80 per cent of all AI projects fail, twice as often as IT projects without AI.
Can I use my existing PDFs and help articles?
Yes, existing articles and PDFs are the ideal basis. They do have to be checked for contradictions and currency beforehand. The same articles that customers read in the public Help Centre serve the AI agent as the direct source for its answers. That way the system catches a large part of the routine requests without human involvement.