Preventing hallucinations in support: what helps technically

Key takeaways

• Unregulated language models hallucinate in up to 27 per cent of cases.
• A grounding layer pushes the error rate in support below 5 per cent.
• The AI may only answer from verified internal sources (RAG).
• When knowledge is missing, the system has to hand over to a human straight away.

Hallucinations as a structural problem

Large language models calculate probabilities of word sequences. They do not access an internal database of verified facts; instead they pick the statistically most likely next token on the basis of their training data. For customer support, this mathematical principle is a problem. Language models are optimised to formulate rhetorically convincing answers, regardless of whether the content is factually correct.

Research distinguishes between two primary forms of error: intrinsic and extrinsic hallucinations¹. Intrinsic hallucinations directly contradict the information provided in the context. Extrinsic hallucinations invent new details that do not appear in the source text at all, such as made-up return periods or product features that do not exist. Without technical safeguards, unregulated language models hallucinate in 15% to 30% of cases². System prompts alone do not solve this problem, because they do not override the underlying stochastic principle of the model.

RAG architecture for local knowledge binding

To reduce the error rate in support drastically, the language model has to be decoupled from its public training data. The technical solution for this is retrieval-augmented generation (RAG). In this approach, the general language model is extended by an external, controlled knowledge store. Instead of generating freely, with every request the model is handed exactly those document sections that are relevant for answering it.

The precision of a RAG architecture stands or falls with the quality of the data preparation and of the search mechanism. During the ingestion process, documents are broken down into sections (chunks) and stored as vectors in a database designed for similarity search. IBM names suboptimal chunking as one of the central weak points of classic RAG pipelines: if the cut is made at fixed points, in the middle of a table, retrieval only returns half the table and the accuracy of the answer is lost³. A well-maintained knowledge base with cleanly cut, semantically self-contained chunks is therefore the precondition for a RAG pipeline to lower the hallucination rate compared with unregulated models at all.

1. 01Document ingestion: automatic cleaning and structuring of internal help articles and policies.
2. 02Smart chunking: breaking the texts into semantically self-contained units of information along the headings.
3. 03Vectorisation and retrieval: mathematical matching of the customer request against the stored chunks via cosine similarity.
4. 04Context augmentation: handing the relevant chunks to the prompter as the sole factual basis.

Grounding and mandatory citations

A RAG architecture on its own is not enough in safety-critical areas. Even when the right documents are retrieved from the database, the language model can make mistakes while summarising. That is why a downstream checking layer is needed: the grounding layer. Benchmark studies on groundedness classification show that generated answers can be classified automatically according to whether they are covered by the source material, and that this step can run after generation with acceptable latency.

In the grounding layer, the draft answer is broken down into individual statements. Each statement is analysed by means of natural language inference (NLI) as to whether it can be derived directly from the source document: the source document serves as the premise, the generated partial statement as the hypothesis, and the check decides whether the statement is covered by the source⁴. In addition, the model is required to supply a specific source citation in the Help Centre for every main statement. This combination of automatic re-checking and compulsory sources lowers the actual error rate in live operation to below 5%².

• Sentence-splitting analysis: breaking the AI answer down into atomic individual facts.
• NLI matching: logical checking of every partial statement against the retrieved context chunk.
• Automatic blocking: discarding the draft answer in the event of logical contradictions or unsupported claims.
• Citation injection: linking to the exact knowledge source directly at the end of the answer.

Confidence thresholds and shadow mode

Not every customer request can be answered unambiguously from the existing documents. To avoid misinformation, technical thresholds (confidence thresholds) have to be defined. The system calculates a confidence value for every retrieval and answering step. If that value falls below the defined limit, the automatic answer is stopped and the case is handed over to a human. In production systems, the usual thresholds lie between 80 and 95 per cent, depending on the risk of the use case⁵.

Before going live, setting up a shadow mode is advisable. The AI agent runs quietly alongside the human workflows on real, incoming cases: it recognises the intent, retrieves knowledge, recommends a solution and drafts the customer reply together with its reasoning, but it does not update any case record, does not send any customer communication and does not change any status⁶. Technical decision-makers can thus measure over several weeks how the hit rate behaves under real conditions, and calibrate the thresholds without any risk to customer satisfaction.

The clean fallback: refusing instead of guessing

The most important safety rule for AI in support is: refusing is always cheaper than guessing. An invented answer leads to wrong customer expectations, unnecessary returns or, in the worst case, to legal liability risks. If the context index does not deliver sufficiently relevant data, the system has to refuse the answer and give clear feedback.

This is where the fallback mechanism takes effect. The AI agent hands the conversation over to human support seamlessly and passes on at least a structured package consisting of the complete conversation history with timestamps, the customer data collected, the CRM profile with earlier contacts, the specific reason for escalation and the steps already attempted⁵. That way customers do not have to explain their concern again. Anyone who plans strategically which workflows can be automated in customer service and where human expertise remains necessary creates dependable processes.

• Transparent refusal: a clear message to the user that the information is not available at present.
• Handover true to context: forwarding the complete chat including the chat history to the inbox.
• No endless loop: immediate escalation to real contacts without repeated standard questions.
• Agent dashboard: highlighting the missing knowledge gap for the support team.

Ongoing evaluation in production

Protection against hallucinations does not end when the system goes live. In live operation, the accuracy of the AI has to be monitored continuously. Quantitative metrics lend themselves to this: the refusal rate, the escalation rate to human support and the error rate of unsupported statements (unsupported claim rate). As a guide for handing over to humans, handoff rates of 10 to 20 per cent at a deflection rate of 40 to 70 per cent apply in practice, whereby a high automation rate is only worth something if customer satisfaction remains stable⁵.

A systematic feedback system is just as important. Every refused answer and every piece of negative customer feedback serves as a signal for missing or outdated information in the knowledge store. If support teams evaluate unanswered customer questions in a targeted way and add to the knowledge base weekly, they can reduce the ticket volume and raise the automation rate continuously.

1. 01Random quality checks: manual review of 5% of the automated answers by support leads.
2. 02Evaluation of fallback signals: recording all requests where the AI aborted for lack of confidence.
3. 03Gap identification: categorising unanswered questions in order to extend the documentation in a targeted way.
4. 04Iterative model optimisation: reworking chunks and removing outdated help articles.

A practical example: an AI layer with a strict tie to knowledge

Anyone looking for dependable automation without the effort of an in-house development department can also buy the protective mechanisms described as a finished platform. Three verifiable properties are decisive here: the AI agent answers exclusively on the basis of the company data stored, supplies the exact source with every answer, and refuses to answer if there is not enough information. In that case the conversation goes straight to the shared inbox.

Infrastructure and data storage are designed for European security standards. All content remains in the EU, processing follows the GDPR strictly and without training public AI models on your customer data GDPR-compliant AI. The pricing model is built transparently: alongside a no-cost Free plan for trying things out, ComLayer Pro offers the full range of features for professional support teams from €49 per month including 500 AI answers.

• Strict knowledge isolation: answers come exclusively from your verified knowledge base.
• Automatic source citation: a direct link to the matching help page beneath every answer.
• Safe fallback: seamless handover to the human team in case of uncertainty.
• EU hosting & GDPR: storage and AI processing in European data centres without passing data on.

Frequently asked questions