Analysing handovers to humans: knowledge gap or outage?

Key insights

• Handovers to humans fall into two strictly separate categories: content decisions by the AI, and technical system outages.
• Content handovers arise from knowledge gaps. They form the direct worklist for your Help Centre.
• A technical outage means the AI never read the question. Even an SLA of 99 per cent allows several days of downtime a year and distorts the rate.
• A sudden jump in the handover rate almost always signals a technical limit, not a sudden deterioration in your knowledge.

One rate. Two completely different causes.

When you report the monthly figures to management as a support lead, one central metric almost always ends up on the slide: the handover rate to the team. In practice, mature AI support systems resolve roughly 50 to 76 per cent of standard enquiries on their own. The rest lands in your human colleagues' inbox.

The problem starts right there: most reporting dashboards throw every enquiry that was not completed automatically into a single pot. Yet a handover rate of 35 or 45 per cent says nothing in isolation about why the agent did not answer. It lumps together two events that have nothing to do with each other operationally.

Two classes. No common denominator.

Every handover to a human necessarily belongs to one of two classes. Either the AI read the question, understood it and deliberately decided to hand over because the stored knowledge is missing. Or the request failed technically before a language model had processed a single word. Anyone mixing both causes into one metric draws the wrong conclusions for staffing and documentation.

If your handover rate climbs noticeably month on month, there are two fundamental causes: either customers are asking new questions for which the Help Centre has no entry at all, or a technical limit is blocking processing. An undifferentiated rate hides that difference completely.

The content handover. Your documented knowledge gap.

A content handover is not a technical fault. It is the intended behaviour of a properly configured RAG system (Retrieval-Augmented Generation). The agent receives the customer enquiry, searches your knowledge base for relevant passages and calculates a confidence value. If the system finds no solid evidence, the AI refuses to invent answers freely.

No guessing. A deliberate handover.

Rather than making a false statement, the agent passes the enquiry on to the human inbox. This is exactly where a properly defined handover moment matters: the customer gets a transparent notice, and your team receives the ticket with its context attached.

In practice it turns out that, as long as the technical infrastructure runs stably, content escalations come down to missing documents or unclear wording in the knowledge base. That is an excellent starting position operationally: every one of these content handovers is a concrete work order for your knowledge base.

• Question understood: the customer's intent is captured semantically.
• Vector search with no hit: no matching section exists in the Help Centre.
• Threshold applies: the calculated relevance falls below the safety minimum.
• Orderly handover: the ticket lands in the team's inbox in a structured form.

Treating these cases as a failure of automation misreads the system. A content handover protects your brand from hallucinations and supplies the raw data for expanding your Help Centre in a targeted way.

The technical outage. When the AI does not read at all.

A technical outage differs fundamentally from a knowledge gap: at no point did the AI examine the customer's question on its merits. There was no comparison against your documentation, and no model assessed the question. The request breaks off at the infrastructure, at administrative barriers or at provider limits.

What matters is that this class is finite and nameable. In practice there are six cases in which the AI never read the question: the agent is not set up at all. The plan booked does not include it. The daily budget for answers is exhausted. The request is too large, because an attachment exceeds the model's limit, for instance. The model returns an empty answer. Or the model request fails with a technical error. Three of these are not a malfunction but a setting — plan, budget and setup are decided by your own company, and it is precisely those three that produce the most inconspicuous outages, because nothing turns red.

Availability in figures. The reality of APIs.

Many support teams assume permanent availability of all cloud services. A look at standard service level agreements (SLAs) shows the mathematical reality: at a guaranteed availability of 99 per cent (two nines), the permissible downtime works out at 3 days and 15 hours a year, or 7 hours and 12 minutes a month¹. Even at 99.9 per cent availability, the permissible downtime adds up to 8 hours and 45 minutes a year, so around 43 minutes a month¹.

If an LLM gateway or an authentication service is unreachable for thirty minutes, every customer enquiry in that window fails. They drop straight into your inbox as an emergency fallback. Those handovers say absolutely nothing about how complete your help articles are.

Telling the two classes apart. A look at the logs.

In the inbox both events look identical at first: an unresolved ticket waiting for human attention. To deliver solid figures in your reports, you have to evaluate the system records.

The handover reason carries the distinction itself.

The clean solution lies not in evaluating HTTP codes, but in the handover reason that the system records anyway. Every handover has a reason in text form, and the reasons of the second class all begin with the same phrase — in effect: AI unavailable, followed by the cause in brackets. The knowledge-gap analysis excludes exactly those rows by testing for that opening phrase.

That sounds like a detour compared with a dedicated column in the database, but it is the more robust route: an additional column would have to be set by every future place that writes a handover — and would be silently wrong the moment someone forgets. The marker inside the reason itself cannot be forgotten, because the reason does not come into being without it.

The price of this solution belongs in the picture too: the opening phrase is user-facing text and marker at the same time. Anyone rewording it changes an analysis along with it, without noticing. So anyone rebuilding this should test that coupling explicitly, rather than documenting it and hoping.

And the reason the exclusion exists at all is no statistical nicety: a malfunction listed as a knowledge gap sends someone off to write a help article against a server outage. The question standing there was never asked — the AI never read it. The work is then not merely wasted, it also dilutes the very list your team is supposed to be working from.

The rate jumps. What you check first.

Monday morning, the weekly dashboard opens: over the weekend the handover rate has shot up from the usual 25 per cent to 60 per cent. In situations like that one rule holds: always check the technology first, never the knowledge base.

Jumps are technical. Drifts are about content.

Your customers' behaviour generally changes continuously. New product features or seasonal campaigns lead to a gradual rise in unanswered questions over days or weeks. A sudden, steep jump in the curve within a few hours is almost invariably an administrative or technical event.

• Step 1: check the status pages of the interfaces and LLM providers.
• Step 2: verify the validity of API tokens, webhooks and authentication keys.
• Step 3: check monthly allowances used up and any active cost limits.
• Step 4: only once technical faults are ruled out, examine the chat transcripts for clusters of content.

Anyone who immediately rewrites help texts after a sudden jump in the rate wastes working time while, in the background, an expired API key or a server error goes undetected.

Analysing knowledge gaps. The worklist for your team.

Once the technical outages have been filtered out, the remaining content handover rate becomes the most precise steering instrument your support has. Each of these tickets documents a real customer question for which your company has so far provided no verified answer.

From ticket to help article. The iterative loop.

Instead of writing documentation on a hunch, you analyse the clusters of unanswered questions. That is exactly how knowledge gaps in the Help Centre can be uncovered and closed systematically. A single new help article on a much-asked topic lowers the handover rate permanently for every future conversation.

1. 01Build clusters: group frequent terms and intents in the unresolved tickets, automatically or by hand.
2. 02Write articles: store short, precise answers directly in the Help Centre.
3. 03Check indexing: make sure the new content has been taken into the vector database.
4. 04Verify the effect: check whether the handover rate for that specific topic drops to zero.

This loop makes your team measurably more productive. Working time no longer goes into answering identical enquiries manually over and over, but into sustainably expanding the central knowledge base.

Analysing outages. Checking limits and budgets.

If the log analysis shows instead that the rise in handovers rests on technical failures, the solution lies not in editorial work but in plan and budget configuration. The most common cause in live operation is hard cost limits or exhausted included volumes.

Understanding consumption. Plans and answer volumes.

Modern support platforms rely on transparent, usage-based billing. The Comlayer Pro plan, for instance, includes 500 AI answers a month (each further answer costs 0.10 euros). If a company sets a rigid limit at exactly 500 answers to guard against budget overruns, the system stops every automatic answer once the limit is reached. All subsequent enquiries are passed through to the team as technical handovers, unexamined.

Ignoring this connection costs you in the end: the supposed saving on software costs is wiped out by expensive manual working time in the support team. It pays here to understand what drives usage-based costs, and from what monthly ticket volume the switch to the Scale plan (with 5,000 answers included and 0.08 euros per further answer) becomes the more economical choice.

The next time you present support metrics, keep the two pillars strictly apart: the content rate shows the maturity of your knowledge base, the technical rate the stability of your infrastructure. That gives leadership a solid basis for decisions instead of misleading aggregate figures.

Frequently asked questions