Skip to content

Separate genuine self-service resolution from frustrated chat abandonment

Stop celebrating an 80% 'deflection rate' when half those users quietly gave up and emailed support ten minutes later.

Copy the prompt and paste it into the AI assistant you already use.

How to calculate honest Verified Resolution Rate

  1. 01

    Clean the denominator of 0-intent bounces

    Exclude widget opens with zero substantive user questions or pure 'Hi' abandonments so containment reflects real support attempts.

  2. 02

    Subtract 72-hour cross-channel recontacts

    Join chat user IDs against Zendesk email, phone, and in-app tickets over the following 72 hours on the same topic to expose false deflection.

  3. 03

    Score implicit resolution on unrated transcripts

    Since under 15% of users click thumbs-up buttons, run an automated transcript evaluator checking whether the user's goal actually completed.

  4. 04

    Report Automated Resolution alongside Escalation Quality

    Some intents (like enterprise contract changes) *should* escalate—grade those sessions on clean triage speed rather than penalizing containment.

How it works in practice

Vendor dashboard vs audited reality (10,000 sessions)

Vendor headline metric: '76% Contained!' (7,600 chats never clicked Transfer to Agent). Yet support email volume dropped only 18%.

Audited resolution breakdown

• 1,400 Non-substantive bounces (excluded from denominator) • Of 8,600 real support chats: - 3,780 Verified Resolved (44.0% True Resolution) - 1,620 Silent Abandonment + Emailed within 48h (18.8% False Deflection) - 800 Dropped mid-troubleshooting (9.3% Friction Drop) - 2,400 Escalated in-chat (27.9%)

Tracking 72-hour email recontact exposes the 18.8% 'false deflection' bucket hiding broken workflows.

Questions and answers

Why do so many chatbot vendors report inflated 75–85% containment numbers?

Counting every chat that ended without an explicit handoff click treats frustrated tab-closures and one-word bounces as 'successful resolutions.'

Is 100% chatbot containment ever a good operational goal?

Never—pushing containment past natural policy boundaries traps customers with complex edge cases and tanks retention and NPS.

How should outcome-based AI pricing vendors ('pay per resolution') be audited?

Verify contractual resolution definitions require substantive answers, zero negative feedback, and zero human recontact within a 48–72 hour window.

Learn more

Related guides and background articles for this task