Skip to content

Configure safe 'I don't know' fallbacks when knowledge gaps appear

Stop your assistant from guessing smoothly when retrieval scores are weak—teach it to abstain honestly and capture the exact question for help.

Copy the prompt and paste it into the AI assistant you already use.

Defense-in-depth against ungrounded guessing

  1. 01

    Gate LLM generation on reranker thresholds

    If no retrieved chunk clears your calibrated cross-encoder relevance threshold, skip open generation and trigger a structured abstention flow.

  2. 02

    Explicitly forbid pre-trained world assumptions

    Instruct the system prompt to answer product behavior strictly from supplied context—never assuming standard SaaS defaults apply to your app.

  3. 03

    Separate known facts from unverified gaps

    When context covers 70% of a question, share the verified part clearly and state explicitly which detail requires human confirmation.

  4. 04

    Pair every abstention with a productive next step

    Never leave users at a dead-end 'I don't know'—offer immediate specialist escalation or async email follow-up with the question pre-attached.

Questions and answers

Why do LLMs hallucinate answers even when told 'only use provided context'?

Without a pre-generation retrieval score gate, models try hard to be helpful on tangentially related chunks and extrapolate missing steps.

Won't raising abstention thresholds lower our self-service containment rate?

Replacing 8% hallucinated 'resolutions' with honest escalations raises customer trust, cuts repeat angry contacts, and reveals exact doc gaps to fix.

How do we calibrate the right similarity or reranker cutoff score?

Evaluate 150 real historical queries—100 answerable in docs and 50 out-of-kb—and set the threshold where false answers drop near zero.

Learn more

Related guides and background articles for this task