Lesson 12: Check retrieved evidence before the answer

The model says normal opening on the 18th, and you consider adding check carefully. First inspect its input. A without closure notice D cannot establish that date’s exception, however carefully it is read.

Lay out the input before diagnosing the answer

Preserve the query and actual candidates. Check for D in date questions and intact C conditions in renewal questions. A correct ID with truncated content is still insufficient evidence.

Ask whether the passage addresses this query, contains needed facts, and applies to its object, date, and version. B is time-related but does not establish renewal extension; E concerns quantity but not the current limit.

The same wrong answer can have different causes

With A/B and an ordinary-opening answer, the complete pack reveals missing D. Repair retrieval or manually supply it in the exercise.

With A/D present, an ordinary-opening answer ignores the exception. Inspect evidence use rather than add unrelated borrowing policies.

Identical outputs can require different fixes. This distinction gives a reason for changing prompts or candidate counts.

Not retrieved versus absent from the collection

Deposits differ: neither current candidates nor the full pack establish an amount, so additional sources are needed. Missing D is retrieval insufficiency; missing deposit facts are a coverage gap.

Say the provided evidence cannot determine the deposit, rather than infer no deposit. Even real links require claim-by-claim support checks; they can point to the wrong rule.

For the three situations, record required evidence, candidates, and cause. Then split a cited answer into claims and locate supporting source text for each.

Three diagnoses

A/B for the date: missing D, inspect retrieval. A/D with ordinary hours: evidence present, inspect use. Deposits: missing collection facts, add sources or acknowledge uncertainty.

Further reading: Microsoft: RAG LLM evaluation phase

← Previous · Learning plan · Fictional library practice sources · Next →