Lesson 03: Why models need external information

The borrowing limit changes from two to three, yet the model answers two. Correcting one conversation may not make the next one aware. It is tempting to put the new rule into training.

Many question-answering tasks can take a more direct route: supply the relevant rule before answering. That is the idea behind RAG.

It may not know the document you have

Training data have a time range and do not include everyone’s internal documents. A new school notice, your notes, or an unpublished team guide cannot be assumed known. Familiarity with libraries does not establish this library’s current policy.

Often we need that specific answer. Current limits need B; opening on a particular date needs an applicable notice. General knowledge can sound plausible while failing the query.

One temporary notice changes the answer

A gives Sunday hours of 09:00–17:00. Alone, it suggests ordinary opening on 18 October. D explicitly announces all-day closure on 18 October 2026 for maintenance. A question about that date requires the notice.

Supplying A and D lets the model answer from available evidence. Contextual documents do not mean its parameters have been updated; another conversation may need the application to supply them again.

RAG adds selection: retrieve relevant passages, then provide them with the question. Updates usually involve maintaining documents and their searchable representations rather than retraining the language model each time.

External evidence still has limits

If D exists but is not retrieved, investigate search. If no document specifies a deposit amount, a different query cannot extract it from this collection. These failures need different next steps.

Sources can be wrong or outdated, and the model can misread them. Check the version and supporting statement after supplying evidence. An open-book answer can still use the wrong page or miss a condition.

Write three evidence-dependent questions: one current policy, one about your notes, and one outside the pack’s coverage. Identify a source for each; this clarifies what belongs in a knowledge base.

An example to compare

Current quantity needs B; opening on the 18th needs D; deposit amount requires additional authoritative evidence. The missing amount does not establish that no deposit is required.

Further reading: Microsoft: RAG and indexes

← Previous · Learning plan · Fictional library practice sources · Next →