vault / agents / retrieval-recall.md

Retrieval recall

Agents & LLMs 47 words 2 outgoing 3 backlinks

Ask whether the expected sources reached the retrieved context. When the answer is wrong, this helps distinguish missing evidence from poor use of available evidence. The expected source set must itself be checked; a score against an incomplete reference can mislead.

Read the full example: related post.

LINKS IN THIS NOTE

[[ RAG ]] Retrieval-augmented generation searches a document collection, then writes an answer using the retrieved context. Keep retrieval and answer quality separate: finding the right document does not guarantee the model uses it, and a plausible answer does not prove the document... [[ Groundedness ]] Check whether the answer’s claims are supported by the retrieved sources. This differs from usefulness: an answer can stay within its evidence and still fail to answer the question. Read groundedness alongside relevance and retrieval recall, rather than reducing all...