vault / agents / groundedness.md

Groundedness

Agents & LLMs 50 words 2 outgoing 3 backlinks

Check whether the answer’s claims are supported by the retrieved sources. This differs from usefulness: an answer can stay within its evidence and still fail to answer the question. Read groundedness alongside relevance and retrieval recall, rather than reducing all three to one score.

Read the full example: related post.

LINKS IN THIS NOTE

[[ RAG ]] Retrieval-augmented generation searches a document collection, then writes an answer using the retrieved context. Keep retrieval and answer quality separate: finding the right document does not guarantee the model uses it, and a plausible answer does not prove the document... [[ Retrieval recall ]] Ask whether the expected sources reached the retrieved context. When the answer is wrong, this helps distinguish missing evidence from poor use of available evidence. The expected source set must itself be checked; a score against an incomplete reference can...