vault / agents / triage.md

Feedback triage

Agents & LLMs 60 words 2 outgoing 4 backlinks

A thumbs-down starts an investigation, not a diagnosis. The workflow I built replays the question, collects evidence, distinguishes a content gap from a configuration issue, and routes the case to an owner. Routing is ordinary code; the model proposes the likely cause. Doubtful cases go to a person rather than turning confidence into authority.

Read the full example: related post.

LINKS IN THIS NOTE

[[ Observability ]] For feedback investigation, record enough of the request to reconstruct what happened: the question, retrieval, response and relevant configuration. A trace helps connect those steps. Dashboards can show a pattern, but a concrete replay is often what tells us which... [[ Human review ]] People approve proposed evaluation records and handle doubtful feedback cases. Keep those jobs explicit: approving a reference answer is different from deciding which owner should investigate an incident. The triage workflow routes missing or conflicting evidence to review instead of...