What Is Candidate Retrieval? Recall@5 vs. a Model's First Choice

What is candidate retrieval?

Candidate retrieval is the stage that pulls possibly relevant items out of a larger pool of content or tools. A correct tool has to enter the candidate set the model can use before it has any chance of being selected; being retrieved, being the first choice and executing successfully are separate observations.

Retrieval is not the final choice

A system may find candidates with BM25 lexical retrieval, embedding semantic retrieval or a hybrid method, then rerank them or hand them to the model to choose from. Some applications simply load the full tool list and have no separate retrieval stage.

So “not called” cannot be read directly as the model comparing tools and disliking one. First save the candidate list that was actually loaded for the model, then examine the selection.

Define the Recall@5 denominator first

In general information retrieval, recall is the share of relevant items that were retrieved out of all relevant items. Recall@5 checks only the top five results.

If each question has exactly one designated target tool, you can record per question whether it landed in the top five, then compute the share of questions with a hit out of all questions. If a question has several relevant tools, or the denominator switches to all relevant documents, state that separately; do not blend them into the same metric.

Related terms

BM25 · Embedding · Reranking · Tool Selection Optimization (TSO) · Tool Metadata