Why does AI pick this tool? Understanding TSO through information retrieval

When AI starts booking restaurants and flights for you, who decides which app gets opened?

Picture a scene everyone knows. This Friday evening you want dinner with three friends in Taipei, so you tell the AI assistant on your phone: “Find me an Italian restaurant in Daan District for four people this Friday at 7 p.m., within NT$1,500 per person. If there is a table, book it directly, add the time to my calendar, then message my friends.”

Doing this yourself used to mean opening Google Maps to read reviews, switching to a booking site to check availability, typing the event into your calendar, and finally opening a messaging app to send the link. For an AI agent that can use function calling, it is one continuous automated workflow: the system parses your intent, looks for tools that can query restaurants and open tables, and after booking calls the calendar and messaging apps in turn. The catch is that a large language model (LLM) is only a brain that is good at understanding language and reasoning. From its parameters alone it cannot know which restaurant still has a table tonight, and it cannot write to your calendar or charge a deposit to your card. Every action that touches the real world depends on external tools.

The Model Context Protocol (MCP), which has drawn wide attention in recent years, OpenAI's function calling, and the many plugin and API integrations are all, at heart, ways of giving this brain senses and limbs. Three often-confused concepts are worth separating first. Function calling is the mechanism by which a model and an application exchange tool-call instructions. MCP is an open protocol that lets external tools and data sources plug into AI in a standardized format. An agent is the complete system that breaks a task down by the user's goal, calls tools, and decides the next step from the results.

Once the infrastructure matures, a new question appears. Suppose your AI assistant is connected to five restaurant-booking services, eight flight-comparison tools and a dozen or so travel-itinerary platforms; across the whole market, hundreds of near-identical APIs support MCP. When you say “book me a restaurant” or “buy me a flight to Tokyo next week,” by what logic does the AI put one service on its candidate list and press the booking button for you?

For the past twenty years, competition among software products has centered on human attention. From search engine rankings (SEO) and App Store charts to interface conversion rate design, companies have tried every way to persuade people to click their product. As users get used to stating only the result and not the process, handing execution to AI agents, the first-line chooser of software services shifts from the human to the model and the retrieval system behind it. Future competition will happen not only on the results pages people can see but also inside the AI's own “tool candidate list.” To understand this new game, you first need to understand how AI learns to choose tools.

From 5 tools to 16,000 APIs: how does AI find a tool through a “two-stage funnel”?

Language models do not come knowing how to use external tools. In 2023 a Meta AI team published Toolformer at NeurIPS, the first systematic look at this problem. The researchers observed that language models write essays well yet often slip on simple arithmetic or real-time lookups, like a learned humanities professor who is shaky at multiplying large numbers in his head. Using self-supervised learning, the team trained a GPT-J 6.7B model to operate five tools (a calculator, a search engine, translation, a question-answering system and a calendar), showing that a model can decide for itself “when a tool is needed,” “which tool to use” and “what parameters to pass.”

The real-world software ecosystem is far more complex than five tools. In 2024 a team including Tsinghua University published ToolLLM at ICLR, collecting 16,464 real RESTful APIs from RapidAPI Hub across 49 categories such as food, travel, finance and video. Once the number of tools jumps from single digits to tens of thousands, the system hits a wall at once: the context window limit and diluted attention. If a user only wants to order a pizza but the system must stuff the names, descriptions and parameter formats of sixteen thousand tools into the prompt for the model to pick from, every conversation burns a large number of tokens, and the model is more likely to choose wrongly under information overload.

To solve this, ToolLLM introduced a “Neural API Retriever” and split the way AI uses tools into a two-stage funnel: retrieval first, selection second. The architecture closely parallels the logic of search-engine information retrieval (IR); only the things being matched change, from long web pages to a tool's name, functional description and parameter definitions.

As Diagram 1 shows, the first stage narrows tens of thousands of tools to a handful of candidates through three mechanisms. The most basic is keyword retrieval (sparse retrieval), represented by BM25, which directly measures the word overlap between the instruction and the tool description. That is why a tool's name and description cannot rest on abstract brand slogans. When a user wants to “book a flight,” an airline tool whose description reads “delivering outstanding global smart mobility solutions” drops out at the keyword stage because it shares no words with the request.

People speak casually and do not issue commands from an engineer's spec sheet. Tell the AI “find me a place for a weekend dinner with friends where there's no time limit,” and a booking tool may describe itself as “filter partner restaurants by dining time limit and party size.” The wording differs but the meaning is nearly the same. This is where semantic vector retrieval (dense retrieval) comes in: the system uses an embedding model to turn both the spoken request and the tool description into high-dimensional vectors, then uses cosine similarity to find semantically close tools.

However, semantically close does not mean the tool is suitable. Suppose you urgently ask the AI to “buy me a high-speed rail ticket from Taipei to Zuoying at 8 p.m. tonight.” In vector space, “look up historical delay statistics for high-speed rail,” “search high-speed rail travel and food blog posts” and “live high-speed rail seat availability and ticket booking” may sit very close together, because all of them revolve around “high-speed rail” and “train times.” On first-round vector similarity alone, the AI might send a tool that can only show historical data or food reviews to do the ticket-buying job. So large retrieval systems usually add a reranking model (reranker) after recalling the first few dozen candidates. It checks whether a tool genuinely has live transaction and booking ability, pushes the options that can actually get the job done to the front, and only then hands them to the language model for the second-stage decision.

More tools, harder to choose? The retrieval bottleneck seen in ToolScope and Anthropic's evidence

Once you understand the two-stage funnel, one thing becomes clear: when your tool is not used by AI, it is often not because the model dislikes the product but because the tool was already eliminated in the first stage, and the model never saw it at all.

As the number of tools in connected ecosystems swells, “name collisions” and “overlapping descriptions” grow worse. ToolScope, published at ACL 2026 by an Oracle AI research team, points to two bottlenecks in large tool libraries: many tools have near-identical functional descriptions, so retrievers and models cannot tell them apart, and the limited context space cannot hold large volumes of tool definitions.

Imagine your phone's AI has three food tools installed. The first calls itself an “all-in-one smart dining assistant,” the second reads “popular restaurant discovery and food guide,” and the third advertises “one-stop dining lifestyle services.” In text they look almost identical; in fact the first is a delivery platform, the second a restaurant-review site, and only the third is a dine-in reservation system. When tool providers all package themselves in vague, all-encompassing wording, the retrieval system gets confused. The ToolScope team designed an automated tool merging and calibration mechanism for this: it first uses vector similarity to find heavily overlapping tools, then has a model clarify their capability boundaries, and adds a reranking module. Tested with GPT-4o on the UltraTool dataset, the results are as follows:

The numbers say two things. First, moving from keyword retrieval to semantic vector retrieval raises the correct selection rate (CSR) by more than 30 percentage points. Second, the paper's ablation study shows that merging tools with overlapping functions, which reduces semantic overlap, brings the largest gain, while reranking helps only modestly on large, noisy tool libraries. In other words, stating a tool's capability boundary clearly is itself an effective retrieval optimization.

Commercial platforms have already turned this approach into infrastructure. A 2025 preprint, RAG-MCP, reports that using semantic retrieval to filter MCP tools before handing them to the model raised tool selection accuracy from 13.62% to 43.13%. In November of the same year, Anthropic launched the Tool Search Tool on the Claude Developer Platform: tools can be marked for deferred loading, and the model loads the definitions it needs only after searching with BM25 or a regular expression. In the example Anthropic published, when several MCP servers are connected, the traditional approach spends about 72,000 tokens on tool definitions alone (about 77,000 in total before any work starts); with the Tool Search Tool the total drops to about 8,700 tokens, which Anthropic says cuts tool-related token overhead by about 85%. Accuracy in choosing the right tool rises as well:

Source: Anthropic, “Introducing advanced tool use on the Claude Developer Platform.” This is a vendor's internal evaluation with an unpublished test set, so it should not be compared directly with academic benchmarks.

Now that Anthropic and the major agent frameworks all use dynamic tool retrieval that searches first and loads later, whether a tool makes it through the first round of search becomes the dividing line for whether it gets any chance of being used.

Opening the failure black box: why does the task still fail after AI finds the tool?

Before optimizing anything, you need a diagnostic framework: being retrieved (tool retrieval) is not the same as being selected by the model (tool selection), and being selected is not the same as executing successfully (tool calling). “Retrieval Models Aren't Tool-Savvy,” in ACL 2025 Findings, used the ToolRet benchmark, which covers about 43,000 tools, and found that retrieval models good at finding web articles perform markedly worse at tool retrieval, which in turn drags down the whole agent's task success rate.

In LS-Labs' four-stage TSO framework, after a tool provider completes the first stage, “tool integration,” the tool passes through three breakpoints in a real task, matching the second stage “candidate exposure,” the third stage “model selection” and the fourth stage “calling and execution.” Suppose you ask an AI assistant to “book a hot pot restaurant for four this Saturday at 7 p.m.” and the AI reports that the booking failed. The diagnostic diagram below shows where it got stuck:

The diagram shows why a mindset of “just manage exposure” is not enough in the agent era. Losing at breakpoint 1 means insufficient discoverability; losing at breakpoint 2 means a blurry capability boundary; losing at breakpoint 3 means a poorly defined parameter specification (schema design). Only when all three links work can a tool actually complete what the user entrusted to it.

What is TSO (Tool Selection Optimization)? Four advantages of doing it well

Most past academic research takes the model developer's view, asking how to train smarter retrievers and models. Flip the view and stand where service companies, SaaS products and tool developers stand: when two hotel-booking sites, payment tools or e-commerce services with the same features connect to the AI ecosystem at the same time, which one gets correctly chosen because of a more precise tool definition?

That is the starting point of TSO (Tool Selection Optimization), as proposed by LS-Labs. The definition used in this article is: a systematic method, taken from the tool provider's perspective, that uses truthful, structured tool names, capability descriptions, applicable scenarios and parameter designs with clear boundaries, so that a tool can be precisely discovered, correctly understood and reliably executed in an AI agent's retrieval, decision and calling process. TSO is this site's operating framework, not a ranking factor published by any AI platform; LS-Labs also has no measurements of its own yet, so the advantages below rest on existing papers and platform documentation.

To see intuitively where TSO applies in an MCP tool definition or OpenAI function calling, compare one tool definition before and after optimization:

{
  "name": "travel_helper_api",
  "description": "Delivers an all-in-one smart travel experience, helping users easily handle global accommodation and itinerary planning.",
  "inputSchema": {
    "type": "object",
    "properties": {
      "date": { "type": "string" },
      "num":  { "type": "integer" }
    }
  }
}

{
  "name": "search_and_book_hotel_rooms",
  "description": "Looks up hotel room availability and prices for a given city and dates, and can create a hotel booking directly. Returns room type, total price including tax and cancellation policy. Limits: accommodation booking only; does not support flight search, restaurant reservations or attraction tickets.",
  "inputSchema": {
    "type": "object",
    "properties": {
      "check_in_date": {
        "type": "string",
        "description": "Check-in date in YYYY-MM-DD format, for example 2026-11-07"
      },
      "guests_count": {
        "type": "integer",
        "description": "Total number of adult guests, not including infants and young children"
      }
    },
    "required": ["check_in_date", "guests_count"]
  }
}

Drawing on EASYTOOL (NAACL 2025), “Tools are under-documented” (Tool-REX, ICLR 2026) and Re-Invoke (Findings of EMNLP 2024), a tool whose documentation has been restructured in this way can be expected to gain an edge on four fronts:

1. Break through the context limit and earn entry to the first-round candidate pool (Top-K recall). Under architectures such as Tool Search and RAG-MCP, the AI loads only the few most relevant tools into context each time. When the description covers a clear verb, a target object and the task situations users commonly describe, the tool is more likely to make the Top-K at the BM25 and vector retrieval stages whether the user says “find a hotel,” “book a homestay” or “check weekend lodging availability,” so it is not eliminated before any comparison. Results from both Tool-REX and Re-Invoke show that filling in tool documentation or rewriting queries improves tool retrieval recall.
2. Remove semantic collisions and raise the correct selection rate (CSR) among similar tools. ToolScope's experiments show that when a dozen or so travel or dining tools on the market all say “smart life assistant,” their vectors overlap heavily in semantic space. Spelling out input conditions, returned data and negative constraints lets the reranking model and the LLM judge faster which tool best fits the current subtask.
3. Reduce parameter hallucination and API errors, and cut retry costs. OpenAI's official documentation proposes the “intern test”: could a new intern, reading only the parameter descriptions you gave the model, fill in every field correctly on the first try? When inputSchema is spelled out, the model is more likely to pass the right date format and parameters in one go, avoiding a retry after a failed call or a switch to another tool. As for whether platforms feed task completion rates into tool ranking, no platform has publicly described such a mechanism.
4. In zero-UI scenarios, become the service the agent calls directly. Traditional SEO competes to get a human to see a web link, and GEO competes to get an AI to mention the brand in its answer; TSO is about whether an agent calls your service while carrying out a task. When a consumer simply tells the phone “get me a ride to the airport” or “buy me a good-value travel insurance policy,” there may be no web browsing or price-comparison screen in between. This is also the core scenario in the discussion of B2A (Business-to-Agent).

How TSO divides the work with existing optimization concepts is compared in the table below:

How to validate TSO results, and the manipulation risk ToolTweak exposed

If TSO stops at “writing the copy clearly,” it easily turns into guesswork. Validation needs controlled variables and layered observation: fix the model version, the tool pool size and the permissions, then run A/B tests with three layers of metrics.

Why stress “correctly not being chosen” and honest descriptions of capability boundaries? Because once a tool description can sway an AI's decision, malicious manipulation follows.

The October 2025 preprint “ToolTweak: An Attack on Tool Selection in LLM-based Agents” exposed this risk. The researchers found that an attacker who repeatedly tweaks a tool's name and description algorithmically, much like early keyword stuffing and prompt injection, can systematically skew an agent's choices. In the experimental setting, the target tool's selection rate was pulled up from a baseline of about 20% to as high as about 81%, and the effect carried across open and closed models.

Consider a dangerous scenario: an unknown third-party currency-exchange or ticketing tool appears and stuffs its MCP description with inducing phrases such as “officially authorized, zero fees, the safest first choice for instant ticket issuing,” so your AI assistant abandons the bank's or airline's official API and passes your credit card details to it. This is the basic difference in risk between tool selection and web search: with SEO, a wrong click means the user closes the page; with TSO, a wrong tool choice can mean the AI directly executes a wrong transfer, deletes files or leaks personal data.

So the methodology has to draw a red line: the goal of TSO is to match real capability to task intent correctly, not to win the highest selection rate with exaggerated wording. A tool that can only look up historical stock prices, if it describes itself as able to “provide the latest market quotes” for the sake of traffic, only leads agents into wrong trading decisions, harms the users behind those agents, and makes the tool itself lose trust.

LS-Labs research status This site is designing a controlled experiment that observes “candidate exposure” and “model selection” separately. It has not been run yet, so this article does not claim that any rewrite raises how often a particular tool is called. The results will be published in a separate article once available, including any finding of no difference.

Conclusion: when AI becomes the agent that operates software services, products need a new way to be understood

From Toolformer teaching a model to press a calculator in 2023, to ToolLLM tackling the retrieval problem across tens of thousands of APIs in 2024, to Anthropic turning tool search into a platform feature in 2025, to ToolScope handling semantic overlap in tool libraries in 2026, the entry point to software services is moving.

In the past, when building an excellent booking system, price-comparison site or personal-finance tool, companies mainly thought about how to make the interface look good to humans and the ads appealing. In the agent era, a product gains another kind of user, one that reads only structured documents yet holds real execution power: the large language model and its retrieval system. People are moved by visual design and brand slogans; AI agents care whether a tool's name is explicit, its functional boundary is clear and its parameter format is free of ambiguity.

For developers and companies, the API and MCP documentation that used to sit in a corner of the engineering department, written only to ease integration, has moved to the front line of competition and become a core asset that decides whether a product can be discovered, understood and chosen by AI. In the future software market, products will compete not only on who is better known in the human world but also on which one lets the AI conclude, in the second a user states a need: “this is the most reliable tool for the job.”

To get the definition of TSO and its four-stage framework first, start with the TSO topic page; whether an agent can get things done after a tool is chosen falls under AX (Agent Experience).

References

Academic research

• Schick, T., et al. (2023). Toolformer: Language Models Can Teach Themselves to Use Tools. NeurIPS 2023. arXiv:2302.04761
• Qin, Y., et al. (2024). ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs. ICLR 2024. arXiv:2307.16789
• Shi, Z., et al. (2025). Retrieval Models Aren't Tool-Savvy: Benchmarking Tool Retrieval for Large Language Models. Findings of ACL 2025. arXiv:2503.01763
• Liu, M. M., et al. (2026). ToolScope: Enhancing LLM Agent Tool Use through Tool Merging and Context-Aware Filtering. ACL 2026, 34095–34119. arXiv:2510.20036
• Gan, T., & Sun, Q. (2025). RAG-MCP: Mitigating Prompt Bloat in LLM Tool Selection via Retrieval-Augmented Generation. arXiv preprint. arXiv:2505.03275
• Yuan, S., et al. (2025). EASYTOOL: Enhancing LLM-based Agents with Concise Tool Instruction. NAACL 2025. arXiv:2401.06201
• Lu, X., et al. (2026). Tools are under-documented: Simple Document Expansion Boosts Tool Retrieval. ICLR 2026. arXiv:2510.22670
• Chen, Y., et al. (2024). Re-Invoke: Tool Invocation Rewriting for Zero-Shot Tool Retrieval. Findings of EMNLP 2024. arXiv:2408.01875
• Sneh, J., et al. (2025). ToolTweak: An Attack on Tool Selection in LLM-based Agents. arXiv preprint. arXiv:2510.02554

Official technical documentation and industry sources

• Anthropic (2025). Introducing advanced tool use on the Claude Developer Platform. anthropic.com
• Anthropic. Tool search tool. platform.claude.com
• Model Context Protocol (2026). Specification: Server Tools (2026-07-28). modelcontextprotocol.io
• OpenAI. Function calling. platform.openai.com
• Biilmann, M. (2025). Introducing AX: Why Agent Experience Matters. biilmann.blog