Back to articles

Gemini API Adds Tool Retrieval for On-Demand Schemas

3 min read

For an agent connected to dozens or hundreds of tools, including every full parameter schema in every request can consume input tokens and add latency. A large candidate list can also make tool selection harder. Google’s preview of Tool Retrieval in the Gemini Interactions API changes when those schemas enter the conversation: a tool can remain discoverable without its complete parameters being present from the start.

How it works

  • Load schemas when needed: Developers add a tool_search tool and mark selected functions with defer_loading: true. The model can still see each deferred tool’s name and description. When it needs one, it requests the tool by name; the API supplies the full schema and the model proceeds with the function call. The search request, returned schema, and function call are visible in the interaction steps, which can help when debugging tool names or descriptions.
  • Mix immediate and deferred tools: Frequently used functions can stay loaded up front, while less common tools are deferred. Google says this approach also works with prefix caching: deferred schemas do not alter the beginning of the prompt, helping keep the cached prefix consistent across turns.
  • Use MCP or bring your own search: A remote MCP server’s tools can be loaded on demand by deferring the server tool. Developers with their own registry, embedding index, or per-user permission checks can instead run client-side retrieval. That option adds a round trip for each search, but does not require every tool to be declared in advance and can support integrations added during a conversation.

Pricing and implications

There is no separate Tool Retrieval charge. Input and output tokens continue to use the model’s standard rates. Enabling tool_search adds a small fixed block of input tokens, and each schema the model loads is billed as input for the rest of the conversation. Savings therefore depend on whether schemas that are never loaded outweigh the search overhead; Google recommends measuring performance against a developer’s own traffic.

The feature is in preview in the Interactions API and works with Gemini 3.6 and later models, including Gemini 4 models. Its broader significance is not simply that fewer schemas may be sent initially. It gives developers a way to scale an agent’s tool catalog without placing the entire catalog’s parameter details into every turn, while keeping ordinary function calling as the execution path. For teams building tool-heavy agents, context use, latency, and selection quality are increasingly connected engineering concerns. Retrieval offers an API-managed option for balancing them, though the benefit will vary with the workload and which tools users actually call.

Google AI Blog

Comments

Checking sign-in status...

Loading comments...

Related articles