Google Built the Answer First and Attached the Question Later

The reason for the inversion is simple: if you write the question first, nobody knows whether it can actually be solved. The paper states that exploration is expensive by nature, offers no guarantee of annotation success, and wastes agent resources. Chain the tools first so that a sequence which really runs already exists, and writing a question to match it takes a single model call.

The difference in the numbers was large. According to the paper, the pass rate under the existing depth-first search approach was 63.8 percent, against 99.8 percent for ToolGrad. The number of tools chained into one sample rose from 2.1 to 3.4, and the cost spent actually calling tools fell from 34.3 to 20.0, a drop of 41.7 percent. More complex data, more of it, for less.

The generation process reads like backpropagation rewritten in words. An API proposer pulls candidates 50 at a time and narrows them to three, three executors call those APIs in parallel for real, and a selector reads the execution reports and appends the one that performed best to the workflow built so far. Finally an updater rewrites the user's question and the answer to match the expanded tool list. The authors call that selection step a textual gradient, meaning the model's judgement, rather than a derivative, decides the next step.

The data made this way runs to only 500 samples. Fine-tuning small models on the set, named ToolGrad-500, produced a Gemma-3-12B that scored 19.6 on ToolBench, the test that asks a model to pick and call the right tools. On the same test Gemini 2.5 Pro scored 11.4, and Claude 4.5 Opus and GPT-5 Nano both scored 15.4. According to the Google Research blog, on the overall BFCL score, which presents tools the model has never seen, ToolGrad-12B came second at 83.1 against 83.2 for Gemini 2.5 Pro, a gap of 0.1, with Claude 4.5 Opus at 82.8 and GPT-5 at 74.4.

The work was led by Zhongyi Zhou, a research scientist at Google XR. He wrote on the blog that "high-quality tool-use datasets can be generated more efficiently and reliably through an answer-first paradigm." The paper puts it more bluntly, with the authors writing of the older approach that "exploration must be expensive by nature." Researchers from the University of Tokyo and RIKEN AIP are among the co-authors, and the paper appears in the Findings of ACL 2026.

The limit the authors set down themselves is that the data starts to resemble itself. The paper concedes that the agent tends to produce similar tool uses inside the training set, and that such repetitive synthetic data will inevitably harm model training once the framework is scaled up. Raising the sample count from 500 to 1,000 and 2,000 did push scores up and then back down. The paper also states that multi-turn tool use and agent use fall outside the scope of this framework.

The part worth watching here is not the score but the fact that the source of the material has changed. What teaches a model to use tools is moving from records people left behind to records a machine produced by running things. ToolGrad pushed even the step of inventing the question to the back of the line, leaving people only the seat where they check whether the result looks right. The paper had two raters look at eight queries and twelve models, scoring 96 items each, and reported a correlation of 0.88 with the machine judge; the design itself says that the human role has shrunk from producing to auditing.

Where small models beat large ones showed up again. METAL has reported that skills teaching agents a procedure drove performance more than knowledge did, and the ToolGrad results point the same way. A Gemma-3-1B that scored 1 in its base state reached 14.1 on the same test not because the model grew, but because it saw worked examples with the procedure spelled out.

The cost structure is another axis of this story. In the comparison table of the paper METAL reviewed, the model-side cost barely moves, from 64.5 to 63.9, while only the tool-side cost falls, from 34.3 to 20.0. What was saved, in other words, is not the cost of thinking but the cost of actually knocking on the APIs. Generation ran on the cheap, fast Gemini 2.5 Flash Lite, and the paper explains that iterations per sample were set at ten because the pass rate stopped climbing somewhere between eight and twelve.

Google released the code, the dataset and the models. It has put the claim that skill at using tools now turns on the design of the training data rather than on model size into a form others can check. The 500 released samples are small by volume, but the order in which each tool was chained is all preserved, so other researchers can take the design apart.

What decides the value of synthetic data is variety, not volume. What ToolGrad showed is that building a verified answer first means almost nothing has to be thrown away, and, at the same time, that the answers built this way come to resemble one another. Teaching AI to use tools has moved from the problem of collecting more examples to the problem of designing examples that differ.