Google's ADK adds automated evaluation for live voice agents

Google has an answer to this problem. The company announced that it has added a new feature to its Agent Development Kit (ADK) that automatically evaluates voice-based live agents. Within the same workflow already used to evaluate text agents, developers can now have voice agents converse with simulated users and receive a score for the results.

A simulated user talks to the agent using a real voice

At the core of this feature is the simulated user. Once a developer defines an evaluation scenario, an LLM plays the role of a user matching that scenario and generates real audio using Gemini TTS to talk to the agent. Because the exchange happens through sound rather than text, the test also captures voice-specific variables like timing and interruptions.

Test cases can be built in two ways. One is the "conversation scenario" approach, where you only need to define a goal and a persona and the simulator carries the conversation forward on its own. For instance, a NOVICE persona is designed to only mention the big picture and wait for the agent to ask for details, which tests how well the agent steers the conversation. The other approach is "fixed conversation," where the user's lines are written out like a script in advance.

Test case type	Characteristics
Conversation scenario	Only goal and persona are specified; the simulator improvises the conversation
Fixed conversation	User utterances are locked into a script, reproducing the same conditions every time

Scoring is done through natural-language rubrics. Voice responses can convey the same meaning in hundreds of different phrasings, so once a person writes out the scoring criteria in plain language, that same standard gets applied automatically to every subsequent conversation.

ADK Web shows the transcript and audio side by side

After a run finishes, ADK Web turns the audio stream into a clean transcript. Each utterance appears in its own bubble with both the text and a playable audio clip, so you can check not just what the agent said but how it said it. This view now has a new toggle for switching between Standard and Live mode, and selecting Live also brings up settings for input type (audio or text) as well as the simulated user's voice and language.

How to try it

Where to start — Begin by installing the additional evaluation components in the ADK package. Run uv pip install -e ".[eval]") to install the eval extras, and set up API credentials for the Live API and Gemini TTS.

Step-by-step usage

1. Build an agent — the example Google published chains together three single-purpose live agents in a graph-style workflow, with each stage running on the gemini-live-2.5-flash-native-audio model.
2. Write evaluation cases — put conversation scenarios or fixed conversations into a JSON-format evaluation set file.
3. Add live_model_config to test_config.json and specify an llm_audio user simulator — the voice you choose here will synthesize each user utterance into audio streamed to the agent.
4. Run the evaluation from the CLI using the adk eval command — results can then be reviewed as transcripts and audio in ADK Web.

Who can use it — This is an open-source feature that any developer already using ADK can try right away, with no separate sign-up required. All you need is access to the Live API and Gemini TTS API. The same pipeline can also be called via AgentEvaluator, letting teams plug it into a CI/CD pipeline to automatically catch regressions before deployment.

What you can do with it — For example, you can automatically verify whether a customer service voice bot keeps context intact across a three-step inquiry flow (intake → confirmation → resolution), and whether it handles user interruptions gracefully, all without needing a person to make test calls every time. You can also rerun the same evaluation set before switching to a new model version to spot where its answers diverge from before.

If you want to follow Google's published example directly, you can download and run it from the live_workflow repository. For more details on user simulation, synthetic audio profiles, and custom evaluation metrics, check the official ADK documentation.

Editor's take

What agent development tools have been racing to fill in lately isn't the "building" side but the "verifying" side. Just as Agent Skills gave Anthropic's ecosystem a way to embed procedures into coding agents, and AWS bolted formal logic verification onto Bedrock, Google has now filled a gap of its own in ADK. Text agents already have reasonably mature evaluation pipelines, but voice agents have largely remained a manual process where a person has to listen and judge for themselves. This update effectively pulls that last manual step into the automated loop.

The practical shift here is straightforward. In the past, every time a voice agent got updated, a QA person had to place test calls scenario by scenario to check for anything off. Now, a simulated user makes those calls instead, and all a person needs to do is write a few lines of natural-language rubric. Regression bugs — like a dropped tool call or broken context in a multi-turn conversation — get caught at the CI stage before anyone has to listen for them. For teams building voice bots or call center automation, this pipeline can meaningfully cut the manual QA cost that used to repeat with every model switch. That said, how reliable the evaluation ends up being depends heavily on how carefully the personas and rubrics are designed to match real user speech patterns — so investing time up front in that setup tends to pay off with faster, more stable deployment down the line.

It's likely that other agent frameworks will roll out similar live-evaluation tools in the coming weeks. As voice interfaces spread beyond chatbots into real operational touchpoints, the shift from "listening and judging" to "measuring and scoring" is a trend this ADK update is unlikely to be the last word on.