Google says AI coding tests must track execution paths

Google has outlined four ways it tested the Google Cloud Developer plugin for AI coding agents. The guidance focuses on how engineers measure agent behaviour, rather than how they build the systems.

The work grew out of evaluation engineering efforts tied to the recently released plugin, which Google used as a test environment for developer workflows.

At the core of the guidance is the argument that testing an agent's final written response is not enough to determine whether it acted correctly. Early assessments used a large language model judge to review the agent's closing reply against a checklist, but that approach missed what the agent actually did during execution.

In one internal evaluation suite of 11 prompts, the automated grader marked the plugin as failing three prompts. A review of tool-call logs and transcripts later showed that it had in fact passed all three tasks.

One example involved a Cloud Run cleanup prompt, where the test rubric required the agent to check command syntax before taking action. The agent ran a help command first, but its final message focused on warning the user about permanent data loss rather than saying it had checked the help documentation.

That led the automated grader to record a failure, even though the required action had taken place. Google concluded that tests should score the execution path itself, including which commands were run and validated, rather than the summary presented at the end.

Execution path

A second lesson focused on the structure of telemetry gathered from agent runs. Flattening activity into a simple sequence of events can hide concurrent tool dispatch, which Google identified as a source of behavioural failure in modern agent systems.

Some agent harnesses batch several tool invocations into a single assistant turn through an array of calls. In one case, an agent was asked to inspect an environment and search documentation, and the harness dispatched a skill-loading call and two documentation searches at the same time.

Because skill activation is a multi-stage process, the search instructions arrived only after the searches had already run. As a result, the system did not give the skill a separate turn in which its guidance could shape the agent's actions.

Google warned that telemetry pipelines which remove the original turn structure can make it difficult for engineers to understand why an agent ignored instructions that appear clear in the record. It recommended preserving that structure so teams can see when parallel dispatch caused actions to happen before guidance entered the context window.

Two conditions

The third point concerned testing under both live and plan-only conditions. Google said engineers need live runs with sandboxed credentials so they can see whether commands execute correctly, whether parameters are handled properly and whether an agent shows restraint before carrying out potentially destructive tasks.

At the same time, relying only on live runs can obscure underlying reasoning problems because permission errors may stop the process before the relevant behaviour appears. For that reason, teams should also run plan-only tests in which credentials are removed and the agent explains what it would do without executing commands.

Google said about half of its most important findings came from comparing those two conditions. In one onboarding prompt, a live run failed immediately on a project list call, while a plan-only run produced an 8,000-character plan that, according to Google, ignored organisation policies, administrative approvals and governance.

The comparison showed how a permission block in a live setting could mask a more basic reasoning flaw. Google said the side-by-side approach helps separate problems in execution from problems in judgement and routing.

Baseline sets

The fourth recommendation was to use "golden sets" as durable baselines for testing. Google said model architectures, agent harnesses and plugin standards change regularly, so evaluation needs to act as a form of continuous integration testing across the full stack.

Rather than examine only the final text output, this method captures the whole trajectory, including the agent's reasoning, the commands it ran and whether it completed the task safely. Those runs are then scored against benchmark sets created by human experts, who define what an acceptable path should look like.

Google also argued against what it called the "one-shot demo", in which a single prompt is treated as evidence that a system performs well. It said reliable assessment requires evaluation suites that test multi-step interactions, permission handling and the avoidance of damaging outcomes.

"At the end of the day, the only thing that outlasts model churn is a stress-tested eval suite."