AI agent teams should be chosen based on the verified outcome of the entire task. Proposed business criterion: the team should improve the quality, turnaround time, or cost of the accepted work under predefined constraints. The number of assistants alone does not prove that improvement.
The prompt for this analysis is the Vals AI study from October 9, 2026: in its set of web apps, team configurations cost 1.8 to 5.1 times more than single-agent setups; a statistically significant score increase was found in one of four comparisons. This is the result of a specific experiment. Source: Vals AI.
This article is intended for technical leaders and product owners. It outlines the study’s limits and an editorial plan for comparing architectures on your own tasks. The editorial team has not run its own models, does not have a client case, and has not measured savings. Sources were checked on October 12, 2026; the plan below has not yet been validated in a production system.
Contents
- What Vals AI actually tested
- How to separate delegation from other changes
- What counts as an accepted outcome
- How to account for cost and time
- Comparison log before rollout
- When it makes sense to keep the team
- Questions about the cost of AI agent teams
- How AIrassvet helps verify agent architecture
- Bottom line: pay for proven value
What Vals AI actually tested
An AI agent performs a sequence of actions using tools; in a team, a lead agent delegates parts of the task to subagents. The test included GPT 6 Sol and Claude Opus 5.5, medium and maximum reasoning modes, and 50 applications. The score is the average share of UI tests passed. Each configuration was run once. Vals AI methodology.
For Sol in medium mode, the score rose from 77.6% to 84.9%: +7.3 percentage points, p=0.005. The other differences were not statistically significant. The intervals reflect variation across applications, not repeated runs. The teams were also given delegation instructions, so the effect of the number of agents was not isolated separately. Different software wrappers make a clean model-to-model comparison harder. The Sol cost calculation does not include a possible surcharge for longer prompts; the completeness of subagent cost accounting is an assumption made by the authors. Study limitations.
The lack of statistical significance does not prove the options are equal. For a rollout decision, it is more useful to ask what practically meaningful gain your test can detect. A small improvement may be valuable in a critical process, but it should be shown reproducibly. The cost of an error and reliability requirements are defined by your task.
How to separate delegation from other changes
We recommend comparing three options: a single agent with current settings, a single agent with stronger reasoning mode, and a team. This is an editorial test design, not a universal recipe for success. It helps avoid attributing to the team a result actually driven by a more expensive model, additional time, or new tools. Stronger reasoning mode also needs validation: it can raise costs without a useful change.
Record model versions, task description, allowed tools, limits, and completion rules. Save the exact instructions for each option. If the team gets a hint about the solution structure, note it separately; if possible, add a single-agent version with the same hint. Otherwise, the comparison will still cover the whole package of changes.
For the chosen architecture, prepare independent acceptance criteria before running the tests. A test invented by the agent after the work is done may miss an inconvenient requirement. Verification should cover the outcome the user actually needs, including access rights and data integrity. A broader principle of choosing simple solutions and adding complexity only when needed is described by Anthropic in its article on effective agents.
What counts as an accepted outcome
Accepted task in the proposed plan means work that passed the predefined mandatory checks and received approval from the person responsible for acceptance. A partially built interface or a successful local test can be counted as an intermediate result. For the final metric, they should be separated from full completion. That way, the architecture does not get credit for a large number of unfinished pieces.
Imagine an internal request portal. One assistant builds the form, another builds the API, and a third handles user roles. Nice-looking screens do not yet mean that the request is saved, visible to the right employee, and inaccessible to unauthorized users. The scenario is hypothetical; it illustrates the proposed acceptance criterion, not an AIrassvet client case.
In this situation, the reviewer follows the path from creating a request to changing its status and trying access from another role. If the pieces work separately but the full flow breaks, the task goes back for fixes. Adding subagents does not change that rule. The acceptor also notes unfinished and stopped runs so they do not disappear from the report.
Related principles for building nodes, links, and result verification are covered in the article on Graph Engineering and AI agent graphs. The new Vals AI experiment adds an observation on this architectural topic from a specific team-mode test.
How to account for cost and time
We recommend keeping two separate counters: money for the full process and time to accepted result. Include model calls, tools, infrastructure, retries, and reviewer effort in the cost. Your company sets the rate for human work. If some costs are unavailable, the report should show the unknown amount and the calculation boundary.
Working formula: cost per accepted task = total expenses for all runs of the selected option / number of fully accepted tasks. Count failed attempts from the same set as well. If there are no accepted tasks, the value cannot be calculated; show the spend and the lack of an accepted result separately. Do not substitute an arbitrary value for a zero denominator.
User wait time and total agent runtime are different metrics. For a deadline, you need the time to complete the entire task; for resources, you need the labor of all participants. Keep both metrics, as well as the time spent on manual cleanup. Define in advance whether queue time, reruns, and approvals are included in the service time.
The average share of tests passed cannot be used as the number of accepted apps without an additional rule. An app with most checks passing can still fail a mandatory access check. Therefore, the financial metric should be calculated based on fixed acceptance of the entire task. The broader cost structure is supplemented by the article on comparing cloud and on-prem LLM TCO.
Comparison log before rollout
Below is a proposed log you can move into a spreadsheet. It defines observation fields, not quality standards. Agree on the acceptable cost threshold, timeline, and mandatory checks with the process owner before running. Every result should have an artifact available to the reviewer.
| Log field | What to save | Why it matters |
|---|---|---|
| Task and input data | Identifier, task version, allowed dataset | Check comparability of conditions |
| Architecture | Model, version, mode, instructions, tools | Separate configuration changes |
| Acceptance | Required Checks, Outcome, Reason for Return | Separate the Full Outcome from the Partial One |
| Cost | Calls, Tools, Repeats, Manual Rework | Account for the Total Process Cost |
| Timeframe | Start, Stop, Acceptance, Waiting | Check the User Deadline |
| Delegation | Task for an Assistant, Artifact Owner, Merge Result | Find the Coordination Cost |
| Decision | Accept the Option, Refine It, or Keep the Original | Connect Observations to the Choice |
Choose real task types where the architecture decision is actually made. Repeat runs so you can assess randomness in behavior; the number of repetitions should meet your confidence requirement. Separate routine and complex tasks in advance while keeping one overall report. Do not remove difficult cases after they have hurt team performance.
Track constraint violations separately: limit overrun, unchecked changes, access outside the allowed scope. Direct execution of risky actions in a production system is not required to compare quality. Use a prepared environment and approved data. These measures are a pilot proposal, not a safety guarantee or legal opinion.
When It Makes Sense to Keep the Team
The suggested selection criterion is a confirmed advantage under the required conditions. A team can be a candidate if the work can be split into independent parts and the result is easy to assemble and verify. If all assistants constantly edit one shared artifact, check the coordination cost. Start by observing a specific task.
Architecture may have a time advantage at a higher cost. In that case, the owner of the process decides whether the speedup is worth the extra spend. Another option wins on quality at the same speed; a third is cheaper because it uses manual review. Do not reduce such cases to a single score without an explicit preference rule.
If the team does not show a stable advantage, keep the original option and investigate the bottleneck. You can test a single assistant for information retrieval or result checking instead of a team for the whole process. The article on subagents in Claude Code adds to the rules for setting up assistants. The existence of this capability alone does not justify using it.
Questions About the Cost of AI Agent Teams
Does the research prove that teams are useless?
No. The choice depends on the task and the kind of gain the process owner needs. Test the full outcome, cost, and timeframe on your own dataset, while keeping failed attempts. A conclusion about a specific implementation requires your own observations.
Can benchmark costs be carried over into the budget estimate?
Use them as a reason to check the calculation, and build the estimate from your own rates, tools, repetitions, and manual work. Service availability, total costs, and quality on Russian-language tasks require separate validation. A benchmark does not set the price of your project.
Should you always strengthen a single agent before using a team?
It is a useful control case for comparison, not a mandatory implementation sequence. The enhanced mode should pass the same acceptance criteria and cost accounting. Choosing it without measurements is just as unsupported as choosing a team based on the number of assistants.
Which metric matters more than token count?
For the proposed plan, the key metric is the cost of a fully accepted task while meeting the time and constraint requirements. Token count helps explain cost, but it does not replace outcome evaluation. When price data is incomplete, keep unknown components explicit.
How AIrassvet Helps Validate Agent Architecture
This section describes AIrassvet's services. The task of choosing one agent or a team includes process audit, data preparation, integration, and testing, which are presented on the official website. Possible discussion topics include acceptance requirements, the task and data set for comparison, and integrating outcome checks with the business system. Support for a specific model and measured gains require separate confirmation.
The first step is to choose one process, describe the current execution method, data sources, constraints, and acceptance criteria. This makes it possible to define the validation task before deciding how many agents to use. Cost, timing, and outcome are determined after the initial conditions are analyzed. Discuss the Task.
Bottom Line: Pay for Proven Value
The Vals AI study is a reason to test team mode as a separate hypothesis. For a practical decision, compare architectures on complete tasks, fixed conditions, and the same acceptance criteria. Keep the costs of stopped runs and manual rework. Then choose the option with the observed advantage that matters to the process owner.
Editorial responsibility for this analysis lies with AIrassvet. If the source changes or you find an error, send the link and specific note through the site contact form. The proposed log is the basis for local validation; the editorial team does not claim published results from its use.
