Short answer. In Project Swap, published by Anthropic on September 24, 2026, Claude agents traded books on behalf of employees. The main limitation was how accurately the agents captured human preferences: in a check of ten books, the order suggested by the agent after a short conversation matched the human’s order for 61% of book pairs. This was one office experiment, not a measure of AI negotiation quality in procurement, sales, or finance.
For business, the practical takeaway is more cautious than the marketing promise that “the agent will negotiate on its own”: before allowing a deal, you need to verify what exactly the person wants, what cannot be conceded, and who confirms execution. Below are the study facts, its limitations, and an editorial framework for a safe pilot. This framework was not tested in a company and is not legal advice.
What happened in Project Swap
The experiment involved 201 Anthropic employees from six offices. Each person brought a book to trade and, in a short conversation, told Claude about their reading preferences. The agent received a model-generated ranking of books and negotiated with other agents on a shared “marketplace.” The trade was executed only after all participating agents agreed. The message and exchange history was saved. For validation, participants ranked ten books themselves; human rankings were not shown to the agents. The main analysis used responses from 188 participants who submitted rankings. Method and sample description.
This was a study of exchanging low-cost physical items among colleagues who knew the context of the experiment. There was no money, no commercial contracts, no outside counterparties, and no competing model vendors in the live exchange. Follow-up runs with other models and settings were conducted in the same limited environment.
Preferences mattered more than bargaining technique
After a short conversation, the model correctly guessed which of two books a person ranked higher in 61% of comparisons; a random order would have been about 50%. This is pairwise ranking agreement, not the probability of a successful deal and not “negotiation accuracy.” The average participant wrote 216 words in eight messages; a more detailed description was associated with better alignment, but that alone does not prove a causal effect. Measurement method.
On the study’s scale, where 1 means the top position in a personal list of ten books and 0 means the bottom, the optimal allocation based on the known researchers’ preferences would have produced 0.89. The actual exchanges produced 0.55. If you calculate the best allocation based on Claude’s inaccurate rankings and then evaluate it using human rankings, the result is 0.60. The authors attribute most of the gap between 0.89 and 0.55 to imprecise modeling of participants’ wishes, and a smaller part to the mechanics of decentralized exchange. These numbers apply only to the chosen scale and sample; they cannot be converted into profit or conversion percentages. based on Claude’s inaccurate rankings, and then evaluate it using human rankings, the result is 0.60. The authors attribute most of the gap between 0.89 and 0.55 to imprecise modeling of participants’ wishes, and a smaller part to the mechanics of decentralized exchange. These numbers apply only to the chosen scale and sample; they cannot be converted into profit or conversion percentages. Breakdown of the results.
| Validation stage | What Project Swap measured | What to check in a working pilot |
|---|---|---|
| Task understanding | Pairwise agreement between human and agent rankings | Several control options and prohibited concessions |
| Negotiation | The position of the received book in the personal ranking | Compliance with hard constraints and an offer log |
| Execution | Whether the participant received the promised book | Confirmation of actual delivery or an entry in the system |
The right column is the editorially proposed framework, not a standard established by Anthropic. In the experiment, some exchanges did not end with a book being handed over: the authors did not have reliable tracking of checkout and returns. So agent agreement in a digital log was not the same as physical execution. Discussion of this limitation.
Why a strong model alone does not solve the problem
In repeated runs, the model influenced negotiation outcomes when effectiveness was measured by rankings created by Claude itself. For example, the average on homogeneous marketplaces was 0.75 for Haiku 4.5 and 0.88 for Opus 4.8, with a theoretical maximum of 0.95 under those same model-based rankings. But when assessed by human rankings , the differences between model variants and marketplace design were small. These two scales cannot be treated as a single business KPI. Results of the repeated runs.
The Stanford Digital Economy Lab study of simulated agent-to-agent deals found different outcomes for users of different agents and scenarios involving overspending or unreasonable agreements. It studied a different experimental environment and does not confirm how often such errors occur in real Russian procurement. And the Wharton Generative AI Labs The study of simulated transactions between agents identified different outcomes for users of different agents and scenarios involving overspending or irrational agreements. It examined a different experimental environment and does not confirm how often such errors occur in real Russian procurement. A experiments with buying watches showed that sources, the order in which they are presented, and injected “user memory” can change the model’s choice. That is another reason to verify the origin of the context from which the agent infers preferences.
How to limit an AI negotiation pilot
Start with a low-risk scenario: selecting several supply options or preparing a counteroffer without the authority to close a deal. For a pilot, you can use the following “LIMIT” framework — an editorial design tool, not a validated certification method:
- Preferences. Ask the person to name priorities and compare several specific alternatives. Record where the agent misunderstood them and correct the profile before bargaining.
- Authority boundaries. Set hard limits separately: price, volume, deadlines, data that cannot be disclosed, and the conditions under which the agent must stop.
- Unit of success. Define the metric in advance: the share of proposals without constraint violations, the share of options approved by a human, and the share of agreements actually executed. Do not replace human evaluation with the agent’s self-assessment.
- Auditability. Keep the input preferences, sources, rule versions, proposals, refusals, and final confirmation. Give the responsible employee a way to reconstruct the decision path.
- Another counterparty. Test behavior when interests conflict and when the input text is hostile. Project Swap involved only Claude models and friendly agents, so its result does not eliminate the risk of an external adversary.
- Action limit. Until local tests show acceptable behavior, keep final approval and execution with a human. It makes sense to expand authority only after logging errors and retesting.
In a pilot, you should distinguish three objects: the proposal, the human-approved deal and the confirmed execution. If you measure only the first, you cannot claim the process negotiated and delivered the result. For operations involving money, personal data, or contractual obligations, access and approval rules must be checked with responsible specialists in your own context.
What the study does not yet prove
Anthropic employees do not represent buyers or negotiators overall. Only 59% of participants answered the final survey, and the agents were built on models from one company. The authors did not change the marketplace rules and did not allow malicious agents. Human rankings of ten books can also express true preferences imperfectly: participation in the ranking was not incentivized. Study limitations.
So there is no proof here that an agent will lower purchase price, increase revenue, comply with the terms of a specific contract, or accurately reflect a client’s preferences. There are also no verified data on cost, upside, or error frequency in Russian companies. It also needs to be measured separately how much time is spent clarifying the task and checking each decision: automatic message exchange is not the same as saving work time.
Frequently Asked Questions
Does 61% mean the agent is successful in 61% of negotiations?
No. This is the share of book pairs for which Claude’s ranking order matched the human order after a short conversation. The deal outcome was assessed with a different metric.
Can you just choose a stronger model?
Stronger models performed better on some repeated runs on the model-based scale, but preference-understanding errors remained the main limitation when evaluated against human responses. Both stages need to be checked separately.
Should the agent immediately accept the counterparty’s offer?
For a business pilot, it is more reasonable to allow only proposal preparation and comparison at first. The limits on concessions and the right of final approval should be set by the process owner; this is a risk-reduction recommendation, not the result of a direct Project Swap test.
What counts as the end of the deal?
Confirmed execution, not just a record of agent agreement. In the book experiment, the absence of delivery tracking made it impossible to guarantee that all books were transferred.
How AIrassvet can help test this scenario
Within the AIrassvet service areas described on the website , the team can: map out a single negotiation process and the sources of preferences; design an agent with access to corporate data and an offer log; integrate review and manual approval into the workflow interface. The first step is to choose one process, document the current workflow, constraints, and pilot acceptance criteria. Discuss the task can be done after this preparation; no numerical impact is promised in advance.
Conclusion
Project Swap demonstrates the possibility of agent-based exchange in a small controlled environment and at the same time shows how much the result depends on correctly understanding the human. For a real-world scenario, first verify preferences and authority limits, then the quality of the offers, and only after that execution. Scaling automated deals based on book-exchange results would be premature.