Briefly: an analysis of an AI agent failure should identify not the last wrong phrase, but the first critical deviation, after which the chain could no longer produce a valid business outcome. Naming the guilty agent remains a hypothesis until a targeted replacement of its step, data, or state fixes the result in a controlled replay.
This material is intended for owners of AI processes, CTOs/CIOs, product, operations, QA, and information security teams. It covers task-oriented chains with a verifiable outcome. Open-ended creative tasks, legal allocation of responsibility, and damage calculations for a specific company are out of scope.
Contents
- What the new DoCtOR paper showed
- Why the last agent is not necessarily at fault
- Attribution, cause, and proof are different things
- What DoCtOR, AgentRx, and DoVer say
- The ISTOK method for failure analysis
- What trace should be preserved
- How to run a counterfactual replay
- What to fix after the cause is confirmed
- Which metrics to use
- Analysis limitations
- Frequently asked questions
- How AIrassvet helps build an analysis framework
- Conclusion
What the new DoCtOR paper showed
On August 28, 2026, researchers from Renmin University of China and Ant Group published the paper Finding Where the Buck Stops, marked as accepted to the main EMNLP 2026 program. The authors proposed DoCtOR: the system first finds the first incorrect step and the agent that performed it, then generates a corrected counterfactual step and adds reflection only to that agent's memory.
The experiment used Magentic-One with frozen GPT-4o-mini agents, Qwen3-1.7B for attribution, and Llama-3.1-8B-Instruct as the reflector. On random samples of 100 tasks from HotPotQA, ChartQAPro, and Mind2Web, across five runs, the authors report improvements over the baseline success rate of 22%, 26%, and 27% respectively. These are results from a specific lab setup, not a forecast for a business process.
The most important practical caveat is in the attribution table. On their own domains, ProFA identified the faulty agent with an accuracy of 0.79–0.85, but on two external datasets — only 0.53 and 0.55. Accuracy for the first incorrect step on external datasets was 0.38 and 0.20. Therefore, automated RCA is useful as hypothesis ranking, but not sufficient for automatic role punishment, changes to shared memory, or releasing a patch.
| Observation | Practical meaning | What it does not prove |
|---|---|---|
| Targeted reflection improved benchmark success | not every agent needs the same postmortem | that the method will transfer to CRM, 1C, or procurement |
| Attribution is stronger on held-in data | familiar failure types are easier to diagnose | that external domains will preserve accuracy |
| Steps after the error produced comparable reflection quality | the diagnostic context can be shortened | that early context is never needed |
| The paper was accepted to EMNLP 2026 | there is conference-level review | that independent production replication was performed |
Why the last agent is not necessarily at fault
In a chain of “search → analysis → decision → action,” the last agent is already seeing a changed state. The analyst may have correctly calculated the conclusion from incomplete data, and the executor may have correctly carried out a poorly worded decision. Their messages look wrong only because the first agent chose the wrong period, misread the tool output, or lost the user's constraint.
That is why it helps to distinguish four levels:
| Level | Question | Example |
|---|---|---|
| Symptom | what the user saw | incorrect final order status |
| Effect | what changed in the system | a refund of the wrong type was created |
| First critical deviation | where the valid trajectory became unreachable | the classifier confused the reason for the request |
| System condition | why the deviation propagated further | the orchestrator did not require confirmation |
A correct downstream action does not become wrong just because it used bad upstream input. But the first unusual step is not necessarily the cause either: the system could have compensated for it later. The critical point is the moment after which a valid outcome is no longer achievable without intervention.
Attribution, cause, and proof are different things
Attribution answers: “which step most likely triggered the cascade?” A causal test asks: “if we change only this factor, will the outcome change?” There is an important gap between the two.
Microsoft Research in the DoVer paper showed that a log-only hypothesis can be wrong, and a single failure can sometimes be fixed by several different interventions. In their experiments, point interventions turned 18–28% of failed trials into successful ones and confirmed or disproved 30–60% of hypotheses about causes. On another dataset and agent framework, they restored 49% failed trials; this is benchmark evidence, not a production SLA.
This leads to a conservative rule:
Trace localizes the hypothesis. Replay tests causality. Business impact determines repair priority.
If replacing the presumed faulty step does not change the result, the investigation continues. If different interventions produce success, the problem may not lie with the “guilty agent” but with the lack of systemic redundancy, constraint checks, or a recovery mechanism.
What DoCtOR, AgentRx, and DoVer say
These papers should not be reduced to one winner: they measure different parts of diagnosis.
| Approach | Primary object | Set/scale | Strength | Limitation |
|---|---|---|---|---|
| DoCtOR | the first wrong step and the agent | 3×100 test tasks; 1,757 failure logs for the training corpus | selective reflection and counterfactual correction | held-out step accuracy 0.20–0.38 |
| AgentRx | the first unrecoverable step and category | 115 labeled failed trajectories | verifiable constraints and evidence log | small benchmark sample |
| DoVer | hypothesis + active intervention | GAIA, AssistantBench, GSMPlus | tests whether the intervention repairs the outcome | intervention success does not make the attribution unique |
| NIST AI RMF | risk and system lifecycle | voluntary framework | monitoring, incident response, recovery, change management | does not define a specific attribution algorithm |
In the AgentRx description Microsoft reports 115 manually labeled failure trajectories from API workflows, incident management, and Magentic-One. The method synthesizes guarded constraints from tool schemas and policy, checks them step by step, and leaves an evidence-backed violation log. This layer is useful where an “error” is defined not by a polished answer but by violating an explicit rule.
The ISTOK method for failure analysis
We propose the method ISTOK. This is an editorial synthesis of failure attribution, incident response, and counterfactual testing; it is not an EMNLP, Microsoft, or NIST standard.
I — Outcome
First, define the expected business outcome, the actual effect, and the tolerance boundary. “The agent answered badly” is too vague. Better phrasing: the order must not change status without approval; the amount must match the document; an impossible operation must fail.
S — Trace
Collect a single trace: model and harness versions, messages between agents, tool calls and outputs, memory states, approvals, retries, timestamps, and the observed effect. A final chat log alone does not let you separate a reading error from an execution error.
T — Point of first critical deviation
Work from the start of the trajectory and mark the first step that violated a verifiable constraint or introduced a fact not present in the available state. Do not automatically call it the cause. Record alternative hypotheses and confidence.
O — Intervention experiment
Repeat the frozen case while changing one factor: swap in a correct step, restore the prior memory, fix the tool output, add approval, or change routing. All other versions and data must remain unchanged. Compare not the text, but the business outcome and forbidden effects.
K — Correction and control
Change only the confirmed owner: role prompt, tool schema, memory entry, policy, orchestrator, or evaluator. Then rerun the original case, nearby normal cases, and adversarial cases. Preserve the decision, residual risk, and rollback.
Minimal incident card:
| Field | What to record |
|---|---|
| Case and version | input, model, harness, tools, policy, memory snapshot |
| Expected outcome | success criterion and forbidden effect |
| Actual outcome | data change, message, action, damage |
| First suspected point | agent ID, step ID, violation, evidence |
| Alternative hypotheses | routing, stale state, tool error, ambiguity, judge |
| Intervention | one changed factor and the control version |
| Replay result | outcome, side effects, latency, cost if measured |
| Decision | owner, patch, regression set, rollback, review date |
What trace should be preserved
A useful trace connects intent, execution, and effect. For each step, it is enough to store not the entire sensitive payload, but the identifiers and evidence needed to reproduce the result in line with data policy.
- Correlation ID: one end-to-end link task → agents → tools → effect.
- Assembly manifest: model ID, prompt hash, tool/schema version, policy, router, and evaluator.
- State lineage: which documents, memory records, and tool outputs the agent saw at the moment of decision.
- Action record: tool request, granted permissions, approval, response, and actual change.
- Decision record: selected path, rejected alternatives, and confidence only if it is calibrated.
- Outcome record: business outcome, human override, complaint, rollback, and recovery time.
The chain-of-thought log is not required and is often unavailable. For investigations, observable inputs, actions, states, and effects matter more. The article on AI agent orchestration separately explains why the runtime harness should be part of the frozen build.
How to run a counterfactual replay
Replay should be like a small experiment, not a free-form prompt rerun.
- Freeze the original build. Record the manifests, inputs, available data, and external system state.
- Describe one hypothesis. For example: “a classification error at step 7 caused the wrong refund.”
- Choose one intervention. Replace only step 7 with a verified value; do not update the model and prompt at the same time.
- Run the replay again. Save the full trace, outcome, and prohibited effects.
- Add a negative control. Change a factor that should not fix the failure, or replay a neighboring normal case.
- Draw a conclusion.
confirmed,rejected,multiple sufficient causesorinsufficient evidence.
For irreversible actions, replay is performed in a sandbox, shadow mode, or on synthetic state. The NIST AI RMF Playbook recommends testing incident response, recovery, and fail-safe behavior, documenting test conditions, and comparing results against risk tolerance (Measure).
What to fix after the cause is confirmed
| Confirmed owner | Precise fix | Mandatory regression |
|---|---|---|
| Role/agent prompt | clarify the rule and allowed actions | adjacent tasks for this role |
| Tool/schema | fix the contract, validation, or permissions | malformed, timeout, and partial output cases |
| Memory/state | delete or scope-limit the record | no-memory and stale-version controls |
| Orchestrator/router | change delegation, stop, or escalation | alternative routes and dead ends |
| Evaluator/policy | fix the acceptance criterion | false accept and false reject cases |
| Human workflow | add approval or a manual fallback | response time and availability of the responsible person |
Do not write the generic text “always double-check” into every role. The DoCtOR work shows the risk of contaminating the memory of agents that are working correctly with incorrect conclusions. But selective repair is safe only after evidence: held-out accuracy of automatic attribution in the same work is far from 100%.
If the incident is related to a hidden channel or trace tampering, self-reflection alone is not enough. The article on hidden collusion among AI agents shows why business impact and control-plane evidence need to be separated from agent text.
Which metrics to use
Do not limit yourself to the share of correctly named agents. A production system needs at least five indicators.
| Metric | What it measures | Risky substitution |
|---|---|---|
| Step localization accuracy | match with the labeled critical point | does not prove causal repair |
| Hypothesis validation rate | share of hypotheses tested by intervention | depends on the quality of the hypothesis set |
| Recovery rate | share of failed cases that became acceptable | may hide a new prohibited effect |
| Regression escape rate | new failures after the fix | requires neighboring and adversarial cases |
| Time to evidence-backed decision | speed from signal to decision | is not the same as time to first explanation |
Cost and latency are useful only together with the outcome. A cheaper diagnosis that assigns the wrong owner and contaminates memory can increase the total incident cost. Universal thresholds for these metrics are unknown: they are set based on process criticality and risk tolerance.
Analysis limitations
DoCtOR is a version v1 published on August 28, although arXiv lists acceptance to the EMNLP 2026 main conference. The main evaluation uses 100 tasks per dataset, small teams, mostly English task-oriented benchmarks, and specific models. The annotation pipeline included GPT-4o and review by three researchers; it is not a flawless oracle.
The metrics for DoCtOR, AgentRx, and DoVer cannot be directly compared as a ranking: they use different datasets, labels, interventions, and metrics. The results do not prove incident frequency in companies, accuracy on Russian-language processes, safety, savings, ROI, or a universal advantage of a multi-agent architecture.
NIST AI RMF is voluntary and does not replace industry, contractual, or Russian requirements. Search volume, difficulty, rankings, traffic, CTR, backlinks, and AI citations for this topic remain Unknown.
Frequently Asked Questions
What counts as the first critical error by an AI agent?
It is the earliest observed step after which the system, without intervention, no longer achieves an acceptable result. It must be tied to a specific violation, state, and downstream effect.
Can the responsible agent be identified automatically?
You can rank hypotheses, but you cannot treat attribution as proven. In a recent paper, held-out step accuracy was 0.20–0.38, so replay or another independent piece of evidence is needed.
Why use replay if the error is visible in the log?
A log shows sequence, but not causality. Replay changes one factor and checks whether the business outcome changes while the other conditions stay the same.
Do we need to store all messages and chain-of-thought?
No. You need versions, accessible state, tool calls, approvals, and effects sufficient for reproduction. Storage of payloads and personal data is limited by policy and legal basis.
Who should the reflection be assigned to after a failure?
Only the role or component whose cause has been confirmed. For everyone else, it is better to keep behavior unchanged and run regression cases than to spread an unverified lesson.
What if several interventions fix the failure?
Document multiple sufficient recovery paths and choose the control based on risk, cost, and reversibility. This is a sign of a systemic vulnerability, not necessarily a single culprit.
How AIrassvet helps build an incident analysis framework
AIrassvet can connect diagnostics with a real-world agent stack:
- audit one process and define expected outcomes, prohibited effects, and approval points;
- link model, tool, RAG, 1C/ERP, or browser automation logs with a single correlation ID;
- build a frozen regression set, sandbox replay, and human escalation rules;
- implement selective repair for prompts, memory, tools, and orchestration with rollback.
A safe first step is to choose one process and define the baseline, data sources, constraints, and acceptance criteria. Discuss the project.
Conclusion
A good analysis of AI agent failures does not look for a convenient culprit. It pins down the missed outcome, reconstructs the full trace, localizes the first critical point, tests the hypothesis with a single intervention, and changes only the confirmed owner.
The recent DoCtOR paper shows the value of selective reflection, but its held-out results also remind us of the limits of automation. So the practical sequence is: trace → attribution hypothesis → counterfactual replay → outcome check → selective repair → regression. Until replay, the cause remains a hypothesis.