MemComposer — a research method for assembling an AI agent’s memory: a separate model decides which past episodes to include in context, how many to take, and in what order to present them. The preprint appeared on arXiv on October 15, 2026. For a support operations manager, the key question is whether a specific set of memories helps close a ticket correctly, not just whether it looks similar to the request. Primary source and version date.
This analysis was prepared by AIrassvet on October 16, 2026. Below are the method design, the limitations of the published experiment, and a proposed validation log for a business process. There was no independent MemComposer run or measurement on customer data here. Installation, training, and hardware selection require separate verification.
*Cover image — an illustration of a fictional support scenario created by another generator, not the MemComposer interface or output.*
What to keep in mind: the usefulness of memory depends on the combination of episodes; reducing the memory block does not mean reducing the entire bill; the rollout decision should be based on completed tasks and total processing cost.
Contents
- What changed in the agent memory approach
- How MemComposer assembles memories
- What the experiment showed and how to read the numbers
- Why memory order needs separate validation
- How to test memory on support tickets
- Common questions about MemComposer
- How AIrassvet helps prepare memory workflows
- Where to start when validating agent memory
What changed in the agent memory approach
In this paper, AI agent memory means stored experience from task execution that can be added to a new request. The researchers work with procedural memory: the sequence of actions, the outcome, and the follow-up analysis. This differs from a factual reference library and from a user’s personal history. The authors explicitly draw that boundary in the Limitations section. Full text of the study.
Standard similarity search answers the question of which records are closest to the new request. MemComposer adds a further choice: which combination of retrieved records is useful to the executor. So even an exact retrieval does not determine the final context composition. Two similar episodes may repeat the same rule, while a less similar one contains the needed constraint.
If your company already uses a reranker in RAG, it is helpful to frame the difference by the selection outcome. A reranker re-scores retrieved candidates; MemComposer separately trains the selection of an ordered set of variable length. The paper studies the usefulness of the whole set through downstream task execution. This is an evolution of a specific memory mechanism, not proof that every document search should be replaced with a learned composer. Method description, §3.
How MemComposer assembles memories
In the published setup, a fixed retrieval mechanism first pulls ten candidates. The composer reads the request and those episodes, then returns an ordered subset. An empty set is allowed: the agent may complete the task without added memory. The executor receives the selected records in the given order and interacts with tools and a simulated user. Method and protocol, §3 and Appendix G.
For training, the authors propose Experience Policy Optimization, or ExPO. The reward accounts for the agent’s downstream success, the size of the selected set, and the correctness of the composer’s output format. Only the composer parameters are updated; the executor, retrieval mechanism, and memory bank remain fixed during this training. So the improvement in the experiment cannot be described as fine-tuning the entire agent system. ExPO, §3.4.
The nature of the stored experience also matters. The authors use both successful and unsuccessful trajectories together with a result analysis. An erroneous episode can sometimes contain a useful action or show the boundary of acceptable behavior. For a corporate process, that leads to an editorial recommendation: store the original action and the error explanation separately so the example does not look like a ready-made instruction. This rule itself was not tested on AIrassvet data.
What the experiment showed and how to read the numbers
The authors evaluated the method on τ²-bench, a multi-step customer service environment. Retail, airlines, and telecom were used; the executors were Qwen3-4B-Instruct and Ministral-3-8B-Instruct, and the user was the GLM-4.7 simulator. The purpose of the benchmark itself is described in the official τ²-bench repository; the specific experimental setup is listed in Appendix G of the MemComposer paper.
The test included 40 retail tasks, 20 airline tasks, and 40 telecom tasks, with four runs per task. The memory bank was built from the training tasks. The pass¹ metric means average success rate, and pass⁴ means the share of tasks successfully completed in all four runs. This is not “success in at least one of four attempts.” Protocol and Table 11.
For Qwen3-4B-Instruct, MemComposer reached an average pass¹ of 50.8 and pass⁴ of 32.5. For Ministral-3-8B-Instruct, the values were 63.5 and 40.0. The average quality metrics in the paper are calculated with equal weight across the three domains: the larger retail sample does not receive more weight. These are the authors’ benchmark results, not a measurement of a Russian-language contact center. Table 1 and §4.3.
Relative to the strongest baseline for each executor, the researchers report a reduction in the average number of memory tokens of 16.9% and 4.9%, respectively. Here they count only the inserted memory block, including its fixed header; the overall token figure is aggregated with weighting by the number of test runs in each domain. It uses a different denominator than average quality. Appendices G and H.1.
These percentages cannot be applied to the agent’s full budget. The memory-block metric by itself does not account for the composer’s total work, training, the rest of the dialog, tool calls, and employee intervention. To claim a reduction in company costs, the whole process would need to be measured on the company’s own task stream.
Why memory order needs separate validation
The study shows that the result is sensitive to the order of the same memories. But that does not mean MemComposer always finds the best order. In the Limitations section, the authors note that training changes both composition and order at the same time, so their contributions are linked; no comparison against all possible permutations was conducted. Study limitations.
There are two more important boundaries. The experiments were run with two relatively small executors; the size of the gain for a more powerful model remains an open question. In addition, procedural memory was tested. Transferring the result to knowledge-base facts or a customer’s personal history requires independent data, even if the storage interface looks similar.
This material is based on version 1 of the preprint. Independent reproduction and peer review of this work were not confirmed in preparing this article. We also did not verify the availability of a ready-made package, the code license terms, or whether it can be used by a specific Russian organization. The article does not promise a turnkey MemComposer integration with a CRM.
How to test memory on support tickets
Let’s imagine a hypothetical request: a customer wants to exchange a desk lamp while keeping the same payment method. The archive contains similar dialogs about lamps, returns, and changing an address. But a less similar episode about limiting exchanges after an order status change may turn out to be decisive. This illustrates the selection problem, not an actual customer case.
For the first comparison, it is useful to keep the agent, rules, tools, and task set unchanged. You can compare stateless execution, current search, and the proposed context assembly. First, you need to define the acceptable outcome of the request: a correct order change, a correct refusal, or escalation to a human agent. The editorial template below helps capture the differences; this is a proposed tool, and its usefulness has not yet been measured.
| What to record | What to compare | How this helps decision-making |
|---|---|---|
| The rules version and the order status | The same conditions across variants | A policy change is not masked as a memory effect |
| The IDs of retrieved episodes | The original candidate set | It is possible to separate a search error from a selection error |
| The selected set and its order | The full set and a permutation of the same records | It becomes clear which factor is being tested |
| The final state of the task | Action, refusal, escalation, and errors | Convincing text does not replace a correct outcome |
| The full processing cost | The composer, executor, tools, and human review | Context savings are evaluated within the total cost |
| Repeated runs and exceptions | Result stability and environment failures | A single successful dialog is not proof |
The acceptance threshold should be agreed before the comparison. For an order-related operation, it is useful to account separately for an impermissible data change: it may require stopping the process even if the average result is good. For an informational response, the criterion will be different. Failed runs and the reasons for exceptions should be saved together with successful ones so the result can be verified.
There is also a project stop condition. If the records are outdated, contain conflicting rules, or the relevant employee is unavailable, the data and access controls should be fixed first. A learnable selection layer should not replace the company’s current operating procedures. For an explanation of the causes, a separate material on AI agent failures may be useful.
Frequently Asked Questions about MemComposer
Does MemComposer replace RAG?
In the studied setup, similarity search remains in place and returns candidates. MemComposer works after that: it chooses the number of episodes, their composition, and their order. The work is about task execution experience; the result cannot be automatically transferred to any document database.
Does reducing memory tokens mean cost savings?
It reduces one component of the executor’s context. The full effect depends on the costs of the composer, training, tools, retries, and human review. Without measuring these components, a reduction in the overall budget remains unproven.
Should unsuccessful episodes be deleted?
In the experiment, unsuccessful trajectories with analysis sometimes contain useful experience. For a company, this is a reason to evaluate the content of the record and its applicability. An erroneous action should remain clearly labeled; it cannot be copied as an approved scenario.
Can the method be implemented in support right away?
The preprint provides a research framework and results from a limited experiment. Before implementation, you need to verify implementation availability and usage terms, suitable data, a secure environment, and your own assessment of the full process. This article does not confirm readiness for production integration.
How AIrassvet helps prepare memory-enabled workflows
For an agent memory task, AIrassvet is relevant for data preparation, corporate knowledge bases, and integrating AI agents with business systems. The discussion can focus on three actions: separate active procedures from request history, define the operations available to the agent, and formulate a verifiable processing outcome. This is task preparation within the studio’s areas of work, without promising deployment of a specific research codebase.
The first step is to choose one support process, describe its current handling, data sources, constraints, and acceptance criteria. That will make it clear whether a learnable memory assembly is needed, or whether search and archive quality should be fixed first. Discuss the task.
Where to start testing an agent’s memory
MemComposer suggests evaluating memory as an ordered set of experience that influences subsequent actions. For a support team, a practical starting point is the log of one process: which episodes were found, which entered the context, and what verifiable outcome they led to. The decision about the next step should be based on task completion quality and the total processing cost. The published reduction in memory tokens is a reason to run such a check, but its result on company data is still unknown.