Fault tolerance for an AI service is the ability of an AI assistant, agent, or RAG system to keep working or recover quickly when a cloud region, provider, or specific model is unavailable. To do this, teams prepare a backup model with another provider in advance, a copy of the knowledge base and vector index outside the primary site, portable configuration, and a manual process mode. Events on October 8–9, 2026 showed that a failure at a single cloud provider site can bring down the services of hundreds of companies running on it.
Prepared by AIrassvet based on open sources, verified October 9, 2026. It is intended for business owners, IT directors, and process leaders who already use AI assistants, chatbots, agents, and enterprise knowledge bases. The recovery situation at the time of verification was still changing; the current status should be checked in the providers’ official channels. AIrassvet is not affiliated with the companies mentioned, and the section at the end describes the publisher’s services.
Contents
- What happened on October 8–9, 2026
- Why an AI service is harder to recover than a website
- In brief: four levels of redundancy
- Backup model and routing layer
- Knowledge base and vector index
- What RTO and RPO to set for AI use cases
- Data and legal requirements during migration
- Drills: how to test backup before an outage
- Checklist for the next week
- Frequently asked questions
- Primary sources and verification method
What happened on October 8–9, 2026
According to company statements, as recounted by Rossiyskaya Gazeta and Fontanka, a fire broke out overnight on October 8, 2026, at Yandex’s data center in Sasovo, Ryazan Oblast, after a drone attack, affecting part of the infrastructure. The data center was fully shut down, and there were no casualties. On the morning of October 9, according to Fontanka, another company data center in Kaluga was damaged, and several of its modules failed.
What this meant for business:
- Yandex Cloud reported power supply issues in availability zone ru-central1-b; due to the incident, there could be difficulties creating new virtual machines, Kubernetes clusters, and databases in other zones, and customers were advised to move workloads to other zones where possible (Fontanka, The Moscow Times).
- According to Downdetector data cited by The Moscow Times, about 130,000 people reported issues accessing Yandex services over the course of the day. The publication also relays reports of outages at third-party companies — marketplaces, banks, developers, e-reporting services, and 1C cloud services.
- On October 9, Yandex said it had agreed with Selectel, K2Cloud, and VK Cloud on accelerated hosting of data and services for affected customers: specialist teams were allocated, flexible financial terms were offered, and onboarding was simplified (Gazeta, iXBT). As of October 9, no exact timeline had been given for restoring the damaged equipment.
Fontanka separately noted the practical point: moving to another provider lets companies resume operations without waiting for repairs, but that can require data backups and additional configuration. This is where companies with AI services run into problems that a standard website does not have.
Why an AI service is harder to recover than a website
A website usually consists of code, a database, and files. An AI service adds several more layers, and each one can be tied to a single site or a single provider:
- Model. If an assistant runs through a model API available only from one provider, there is nothing to replace it with if that provider goes down. If the model is deployed on dedicated GPUs in one zone, recovery requires free GPUs elsewhere — and there may not be enough when many customers move at once.
- Vector index and knowledge base. A RAG system stores documents, chunked passages, and their vector representations. The index can be rebuilt from the documents, but that takes time and requires the same embedding model; otherwise, search will work differently.
- Prompts, tools, and agent configuration. System instructions, tool definitions, and routing settings often live in the provider’s console rather than in the company’s repository.
- Keys, limits, and access. A new provider requires a contract, an account, API keys, and limits; during a large-scale migration, approval can take time.
- Integrations. An agent that works with a CRM, 1C, or email depends on those systems too — if they are on the same site, they fail together.
The main takeaway: a data backup is not the same as a backup AI service. You also need a backup model, a copy of the index or a procedure to rebuild it, configuration in your own storage, and pre-approved access to another provider.
In brief: four levels of redundancy
| Level | What it protects against | What is required | Cost and complexity |
|---|---|---|---|
| Multiple zones with one provider | Failure of one site | Hosting services and data in at least two availability zones | Lower than the others, but it does not help if the provider itself has problems |
| Backup provider | Provider outage, account lockout | Account, contract, and keys with a second provider; portable configuration; data copy outside the primary cloud | Medium; requires regular testing |
| Backup model | Failure or shutdown of a specific model | A second model with proven quality for your workloads and a layer that switches requests | Medium; requires a test question set |
| Manual process mode | Failure of the entire AI stack | A documented workflow without AI, assigned owners, and templates | Low, but the process runs more slowly |
Most small and midsize businesses do not need everything at once. A sensible minimum is a manual mode for each AI use case, a backup model with another provider for critical scenarios, and a copy of the knowledge base outside the primary site.
Backup model and routing layer
A model routing layer is an intermediary service between the application and the models that sends a request to the primary model and, if there is an error or timeout, to the backup one. It also lets you switch models without rewriting the application. For a detailed look at this approach, see the article “AI Model Router for Business”.
What you need to build in:
- Compatible interface. Many providers in Russia and abroad offer an API compatible with OpenAI. That makes switching easier, but it does not guarantee the model will behave the same way.
- Set of benchmark questions. 30–100 typical requests with reference answers for each scenario. Test the backup model against them in advance, not on the day of an outage.
- Separate prompts for the backup model. Instructions that work well with one model may produce a different response format with another.
- Degradation rules. If the backup model is weaker, some functions can be temporarily disabled: for example, keep knowledge-base answers but turn off the agent’s autonomous actions.
Knowledge base and vector index
For RAG systems, the recovery order depends on what exactly has been saved:
- Source documents — make sure to keep a copy off the primary platform. Without them, there is nothing to restore.
- Chunked fragments and metadata — save them along with the documents so you do not have to reprocess everything.
- Vector index — either a regular copy with another provider, or a tested rebuild procedure with measured timing.
- Embedding model — if it is available only from one provider, migrating will require rebuilding the index on another model and rechecking search quality.
Simple rule: you need to know the index rebuild time before an outage. If rebuilding takes a day, that is the minimum recovery time for a scenario without an index copy.
What RTO and RPO to set for AI scenarios
RTO is the acceptable time to restore the service. RPO is the acceptable data loss, meaning how much change can be lost over a given period. For AI scenarios, it is best to set them based on how critical the process is.
| Scenario | Example | RTO target | RPO target | Minimum protection |
|---|---|---|---|---|
| Customer channel | AI assistant in the website chat and messaging apps | Hours | One day for the knowledge base | Backup model, manual mode with routing to operators |
| Internal assistant | Search across policies and procedures for employees | One day | One week | Off-platform document copy, rebuild procedure |
| Agent in accounting systems | Processing requests, filling out forms in 1C | Hours or one day | Depends on the accounting system | Manual mode, log of unfinished actions |
| Analytics and reports | Weekly summaries | Days | One week | Copy of data and scripts |
The values in the table are editorial guidelines for discussion, not standards. The exact figures are set by the process owner based on the cost of downtime: how many inquiries, orders, or documents are lost per hour without the service.
Data and legal requirements during migration
A rushed migration is a common cause of data mistakes. Before moving an AI service to another provider, check:
- Where personal data will be processed. The new provider must meet the same requirements under Federal Law No. 152-FZ as the previous one. For rules on transferring data to AI services, see the article “152-FZ and Foreign Neural Networks”.
- Whether the data leaves the country. A backup model from a foreign provider may be a fast technical fix, but it means cross-border data transfer. For scenarios involving personal data, this must be decided in advance, not on the day of an outage.
- Transitional rules under 243-FZ. Under Federal Law No. 243-FZ starting March 1, 2027, the government will determine cases where only sovereign or national models are allowed; for systems operating on that date, the transitional period until September 1, 2032 applies provided that data is processed and stored in Russia. The backup setup should be built with this condition in mind.
- Contract and provider obligations. What the provider promises in the event of a zone failure, how it compensates for downtime, and how long it retains backup copies.
Drills: how to test the backup before an outage
A backup that has never been turned on is unlikely to work. Minimum drill program for an AI service:
- Once a quarter switch one scenario to the backup model for one hour during business hours and compare the answers with the reference.
- Twice a year restore the knowledge base and index with the backup provider from a copy and measure the time.
- Twice a year run a “day without AI” for one process: employees work according to the manual procedure, and you note where it is not documented.
- After each drill update the instructions, contacts, and keys.
This approach is close to what we described in the article “AI Service Exit Plan”: there we talked about a planned departure from a vendor, here about an unexpected failure, but in both cases preparation before the event is what matters most.
Checklist for the Coming Week
- Make a list of all AI use cases and, for each one, specify the provider, model, and the region or site where the knowledge base documents are stored.
- For each use case, name the owner and describe the manual fallback process in one paragraph.
- Check whether there is a copy of the knowledge base documents and agent configurations outside the primary cloud.
- Open an account with a second provider for critical use cases and test the backup model on control questions there.
- Set up model availability monitoring so you learn about outages before your customers do. Read more about monitoring in the article “Monitoring LLMs and AI Agents”.
Frequently Asked Questions
What is AI service resilience?
It is the ability of an AI assistant, agent, or RAG system to keep operating or recover quickly after a cloud region, provider, or model failure. It is ensured by a backup model, a copy of the knowledge base outside the primary site, portable configuration, and a manual fallback process.
Is it enough to host an AI service across multiple zones within one cloud?
That protects against a failure of one site, but not against a provider-wide issue, account suspension, or model shutdown. For critical use cases, you also need a backup provider or a standby model.
How quickly can you switch to a backup model?
Through a routing layer that sends the request to the backup model in case of an error or timeout. Failover only works if the backup model has been tested in advance with control questions and has its own instructions prepared.
What should you do with the vector knowledge base when moving to another provider?
Restore it from a backup or rebuild it from the source documents. If the embedding model is available only from the previous provider, the index will need to be rebuilt on a new model and search quality tested again.
Can a foreign model be used as a backup?
Technically yes, but for scenarios involving personal data, that means cross-border data transfer and requires a separate decision taking 152-FZ into account. In addition, starting in 2027, 243-FZ rules take effect for cases where only sovereign or national models are allowed.
Where should you start if you have no backup plan?
Start with a list of AI use cases, the owners, and a description of the manual fallback process for each one. Then create a copy of the knowledge base outside the primary cloud and test the backup model for the most critical use cases.
Source Documents and Verification Method
Verified on October 9, 2026:
- Rossiyskaya Gazeta, October 8, 2026 — a press release from Yandex about the fire and shutdown of the data center in Sasovo.
- Fontanka, October 8, 2026 — data center shutdown, a Yandex Cloud statement about the ru-central1-b zone, and recommendations to move workloads.
- The Moscow Times, October 8, 2026 — Downdetector data and a summary of reports about outages at third-party companies.
- Fontanka, October 9, 2026 — damage to the data center in Kaluga.
- Gazeta, October 9, 2026 and iXBT, October 9, 2026 — Yandex’s agreement with Selectel, K2Cloud, and VK Cloud to migrate customers.
- ConsultantPlus, 243-FZ — the timing of the rules on sovereign and national models and the transition period.
The incident description is based on company and media reports as of October 9, 2026; subsequent recovery is not reflected in the article. The redundancy levels, RTO/RPO benchmarks, exercise program, and checklist are AIrassvet editorial analysis; the article does not provide its own measurements or project results.
How AIrassvet Helps Make AI Services Resilient
If your AI assistant, agent, or knowledge base depends on a single site or a single model, the task can be broken down step by step. On the AIrassvet website, which also uses the name “Vladimir Lomtev’s AI Studio,” the following directions are described as applicable:
- AI use case audit: which models, platforms, and integrations are used, and where there is no fallback option.
- Integration of multiple models, including domestic ones, through a single layer so that switching to a backup model does not require rewriting the application.
- Deploying a language model on the company’s servers or in a secure cloud as a backup environment for scenarios involving confidential data.
- Migration and recovery of the corporate knowledge base and RAG system outside the primary site.
The first step is to choose one process where AI is already in use, describe its current metrics, data sources, constraints, and the criterion by which you will consider the backup operational.
Bottom line: prepare AI backup before an outage
The events of October 8–9, 2026 showed that a cloud provider site outage affects not only its own services, but also the companies operating there. Providers organized accelerated customer migration, but moving still requires backups and preparation.
For AI services, that is not enough: you need a standby model, copies of the knowledge base and index, configuration in your own storage, and a manual fallback process. Start with the most critical use case—describe its manual fallback process and test the backup model on control questions while the primary one is still running.