An LLM firewall is a single control point between users, internal services, and language models that inspects every request to the model and every response from it: it looks for prompt injection and jailbreak attempts, strips personal data, passwords, and keys, and if a threat is detected, it warns, masks data, sends the request to quarantine, or blocks it. It is needed when a company has dozens or hundreds of AI applications and agents, and traditional DLP and WAF tools do not understand the meaning of text-based prompts. But a firewall is one layer of defense, not a substitute for least-privilege access for agents, logging, and human review of risky actions.
Prepared by AIrassvet October 15, 2026 based on Rostelecom’s request for information on an LLM application communications control system (TAdviser, October 2026) and on the OWASP Top 10 for LLM Applications 2025 risk list. It is intended for IT directors, information security leaders, and product owners whose AI assistants, chatbots, and agents are already working with corporate data. We did not test commercial LLM firewalls and do not compare vendors: the requirements checklist and implementation plan are the editorial team’s methodology. AIrassvet is not affiliated with Rostelecom, TAdviser, or OWASP; the section at the end of the article describes the publisher’s services.
Contents
- In Brief
- Why LLM firewalls are being discussed now
- What an LLM firewall is and how it differs from DLP and WAF
- How an LLM firewall works
- What risks it protects against, and what it does not
- LLM firewall requirements checklist
- Step-by-step implementation plan
- How to tell whether the protection is working
- Frequently asked questions
- Primary sources and verification method
In Brief
- The growth in AI applications is creating a new attack surface. According to Rostelecom, at least 1,500 solutions based on large language models are operating in its environment—from internal assistants to customer chatbots.
- An LLM firewall analyzes meaning, not just signatures. It checks both the incoming prompt and the generated response, because data leakage can happen in either direction.
- The response is not necessarily blocking. The system described in Rostelecom’s request includes warning, data masking, quarantine, and blocking, with an incident ticket sent to the SOC and SIEM.
- There is no complete protection against prompt injection. OWASP states directly that no reliable prevention methods are currently known, so firewalls are combined with least privilege and human review.
- It makes sense to start with inventory and monitoring moderather than strict blocking: first, you need to see real traffic and the rate of false positives.
Why LLM firewalls are being discussed now
In October 2026, TAdviser reported that Rostelecom was exploring the creation of a “system for controlling communications of LLM applications” and had issued a request for information to suppliers; applications are being accepted until October 19 (TAdviser; the news was also published on Habr on 10/15/2026). From the technical requirements, it follows that:
- the system must protect against Prompt Injection and Jailbreak attacks, as well as leaks of personal data, trade secrets, and credentials through requests to language models;
- it sits as a single traffic gateway between users, internal services, external language models, and knowledge bases (RAG), and the actions of autonomous AI agents are also monitored;
- inspection happens in real time: incoming prompts and model responses are analyzed, and attempts to extract system settings or secretly exfiltrate restricted data through leading questions are stopped;
- the content filter removes customer data, trade secrets, passwords, access tokens, and API keys;
- the software must be included in the Ministry of Digital Development registry, and for heavy-duty request analysis, the environment provides servers with NVIDIA H100-class accelerators or domestic equivalents.
Rostelecom’s Information Security Projects Director Ekaterina Filippova told TAdviser that the baseline level of protection is already in place and that the company is piloting solutions for additional control of LLM services. Experts in the same piece note that traditional DLP and WAF tools can no longer handle the new challenges, and Vladislav Tushkanov of Kaspersky calls multilayered defense the foundation of security, with guardrails, for example within an LLM firewall, being one option.
For smaller businesses, the story points to something else: the problem appears not when a company has one model, but when AI services are introduced piecemeal across different departments without centralized visibility. We covered how to find these integrations in the article on shadow AI in the company.
What an LLM firewall is and how it differs from DLP and WAF
An LLM firewall is a security gateway for language models: it routes requests to models and their responses through itself and applies policies to them based on the meaning of the text. Unlike a network firewall, it does not look at addresses and ports, but at the content of the conversation: what the user is asking for, what data is being sent to the model, and what the model returns.
Guardrails are rules and mechanisms that constrain the model’s behavior: banned topics, required response format, sensitive data filters. An LLM firewall is one way to apply guardrails centrally instead of building them into every application.
| DLP | WAF | LLM firewall | |
|---|---|---|---|
| What it protects | Data transfer channels: email, USB, cloud services | Web applications from HTTP-based attacks | The dialogue with the language model and agent actions |
| How it detects a threat | Patterns and labels for confidential data | Signatures and rules for HTTP requests | The meaning of the request and response text, and the context of the conversation |
| Does it detect prompt injection? | No | Usually no: an injection looks like ordinary text | Yes, that is its main job |
| Does it inspect the model’s response? | Only if the response goes through a controlled channel | No | Yes |
| Typical reaction | Blocking data transfer | Blocking the request | Warning, masking, quarantine, blocking |
The table is the editorial team’s summary of the purpose of the solution classes, not a comparison of specific products.
How an LLM firewall works
The diagram below is based on Rostelecom’s request description and OWASP recommendations. This is the logic of operation, not the architecture of a specific product.
- Single entry point. All applications and agents access models only through the gateway: direct keys to external APIs are revoked from departments.
- Source identification. The gateway knows which application, which user, and which agent sent the request—without this, neither policy enforcement nor incident investigation is possible.
- Incoming request inspection. Detection of prompt injection and jailbreak indicators, attempts to extract the system prompt, and sensitive data: personal data, trade secrets, passwords, tokens, and keys.
- External content inspection. Documents from the knowledge base, web pages, and files that the agent pulls into context are checked separately: this is how indirect injection gets in.
- Model response validation. The response is checked against policy before it reaches the user or another system: does it expose restricted data, does it match the expected format.
- Response by risk level. Warn the user, mask the fragment, place the request in quarantine for manual review, or block it.
- Logging and incidents. Every trigger is written to the log, and serious ones are turned into incident tickets in SOC and SIEM.
Why the response is checked as strictly as the prompt: leakage can happen both ways. The user may not submit anything secret, but by asking leading questions they can get the model to repeat what it found in the corporate knowledge base.
Latency is a separate issue. Semantic validation requires compute, so Rostelecom’s request for analysis includes servers with accelerators. For a smaller company, that means latency and validation cost need to be measured on your own traffic before turning the firewall on for all applications.
What it protects against, and what it does not
OWASP, the open application security community, maintains a list of the ten biggest risks for LLM-based applications (OWASP Top 10 for LLM Applications 2025). Below is the editorial assessment of which of them are covered by a gateway that sees prompts and responses, and which require other measures.
| OWASP 2025 Risk | How an LLM firewall helps | What you still need beyond it |
|---|---|---|
| LLM01 Prompt Injection | Recognizes some direct and indirect injections, separates external content | Least privilege, human review for risky actions, regular red-team attacks |
| LLM02 Sensitive Information Disclosure | Masks and blocks sensitive data in prompts and responses | Data classification, knowledge base access controls |
| LLM05 Improper Output Handling | Checks response format and content before passing it on | Code-based response validation in the receiving system |
| LLM06 Excessive Agency | Sees and logs agent actions | Limiting the agent’s tools and permissions is handled elsewhere, not by the filter |
| LLM07 System Prompt Leakage | Stops attempts to extract system settings | Do not store secrets in the system prompt |
| LLM10 Unbounded Consumption | Can limit request volume and frequency | Provider budget limits, cost monitoring |
| LLM03 Supply Chain, LLM04 Data and Model Poisoning | Almost no help | Checking models, libraries, and training data |
The main limitation is stated by OWASP itself: because of the probabilistic nature of models, it is unclear whether reliable methods exist to prevent prompt injection (OWASP, LLM01). OWASP recommends a set of measures that reduce damage: constrain model behavior, validate output format in code, filter input and output, give the application least privilege, require human approval for risky actions, separate external content, and run adversarial testing regularly. Filtering is only one of seven measures.
So an LLM firewall does not replace architectural decisions. If an agent can delete files or send an email, that permission has to be restricted in the system itself, rather than hoping the gateway will recognize a malicious instruction. We covered how attacks through images and documents work in the article on multimodal prompt injection, and the basic rules for securing an AI chat are in the piece on protecting AI agents.
LLM Firewall Requirements Checklist
This checklist was prepared by the editorial team based on the Rostelecom request description and OWASP measures. It can be used as the basis for a technical specification or a vendor questionnaire.
Coverage
- Can all applications, chatbots, and agents be routed through the gateway, including requests to RAG and agent tools?
- Are the required models and providers supported, including local models in your own environment?
- Are both prompts and responses checked, as well as external content that is added to context?
Detection quality
- How does the system handle Russian and mixed Russian-English prompts?
- Can you add your own sensitive-data categories: contract numbers, internal codes, project names?
- How does the vendor show the rate of false positives and misses on your examples, not just demo ones?
Responses and processes
- Are there different response levels: warning, masking, quarantine, blocking?
- Who reviews quarantined items and within what timeframe?
- Are incidents sent to the SOC and SIEM you already use?
Operations
- How much latency does validation add, and how does it scale under peak load?
- Where are chat logs stored, who has access to them, and how long are they retained? The prompt log itself contains sensitive data.
- Does the product need to be listed in the Russian software registry, and is it compatible with your infrastructure?
If the company sends data to external providers, separately check what happens to it on the provider side; we wrote about no-storage modes in the article on Zero Data Retention, and about the rules for transferring customer data in the piece on anonymization.
Step-by-step implementation plan
This is the editorial team’s plan, not the result of our own implementation. The timing of each stage depends on the number of applications and is not provided.
- Inventory. Create an inventory of AI applications and agents: who owns them, which models are called, what data is transmitted, and whether they have access to tools. Without an inventory, it is unclear what exactly needs to be protected.
- Single exit point to models. Route model requests through a single gateway. This is useful even without filters: you get request and cost tracking. We described how this routing layer works in the article on model routing.
- Monitoring mode. Turn on logging-only checks. Several weeks of real traffic will show which data actually reaches the models and how many false blocks there would have been.
- Masking and soft responses. Start by masking clearly sensitive data and warning users, then enable quarantine.
- Blocking for high risk. Enable hard blocking for categories where the false positive rate on your traffic is acceptable.
- Adversarial testing. Regularly test your defenses with a set of Russian-language injections and jailbreaks, including injections through knowledge base documents.
How to tell whether the defense is working
The metrics below are guidelines for reporting, not requirements. Thresholds should be defined based on your own traffic during the observation phase.
| Metric | What it shows |
|---|---|
| Share of applications and agents connected through the gateway | Coverage completeness |
| Number of triggers by category: injections, personal data, secrets | Where the real risks are |
| False positive rate based on quarantine review results | Whether the defense is disrupting operations |
| Misses found through adversarial testing | Actual detection effectiveness |
| Median and 95th percentile added latency | The cost of protection for users |
| Quarantine review time | The workload on the security team |
Frequently asked questions
What is an LLM firewall?
It is a security gateway between users, applications, and language models. It checks prompts sent to the model and its responses for meaning, looks for prompt injections, jailbreaks, and sensitive data, and depending on the risk either warns, masks data, sends the request to quarantine, or blocks it.
How is an LLM firewall different from DLP?
DLP monitors data transfer channels and looks for confidential data using patterns and labels. An LLM firewall works inside the conversation with the model: it understands attempts to manipulate the model and checks its responses. These systems complement each other; they do not replace one another.
Does an LLM firewall protect against all prompt injections?
No. OWASP notes that no reliable methods for preventing prompt injections are known yet. A firewall reduces the risk, but it should be combined with least privilege for applications and agents, human review of risky actions, and regular adversarial testing.
Do you need an LLM firewall if the company has only one AI application?
Not always. With one application, the rules can be built into it directly. A single control point pays off when the number of models, applications, and agents grows, different departments connect them, and you need a common log and unified policies.
Does an LLM firewall slow down model responses?
Semantic checking requires compute and adds latency. The amount depends on the product, the verification models, and the load, so you should measure latency on your own traffic in observation mode before enabling blocking.
Where should you start when implementing an LLM firewall?
Start by inventorying AI applications and agents and routing model requests through a single gateway. Then move to observation mode without blocking, masking and soft responses, and only after measuring false positives enable blocking for high risk.
Primary sources and verification method
Method: on October 15, 2026, we read the Rostelecom project description on TAdviser and the Habr article, extracted the system requirements, and compared the risk and protection wording with current OWASP pages. AIrassvet did not conduct its own tests of LLM firewalls; the comparison tables, checklist, implementation plan, and metrics are the editorial team's methodology.
- TAdviser: Rostelecom will build a communications control system for 1,500 AI applications — description of the information request, requirements, and comments from Rostelecom and Kaspersky Lab.
- Habr, 10/15/2026: news about the communications control system for LLM applications.
- OWASP: Top 10 for LLM Applications 2025, LLM01 Prompt Injection, LLM02 Sensitive Information Disclosure.
Limitations: the Rostelecom project is still at the information request stage, so the final architecture and vendor are unknown; the detection effectiveness of specific products was not measured in the article.
How AIrassvet helps protect enterprise AI applications
An LLM firewall works best when the AI applications themselves are designed with it in mind: it should be clear what data goes where and what agents are allowed to do. On the AIrassvet website the following directions are described as directly relevant to this task:
- auditing processes and preparing data: an inventory of AI applications, a map of the data sent to models, and classification of sensitive information;
- deploying language models on the customer’s servers for scenarios where data cannot be sent to external providers;
- developing AI agents integrated with enterprise systems, with a limited set of tools and human approval for risky operations;
- integration and testing before launch, including testing against prompt-injection scenarios.
The first step is to choose one AI application with real users, describe what data it sends to the model and from which systems, what security constraints apply, and what criterion will count as sufficient protection.
Bottom line: a firewall as a layer, not a wall
The Rostelecom request shows where large companies are headed: from fragmented AI services to a single control point that sees every request to the model and every response. An LLM firewall covers what DLP and WAF cannot see: manipulation of the model and leaks within the conversation.
But OWASP explicitly warns that there is no complete protection against prompt injections. That is why a firewall should be implemented together with least privilege for agents, logging, and human review of risky actions: start with inventory and observation, and turn on blocking based on false positive data.