RAG System for Business is an application that, before answering, searches for user-authorized fragments from corporate sources, passes them to a language model as context, and shows the basis for the answer. Unlike model fine-tuning, knowledge is updated through the index and source content, not by changing the model’s weights.
A working RAG system is not just a “vector database.” It needs two main pipelines: content ingestion and updates, plus query processing. A control layer sits above them — access rights, versions, evaluation, observability, and incident response.
Short answer: start with one process, a source registry, and a set of questions with known answers. First prove that retrieval finds the right fragment and does not return anything restricted; then verify that the answer is supported by the retrieved context. Only after that evaluate usability, speed, and economics.
Key takeaways in one minute
- RAG is a good fit for changing or private knowledge, but it does not fix bad source content.
- User permissions are applied before retrieval and context assembly.
- Chunking, metadata, hybrid search, and rerank should be tested against their own questions.
- Evaluate retrieval and generation separately: otherwise the source of the error is lost.
- A citation in the answer is useful only if the fragment actually supports the conclusion.
- The system should be able to clarify the question, refuse to answer, and hand off to a human.
- Freshness is a measurable process with an owner, not a promise that “the database updates itself.”
Contents
- When a Business Needs RAG
- The KONTUR RAG Method
- Three Architecture Layers
- Source Registry
- Access Rights Before Retrieval
- Document Preparation
- Search and Rerank
- How the Answer Is Generated
- How to Measure Quality
- Freshness and Observability
- RAG Security
- Pilot Plan
- What to Accept from a Vendor
- Frequently Asked Questions
- How AI Rasvet Designs RAG Systems
- Conclusion
When a Business Needs RAG
RAG makes sense when the answer needs to rely on internal policies, contract templates, technical documentation, a product catalog, ticket history, or other sources that change and have owners. Typical interfaces include an employee assistant, knowledge base search, an operator copilot, and a customer-facing chat with a limited scope.
Do not start with RAG if the task can be solved with a precise database filter, standard full-text search, or a deterministic business rule. If the answer must be numerically exact and is already stored in a structured field, it is better to call the API and display the value instead of asking the model to restate it.
RAG also does not fix conflicting documents. If two active policies give different answers, the system should surface the conflict or choose a source based on a predefined priority — but the process owner defines that priority.
The KONTUR RAG Method
KONTUR RAG is six checks for a production system:
- K — Corpus: the sources, format, owner, and authoritative priority are known.
- O — Constraints: identity, tenant, role, and document ACLs are enforced before retrieval.
- N — Normalization: parsing, OCR, chunking, metadata, and versions are reproducible.
- T — Targeted Search: lexical and semantic candidates are combined and reranked.
- U — Authorized Answer: the output is supported by context and includes a citation or a refusal.
- R — Regression: frozen eval runs after changes to data, the model, or configuration.
This framework helps localize the problem. If the needed fragment is not in top-k, changing the prompt will not fix retrieval. If the fragment is found but the model makes an unsupported conclusion, that is a generation or answer-policy error.
Three Architecture Layers
| Layer | Input | Key Steps | Output |
|---|---|---|---|
| ingestion | document or record | parse/OCR → chunk → metadata/ACL → embedding/index | fragment version |
| query | user and question | auth → filters → retrieve → rerank → context → generate | answer, links, status |
| control | events and evaluations | eval → monitoring → review → rollback/reindex | decision and log |
Microsoft Architecture Center separates the application flow from the data flow: documents go through chunking, enrichment, embedding, and storage in an index, while the query goes through search, context assembly, and model invocation. For business use, you need to add identity, ACLs, corpus versioning, and a decision log to this flow.
A simple RAG system performs one search over one index. Agentic retrieval can decompose a complex question, select sources, and execute multiple queries. A more complex setup is justified only after the basic pipeline has been measured: extra steps increase cost, latency, and the number of failure points.
Source Registry
Before development, create a source registry:
| Field | Why It Matters |
|---|---|
| source_id and URI | find the original and remove it from the index |
| owner | confirm the content and accept corrections |
| authoritative priority | resolve version conflicts |
| audience/ACL | do not show another person’s document |
| updated_at/version | check freshness and reproduce the answer |
| parser/type | understand the risk of losing tables, footnotes, and structure |
| expiry/SLA | exclude expired knowledge |
PDFs with scans, a table, a case database, and a wiki require different handlers. A parsing error can leave you with a nice-looking page and no useful text, so selectively compare the indexed fragment with the original.
The practice of managing knowledge owners and the knowledge lifecycle is covered in more detail in the article about a corporate knowledge base with an AI agent.
Access rights before retrieval
The right sequence is: authenticate the user, retrieve their attributes, build the required filters, and only then search for candidates. Filtering after retrieval is dangerous: forbidden text may already have entered the context, log, or intermediate cache.
Azure AI Search describes document-level security trimming, inheritance of permission metadata, and query-time filtering. The exact implementation depends on the platform, but the invariant is the same: the generator sees only what the request subject is allowed to see.
Test not only positive roles. Your eval should include a user with no group, a terminated employee, a department change, two tenants, a closed document with a similar title, and a previously allowed document after access has been revoked. For access leakage, the criterion is usually binary: forbidden content must not appear in the answer, the citation, or the debug interface.
Document preparation
A chunk is not an arbitrary span of characters, but the smallest fragment that can be found and understood with enough context. For a policy document, the section heading, clause number, product, region, effective date, and link to the original are useful. For a table, you may need to preserve row and column headers inside each fragment.
Test several strategies on one frozen set: by section, by sentence, with a fixed window and overlap, or with a layout-aware parser. A larger chunk provides more context, but adds noise and consumes model context window; a too-small chunk loses conditions and exceptions. There is no universal size.
Metadata should support filters and diagnostics, not just decorate the index. Minimum: source, version, section, language, product, access labels, and timestamps. Mark automatically generated summaries and keywords as derived data and recreate them together with the parser/model version.
Search and rerank
In corporate data, you’ll find both semantic questions and exact part numbers, contract numbers, and abbreviations. That is why the basic testable hypothesis is often hybrid retrieval: keyword search preserves exact matches, vector search finds semantic matches, and the reranker resorts the combined set.
The official Microsoft overview of RAG directly recommends evaluating hybrid queries and semantic ranking. This does not guarantee the best result on any corpus: compare the options on your own queries and lock in the experiment configuration.
Retrieval diagnostics:
- source absent — the document is not connected or not updated;
- parse miss — the needed text was lost;
- chunk miss — the condition was split apart;
- filter miss — metadata or ACL excluded the document;
- recall miss — the candidate did not make it into the set;
- rank miss — the candidate was found but ended up below the context cutoff;
- query miss — the user’s term was not matched to the document’s term.
How the answer is formed
The orchestrator assembles the question, the allowed top fragments, the instruction, the policy, and the answer format. The model must distinguish four outcomes: answer, clarify, abstain, handoff. The requirement to “always be helpful” encourages guessing where there is no source.
A citation is provenance, but not automatic proof. The reviewer checks that the cited fragment supports every material claim, is not outdated, and refers to the correct product or contract. If there is a conflict, the system shows the conflict and versions, or follows the published priority rule.
RAG reduces dependence on knowledge memorized by the model, but it does not eliminate hallucinations. GOV.UK AI Insights separately lists retrieval quality, hallucinations, latency, security, and evaluation as distinct aspects of the system.
How to measure quality
Do not reduce quality to a single percentage. Build a frozen eval set from real, synthetic, and negative queries, removing personal data. For each question, store the allowed sources, expected meaning, user role, required outcome, and critical-fail label.
| Layer | Test question | Example signal |
|---|---|---|
| retrieval | was an allowed source found | recall@k, rank, filter correctness |
| grounding | is the output supported by the context | claim-support review |
| answer | is the answer correct and complete | rubric reviewer |
| abstention | did it refuse when knowledge was absent | correct abstain |
| access | was forbidden content not disclosed | leakage count |
| operations | is the service stable | latency, errors, cost per accepted answer |
Automated judges speed up regression testing, but they are calibrated against human labeling. Microsoft guide recommends evaluating stages separately and documenting hyperparameters and results. Acceptance thresholds are set by the process owner based on the cost of error; there are no universal values.
Freshness and observability
For every change, a useful chain is: event in the source → ingest job → parse status → index version → eval → activation. The dashboard shows lag by source, parsing errors, documents without an owner, expired versions, empty ACLs, the share of answer/clarify/abstain/handoff, and problematic queries.
In the answer log, store the minimum allowed set: request id, user/role hash, query class, index/model/prompt versions, document ids, scores, outcome, and feedback. Query and fragment content may be sensitive, so retention and log access are defined separately.
Rollback should restore a consistent bundle — index, parser, retrieval settings, prompt, and policy. Rolling back only the model will not help if the defect was caused by corrupted indexing.
RAG security
The retrieved document is untrusted input. Inside a file there may be an instruction for the model, intentional or accidental. OWASP LLM01:2025 notes that RAG and fine-tuning do not fully eliminate prompt injection.
Minimum controls:
- separate system policy, user query, and retrieved content;
- do not let a document change the tools list or permissions;
- restrict actions with an allowlist, schema validation, and approval;
- isolate tenants and caches;
- scan sources and verify provenance;
- redact secrets/PII in logs;
- test indirect injection and poisoned documents;
- have a kill switch, incident owner, and reindex procedure.
If RAG is used in a customer-facing interface, additional requirements for handoff and the widget are covered in the article about an AI chatbot for a website.
Pilot plan
- Choose one process and one user group.
- Establish a baseline: search time, errors, escalations, and process cost based on the defined methodology.
- Create a source registry, owners, ACLs, and authoritative priority.
- Prepare a frozen eval with answerable, unanswerable, conflicting, and unauthorized queries.
- Build ingestion and verify sample chunks against the originals.
- Compare retrieval configurations without generation.
- Add answer/clarify/abstain/handoff and citation checks.
- Launch a shadow test or a limited beta group.
- Set up freshness, monitoring, feedback, and an incident runbook.
- Make the
scale / revise / stopdecision according to preapproved criteria.
Initial sources, constraints, and acceptance criteria can be identified as part of the business process audit before AI implementation.
What to accept from the contractor
- a map of ingestion/query/control and a list of components;
- a source registry, owners, ACL mapping, and version policy;
- a description of parser/chunking/metadata and a reproducible reindex;
- a frozen eval with labeling and layer-by-layer reports;
- a threat model, access tests, and prompt injection tests;
- an observability dashboard, retention, and audit log;
- a runbook for stale sources, leakage, outages, and rollback;
- a CAPEX/OPEX estimate with clearly stated assumptions;
- documentation, access to configuration, and a handoff procedure.
The service page AI Dawn RAG systems describes the development and integration format. This article serves as a technical checklist for selecting and accepting the solution.
Frequently asked questions
Is RAG a vector database?
No. A vector store is just one possible retrieval component. A production system also includes ingestion, identity and ACLs, an orchestrator, generation, citations, eval, monitoring, and operations.
Is vector search always necessary?
No. For exact codes and structured fields, keyword search, a filter, or an API may be better. The choice is validated on representative queries; a hybrid approach is often tested.
Does RAG eliminate hallucinations?
No. It gives the model external context, but retrieval can return the wrong passage and the model can still draw an unsupported conclusion. Grounding review, abstention, and regression eval are needed.
How often should the index be updated?
It depends on source criticality and acceptable lag. The policy should have an owner, a trigger, an SLO, and failed job monitoring. There is no single interval for all documents.
Can all company documents be loaded into RAG?
Technically, the volume can be expanded, but a pilot is better limited to one process. Bulk loading without owners, ACLs, versions, and priorities increases the risk of conflicts and leaks.
How do you estimate the cost of RAG?
Split discovery, data preparation, integration, eval, and launch as CAPEX; and models, search, storage, reindexing, monitoring, and knowledge operations as OPEX. A general budgeting model is in the article on the cost of AI implementation.
How AI Dawn designs RAG systems
AI Dawn can audit the process and sources, design ingestion and permission-aware retrieval, prepare RAG and integrations, build eval, set up monitoring, launch, training, and support.
The safest first step is to choose one process, establish a baseline, sources, permissions, constraints, and one verifiable acceptance criterion. After that, you can compare retrieval options on a frozen eval and decide whether to move forward with a pilot. Discuss the project.
Conclusion
A business RAG system is a governed knowledge layer, not just “embeddings plus chat.” The KONTUR RAG method connects the corpus, constraints, normalization, precise retrieval, an approved answer, and regression testing.
Start with one process and a source registry. Apply ACLs before retrieval, measure search separately from generation, require citations and a proper refusal, and manage freshness and versions. Then you can make decisions based on observed errors and acceptance criteria, not on the impression from a successful demo.