Checklist for Choosing an Embedding Model for RAG

The embedding model for RAG should be chosen based on tests on your own documents and questions, not by where it ranks overall. MTEB and ruMTEB rankings help narrow the shortlist, but the final decision comes down to the share of correctly retrieved passages on your test set, vector size and storage cost, input length, license, and where the data is processed. Below is a seven-step checklist with owners and “done” criteria, plus a copyable template.

Prepared by AIrassvet October 23, 2026 for project leaders, IT directors, and ML engineers launching a knowledge base search system or a Russian-language RAG assistant. Model specifications were taken from their Hugging Face model cards and documentation, verified on that date. We are not publishing our own comparative model benchmark: results on other people’s data do not transfer well to yours.

In Brief
Who This Checklist Is For and What It Does Not Cover
What You’ll Need Before You Start
Stage 1. Requirements and Constraints
Stage 2. Shortlist of Candidates
Stage 3. Test Set on Your Own Data
Stage 4. Runs and Metrics
Stage 5. Storage, Speed, and Hardware
Stage 6. License and Data
Stage 7. Decision and Version Lockdown
Copyable Checklist
Common Mistakes
Frequently Asked Questions
Primary Sources and Method

In Brief

A ranking is a filter, not the answer. The MTEB authors note that no embedding method leads on every task, so the final choice should be made on your own data.
Compare models with the same chunking and with the correct prefixes or instructions: for many models, search quality is lower without them.
Vector size costs money. In float32, a vector of 1,024 numbers takes about 4 KB, and 4,096 numbers take about 16 KB, excluding the index.
Switching models means reindexing the entire database: vectors from different models are incompatible.
Lock the model version, prefixes, dimensionality, and normalization in the project documentation.

Who This Checklist Is For and What It Does Not Cover

This checklist is for when a company is building semantic search or a RAG assistant for Russian-language documents and choosing a model that turns text into vectors. We explained what embeddings and a vector database are in the article “What Is a Vector Database”. This does not cover document chunking, result reranking, or answer generation — separate materials for those are linked throughout the text.

Terms. *Embedding* — a vector of numbers that represents the meaning of text. *Dimensionality* — the number of numbers in a vector. *Recall@k* — the share of questions for which the correct passage appears in the first k search results. *MRR* — the average reciprocal rank of the first correct passage. *MRL* (Matryoshka Representation Learning) — training that lets you shorten the vector without retraining the model.

What You’ll Need Before You Start

Roles. The process owner knows what questions employees or customers ask. The ML engineer runs the models and calculates metrics. DevOps is responsible for servers and the vector database. The information security specialist or lawyer checks licenses and where data is processed.
Documents. An export of the real corpus or a representative subset of it, already chunked into passages. How to do this is described in the document chunking guide for RAG.
Questions. Real wording from email, chats, tickets, or portal search.
Test environment. A server or cloud machine where several models can be run, plus a library for calculating metrics.

Stage 1. Requirements and Constraints

Owner: the process owner and the information security specialist. Done when: the constraints are recorded in a single document and signed off.

☐ Corpus and question languages: Russian only, or Russian plus English.
☐ Document types and average passage length in tokens.
☐ Where text processing is allowed: external API, Russian cloud, or only your own servers.
☐ Acceptable model licenses for commercial use.
☐ Expected volume: number of passages now and in one year.
☐ Search latency requirements and hardware budget.

Stage 2. Shortlist of Candidates

Owner: the ML engineer. Done when: three to five models have been selected, and for each one the dimensionality, input length, license, and text input rules have been recorded.

☐ Filter models by Russian-language retrieval tasks in the MTEB ranking, which includes the ruMTEB set.
☐ Exclude models that do not meet the Stage 1 constraints on license or processing location.
☐ Check the model card for the maximum input length and compare it with the length of your passages.
☐ Write down any required prefixes or instructions for queries and documents.
☐ Include at least one model trained specifically for Russian and one multilingual model.

ruMTEB is the Russian-language extension of the MTEB benchmark from the SaluteDevices team: according to the authors’ description, it includes seven task categories, including retrieval, reranking, classification, and semantic similarity. The ranking is useful as a filter, but its datasets do not match your documents.

Starter candidates: specifications from model cards as of October 23, 2026

Model	Developer	Vector size	Input length	License	How to input text
multilingual-e5-large	intfloat	1024	512 tokens	MIT	Prefixes "query: " and "passage: "
bge-m3	BAAI	1024	8192 tokens	MIT	No instructions; supports dense, sparse, and multi-vector modes
USER-bge-m3	deepvk	1024	8192 tokens by configuration	Apache 2.0	A compressed version of bge-m3 trained for Russian; the authors did not evaluate other languages
ru-en-RoSBERTa	ai-forever	1024	512 tokens by configuration	MIT	Prefixes "search_query: " and "search_document: "
FRIDA	ai-forever	1536	512 tokens	MIT	Same prefixes; the authors did not evaluate quality in English
Qwen3-Embedding-0.6B	Qwen	up to 1024, can be shortened to 32	32K tokens	Apache 2.0	Instruction in the query, documents without one
Qwen3-Embedding-8B	Qwen	up to 4096, can be shortened to 32	32K tokens	Apache 2.0	Instruction in the query, documents without one
GigaChat Embeddings	Sber	1024	512 tokens	API terms	Cloud service; texts are sent to the provider

This table is not a quality ranking. It is a starter list that you need to validate on your own data in stages 3 and 4.

Stage 3. Test set on your own data

Owner: process owner and subject matter experts. Done when: for each question, at least one correct passage is marked, and the set is aligned with the expert.

☐ Collect several dozen real questions, and more for critical scenarios.
☐ Mark the passages that contain the answer for each question.
☐ Include difficult cases: contract numbers and part numbers, abbreviations, jargon, and questions in English if they occur.
☐ Add questions that have no answer in the knowledge base so you can spot false matches.
☐ Save the set in a versioned file so you can rerun the test after changes.

This same set will later be useful for testing the entire RAG system — our article on RAG accuracy and hallucinations explains that in more detail.

Stage 4. Runs and metrics

Owner: ML engineer. Done when: recall@5, recall@10, and MRR have been calculated for each model on the same set and recorded in a table.

☐ Use the same document chunking for all models.
☐ Feed text exactly as specified in the model card: with the required prefixes or instruction in the query.
☐ Check whether the chunk is being truncated: with a 512-token limit, long text loses its end.
☐ Normalize vectors and use the similarity metric recommended in the model card.
☐ Manually review the questions where all models failed: often the issue is chunking or the documents themselves.
☐ Repeat the best setup with reranking and check whether it is needed — see the article about rerankers in RAG.

The Qwen3-Embedding authors note in the model card that without an instruction on the query side, search quality drops in most scenarios. For multilingual-e5-large, ru-en-RoSBERTa, and FRIDA, prefixes serve the role of the instruction. Comparing models without them would be unfair.

Stage 5. Storage, speed, and hardware

Owner: DevOps and ML engineer. Done when: storage size, corpus indexing time, and query latency have been calculated for the two best models.

☐ Calculate the vector storage volume: number of chunks × size of one vector.
☐ Check the vector database's limits on index dimensionality.
☐ Measure the time it takes to vectorize the entire corpus on your server.
☐ Measure the latency of vectorizing a single query under load.
☐ For MRL models, check how much quality is lost with a shorter vector.

How to calculate the storage volume. In pgvector, a vector of type vector takes 4 × dimensionality + 8 bytes, and type halfvec takes 2 × dimensionality + 8 bytes. For one million chunks, that is about 4.1 GB at 1024 dimensions, 16.4 GB at 4096, and about 2.1 GB at 1024 dimensions in halfvec (AIrassvet calculation, excluding the index). In addition, the pgvector HNSW index for type vector supports up to 2000 dimensions, and for halfvec — up to 4000. So the Qwen3-Embedding-8B vector at 4096 dimensions will need to be shortened or stored differently.

Stage 6. License and data

Responsible: lawyer or information security specialist. Done when: the license text for the selected model is saved with a date, and the data processing location is approved.

☐ Save a copy of the model license from its repository.
☐ For a cloud API, check what data is sent to the provider and what the contract says about storage.
☐ If the documents contain personal data, coordinate processing with the person responsible for Federal Law 152-FZ.
☐ Make sure the candidate model was not downloaded from an unofficial mirror.

For more on how to read open model licenses, see our FAQ on Open LLM Licenses.

Stage 7. Decision and version control

Responsible: process owner. Done when: the decision is recorded in the project documentation, and the reindexing plan is approved.

☐ Record the model with its version (repository revision), prefixes, dimensionality, normalization, and similarity metric.
☐ Save a table of metrics for all candidates and the test set.
☐ Describe how to reindex the database when changing the model: vectors from different models are not compatible.
☐ Set a review date: when the corpus grows, new document types appear, or new models are released.

Copyable checklist

EMBEDDING MODEL SELECTION — project: ________  date: ________

Stage 1. Requirements (process owner, information security)
☐ Corpus languages and question languages
☐ Document types, average chunk length in tokens
☐ Where texts can be processed: API / Russian cloud / your own servers
☐ Allowed licenses
☐ Volume: chunks now / in one year
☐ Search latency and hardware budget
Done: constraints signed off

Stage 2. Candidates (ML engineer)
☐ 3–5 models from the MTEB/ruMTEB rankings that meet the constraints
☐ For each: dimensionality, input length, license, prefixes
☐ At least one Russian and one multilingual model
Done: candidate table

Stage 3. Test set (process owner, experts)
☐ Real questions with the correct chunks marked
☐ Hard cases: numbers, abbreviations, jargon
☐ Questions with no answer in the database
☐ Versioned dataset file
Done: set approved by an expert

Stage 4. Run test (ML engineer)
☐ Same chunking for all models
☐ Prefixes/instructions from the model card
☐ Check truncation of long chunks
☐ recall@5, recall@10, MRR in one table
☐ Review common errors, check reranking
Done: metrics table

Stage 5. Storage and speed (DevOps)
☐ Vector volume = chunks × (4 × dimensionality + 8) bytes
☐ Index dimensionality limits
☐ Corpus indexing time, query latency
Done: calculation for the two best models

Stage 6. License and data (lawyer, information security)
☐ Copy of the license with date
☐ Data processing location and Federal Law 152-FZ
Done: approved

Stage 7. Decision (process owner)
☐ Model + revision, prefixes, dimensionality, normalization
☐ Reindexing plan and review date
Done: recorded in project documentation

Common mistakes

Choosing the model that ranks first overall. An average score across dozens of tasks does not show how the model will find answers in your internal regulations.
Comparing without prefixes. A model that is not given “query: ” or an instruction is unfairly penalized.
Testing on generated questions. Questions written from the chunk text repeat its wording and inflate quality.
Truncated chunks. A chunk longer than the model limit is vectorized without the end of the text, and the answer from that part will not be found.
A large vector “for future growth.” 4096 dimensions cost four times more memory than 1024 and may not fit into the database index.
Changing the model without reindexing. Queries encoded by the new model cannot be searched among vectors from the old one.

Frequently asked questions

Which embedding model is best for Russian?

There is no universal answer: the best model is determined by testing on your own documents. To start, it makes sense to compare models trained for Russian, such as FRIDA, ru-en-RoSBERTa, and USER-bge-m3, with multilingual models like bge-m3, multilingual-e5-large, and Qwen3-Embedding. The authors of the MTEB benchmark note that no single method leads in every task, so recall on your own question set is what matters.

Can embeddings be used through a cloud API?

Yes, if the documents are allowed to be sent to the provider. A cloud service such as GigaChat Embeddings removes the need for your own servers, but the texts of all chunks and queries are sent to it. For internal information and personal data, this must be approved by an information security specialist, and in some cases you should choose an open model on your own servers instead. Compare the cloud and local models on the same test set.

What if the documents are longer than 512 tokens?

First split them into chunks that fit within the model limit. If the chunks need to be longer for semantic reasons, choose models with a long input: bge-m3 supports 8192 tokens, and Qwen3-Embedding supports 32,000. But a long chunk blurs the vector’s meaning, so length should be validated by testing, not chosen by the maximum.

How often should the embedding model be reviewed?

Review it when there are noticeable changes: the document base has grown or changed, new question types have appeared, or new models have been released that meet your constraints. Re-selection is inexpensive if the test set and metrics table are preserved. Reindexing the entire database is still the costly part, so a new model should be implemented only if it clearly outperforms the current one on your test.

Primary sources and methodology

We checked the model cards, configurations, and licenses on Hugging Face, the pgvector documentation, and the GigaChat API, as well as benchmark articles, on October 23, 2026:

MTEB: Massive Text Embedding Benchmark — Muennighoff et al., 2022.
ruMTEB and the ru-en-RoSBERTa model — Snegirev et al., SaluteDevices, 2024.
Model cards: multilingual-e5-large, bge-m3, USER-bge-m3, ru-en-RoSBERTa, FRIDA, Qwen3-Embedding-0.6B, Qwen3-Embedding-8B.
pgvector documentation — vector size and index limitations.
The Embeddings model in the GigaChat API documentation.

We took the input length for USER-bge-m3 and ru-en-RoSBERTa from the model configuration because it is not explicitly listed in the model card. Limitations: we did not measure model quality on our own dataset or compare speed on specific hardware—that is done in stages 4 and 5 using your data.

How AIrassvet helps you choose an embedding model for a knowledge base

Model selection is part of a RAG project, and it is easier to do on real company documents. On the AIrassvet website you’ll find the areas that cover this task:

RAG systems and enterprise knowledge bases: corpus preparation, chunking, and comparing embedding models on your questions;
deploying models on the customer’s servers if the texts cannot be sent to an external API;
process audit, data preparation, and testing: building a test set and a metrics table before launch.

The first step is to choose one process where employees search for answers in documents and record the current result, data sources, data-processing constraints, and the acceptance criterion.

Discuss the project

Bottom line: choose an embedding model through testing

The ranking helps you identify which models to start with, but it does not make the decision for you. The decision comes down to recall on your questions, input length, vector storage cost, licensing, and where the data is processed.

Go through the seven stages of the checklist, save the test set and metrics table, and record the chosen model with its version and prefixes. Then revisiting the model in a year will take days, not a full rebuild from scratch.