Synthetic data is artificially created records, text, or images that replicate the properties of real data and are used to train and test AI models. They help when there are too few real examples, rare cases almost never occur, or the data cannot be used directly. But they do not replace validation on real data and do not guarantee privacy on their own.
Prepared by AIrassvet October 25, 2026 for executives and data owners who hear from a contractor, “there isn’t much data, we can add synthetic data,” and want to understand what that means. The answers are based on publications in Nature and on arXiv, technical reports from NVIDIA and Microsoft, the SDV library documentation, and the text of Federal Law No. 152-FZ. We did not conduct our own experiments. This is not legal advice: evaluate the legal status of a specific dataset with a lawyer.
Contents
- Quick take
- What this article covers and does not cover
- Fundamentals questions
- Application questions
- Risk, quality, and legal questions
- Synthetic data by task type: summary table
- Primary sources and verification method
Quick take
- Where it helps: rare cases and defects, computer vision through rendering, instructions for fine-tuning language models, test sets for system validation.
- Who uses it: according to NVIDIA’s technical report, more than 98% of the data used in the alignment stage of the Nemotron-4 340B model was synthetic.
- Main quality risk: a model trained on synthetic data may perform worse on real-world traffic. Final validation should be done only on held-out real data.
- Main privacy risk: a study by Stadler and co-authors showed that synthetic data either does not protect against inference attacks or loses usefulness.
- Long-term risk: uncontrolled training on generated data leads to “model collapse,” according to a 2024 Nature article.
What this article covers and does not cover
We answer questions about synthetic data for training and testing models: tabular, text, and image data. We do not go into the generation algorithms, differential privacy, or the legal regime for anonymized data in detail — we wrote separately about anonymization and the transfer of customer data to AI in an article about customer data and neural networks.
Terms. *Real data* — data collected from the company’s operations: sales, product photos, customer inquiries. *Generator* — a model or program that creates synthetic data. *Held-out set* — real data that is not used for training and is needed only for final validation. *Augmentation* — modifying existing examples: rotation, cropping, noise.
Fundamentals questions
What is synthetic data?
It is data created by an algorithm rather than collected from real events. The generator learns the properties of the source dataset or works according to predefined rules and produces new records: table rows, text, or images. The goal is to preserve patterns useful for training a model while getting more examples, needed rare cases, or a dataset without direct information about real customers.
What kinds of synthetic data are there?
By content type, there are four main groups. Tabular data — rows with the same columns and distributions as in the source table. Time series and sequences — sales, sensor readings, action history. Text data — questions, answers, dialogues, and instructions generated by a language model. Images and video — 3D model rendering or neural network generation. Each type has its own tools and its own validation methods.
Do major model developers use synthetic data?
Yes, and it is documented. In the June 2024 technical report for Nemotron-4 340B, NVIDIA says that more than 98% of the data used in the model’s alignment stage was synthetically generated. Microsoft trained the phi-1 programming model on 6 billion tokens of selected web data and 1 billion tokens of synthetic textbooks and exercises created by GPT-3.5. The 1.3 billion-parameter model achieved 50.6% pass@1 on the HumanEval benchmark.
How is synthetic data different from augmentation?
Augmentation modifies an existing example, while synthetic data creates a new one. Rotating a product photo, changing brightness, or adding noise is augmentation: the object and the defect remain the same. Rendering the part with a scratch in a new location, under different lighting, and on a different background is synthetic data. In practice, both approaches are often combined: augmentation is inexpensive, while synthetic data provides variations that do not exist in real data at all.
Application questions
We have little data — will synthetic data help?
It can help, but it will not fully replace real examples. Synthetic data is useful when there are few rare cases: defects, failures, unusual customer inquiries. If there is no real data at all, the generator has nothing to rely on, and there will be nothing to validate the model against. At a minimum, you need a small real dataset to tune the generator and a held-out real sample for final evaluation. We wrote about collecting and labeling real data in an article on preparing data for AI.
Can computer vision be trained on synthetic images?
Yes, this is one of the most well-developed use cases. In a 2017 study by Tobin and co-authors, an object detector was trained only on simulator images with random unrealistic textures, and on real photos it identified the object’s position with 1.5 cm accuracy. The method is called domain randomization: if synthetic data has enough variation, the real world looks like just another variant to the model. Real photos are still needed for validation and labeling.
Can data for fine-tuning a language model be generated using another model?
Yes, that is how questions, answers, and instructions are prepared for fine-tuning and distillation. A strong model generates examples, specialists review the sample, and then a smaller model is fine-tuned on them. Before doing this, check the generating model’s license: some providers place restrictions on using outputs to train other models. More details are in the materials on LLM fine-tuning and distillation.
Are synthetic data suitable for tabular tasks such as forecasting, scoring, and churn?
Yes, but with caution. For example, the SDV library can synthesize single tables, linked tables, and sequential data. This is useful for test environments, data sharing with a contractor, and boosting rare classes. For demand forecasting or churn prediction, synthetic data is risky: the generator may smooth out seasonality, outliers, and relationships between features. Always compare model quality with a model trained only on real data.
Can AI systems be tested on synthetic data?
Yes, and it is often safer than training. Synthetic inquiries, documents, and dialogues help test edge cases: typos, unusual formats, attempts to bypass rules. This is a good fit for a test environment where real customer data cannot be used. But system acceptance should be done on real examples from the production flow: a synthetic test shows that the system handles invented cases, not yours.
Risk, quality, and legal questions
Are synthetic data personal data?
It depends on whether a specific person can be identified from them. Under Article 3 of Federal Law No. 152-FZ, personal data is any information relating to a directly or indirectly identified or identifiable individual. If synthetic records are too close to real ones or the generator has “memorized” rare customers, the link to a person may remain. That is why the status of a dataset is assessed based on the results of testing, not on the label “synthetic.”
Does synthetic data protect against personal data leakage?
Not necessarily. In their paper “Synthetic Data — Anonymisation Groundhog Day,” Stadler, Oprisanu, and Troncoso evaluated several generative models and concluded that synthetic data either does not prevent inference attacks or loses utility. The authors also note that the privacy benefit is hard to predict in advance. That is why synthetic data for data sharing is combined with attack testing and formal protection methods.
What is model collapse, and is it a risk for business?
This is the degradation of models trained on data generated by previous models. A July 24, 2024 Nature paper by Shumailov and coauthors showed that uncontrolled use of generated content causes irreversible defects, and the model loses the “tails” of the original distribution. The practical takeaway for business: keep real data, label synthetic data, and do not train a model only on its own generated output.
How do you check the quality of synthetic data?
It is checked in three areas. Similarity: do the distributions and relationships between features match the real ones. Utility: train a model on the synthetic data and test it on a held-out real dataset, then compare it with a model trained on real data. Privacy: are there records that are almost identical to real ones, and is the dataset resilient to inference attacks. The held-out real sample should never be fed into the generator.
How much synthetic data should be added to real data?
There is no universal ratio: it is selected experimentally. Train several model variants with different amounts of synthetic data and compare them on the same held-out real dataset using the metric that matters to the business. If adding synthetic data stops improving results or makes performance worse on real data, stop. Also pay attention to rare classes separately: synthetic data may improve their recognition but hurt common cases.
What tools are used to generate synthetic data?
The choice depends on the data type. For tabular data and sequences, there is the SDV library, for example; it is distributed under a Business Source License, which prohibits using it for a commercial synthetic-data generation service for third parties. For text, language models are used; NVIDIA has released a synthetic data generation pipeline used for Nemotron-4. For images, 3D rendering and generative models are used. Check the tool license before starting the project.
How much does it cost to prepare synthetic data, and how long does it take?
There is no universal price or timeline: they depend on the data type, volume, and validation requirements. The main cost items are preparing the real sample, configuring the generator or 3D scene, computing, quality and privacy checks, and retraining the model. The most common mistake is not budgeting time to compare against real data. A small pilot on one task with a preselected metric gives a realistic estimate.
Synthetic data by task type: summary table
| Task | How synthetic data is generated | Main risk | How to verify |
|---|---|---|---|
| Computer vision | 3D rendering with randomized lighting, background, and position; generative models | The gap between the render and the real camera | Metrics on held-out real images from the same line |
| LLM fine-tuning | Generation of questions, answers, and instructions by a strong model | Generator errors, license restrictions | Selective expert review, test on real questions |
| Tabular models | Statistical and neural table generators | Loss of seasonality, outliers, and relationships | Comparison with a model trained on real data |
| System testing | Generation of edge cases and documents | The test does not reflect the real flow | Acceptance on real examples |
| Data sharing with a contractor | Synthetic copy of structure and distributions | Residual link to real people | Checking for close records and inference attacks |
The table is our summary based on the sources below, not an industry standard.
Primary sources and verification method
We verified the claims on October 25, 2026, against the primary sources:
- Shumailov I. et al. AI models collapse when trained on recursively generated data, Nature, published on 07/24/2024;
- NVIDIA. Nemotron-4 340B Technical Report, arXiv:2406.11704, June 2024;
- Gunasekar S. et al. Textbooks Are All You Need, arXiv:2306.11644 — phi-1 model;
- Tobin J. et al. Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World, arXiv:1703.06907;
- Stadler T., Oprisanu B., Troncoso C. Synthetic Data — Anonymisation Groundhog Day, arXiv:2011.07018;
- Federal Law No. 152-FZ “On Personal Data”, Article 3;
- README and license of the SDV library.
The numbers are given as reported by the authors, under the conditions they specified. The guidance on validation and ratios is our editorial interpretation, not the result of our own experiment.
How AIrassvet helps when an AI project lacks data
The decision to “add synthetic data” should be made only after it is clear which examples are missing and how the result will be evaluated. On the AIrassvet website the listed service areas include process audits and data preparation, computer vision solutions, ML forecasting, as well as LLM deployment and fine-tuning. For this task, that means:
- auditing the available data: which cases are missing and whether real examples can be collected;
- preparing the data and a held-out real dataset, where models with and without synthetic data are compared;
- a computer vision, forecasting, or fine-tuned LLM prototype, validated on real data before launch.
The first step is to choose one process that lacks data and document the current result, data sources, constraints, and acceptance criteria.
Bottom line: synthetic data complements real data, but does not replace it
Synthetic data is a practical tool: it is used to balance large language models, train object detectors, and test systems. But three rules remain unchanged: final validation must be done only on held-out real data, privacy must be checked rather than assumed, and real data must be preserved to avoid model collapse.
If a contractor proposes synthetic data, ask which metric and which real dataset will be used to prove the effect. That is the key question that separates a justified solution from a buzzword.
