Built a unified knowledge base for a Russian microelectronics company that pulls information from documents, spreadsheets, and internal sources, regularly updates the data, and provides access to employees and AI agents.
Before implementation, critical information was spread across PDF files, Excel spreadsheets, work chats, and individual specialists. Documents were stored in different places, some data was outdated, and finding the right information depended on whether an employee knew who had it and where to look.
We brought these sources together into a centralized system, set up ETL processes, indexing, vector search, and cross-validation of data. As a result, the knowledge base became a single source of truth for employees, internal services, and external AI agents.
Challenge
The client needed to organize a large volume of fragmented technical and corporate information.
The company used:
- PDF documents, some of which were not suitable for automated reading;
- Excel spreadsheets with different structures and update schedules;
- technical descriptions and specifications;
- materials from work chats;
- instructions and internal policies;
- knowledge that, in practice, existed only with individual employees.
It was necessary to create a single knowledge base that could:
- collect information from different sources;
- convert documents into machine-readable form;
- determine the relevance and reliability of data;
- sync automatically with updated files;
- provide convenient search for employees;
- be used by AI agents;
- become a centralized source of information on products, processes, and technical solutions.
The main business goal was not simply to create a corporate file repository. The need was for a system that understands document content, links data from different sources, and makes it possible to get accurate answers in natural language.
Problem
Information inside the company was fragmented.
Some information was in PDF documents, while other information was in multiple versions of Excel spreadsheets. Some materials were shared through chats, and some technical details existed only in message threads or in employees’ memories.
As a result, several problems arose.
Employees spent a lot of time searching for the right documents and clarifying information with colleagues. Different departments could use different versions of the same file. It was difficult to determine which information was current and which was already outdated.
This problem became especially critical when working with complex technical products. In microelectronics, component parameters, specifications, solution compatibility, and implementation details can be spread across dozens of documents and spreadsheets.
Standard file search did not solve the problem. It could find a document by name, but it could not answer a specific question, compare information from multiple sources, or identify conflicts between data versions.
Without a unified system, the company would continue to depend on manual search and individual experts. That slowed down the work of managers, marketers, technical specialists, and other employees.
Solution
We developed a web platform with a unified knowledge base and connected internal and external AI agents to it.
The system automatically pulls data from documents and spreadsheets, converts it into a unified format, validates it, indexes it, and stores it in a centralized repository.
To process the information, we implemented ETL processes: loading, extraction, cleansing, normalization, and data updates. For complex documents, separate processing workflows are used for PDFs, spreadsheets, and files with nonstandard structures.
After ingestion, the materials are broken into logical chunks, enriched with metadata, and placed into search indexes. Vector representation makes it possible to find information not only by exact word matches, but also by the meaning of the query.
To improve reliability, we implemented advanced cross-validation. The system compares data from multiple sources, takes document versions, update dates, and source priorities into account.
The resulting knowledge base is used as a single source of truth. All connected agents receive information from one system and work with the current version of the data.
What we implemented
A centralized knowledge base
All corporate materials were brought together into a single information environment.
The knowledge base can ingest:
- PDF documents;
- technical descriptions;
- instructions and policies;
- Excel spreadsheets;
- text files;
- internal company materials;
- structured and unstructured data.
Each source is processed using a separate workflow. The system extracts the content, identifies the document structure, adds metadata, and prepares the information for search and for AI agents to use.
Instead of a set of disconnected files, the client received a manageable knowledge base with consistent rules for storing and updating information.
Processing PDFs and non-machine-readable documents
A significant portion of the information was in PDF files that could not be properly processed with standard text extraction.
For these documents, we built a separate processing pipeline. The system recognizes page structure, extracts text blocks, tables, and related elements, then converts the data into a format suitable for indexing.
Processing takes the location of information within the document into account. This helps preserve the semantic relationships between headings, tables, explanations, and technical specifications.
As a result, even complex technical materials became available for semantic search and analysis by agents.
Synchronizing Excel spreadsheets
The company used several Excel files that were updated on different schedules.
We developed ETL processes that:
- track changes in spreadsheets;
- load new versions;
- normalize columns and values into a single structure;
- check data accuracy;
- identify duplicates and conflicts;
- update search indexes;
- store information about the source and the time of change.
This made it possible to eliminate manual updates to the database after every spreadsheet change.
If the source file is updated, current data automatically flows into the knowledge base and becomes available to employees and agents.
Cross-validating data
One of the main challenges of the project was that identical or related information could appear in multiple sources at once.
For example, a technical parameter might be listed in a PDF specification, an Excel catalog, and an internal document. At the same time, the values could differ because of different update dates or errors during manual entry.
To solve this problem, we implemented a cross-checking mechanism.
The system takes into account:
- the source type;
- the creation and update date;
- the document version;
- the source priority;
- matches and discrepancies between data;
- relationships between technical entities.
When conflicts are detected, the information is not merged indiscriminately. The system preserves data provenance and makes it possible to determine which source a specific answer is based on.
This improved the reliability of the knowledge base and reduced the risk of using outdated specifications.
Indexing and Vector Search
After processing, documents and spreadsheets are broken into meaningful chunks and added to search indexes.
For each chunk, the following are stored:
- content;
- source;
- document type;
- last updated date;
- version;
- related entities;
- technical parameters;
- vector representation.
The system combines traditional full-text search with semantic similarity search.
This lets users ask questions in natural language without knowing the exact document name or wording from the source file.
For example, an employee might ask about component compatibility, the features of a specific solution, or the differences between several product options. The system finds related information across different documents and generates an answer based on current sources.
Web Interface for Employees
A web interface was developed in Astro for working with the knowledge base.
Through it, employees can:
- search for information across the entire knowledge base;
- ask questions in natural language;
- review the sources found;
- open the source documents;
- check what data the answer is based on;
- work with technical information without knowing the storage structure.
The interface is designed not only for technical specialists. Managers, marketers, and other employees who need to quickly understand product features or implementation details can use it.
Instead of reaching out to developers or engineers, an employee asks the agent a question and gets an answer with links to company sources.
Internal AI Agents
After the knowledge base was built, internal AI agents were connected to it.
They help employees:
- find technical information;
- understand product specifications;
- compare components and solutions;
- get explanations of internal processes;
- prepare materials for clients;
- clarify implementation details;
- get new specialists up to speed faster in the work context.
The agents have been especially useful for non-technical teams.
Managers no longer need to manually review dozens of specifications. Marketers can get explanations of complex technical advantages in clear, plain language. New employees can ramp up faster on the company's product and technical knowledge base.
External Agents on the Website
The unified knowledge base was also connected to external agents on the company's website.
The agent answers user questions based on current company information. It can explain products, describe specifications, and help users choose the right solution.
These answers are generated not from the language model's general knowledge, but from verified company data.
If the source documents or spreadsheets are updated, the new information is automatically available on the website. The client does not need to separately update each agent's instructions.
This makes it possible to use the knowledge base as a central layer between corporate data and any user interfaces.
Source and Freshness Control
For each answer, the system preserves a link to the source materials.
The user can see which document, spreadsheet, or section the information came from. This is especially important when working with technical specifications and internal documentation.
In addition, the system tracks source update dates and versions. Outdated data can be excluded from results or used with lower priority.
This approach reduces the risk that the agent will produce a convincing but incorrect answer based on an old document.
Technical Foundation
The project is built on a modular architecture and containerized with Docker.
The project used:
- Astro;
- Docker;
- PDF and document processing tools;
- OpenAI SDK;
- Codex SDK;
- ETL pipelines;
- full-text search indexes;
- vector storage;
- file storage;
- cross-validation mechanisms;
- background data synchronization processes.
The architecture is divided into several functional layers:
- ingesting source materials;
- extracting information;
- normalizing data;
- verification and cross-validation;
- storing files and metadata;
- building search indexes;
- creating vector representations;
- finding relevant context;
- interacting with AI agents;
- web interface for users.
This structure makes it possible to expand the system without fully rebuilding the platform.
You can connect new data sources, add document formats, create agents for individual departments, and roll out additional automation scenarios.
Result
The client received a unified knowledge base that became a centralized source of corporate and technical information.
Documents, spreadsheets, and other materials no longer exist as separate, disconnected files. They go through automated processing, validation, and indexing, becoming part of a single information environment.
Employees can find the information they need through search or ask the AI agent questions. They no longer need to know the exact document name, file location, or the employee who has the needed information.
The external agents on the website use the same database and answer customers based on current data. Internal agents help managers, marketers, and other non-technical specialists understand complex product and implementation details.
Updates to source documents are automatically reflected in the system. As a result, all interfaces and agents work with a single up-to-date version of the knowledge base.
What the client received
- A single knowledge base instead of scattered files and spreadsheets.
- Automated processing of PDFs and other complex documents.
- ETL processes for synchronizing regularly updated data.
- Cross-validation of information from multiple sources.
- Full-text and vector search.
- A web interface for employees.
- Internal agents for managers, marketers, and technical specialists.
- External agents for the company website.
- Answers with links to source materials.
- Version control, source tracking, and data freshness monitoring.
- An architecture that can be expanded with new sources and use cases.
The project transformed the company’s accumulated knowledge from a collection of documents, spreadsheets, and employees’ personal expertise into a manageable digital system.
Now the knowledge base serves as a single source of truth: data is loaded and updated centrally, employees get the information they need faster, and internal and external AI agents generate answers based on current, verified corporate materials.