AI Agent for Data Analytics: Why You Need a Semantic Layer

AI Agent for Data Analytics — a system that takes a business question in plain language, finds approved data, builds a query, checks the result, and explains the conclusion. For such an agent to answer not only technically correctly but in the company’s own terms, it needs a semantic layer: a versioned contract for metrics, dimensions, relationships, and access rules.

This material is intended for executives, BI owners, analysts, and data teams. It describes the architecture and acceptance testing of an analytics agent, but it does not compare specific pricing plans and does not replace an audit of your data model.

Editorial content from AgentSunrise. Sources and product pages were checked on September 16, 2026. Commercial affiliation is transparent: the penultimate section describes the publisher’s capabilities. We did not measure production deployment or Russian accuracy statistics.

Key takeaways: access to a table does not yet determine what the company means by revenue, active customer, or margin. Before connecting AI, define metric meanings, owners, allowed dimensions, and control answers; then verify not only SQL execution, but also alignment with business meaning.

What changed after the launch of OpenAI Data agent
Why correct SQL can produce the wrong answer
What a data semantic layer is
What the research shows
The PRAVDA method for preparing metrics
Analytics agent architecture
How to run acceptance testing on real questions
What pilot metrics to track
Where a human is needed
Evidence limitations
Frequently asked questions
How AgentSunrise helps prepare data and the agent
Conclusion

What changed after the launch of OpenAI Data agent

On September 10, 2026, OpenAI introduced Data agent in ChatGPT Work. According to the vendor, the agent connects to approved sources, respects business terms and user-defined calculations, builds interactive dashboards, and can send insights to other tools after approval. Queries must inherit the restrictions of the connected account at the table, row, and column levels. OpenAI: Now everyone can put data to work.

The key part of the announcement is not the conversational interface itself, but the reliance on semantic layers and trusted sources. OpenAI also says that its own data team first created shared business definitions, access rules, and protections for sensitive data. That is the vendor’s statement about its own practice, not an independent assessment of product accuracy.

The growing availability of tools like this changes the implementation question. Previously, the company had to decide whether a model could write SQL. Now it must decide, which metric definition, data version, and set of permissions the agent must follow, and then prove it on a control set of questions.

Why correct SQL can produce the wrong answer

SQL can run without an error and return numbers, but the answer can still be wrong for the business. The reason is ambiguity in terms, not in syntax.

For example, the question “What was revenue in August?” can be calculated in different ways:

Possible definition	What it includes	When the answer changes
Orders	Sum of created orders	Excludes cancellations and refunds
Payments	Payments actually received	Payment may fall into a different period
Shipment	Value of delivered goods	Recognition date differs from payment date
Net revenue	Recognized amount minus refunds and discounts	Recognition and period-close rules are required

Row and column permissions answer what the user can see. The metric definition answers what exactly needs to be counted. These two controls complement each other, but one does not replace the other.

There is also a third problem: the same word can mean different entities across departments. “Active customer” in product means a user with an event in the last 30 days; in sales, it is an account with an open opportunity; in finance, it is a counterparty with recognized revenue. The agent cannot choose a definition based on name similarity without explicit context or clarification.

What a data semantic layer is

Data semantic layer — an executable agreement between physical tables and business language. It records the metric name, formula, unit, allowed dimensions, calendar, filtering rules, owner, version, and access restrictions.

A minimum metric record should answer eight questions:

What is it called for the user, and what synonyms are allowed?
Which formula and unit of measure are approved?
Which date defines the period: creation, payment, shipment, or recognition?
Which rows are excluded, and how are refunds, test accounts, and duplicates handled?
Which dimensions is it allowed to group by?
Who owns the definition, and who approves changes?
Which version applies to the selected period?
Which users can see source and aggregated data?

A semantic layer does not have to be a separate product. It can be a governed model in BI, a dbt project, a metrics catalog, or an API with typed parameters. There is one criterion: the agent uses an approved description and a reproducible calculation path, instead of inventing business logic from scratch for every question.

Figure 1. Proposed separation of meaning and execution:

User question
        ↓
approved context + metric dictionary
        ↓
plan: metric / period / dimension / filters
        ↓
semantic layer → verifiable SQL or API call
        ↓
result + definition version + sources + warnings

If the layer does not know the needed term or allows two incompatible definitions, the correct behavior is to ask for clarification or decline to give a number. A confident default choice hides uncertainty.

What the research shows

Research does not give a single “text-to-SQL accuracy,” but several different slices. They cannot be added together or transferred directly to your own database.

Source	Conditions	Observed result	What it does not prove
EntSQL, June 2026	1,066 Chinese-English examples, five business domains, long internal documents	The best tested option was 15.9% on English questions with long documents	Accuracy of all agents or of a Russian-language company
LinkedIn, July 2025	Internal Agent, Knowledge Graph, and 300+ Weekly Users	Expert assessment: 53% of answers are correct or close to correct	Autonomous decision-making without review
Semantic-layer agent, June 2026	547 Spider2-snow tasks, Gemini 3 Pro, prepared data models	515 out of 547, or 94.15%, by execution accuracy	The quality of explanations, business decisions, and portability to a new company

EntSQL specifically tests cases where a database schema alone is not enough and internal definitions, reporting rules, and documents are needed. The authors report 1,066 examples across five domains; the best result with long context remains 15.9%. The work is a preprint, and the dataset is bilingual, so this is a signal of complexity rather than a forecast for a Russian company. EntSQL.

The LinkedIn team described an internal assistant with a knowledge graph built from metadata, query logs, wiki pages, and code. It had more than 300 weekly users, and experts judged 53% of answers to be correct or close to correct on an internal dataset. The "close to correct" category is softer than an exact match, and the dataset itself was not published. Text-to-SQL for Enterprise Data Analytics.

Another preprint uses a prepared semantic layer and a deterministic intermediate-representation compiler to SQL. On Spider2-snow, the system correctly executed 515 of 547 tasks. However, on the small native Snowflake segment, the result was 10 out of 18, or 55.6%; the authors separately note dependence on the maturity of the semantic layer and the risk of overfitting to the dataset. Semantic-Layer-Mediated Agent.

Practical takeaway: the model matters, but the quality of the enterprise contract and validation set can matter more than the model name. High execution accuracy confirms that the query result matches the reference for a specific test, but it does not prove that the recommendation is useful to a manager.

The TRUTH method for preparing metrics

TRUTH is an editorial framework for preparing a single analytics workflow. It is not an industry standard or a quality certification.

Concept. Write the business definition, formula, unit, and metric synonyms.
Dimensions. Specify the allowed dimensions, filters, calendar, and granularity.
Owner. Assign the definition owner, source owner, and change approver.
Version. Link the answer to the data model version, update date, and period of applicability.
Tolerances. Describe the margin of error, delay, incomplete data, and conditions for mandatory refusal.
Acceptance record. Prepare control questions, reference answers, and the go-live decision.

For a pilot, you do not need a company-wide catalog. Choose one metric that is used regularly for decisions, and build its TRUTH card. If the owner and formula are disputed, the agent will only speed up the spread of the dispute.

Minimum contract example

metric: net_revenue
label_ru: Net revenue
formula: recognized_revenue - refunds - approved_discounts
unit: RUB
time_dimension: recognition_date
allowed_dimensions: [region, product_line]
owner: finance_analytics
freshness_sla: P1D
version: 2026-09-01

This is an illustration of structure, not a ready-made definition for accounting or tax reporting. The formula and recognition point must be approved by the responsible function within the company.

What we tested in the local demo

On September 16, 2026, we tested a minimal deterministic contract executor on three synthetic orders. The formula recognized - refund - discount produced total net revenue of 2,150 notional rubles; the allowed region dimension returned 1,550 and 600. An unknown metric and a forbidden order-number dimension were rejected. All four predefined expectations matched the result.

The demo only tests the application of an explicit formula and an allowlist of dimensions. It does not include an LLM, natural language, a real database, user permissions, update latency, or accounting rules. So 4 out of 4 is the result of a unit test of the selected logic, not the accuracy of an analytics agent. Reproducible file: seo_pipeline_v2/4_audit/ii-agent-analitika-dannyh_test/test_metric_contract.py.

Analytics agent architecture

A production workflow is conveniently divided into six components.

Component	Responsibility	Verifiable evidence
Identification	Who asked the question and on whose behalf	User, role, session
Routing	Which domain and owner the question belongs to	Selected set of metrics and the reason
Semantics	How the metric, period, and dimension are defined	Contract ID and version
Execution	How the result was obtained	SQL/API, parameters, source snapshot
Validation	Whether constraints and checksums passed	Rule decisions, warnings, tests
Presentation	What the user saw	Number, unit, period, source, caveats

SQL generation should be separated from the right to execute it. Before running, check allowed tables, read volume, timeout, query type, and lack of write access. For recurring metrics, compilation from an approved intermediate representation is preferable, while ad hoc SQL should be a controlled exception.

If the agent can do more than analyze and can also send a report, change a plan, or create a task, the output becomes the input to the next action. Then a separate approval is needed: who may receive which result, with what metric version, and at what confidence level.

How to run acceptance testing on real questions

Collect 30–100 questions from analyst logs, emails, and meetings. The number is chosen before reviewing results and depends on process diversity; there is no universal minimum. Include simple requests, ambiguous terms, rare dimensions, missing data, and questions the system should refuse to answer.

For each question, record:

the approved metric and definition version;
the allowed period and dimension;
the expected result or range;
the source and snapshot time;
required explanations;
whether an automatic answer, clarification, or refusal is allowed.

Test step by step. First, check whether the agent selected the correct metric. Then verify whether the query plan is correct. After that, confirm whether the numerical result matches. Finally, make sure the explanation did not distort causality: correlation in a dashboard does not prove that one factor caused another to change.

Use paired tests. Change the period, region, or wording in the question while keeping the rest of the context the same. If the answer does not change where it should, the agent may have ignored the condition. If the number changes because of an irrelevant synonym, routing is unstable.

Diagram 2. Acceptance Funnel:

100 business questions
  ├─ choosing the right metric
  ├─ the correct period and slice
  ├─ the result matches the benchmark
  ├─ correct sources and version
  └─ useful explanation without unnecessary causality

Which pilot metrics to keep

Metric	Definition
Route accuracy	Questions with the correct metric contract / all verified questions
Result accuracy	Answers within the benchmark / questions with a defined benchmark
Source completeness	Answers with source, version, and data time / all answers
Correct refusal	Correct clarifications and refusals / questions where a number cannot be provided
False confidence	Incorrect answers without warning / all incorrect answers
Paraphrase stability	Equivalent questions with an equivalent plan / all pairs
End-to-end cycle time	From question to a user-accepted result
Cost per accepted answer	Infrastructure and validation / accepted results

Define denominators and labeling rules in advance. Track manual analyst corrections separately: they show not only model quality, but also gaps in the contract. Reducing time without preserving correctness is not a success.

Do not mix up “the query ran,” “the number matched,” and “the decision was useful.” These are three different levels. The first is checked automatically, the second against a benchmark, and the third by the process owner and the downstream outcome.

Where a human is needed

Human confirmation remains mandatory when:

the metric definition changes or conflicts across functions;
the question allows multiple materially different interpretations;
the data are incomplete, delayed, or going through period close;
the output concerns hiring, credit, medicine, legal position, or another sensitive area;
the recommendation triggers an irreversible action;
the agent detects an anomaly but cannot separate a data error from a real event.

A human should not manually recheck every line forever. Their job is to own the definitions, exceptions, and escalation thresholds. Repeatable checks are turned into tests; disputed decisions stay with the assigned owner.

Evidence limitations

The OpenAI announcement is a primary source on features and internal use, but it is also the vendor’s marketing material. The article does not include an independent public evaluation of Data agent.
EntSQL and the paper on the semantic-layer agent are preprints. Their results depend on the chosen models, datasets, prompts, and prepared context.
The 94.15% figure refers to execution accuracy on Spider2-snow; it does not measure the correctness of business interpretation, explanation, or the downstream decision.
On LinkedIn, 53% combines fully correct and nearly correct answers; the closed set and criteria do not allow the number to be independently reproduced.
Russian production accuracy, semantic-layer preparation cost, time savings, ROI, and error frequency were not observed.

Search volume, keyword difficulty, rankings, traffic, CTR, backlinks, and AI citations also Unknown. The article offers a validation method rather than predicting SEO or business results.

Frequently Asked Questions

What is an AI agent for data analytics?

It is a system that takes a question in natural language, selects approved sources and metrics, builds and runs a query, and then shows the result with provenance and caveats.

How is a semantic layer different from a data catalog?

A catalog helps find and describe assets. A semantic layer additionally defines executable metric definitions, relationships, dimensions, and calculation rules. In a specific platform, the functions may overlap.

Is read-only access enough for the agent?

No. Read-only reduces the risk of changing the database, but it does not prevent choosing the wrong table, formula, period, or slice. You need a semantic contract and acceptance tests.

Can an agent replace a BI analyst?

It can speed up repeatable queries and report preparation. Ownership of definitions, ambiguity handling, causal analysis, and sensitive decisions remains with people.

How do you get started without a full warehouse redesign?

Choose one process and one management metric. Lock down its formula, owner, source, version, slices, and 30–100 control questions, then launch a limited read-only pilot.

Do users need to see the SQL?

Not for every user. But the system should store a reproducible plan, parameters, metric version, and source; an analyst or auditor should be able to verify them.

How AgentSunrise helps prepare the data and the agent

AgentSunrise can help connect the corporate workspace, AI agent, BI, and data sources into one verifiable process:

describe the selected analytics process, metric owners, sources, constraints, and current baseline;
prepare a glossary and versioned contracts for key metrics together with the client team;
build an MVP agent with approved connections, provenance logging, tests, and limited execution;
integrate the result with corporate systems, perform functional acceptance, and train the team.

The first step is to choose one metric and gather its formula, sources, owner, constraints, and accepted-answer criterion. Discuss the task.

Conclusion

A conversational interface makes analytics more accessible, but it does not automatically create a single shared meaning for the data. OpenAI’s recent launch underscores the role of semantic layers and active permissions, while research shows a wide spread of results between long corporate context, an internal system, and a carefully prepared test layer.

The reliable implementation order is: metric definition → owner and version → allowed slices and permissions → reproducible query → reference answer → go-live decision. Start with one metric and prove the full path from question to accepted result. Only then expand to other domains and allow the agent to pass conclusions into actions.