Topic: Part 19. Semantic layer, metrics, and exposures
Difficulty level: Medium
Estimated study time: 3-4 hours
Prerequisites: Understanding of data mart architecture
Experience with dbt or similar transformation tools
Knowledge of Data Quality principles and check facts
Basic understanding of YAML and Markdown for documentation
Experience with BI tools and reporting
Learning objectives: Distinguish between a metric as a column and a metric as a semantic contract with a formula, granularity, owner, and consumers
Identify when a team truly needs a semantic layer and distinguish it from a bureaucratic layer
Create a semantic registry in Markdown to document shared metrics
Use exposures to capture consumers of a data product and manage breaking changes
Apply a Qwen prompt to search for and validate shared metrics in data marts
Overview: Part 19 of the course is devoted to the semantic layer, metrics, and exposures — concepts that become critically important when the same metrics start being used in multiple reports, applications, or teams. Before repeatable consumption emerges, well-defined data marts and check facts are sufficient. But when fields like total_balance_rub, active_consent, or risk_event_count start appearing in different contexts, there arises a need to centralize their definitions. A metric is not just a column, but a full-fledged contract: a name, a formula, granularity, filters, freshness, an owner, and a list of consumers. dbt supports semantic models and metrics, but in the learning track it is more important to learn how to capture semantic intent before adding a technical layer. Exposures complete the picture by showing exactly who reads the data product — a dashboard, API, notebook, or report — which is critical for managing contract drift and breaking changes.
Key concepts: Semantic layer: A centralized level of metric definitions that eliminates discrepancies between teams. Introduced only when the same metrics start being used in multiple contexts. It is not an end in itself — it is a tool against definition drift.
Metric as a contract: A metric is not a renamed column, but a full-fledged semantic contract that includes: a name, a calculation formula, granularity (grain), applied filters, expected data freshness, the owner of the definition, and a list of consumers.
Granularity (grain): The level of detail of a metric: per row, per client, per scope, per product, per day, etc. Without explicit granularity, it is impossible to guarantee that different reports interpret the metric the same way.
Dbt semantic models: A mechanism built into dbt for describing metrics through YAML configurations. It includes entities, measures, and dimensions. Used in mature stacks, but requires preliminary alignment on the meaning of metrics.
Exposure: A documented link between a data product (data mart) and its consumer: a dashboard, API, notebook, regular report. It makes it possible to see who will be affected by a schema change or grain change in the data mart.
Semantic registry: A Markdown document or YAML file that captures definitions of shared metrics in the form of: name, definition, granularity, owner, consumers, check fact. A minimally sufficient alternative to a full semantic layer.
Contract drift: A situation in which the same metric name has different definitions or formulas in different teams. For example, one team counts active consent by null in revoked_at, another excludes old consents by date — these are two different metrics with one name.
Check fact for a metric: A concrete assertion or test that validates the correctness of metric computation. For example, the sum of all positive values must match the aggregate from the source.
Breaking change: A change in a data mart or metric that affects downstream consumers: renaming a field, changing the grain, changing the formula. Exposures help identify such consumers in advance.
Qwen prompt for metrics: A standardized request to an LLM that reads data marts and specs, finds fields that look like shared metrics, and suggests for them a definition, grain, owner, consumer, and check fact — without automatically creating semantic models.
Important dates: Stage 1: before reuse: Data marts and check facts are sufficient. Metric definitions can be kept next to the data mart.
Stage 2: appearance of a second consumer: The moment to start thinking about a semantic registry. Consumers diverge if definitions are not explicitly captured.
Stage 3: several teams with one metric: The point at which the absence of a semantic layer creates drift, and its presence becomes a prerequisite for data trust.
Practice exercises: Name: Filling out the semantic registry for active_consent_count
Problem: The mart_open_api_consents data mart has a field that reflects active consents. Fill in the semantic registry: specify the definition, granularity, owner, consumers, and check fact. If the owner or consumer is unknown, do not add a semantic model.
Solution: 1. Open the spec of the mart_open_api_consents data mart and find the field description. 2. Formulate the definition: 'the count of consents where revoked_at is null'. 3. Determine the grain: at the scope level (for example, consent type). 4. Ask the Open API product owner who is responsible for this definition. 5. Ask the Open API monitoring team whether it is a consumer. 6. Formulate a check fact: 'the sum of active_consent_count over a period equals the number of rows in the source where revoked_at is null'. 7. If at least one of steps 4-5 cannot be confirmed, stop and clarify the product promise.
Complexity: intermediate
Name: Detecting drift in the total_balance_rub metric
Problem: Two reports use the total_balance_rub field but produce different totals. Determine where the drift occurs and propose how to eliminate it through a semantic registry.
Solution: 1. Compare the SQL queries of both reports against the data mart. 2. Find the discrepancy: for example, one filters by client_status='active', the other includes closed accounts. 3. Capture that total_balance_rub is a column name, not a metric. 4. Create two separate entries in the registry: 'total_balance_rub_active' (only active clients) and 'total_balance_rub_all' (all clients) with explicit grains and definitions. 5. Agree with the teams on transitioning to the new names. 6. Capture a check fact: the sum over the new metric must match the aggregate from the source under the same filters.
Complexity: intermediate
Name: Applying a Qwen prompt to find shared metrics
Problem: Use the Qwen query template to analyze three project data marts and find fields that could be shared metrics. Do not create semantic models — only propose candidates.
Solution: 1. Formulate the query: 'Read the data marts and specs. Find fields that look like shared metrics. For each, suggest a definition, grain, owner, consumer, and check fact. Do not create semantic models without confirmation.' 2. Pass the descriptions of three data marts into the query (for example, mart_clients, mart_accounts, mart_transactions). 3. Obtain a list of candidates: total_balance_rub, active_client_count, transaction_volume, avg_transaction_amount. 4. For each candidate, check: does the field appear in multiple data marts or reports? If yes — it is a candidate for a metric. 5. Write the result to a registry draft, marking each entry with the status 'requires owner confirmation'.
Complexity: intermediate
Name: Decision: whether to add a semantic layer
Problem: A team of 3 analysts works with 5 data marts. Each data mart is used by a single report. Is it worth implementing a full dbt semantic layer? Justify the decision.
Solution: 1. Check whether there are shared metrics: take the list of fields from all 5 data marts and compare it with the list of fields from all reports. 2. If there are no shared metrics (each data mart serves a single report), the semantic layer becomes bureaucracy. 3. Recommendation: leave definitions next to the data marts in the form of dbt doc blocks or comments in the spec. 4. Capture a trigger: as soon as a second consumer appears for any metric, create an entry in the semantic registry. 5. Revisit the decision when the number of consumers exceeds 3-4 teams.
Complexity: beginner
Case studies: Name: Drift of the 'active client' metric in a fintech startup
Scenario: A fintech startup with 4 analytics teams. The risk team counts active clients using the criterion 'made at least one transaction in the last 90 days'. The marketing team uses the criterion 'has not closed the account and logged into the app in the last 30 days'. Both teams call the metric active_client_count and publish it in dashboards. Management sees an 18% discrepancy between reports and loses trust in the data.
Challenge: The absence of a single semantic contract for the same metric. Different teams apply different filters, grains, and formulas, but use one name. The BI tool cannot solve it, since the problem arises at the level of definitions, not visualization.
Solution: 1. The Data Lead conducted an audit: collected all definitions of active_client_count across all teams. 2. Captured three separate metrics in the semantic registry: active_client_90d (transactions), active_client_30d (login), active_client_strict (both conditions). 3. Assigned an owner for each metric: risk, marketing, product. 4. Added exposures in dbt: risk dashboard → active_client_90d, marketing dashboard → active_client_30d. 5. Captured a check fact for each metric: the sum across the grain matches the aggregate from the source under the same filters. 6. Did not implement a full dbt semantic layer — limited themselves to a Markdown registry, since there were fewer than 10 metrics at that point.
Result: After 6 weeks, the drift was eliminated: team reports became consistent, management gained a unified understanding of what each metric means. The risk and marketing teams began coordinating metric changes through owners, not through chat arguments. Trust in data was restored. After 4 months, when the number of shared metrics grew to 12, the team decided to transition to dbt semantic models — but only because real consumers and repeatable definitions had emerged.
Lessons learned: A semantic layer is needed not for the sake of stack completeness, but when there are multiple consumers of one metric
Drift occurs not in BI, but in the absence of an explicit contract at the level of definitions
Exposures are a way to see who will be affected by a metric change and warn them in advance
Capture the intent first (Markdown registry), and only then choose the tool (dbt semantic layer)
Related concepts: Semantic registry
Contract drift
Exposure
Granularity
Check fact for a metric
Name: Transition from ad-hoc data marts to a semantic layer in a bank
Scenario: A mid-sized bank, 15 analytics teams, 40+ data marts in dbt. The risk_event_count metric is used in 8 reports: regulatory reporting, anti-fraud, operational risk, scoring. Definitions differ: some teams count from the events table, some from the alerts table, some include manual reviews, some count only automated triggers.
Challenge: The regulator requested a report on risk_event_count, and the bank could not quickly produce a consistent figure: different teams produced values differing by 40%. Reputation and regulatory deadlines were at risk. Implementing a full dbt semantic layer seemed excessive, but the absence of a contract created operational risk.
Solution: 1. A working group was created: Data Steward plus the owners of the 4 consumer teams. 2. A series of meetings was held to align on the definition of risk_event_count: source (events), filters (event_type in scope), grain (client_id, date), formula (count distinct). 3. Captured in the semantic registry: definition, grain, owner (operational risk team), 8 consumers via exposures, check fact (the sum across the grain over a period equals the aggregate from source.events). 4. A practice was introduced: any change to risk_event_count is agreed with the owner, and notifications go to all exposure consumers. 5. Only after 3 months, when such 'shared' metrics numbered 7, did the bank begin piloting dbt semantic models for them, leaving the rest in the Markdown registry.
Result: The regulatory report was submitted on time with a consistent figure. The bank avoided a fine. After six months, the number of 'semantic' metrics grew to 15, and the bank standardized the approach: every shared metric first appears in the registry, and only then, with 3+ consumers, is it moved into the dbt semantic layer. Breaking changes now automatically notify all exposure consumers.
Lessons learned: A semantic layer is not about technology (dbt vs Markdown), but about aligning meaning between teams
Exposures make it possible to see the blast radius of a change and reduce operational risk
A full dbt semantic layer makes sense to implement only after real shared metrics emerge, otherwise it is bureaucracy
The owner of a metric is not the analyst who computed it, but the product stakeholder responsible for the definition
Related concepts: Exposure
Breaking change
dbt semantic models
Metric owner
Semantic registry
Study tips: Start with a check: take 2-3 data marts from your project and find fields that appear in multiple reports — these are candidates for shared metrics
Do not rush to implement a dbt semantic layer: first capture the intent in a Markdown registry, and only with 3+ consumers consider automation
Ask 5 questions for each metric: name, formula, grain, owner, consumers — if at least one has no answer, the metric is not ready
Use a Qwen prompt as an assistant for drafting the registry, but do not trust it to automatically create semantic models — always confirm the owner
Think of exposures as a dependency map: every exposure is a commitment to warn the consumer before a breaking change
Distinguish a 'metric' from a 'column': a column is a physical field, a metric is a contract for its interpretation
A sign of drift: one team says 'the metric grew', another says 'it fell', while the data is the same — meaning the definitions have diverged
Do not confuse a semantic layer with a name catalog: a metric registry is about meaning and formulas, not about renaming columns
Additional resources: Dbt semantic layer (official documentation): https://docs.getdbt.com/docs/build/dimensions — official guide on entities, dimensions, and measures in dbt
Dbt metrics (deprecated, but useful for understanding the evolution): https://docs.getdbt.com/docs/build/metrics — historical context of the semantic layer in dbt
Book 'Fundamentals of Data Engineering' (O'Reilly): Joe Reis and Matt Housley — chapter on the semantic layer and metrics in the modern stack
Article 'What is a semantic layer?' (Towards Data Science): Practical explanation of the concept with examples from real companies
Cube.dev — an alternative semantic layer: https://cube.dev/docs — an approach alternative to the dbt semantic layer
dbt Labs blog: 'The Rise of the Semantic Layer': https://blog.getdbt.com — a series of articles from the dbt team on the evolution of the approach
Course materials: part 18 (data marts and check facts): The previous part of the course, necessary for understanding the context of the semantic layer
Summary: Part 19 of the course introduces the concept of the semantic layer, metrics, and exposures as a response to definition drift during data reuse. The key idea: a metric is not a column, but a semantic contract with a formula, granularity, owner, and consumers. A semantic layer makes sense to introduce only when there are several consumers of shared metrics; otherwise it turns into bureaucracy. In the learning project, a Markdown registry that captures intent is sufficient before moving to dbt semantic models. Exposures show who reads the data product and help manage breaking changes through consumer notifications. The main practical takeaway: first record the definition, grain, owner, consumers, and check fact — and only then decide whether a formal semantic layer is needed.