Reading: Part 10. dbt Staging Models

Lesson 1 of 5 in module «Part 10. dbt Staging Models»
You are viewing the lesson without signing in. Sign in to save progress and take tests.

Part 10. dbt Staging Models

Staging turns raw facts into stable technical models. It is not the place for complex business logic, but it is the place for clear types, naming, primary keys, and minimal normalization. Good staging makes the SQL further down the chain boring and predictable.

What is allowed in staging

  • align column names with the project convention;
  • cast types (date, integer, decimal);
  • explicitly handle the semantics of empty values and null;
  • preserve source keys;
  • flag PII fields, but do not hide them without a reason.

Example:

try_cast(nullif(cast(revoked_at as varchar), '') as date) as revoked_at

This line does not solve a product question on its own. It technically turns an empty value into null, while the meaning "null in revoked_at = active consent" must be reflected in the specification and the mart.

What is forbidden in staging

  • aggregate customers;
  • compute a risk score;
  • choose the final grain of the mart;
  • delete rows "for tidiness";
  • hide contract drift.

If staging starts answering consumer questions, the layer has become too smart.

dbt tests

Minimum:

  • not_null and unique for source keys;
  • not_null for required amounts and dates;
  • accepted_values where the domain is small;
  • singular tests for complex constraints.

Tests in staging should catch technical issues of the source, but not replace data product validation.

Walkthrough for the reader

Staging is the first layer where data begin to speak the language of the platform. Here stable column names appear, explicit types, clear handling of empty values, and basic keys. But staging should not turn into a small business mart. Its task is to make the source usable for further work without claiming decisions that belong to the product specification.

A good staging model is boring: select fields, cast types, explicitly handle empty values and null, preserve synthetic identifiers, do not change the grain. Boredom here is useful. If complex aggregation appears in staging, a "only active" filter, risk classification, or a business metric calculation, the reviewer should ask why this decision was made before the model specification.

Empty-value handling is a good learning example. An empty revoked_at can break type casting. You can quickly replace it with null and forget about it. But the SDD approach requires

more: record that an empty value appears in raw; explain why it becomes null in staging; verify that the mart counts active and revoked consents consistently. Then a technical fix becomes a documented decision.

Staging also sets the SQL style for the agent. Models should be small, read top to bottom, and use ref()/source() where appropriate. If staging is hard to read, the models further down the chain will be even worse. If staging is clear, the reviewer can quickly separate technical normalization from product meaning.

Practice

Pick one staging model and label every transformation: renaming, type casting, null handling, or business logic. If you find business logic, check whether there is a specification for it.

Qwen prompt

Read the Schema Manifest and raw sources.
Create or review staging models.
Do not add business aggregations. For every type cast, explain the source fact it
is based on.

Minimum output

After the chapter:

  • staging models exist for the main sources;
  • keys are covered by dbt tests;
  • decisions on empty values and null are documented;
  • PII remains visible only where it is needed to demonstrate the policy.

Typical mistake

Hiding business logic in staging. For example, `case when amount_rub > 100000 then 1 else 0 end as risk_event` is acceptable only if the rule is already recorded in the specification or the source manifest. Otherwise the agent invents a risk policy.

Review questions

  1. What decisions can staging make without approval?
  2. Why does PII sometimes remain in staging?
  3. How does a staging test differ from a mart validation fact?
My notes
0 / 10000

Notes are saved in this browser. They will not appear on another device.

Course menu

Course

SDD Data. Bank Data Platform with Qwen Code and dbt
Progress 0 / 110