Reading: Part 15. Gold layer: business entities and grain

Lesson 1 of 5 in module «Part 15. Gold layer: business entities and grain»
You are viewing the lesson without signing in. Sign in to save progress and take tests.

Part 15. Gold layer: business entities and grain

Gold layer is the data products layer. Here the downstream consumer sees stable data marts rather than intermediate computations. Therefore, gold starts with grain and contract, not with a convenient select.

What makes a gold model ready

A gold model is ready if:

  • it has an explicitly named consumer;
  • grain is recorded and verified;
  • required fields are tied to a contract;
  • PII policy is applied;
  • verification facts cover key risks;
  • the reviewer's report separates facts from assumptions.

For mart_customer_360 the consumer is customer and portfolio analytics. For mart_payment_risk_signals the consumer is risk signals monitoring. For mart_open_api_consents it is consent state control. These consumers should not accidentally receive the same grain.

Grain as protection

Grain answers the question "what does one row mean." If the answer changes, this is not cosmetics. It is a data API change.

Examples:

  • one row per customer_id;
  • one row per transaction_id;
  • one row per scope;
  • one row per day and channel.

If a mart is called customer_360 but contains multiple rows per customer, the name deceives the consumer.

Gold acceptance facts

  • grain primary key passes not_null;
  • grain primary key passes unique if the grain implies one row per key;
  • required metrics pass not_null;
  • no direct PII;
  • contract fields are present;
  • lineage is readable through a dbt graph or reviewer note.

Qwen query

Read specs/models and models/marts.
For each mart, determine grain, key verification facts, PII policy, and impact on the contract. Find marts where grain is not proven by a test or manual reviewer fact.
Do not modify files.

Minimum output

Compose a gold layer registry:

# Gold layer registry

## mart_customer_360
Consumer:
Granularity:
Required fields:
PII policy:
Verification facts:

Breakdown for the reader

Gold layer is the moment when engineering work becomes a public promise. Before it, you can talk about intermediate entities, normalization, and preparation. In the gold layer, a consumer appears: analyst, risk team, reporting, consent monitoring. That is why any ambiguity here costs more. If grain is not recorded, the consumer will invent it themselves. If PII policy is not verified, an extra column can end up in a report. If required metrics are not covered by tests, the mart only appears ready on the surface.

A gold model should be narrow. This sounds strange, because you want to make the mart "richer": add a product, channel, segment, last operation, risk flag, credit stage. But each extension changes the question the table answers. Customer 360 can be a customer slice, or it can quietly become a customer-product mart. The name will stay the same, but the meaning will not.

That is exactly why it is useful to write the public promise in a single line. If this line cannot be formulated without commas and exceptions, the gold model is too wide. "One row per customer for portfolio analytics without direct PII" is a good frame. "A useful customer mart with accounts, payments, risks, and everything that might be needed" is an invitation to drift.

Reviewing the gold layer should be stricter than reviewing staging. In staging you can discuss technical normalization. In gold you need to ask about the consumer, contract, SLA, PII, grain, and evidence. If the reviewer cannot understand to whom the mart is promised and how it will be used, the model is not yet gold, even if it sits in models/marts/.

Practice

For each mart from the README, write one public promise in a single line. If the promise does not contain a consumer and grain, the mart is not yet ready to be considered a gold data product.

Typical mistake

Calling any table in models/marts/ the gold layer. Gold is not a folder. It is a promise to the consumer, anchored by specification, contract, and verification.

Control questions

  1. Why is grain part of the public contract?
  2. What tests are needed for a customer-level mart?
  3. When can a nullable field still be a breaking change?
My notes
0 / 10000

Notes are saved in this browser. They will not appear on another device.

Course menu

Course

SDD Data. Bank Data Platform with Qwen Code and dbt
Progress 0 / 110