Reading: Part 7. Specification of the first data product

Lesson 1 of 5 in module «Part 7. Specification of the first data product»

You are viewing the lesson without signing in. Sign in to save progress and take tests.

Source

Part 7. Specification of the First Data Product

The book's first data product is customer_360. It should give analysts a customer slice without direct PII. At this step, it is important not to write SQL. First, you need to formulate the product's promise: who is the consumer, what is the output port, what grain, which fields are required, what freshness, and which checks will prove readiness.

Artifacts

specs/customer_360_product.odps.yaml
specs/customer_360_contract.odcs.yaml
specs/models/mart_customer_360.md
specs/validation/customer_360.md

ODPS describes the product. ODCS describes the technical contract. The model specification describes the grain and acceptance facts. The validation note ties the specification to the commands and reviewer checks.

Grain

For mart_customer_360 the grain is simple:

one row per `customer_id`

If you add account_id, product_code, or scope, this becomes a different mart. Such a change requires confirmation, because the downstream consumer will otherwise receive a different row unit.

PII Policy

Staging may read pii_email to show the risk. The mart must not expose it. The rule must be verifiable:

models/marts/mart_customer_360.sql does not contain direct PII fields.
the dbt singular test assert_customer_360_no_direct_pii returns 0 rows.

SLA and Freshness

In the training example, freshness is 24 hours. Locally we are not building a scheduler, so freshness remains a product promise and a release note. In a production environment, it needs to be backed by load metadata, source freshness, or an orchestrator.

Prompt for Qwen Code

Read specs/customer_360_product.odps.yaml,
specs/customer_360_contract.odcs.yaml, and specs/models/mart_customer_360.md.
Formulate the grain, PII policy, SLA, mandatory fields, and acceptance facts.
Do not modify the files.

Minimum Output

After the chapter there should be a table:

Field	Value
Data product	`customer_360`
Mart	`mart_customer_360`
Grain	one row per `customer_id`
PII policy	no direct PII in the mart
SLA	freshness 24 hours
Main check	unique/not_null on `customer_id`

This table will serve as a reference for parts 10–16.

Walkthrough for the Reader

A data product specification is the place where a technical table first

becomes a promise. Before it, mart_customer_360 looks like a model name. After it, it is already a contract with the consumer: who needs the mart, what a row means, which fields are required, which fields are forbidden, what data age is acceptable. If this contract is not recorded, SQL starts to play the role of a product, even though SQL only shows the method of computation.

ODPS and ODCS separate two different conversations. ODPS talks about the product: who the consumer is, why the mart exists, what result it promises. ODCS talks about the technical contract: which fields, types, SLA, and change rules must be preserved. When these layers are mixed, review becomes murky. You cannot tell whether the product meaning broke or only the technical form.

Grain should be written before SQL, because after SQL the author already sees a specific join, a specific grouping, and a specific convenient set of columns. At this stage, it is easy to justify extra granularity: "it might come in handy", "it does not hurt", "it is easier to check this way". But for the consumer, this can be a different product. One row per customer and one row per customer-product are different promises, even if both tables are called Customer 360.

A good specification does not try to describe the entire future. It captures the minimum needed for the first version: consumer, grain, PII policy, mandatory fields, SLA, and acceptance facts. That is enough for an agent to implement the mart and for a reviewer to reject it. If a reviewer cannot name a reason for rejection, the specification is too vague.

Practice

Compare ODPS, ODCS, and the model specification for Customer 360. Find one field each that belongs to the product, the technical contract, and model-level acceptance. If a field appears in the wrong layer, write down why this is a risk.

Typical Mistake

Recording "customer 360" as a general slogan. For an agent, this is too broad. You need to explicitly state which sources are included, what grain, which fields are forbidden, and what is considered readiness.