Part 9. Feature Validation: From Specifications to Facts

Feature validation is not a formality after writing code. It is a separate mode of work where textual specifications become verifiable facts. A specification explains intent, but does not by itself prove that intent is implemented. A fact is a statement that a machine or human can verify without reinterpreting lengthy prose.

Short formula:

Specifications guide.
Facts gate the merge.

flowchart LR
  A["Textual specification<br/>intent and boundaries"] --> B["Set of facts<br/>validation.md"]
  B --> C["Checks<br/>commands, tests, manual scenarios"]
  C --> D{"Facts confirmed?"}
  D -- "yes" --> E["Branch can be merged"]
  D -- "no" --> F["Fix code<br/>or clarify specification"]
  F --> B

For working with agents that write code, this is critical. A model may read the same textual specification differently across sessions or versions. A test, exit code, HTTP status, database invariant, or explicit contract are less dependent on interpretation. Therefore, validation.md should not be merely a checklist, but a set of facts that gate the merge.

What is a fact

A fact is an executable or unambiguously verifiable statement:

• npm run typecheck exits with code 0;
• GET / returns 200;
• response contains <h1>AgentClinic</h1>;
• expired JWT returns 401;
• feedback submission without message fails validation;
• migration can be run twice without unexpected schema changes;
• for every initial agent, the details page returns the associated list of ailments.

Bad checklist item:

Make sure the page looks good.

Better:

At 375px width, header, main content, and footer do not overlap.
Main heading remains visible without horizontal scrolling.

Levels of fact sets

Use four levels.

Examples verify specific input-output pairs:

curl -s http://localhost:3000 | rg "<<h1>AgentClinic</h1>"

Invariants describe what must always be true:

Every feedback record has a non-empty message.

Properties verify a class of cases:

Any rating outside the range 1..5 is rejected.

Contracts fix a precondition, action, and postcondition:

If the session is not authenticated,
when requesting GET /dashboard

the response redirects to /login.

Not every feature requires all four levels. But every feature must have at least several machine-verifiable facts.

What level of facts is needed — by risk type

Not all features are equal. A simple UI banner and a database migration require different check density. The minimally sufficient level of facts can be chosen using a matrix:

How to read the table:

• Example — a specific input-output pair (one command, one curl, one approved test).
• Invariant — what is always true after an action. For migration: "rerunning does not change schema". For background jobs: "after successful run, counter does not decrease".
• Property — verifies a class of cases. For validation: "any rating outside 1..5 is rejected". For authorization: "any request without session returns 401".
• Contract — formal binding "given condition X, action Y leads to Z".
• Manual fact — what a human checks by eye or by hand. Required for UI because automatic verification of visual hierarchy is often inadequate.

The goal of the matrix is not to turn it into a mandatory checklist, but to help see what you missed. If a "data migration" feature passes only with an example, without an invariant — that is a signal to rewrite validation.md.

Structure of validation.md

Example:

# Validation — Feedback Form

## Fact Set

### F1 — TypeScript compiles

- Command: `npm run typecheck`
- Expectation: exit code 0
- Owner: automatic check
- Status: draft

### F2 — tests pass

- Command: `npm test`
- Expectation: exit code 0
- Owner: automatic check
- Status: draft

### F3 — empty message is rejected

- Command: `npm test -- feedback`
- Expectation: POST /feedback with empty message returns 400
- Owner: automatic check
- Status: draft

### F4 — valid feedback is saved

- Command: `npm test -- feedback`
- Expectation: valid POST /feedback adds one row and redirects to /feedback
- Owner: automatic check
- Status: draft

### F5 — page remains usable on mobile screen

- Check: open /feedback at 375px width
- Expectation: form fields and submit button are visible without horizontal scrolling
- Owner: manual check
- Status: draft

## Readiness Criteria

- All automatic facts pass.
- Manual facts are checked.
- Facts that cannot be implemented are removed from scope or returned to draft with explanation.
- Roadmap and changelog are updated before merge.

Lifecycle helps not to mix intent and proof:

• draft: fact proposed but not yet anchored;
• required: fact accepted as mandatory for the feature;
• implemented: has test, command, or manual confirmation;
• deferred: fact consciously moved to a future phase.

Start with the diff

git status --short
git diff --stat main...HEAD

Ask Qwen Code to check not only specification compliance, but also fact set statuses:

/clear
Compare this branch with @specs/2026-05-01-hello-hono/validation.md.

Show:
1. facts that are implemented and pass;
2. facts that are missing;
3. facts that are ambiguous and require rewriting;
4. implementation decisions not described in requirements.md;
5. outdated specification statements.

Do not modify files yet.

If Qwen Code cannot determine pass/fail for a checklist item, it is not a fact, but a prose wish. Rewrite it.

Human-in-the-loop validation

An agent can find mechanical mismatches, but a human must evaluate product and architectural concerns:

• does the page match the mission;
• has the scope of the task grown too much;
• is there an unagreed dependency;
• is it clear to a new developer why files are organized this way;
• is there a fact for every risky behavior;
• have important decisions remained only in chat.

Typical example: after first implementation, it often becomes clear that the page structure is needed not as one monolithic component, but as a set of Header, Main, Footer. This is not just "fix code": you need to update plan.md and facts in validation.md, so a future session does not revert to the old interpretation.

Prompt for fixing code, specification, and facts together

The implementation needs a clearer page structure.

Update @specs/2026-05-01-hello-hono/plan.md and require:
- Layout component;
- Header component;
- Main component;
- Footer component;
- linking static/style.css from Layout.

Update @specs/2026-05-01-hello-hono/validation.md with facts that check:
- response contains header/main/footer landmarks;
- /static/style.css is served by the server;
- npm run typecheck exits with code 0.

Then update the implementation to match the new plan and facts.
Keep changes within the boundaries of this feature.

This simultaneously prevents specification drift and fact drift.

Automatic checks

Minimum:

npm run typecheck

If tests already exist:

npm test

If there is a dev server:

npm run dev
curl -s http://localhost:3000
curl -s http://localhost:3000/static/style.css

Record exact commands and expected results in validation.md. Do not write "check that it works" when you can write the command and expected exit code.

Manual facts

Manual facts are no weaker than automatic ones, if they are specific. Weak manual check:

Check the interface.

Proper manual fact:

At 375px width, the /feedback page shows name field, message field,
submit button, and three latest records without horizontal scrolling or overlaps.

Manual facts are useful for tone, visual hierarchy, basic accessibility check, and scope creep detection. But if a manual fact repeats in every feature, consider automating it via Playwright or unit and integration tests.

CI validation

Facts must reach the merge gate. For a small project, local commands are enough. For a team, add CI:

Merge request cannot be accepted until:
- npm run typecheck passes;
- npm test passes;
- required route tests pass;
- fact set in validation.md is updated.

This is the practical answer to the criticism "specifications are interpreted". Textual specification guides the agent, but facts decide the merge.

Evidence bundle for merge

When a feature approaches merge, the reviewer should have one compact artifact showing: what exactly was implemented, which facts passed, which were deferred. This artifact is conveniently called an "evidence bundle" (in English-language sources — evidence bundle).

This is not a separate new file — it is a format for describing the merge request. A good evidence bundle consists of:

• link to specification folder (specs/YYYY-MM-DD-feature/);
• list of facts from validation.md with statuses: confirmed, failed, deferred;
• command run traces: command names, exit codes, last output line or short excerpt;
• manual check results: what exactly the human checked, on which screen, what they saw;
• list of decisions made during implementation that were not in the original specification;
• links to commits where these decisions are reflected.

A template for such a merge request description is in Appendix C. Main idea: the reviewer should not need to rerun everything to verify readiness. They should be able to understand from the evidence bundle what exactly the author did and checked, and selectively rerun specific commands if in doubt.

If the evidence bundle contains an item "fact changed after failure", this is not a reason to hide it. On the contrary: an explicitly explained fact change is a normal part of SDD, a hidden change is an antipattern (see part 20).

Updating the roadmap

After passing the fact set:

## Phase 1: Hello Hono (completed)

- [x] Install Hono and tsx.
- [x] Create GET / route.
- [x] Return minimal HTML with server-side rendering.
- [x] Add type check script.

Commit:

git add specs/roadmap.md specs/2026-05-01-hello-hono
git commit -m "Validate Hello Hono feature"

Merge:

git checkout main
git merge phase-1-hello-hono

git branch -d phase-1-hello-hono

Practice

1. Run all automatic facts.
2. Ask Qwen Code to compare code with validation.md.
3. Rewrite ambiguous checklist items as facts.
4. Fix code or specification if they diverge.
5. Mark fact statuses.
6. Mark phase in the roadmap.
7. Perform the merge.

Review questions

1. Why should textual specification not be the only gate to merge?
2. How does a fact differ from a wish in a check?
3. When can a manual check be considered a fact?
4. What to do if tests pass but implementation does not match requirements.md?
5. Why is "specifications guide, facts gate the merge" better than simply "write better specifications"?