Topic: Applied Part 13. Practical Exam: Assemble a production SDD pipeline
Difficulty level: Medium
Estimated study time: 6-8 hours
Prerequisites: Successful mastery of chapters 1-12 of the second volume of the course
Understanding of the concepts of SDD (Spec-Driven Development) and the AgentClinic-production framework
Skills working with the terminal and launching runnable scripts (Python/Shell)
Basic understanding of the principles of AI agents, CI/CD and post-mortems
Learning objectives: Independently assemble a coherent package of artifacts (capstone/) for completing a production SDD scenario.
Learn to restore requirements with provenance (genealogy.md) and form controllably defective specifications.
Master the configuration of constitution rules (constitution.md), budget control and anti-Goodhart metrics.
Validate a case through runnable scripts and produce file arbitration records (judgment.md).
Evaluate package readiness on a 30-point scale, guaranteeing its reproducibility for an independent reviewer.
Overview: This module is the final practical exam of the second volume. Its main goal is to prove that you are capable of completing a real production SDD (Spec-Driven Development) scenario, from a legacy trail to a final solution that is justified by strict facts rather than the blind confidence of an AI agent. Instead of studying new theoretical material, you will assemble a single verifiable route (the capstone/ package) based on a high_memory_usage or autoscale_200pct incident. The exam is considered successfully passed if an external reviewer can open your package and, without additional verbal explanations, understand the verification logic, the source of requirements and the reasons for blocking or admitting to production.
Key concepts: Sdd pipeline (spec-driven development): A methodology of development and operations in which every automatic action of an agent relies on a verifiable specification and immutable rules, rather than on heuristics.
Provenance (genealogy): The principle of traceability of the origin of any requirement. The genealogy.md artifact links a requirement to its sources (post-mortem, QWEN.md, oral practice) and estimates the level of confidence.
Readiness gate: The final check of a solution before admission to production. It includes a readiness score, a list of blocking conditions and evidence (evidence_ref).
Controllably defective specification (poisoned spec): A specification into which exactly one defect is intentionally introduced (for example, a priority conflict or a hidden limit violation) to test validation and correction mechanisms (fixed-spec).
Constitution: A set of system rules, clearly divided into immutable_principles (immutable laws) and mutable_rules (changeable rules with a defined TTL lifetime, scope of action and rollback conditions).
Anti-goodhart control: A mechanism for protecting against metric distortion, when excessive optimization of a target KPI leads to negative consequences. It is controlled through the introduction of guard metrics (for example, token_health).
Practice exercises: Name: Package initialization and routing
Problem: Create the capstone/ directory and a README.md file. Formulate 5 key stub lines for the high_memory_usage incident: Incident-case, Main risk, Key check, Main blocker, Next fix.
Solution: 1) Create the folder mkdir -p capstone. 2) Create the file capstone/README.md. 3) Enter the stub: Incident-case: high_memory_usage; Main risk: auto-remediation without a complete audit_trace; Key check: python3 scripts/check_readiness.py; Main blocker: stateful workload without backup_verified; Next fix: add evidence_ref for backup. This will focus further work.
Complexity: beginner
Name: Restoring provenance and defect
Problem: Based on an abstract post-mortem, create a genealogy.md file (one requirement, 2 sources) and a poisoned-spec.md / fixed-spec.md pair with one cyclic defect and its fix.
Solution: 1) In genealogy.md, specify the requirement 'Clear cache when memory reaches 90%', sources: 'Grafana Alert' and 'SRE oral practice', confidence level: High. 2) In poisoned-spec.md, create the rule: 'If memory >90%, clear the cache; if the cache is empty, request memory from the system'. 3) In fixed-spec.md, specify: 'Cycle fix: when the cache is empty, run GC, not request memory'.
Complexity: intermediate
Name: Running validation and recording the result
Problem: Run the check_readiness.py verification script from the runnable examples for readiness_block_stateful.json. Transfer the result to validation.md in the format: command, expected fact, actual result, blocker.
Solution: 1) Run python3 scripts/check_readiness.py --readiness fixtures/readiness_block_stateful.json. 2) The command will return code 1 (error/blocker). 3) In validation.md write: Command: check_readiness...; Expectation: Blocking of stateful; Result: Exit code 1; Blocker: No backup confirmation. This proves that the readiness gate works correctly.
Complexity: advanced
Case studies: Name: Incident high_memory_usage: from alert to secured deploy
Scenario: The AgentClinic-production system received a critical alert from Grafana about exceeded memory consumption. Legacy traces are partially lost: part of the recovery rules are known only from the old QWEN.md documentation and oral knowledge transfer. Automatic remediation (cleanup and restart) looks like an attractive solution.
Challenge: Automatic intervention may violate the blast radius limit or the tier budget. The team needs to prove that the proposed specification is safe, contains no hidden defects and is ready for admission to production without the risk of data loss.
Solution: Engineers assembled the capstone/ package. They restored the origin of requirements in genealogy.md. They created an intentionally defective specification and fixed it, recording the rules in constitution.md. They ran a test pass (dry-run) with the check_readiness.py script, which blocked the action due to missing backup_verified labels on the stateless workload. The results were moved to judgment.md with the status DEFERRED.
Result: The evidence package scored 26/30 points. A dangerous deploy was prevented thanks to the correct operation of the readiness gate. The decision was deferred until evidence_ref for backups was added, which made it possible to avoid downtime in production.
Lessons learned: A readiness score without evidence (evidence_ref) is meaningless and cannot serve as admission.
One blocker detected at the testing stage is more valuable than a formal APPROVE without running checks.
Logs out/ should not be included in the final reviewable trail; the reviewer needs only the validation.md digest.
Related concepts: Readiness Gate
Genealogy
Poisoned Spec
Validation
Name: Dispute resolution during autoscale_200pct scaling
Scenario: Under peak loads, the AI agent (Implementor) proposed scaling resources by 200% based on a template from QWEN.md. The Safety agent imposed a veto, citing the risk of exceeding the budget and a possible Goodhart effect, when the system will scale resources for the sake of the metric rather than real benefit.
Challenge: A role conflict arose. It is required to conduct file arbitration through a duel and make a decision based on the system constitution and metrics, rather than the authority of the agents.
Solution: The tribunal mechanism was launched: run_duel.py to record arguments and check_invariants.py to check budgets. In goodhart-note.md it was recorded that the KPI 'response speed' lies, and a guard metric 'cost/request' was added. In judgment.md the verdict DENY was issued, which automatically created a precedent in precedents.md (with case_id and applies_to fields).
Result: The conflict was resolved in a formalized way. A precedent was created that in the future allows automatically rejecting similar requests without spending tokens on a duel. Budget-note.md clearly showed the limits of the cheap tier.
Lessons learned: A Safety-veto cannot be bypassed by a simple majority of votes.
Any recurring dispute should turn into a recorded precedent.
Anti-Goodhart control (a KPI and guard metric bundle) is vital for automatic scaling.
Related concepts: Arbitration and roles
Judgment
Anti-Goodhart
Precedents
Study tips: Start from the end: first fill in 5 stub lines in README.md to set the framework, and only then create the remaining artifacts.
Do not mix incidents. Choose strictly one case (for example, high_memory_usage) and do not add elements from other scenarios to it.
Use the 'Golden Path': open examples/templates/capstone-dossier.md as a sample of how concise your package should be. Do not bloat the documentation.
Simulate someone else's review: after assembling capstone/, put the work aside for an hour and then try to answer 5 control questions looking only at your files, without referring to the AI chat history.
If you get stuck on an artifact, return to the corresponding chapter (1-12), finding the block 'How this gets into capstone/'.
Additional resources: Examples/templates/capstone-dossier.md: A reference example (golden path) demonstrating the ideal balance of detail for the exam.
Part 00: production-lab: The basic laboratory framework of AgentClinic-production. Required reading if the context seems lost.
Part 11: real-api-deployment: Instructions for launching runnable scripts for the main case high_memory_usage.
Part 08: multiagent-tribunal: Theory and practice of file arbitration, duels and work with precedents for the autoscale_200pct case.
Summary: The practical exam 'Assemble a production SDD pipeline' is a test of your ability to embed a culture of evidentiality in the processes of AI agents. You have learned to form a compact, reproducible package of artifacts (capstone/) that links incidents, defects, constitution and validation into a single verifiable route. Key takeaway: readiness for production is confirmed not by perfect metrics or the volume of documentation, but by the presence of transparent provenance, working blockers and clear answers to the question 'Why is this safe?'.