The method
Show the number.
A demo shows that something worked once. A measurement shows how often, under which conditions, and how you could check it yourself. That is the whole method, and the book is built on it.
143 measured findings, chapter by chapter
Each mark on the dial is one finding in the book: a number that came from a run. Pick a chapter to see how many it holds.
How this was counted: a finding is one row of the findings table in the book's working record that a chapter or an appendix uses. Each one is placed in the first chapter that uses it. Rows used only by the course are not counted.
Findings by chapter
| Chapter | Title | Findings |
|---|---|---|
| 1 | Why AI Development Feels Different | 4 |
| 2 | Picking Your Stack Without Getting Paralyzed | 6 |
| 3 | Structured Output: Getting Data Back, Not Text | 10 |
| 4 | Your First Agent: A Loop, a Tool, and a Stop Condition | 20 |
| 5 | RAG: When to Retrieve, and How to Tell It Failed | 20 |
| 6 | Vector Storage: An Index That Outlives the Process | 13 |
| 7 | Conversation State: What to Keep and What to Drop | 5 |
| 8 | Multi-Agent Systems: When to Split, and When Not To | 10 |
| 9 | MCP: Using Tools You Did Not Write | 8 |
| 10 | Evaluation: Catching Regressions in a System That Won't Repeat | 7 |
| 11 | Security: Bounding What the Model Can Do | 7 |
| 12 | Observability: Tracing What Actually Happened | 7 |
| 13 | Cost: What It Spends, and How to Spend Less | 9 |
| 14 | Streaming, Async, and Jobs: Handling the Wait | 15 |
| 15 | Your Own System: The First Three Moves | 0 |
| Appendix C | 2 | |
| Total | 143 | |
What a measured claim carries
The same subject, said two ways. The first is how a demo would put it. The second is one finding from chapter 1 of the book, with the four things every finding in it carries.
A demo claim
The model gives a good answer to the customer's question.
How often? Compared with what? Could you run it again?
An illustration of how demos are usually worded, not a quotation.
A measured claim
10 distinct answers out of 20
- What happened
- 10 distinct answers in 20 calls. The most common one came back 8 times.
- Why the gap
- A model picks its next words by sampling, so by default the same prompt does not give the same text twice. All 20 answers agreed in meaning: only the wording moved.
- How it was measured
- 20 calls with one prompt to claude-opus-4-6. Identical replies were grouped and the groups counted.
- In the book
- Chapter 1 of the book runs the same 20 calls under a second sampling setting and counts again.
- Run file:
- runs/ch1-sampling-2026-09-07.txt
- Script:
- scripts/ch1_sampling.py
Check it yourself
The book's rule is that every published number has a path back to a row in a recorded run. When the book launches, the run files will be public in a companion repository, so a reader can follow that path instead of taking the author's word.
About the bookJoin the waitlistLaunching soon