Measured

The method

Show the number.

A demo shows that something worked once. A measurement shows how often, under which conditions, and how you could check it yourself. That is the whole method, and the book is built on it.

143 measured findings, chapter by chapter

Each mark on the dial is one finding in the book: a number that came from a run. Pick a chapter to see how many it holds.

Chapter
123456789101112131415C143measured findings

143 findings across 15 chapters and the appendices. Behind them: 134 run files and at least 1,123 model calls on record.

Chapter 1

Why AI Development Feels Different

4 measured findings

One of this chapter's findings is a question on the home page. Guess it before you read it. Go to the three questions

Chapter 2

Picking Your Stack Without Getting Paralyzed

6 measured findings

Chapter 3

Structured Output: Getting Data Back, Not Text

10 measured findings

Chapter 4

Your First Agent: A Loop, a Tool, and a Stop Condition

20 measured findings

Chapter 5

RAG: When to Retrieve, and How to Tell It Failed

20 measured findings

Chapter 6

Vector Storage: An Index That Outlives the Process

13 measured findings

Chapter 7

Conversation State: What to Keep and What to Drop

5 measured findings

Chapter 8

Multi-Agent Systems: When to Split, and When Not To

10 measured findings

One of this chapter's findings is a question on the home page. Guess it before you read it. Go to the three questions

Chapter 9

MCP: Using Tools You Did Not Write

8 measured findings

Chapter 10

Evaluation: Catching Regressions in a System That Won't Repeat

7 measured findings

Chapter 11

Security: Bounding What the Model Can Do

7 measured findings

Chapter 12

Observability: Tracing What Actually Happened

7 measured findings

Chapter 13

Cost: What It Spends, and How to Spend Less

9 measured findings

Chapter 14

Streaming, Async, and Jobs: Handling the Wait

15 measured findings

One of this chapter's findings is a question on the home page. Guess it before you read it. Go to the three questions

Chapter 15

Your Own System: The First Three Moves

No finding is placed in this chapter.

Appendix C

2 measured findings

How this was counted: a finding is one row of the findings table in the book's working record that a chapter or an appendix uses. Each one is placed in the first chapter that uses it. Rows used only by the course are not counted.

Findings by chapter
Findings by chapter
ChapterTitleFindings
1Why AI Development Feels Different4
2Picking Your Stack Without Getting Paralyzed6
3Structured Output: Getting Data Back, Not Text10
4Your First Agent: A Loop, a Tool, and a Stop Condition20
5RAG: When to Retrieve, and How to Tell It Failed20
6Vector Storage: An Index That Outlives the Process13
7Conversation State: What to Keep and What to Drop5
8Multi-Agent Systems: When to Split, and When Not To10
9MCP: Using Tools You Did Not Write8
10Evaluation: Catching Regressions in a System That Won't Repeat7
11Security: Bounding What the Model Can Do7
12Observability: Tracing What Actually Happened7
13Cost: What It Spends, and How to Spend Less9
14Streaming, Async, and Jobs: Handling the Wait15
15Your Own System: The First Three Moves0
Appendix C2
Total143

What a measured claim carries

The same subject, said two ways. The first is how a demo would put it. The second is one finding from chapter 1 of the book, with the four things every finding in it carries.

A demo claim

The model gives a good answer to the customer's question.

How often? Compared with what? Could you run it again?

An illustration of how demos are usually worded, not a quotation.

A measured claim

10 distinct answers out of 20

What happened
10 distinct answers in 20 calls. The most common one came back 8 times.
Why the gap
A model picks its next words by sampling, so by default the same prompt does not give the same text twice. All 20 answers agreed in meaning: only the wording moved.
How it was measured
20 calls with one prompt to claude-opus-4-6. Identical replies were grouped and the groups counted.
In the book
Chapter 1 of the book runs the same 20 calls under a second sampling setting and counts again.
Run file:
runs/ch1-sampling-2026-09-07.txt
Script:
scripts/ch1_sampling.py

Check it yourself

The book's rule is that every published number has a path back to a row in a recorded run. When the book launches, the run files will be public in a companion repository, so a reader can follow that path instead of taking the author's word.

About the bookJoin the waitlistLaunching soon