Skip to content
Final StatePrediction vs Decision: The Machine Forecasts, You Judge
VOL. I  ·  NODE 117▢  ATLAS

THE UMBRELLA

Prediction vs Decision: The Machine Forecasts, You Judge

Prediction is a claim about what may happen; decision is the accountable choice about what to do.

CLAIM AND COMMITMENT

A claim about the world is not a commitment to act

Split diagram contrasting prediction cards with signed decision cards.Prediction is shown as a probability claim with no owner yet; decision is shown as an accountable act carrying a signature.PREDICTIONCLAIMDECISIONCOMMITMENTINVOICEOFF CONTRACTPROPERTYNEARS AUCTIONI DISPUTE ITSUBMIT BIDJUDGMENTVALUES · WEIGHTSERROR COST
  • Prediction: this invoice probably doesn't match
  • Decision: I dispute it — and I answer for that
  • Between them sits judgment: values, weights, the cost of each error

A decision rests on a decision premise: the values that say what counts as a good outcome. A model can apply an encoded premise; it does not choose or answer for it.

PREDICTION MACHINES

When a component gets cheap, its complements can become more valuable

Curve chart showing prediction getting cheap while judgment can become a more valuable complement.The figure visualizes the Agrawal, Gans, and Goldfarb frame: as prediction cost falls, judgment can become a more valuable complement.PREDICTION GETS CHEAPERJUDGMENTMAY RISEPREDICTIONCOST FALLSCOMPLEMENTS MAY GAIN VALUE
concept-based: Agrawal, Gans and Goldfarb, Prediction Machines, 2018.
  • Agrawal, Gans and Goldfarb — Prediction Machines, 2018
  • Their move: read AI as a fall in the cost of prediction
  • Judgment, data, and action are complements whose value may rise as prediction expands

Cheap prediction can shift the bottleneck toward complements such as data, action, and accountable judgment; it does not guarantee that any one complement becomes more valuable.

A probability does not say what the outcomes are worth.

Automated systems can execute decisions, but accountability for the objective, error tradeoff, and authority remains a governance choice. A confident prediction is still only evidence for a commitment; sounding sure does not assign responsibility.

INSIDE AND OUTSIDE

Estimate from the crowd of similar cases, not from this one's story

Inside-view story panel beside an outside-view field of similar cases.The figure contrasts estimating from this case's narrative with anchoring on outcomes from a validated set of comparable cases.STORY OR SERIES?INSIDE VIEWTHIS PROJECTTHIS TEAMTHIS STORYOUTSIDE VIEWSIMILAR CASESANCHOR ON THE SERIESBEFORE THE STORY
concept-based: Kahneman and Tversky, Intuitive Prediction: Biases and Corrective Procedures, 1979; curriculum example retold by Kahneman, Thinking, Fast and Slow, 2011.
  • Kahneman and Tversky: the inside view versus the outside view
  • Inside: built from this case's own details and often vulnerable to optimism
  • Outside: anchor on outcomes from a relevant reference class

The inside view sees only this story; the outside view sees the series it belongs to. It works because what is rare for you is common for the corpus — the class exists even when your own experience of it does not.

STORY VERSUS STATISTICS

Story versus statistics — once the class is real

Candidate analogues being retrieved and narrowed into a validated reference class with a base rate.The machine can cheaply propose similar cases, but the human must validate comparability before treating them as an outside view.REFERENCE-CLASS FORECASTCANDIDATESVALIDATEDCLASSBASE RATERETRIEVE CANDIDATESVALIDATE THE CLASS
method-based: Flyvbjerg, From Nobel Prize to Project Management, 2006; HM Treasury, The Green Book, 2026, and optimism-bias supplementary guidance.
  • Flyvbjerg applied reference-class forecasting to major projects
  • The UK Green Book requires explicit adjustment for optimism bias
  • The weaker the case-specific evidence, the more weight a relevant base rate deserves

Assembling candidate comparisons can be costly. Search or retrieval systems may lower that cost, but retrieval does not prove comparability or produce a valid base rate by itself. Past any defensible reference class lies Knightian uncertainty: cases for which no stable class is available.

WHAT A CORPUS CAN INDUSTRIALIZE

The machine proposes candidates. You validate the class.

Corpus handoff from reference class and base rate to owner signature, with verdict withheld.The engine carries the population it speaks from, but the owner still decides the error tradeoff and signs the commitment.CLASS, NOT VERDICTCORPUSREFERENCECLASSBASE RATEOWNERERRORTRADEOFFNO VERDICTTRACE THE CLASSOWNER KEEPS THE COMMITMENT
  • Automatable: candidate retrieval and outcome summarization
  • Still required: provenance, comparability, and a defensible base-rate calculation
  • Still owned: the error tradeoff, authority, and accountable commitment

A corpus can propose the comparison set; the value premise remains the decision premise it does not carry.

The machine can propose candidates for an outside view, not certify the verdict.

That line separates retrieval from authority: a system can surface possible comparisons, while provenance and comparability determine whether they form a valid class. Better prediction does not assign the value premise or accountability.

WHERE THE VALUE WENT

Cheap prediction can move the bottleneck to accountable judgment.

  • Use systems for prediction and candidate retrieval where they perform
  • Validate provenance and comparability before claiming an outside view
  • Operator test: what class, what base rate, what error tradeoff, whose authority?

Return to the measurable wins becoming table stakes: value can move past cheap prediction toward judgment, but the shift must be observed rather than assumed.

Read the transcript

01 · THE UMBRELLA

The machine tells you there is a seventy percent chance of rain. Useful. But that probability alone does not tell you whether to carry the umbrella. The action depends on how much you mind getting wet, weighed against the nuisance of carrying something you may not need. Two people can hear the same forecast and reasonably make opposite choices. The prediction describes uncertainty about the world. The decision combines that evidence with values, stakes, and authority. Between the forecast and the umbrella runs a seam. A system may operate on both sides when people configure it to act, but the probability never settles the objective or the responsibility by itself.

02 · CLAIM AND COMMITMENT

Pull the two apart, because the difference matters. A prediction is a claim about how the world is, or may be. This invoice probably does not match the contract. This property is probably approaching auction. A decision is a commitment that triggers an action and sits within someone's authority. I dispute the invoice. I submit the bid. Between the claim and the commitment lie an objective, an error tradeoff, and a rule for acting. A model can apply a rule that people encode, and an automated system can execute the resulting action. What it does not do is originate or answer for the authority under which that rule operates.

03 · PREDICTION MACHINES

This is also an economic distinction. In 2018, Ajay Agrawal, Joshua Gans, and Avi Goldfarb published Prediction Machines. Their framing was to treat advances in artificial intelligence as a fall in the cost of prediction: using available information to generate information that is missing. When a component becomes cheaper and is used more, complements can become more valuable. Their complements to prediction include judgment, data, and action. Judgment includes specifying what outcomes are worth and how costly different errors are. This is not a law that every human judgment becomes dearer. It is a way to look for the next bottleneck as prediction gets cheaper.

04 · NO VALUES IN A FORECAST

Here is the line to keep. A probability does not say what the outcomes are worth. A forecast can reflect choices about data, targets, and thresholds, so it is not value-free in its construction. But its number alone does not determine the error tradeoff, the authority to act, or who answers for the result. Automated systems can execute decisions. The governance question is who set the objective and rule, who may override them, and who remains accountable. A prediction that sounds certain is still evidence for a commitment, not an assignment of responsibility.

05 · INSIDE AND OUTSIDE

The second half is a forecasting discipline with its own history. Kahneman and Tversky contrasted an inside view, built from the plan and details of this case, with an outside view grounded in the outcomes of similar cases. Kahneman later told the story on his own curriculum team. They expected to finish in roughly two years. A colleague with relevant experience reported that comparable teams had taken seven to ten years, and that many never finished. The lesson is not that the outside view is automatically right. It is that a relevant reference class can correct optimism that the case narrative hides. The work is choosing a class comparable enough to carry that weight.

06 · STORY VERSUS STATISTICS

Bent Flyvbjerg applied reference-class forecasting to major projects: identify a relevant class, establish the outcome distribution, and compare the proposed project with that distribution. The United Kingdom's Treasury now requires appraisals to adjust explicitly for optimism bias by changing estimated costs, durations, and benefits. Its supplementary guidance uses evidence from past or similar projects when robust primary evidence is absent. Search and retrieval systems may reduce the cost of finding candidate analogues and outcome records. But retrieval is not validated reference-class forecasting. Provenance, selection rules, and comparability determine whether the candidates support a defensible base rate.

07 · Advertisement · Twain GTM Agents

A good outbound message should know why it is being sent. Twain GTM agents research accounts, qualify contacts, and turn live signals into relevant outreach. They work with your CRM, personas, and GTM stack, so the message is not just personalized. It is backed by a specific reason to start the conversation.

08 · WHAT A CORPUS CAN INDUSTRIALIZE

Now the two halves meet. Cheap prediction can move a bottleneck toward complementary judgment. An outside view contributes evidence to that judgment: a relevant class, its outcomes, and a base rate. A retrieval system can help surface candidate cases and summarize recorded outcomes. But candidates become a reference class only through explicit inclusion rules, traceable provenance, and a test of comparability. Then another part remains: the value premise. What counts as a good outcome here? Which error is more costly? Who has authority to act? A base rate informs those choices. It does not own them. The system can propose candidates for an outside view. It cannot certify its own verdict.

09 · NOT A VERDICT

Hold the node in one sentence. A system can propose candidates for an outside view, not certify the verdict. It can retrieve records and summarize what happened. Whether those records form a valid class depends on provenance, selection, and comparability. What to do then depends on an objective, an error tradeoff, and legitimate authority. A model may be configured to compute or execute that rule. It still does not make the organization accountable. Better prediction can narrow uncertainty; it cannot assign responsibility for the commitment built on top of it.

10 · WHERE THE VALUE WENT

The argument ends at one seam. Prediction and candidate retrieval may become cheaper. Use them where they perform, but validate the evidence before calling it an outside view. Ask where each case came from, why it belongs, and how the base rate was calculated. Then make the commitment explicit: what outcomes matter, which error costs more, who may act, and who answers for it. Cheap prediction can move the bottleneck toward those choices, but does not guarantee their value. Keep four questions on your side: what class, what base rate, what error tradeoff, whose authority. The forecast informs the verdict. It does not sign it.

01 / 10 · THE UMBRELLA0:00 / 7:44