Skip to content
Final StateThe 70% Problem: Fast to Prototype, Hard to Finish
VOL. I  ·  NODE 121▢  ATLAS

THE DEMO THAT DAZZLES

The 70% Problem: Fast to Prototype, Hard to Finish

The 70% problem is Addy Osmani's field observation that AI can make a prototype arrive quickly while production work resists the same shortcut.

THE SHAPE, NOT A MEASURE

Rapid first pass, disproportionately stubborn finish

Illustrative progress curve moving quickly toward seventy percent before flattening into a difficult production finish.The curve marks seventy percent as Osmani's heuristic wall rather than a measured allocation, then names the production tasks that make further progress uneven.FAST FIRST PASS · STUBBORN FINISHTIME + ITERATIONPRODUCTION PROGRESS70% HEURISTICWALLDEBUGARCHITECTSECUREPERFORMPOLISHILLUSTRATIVE SHAPENOT A MEASURED SPLIT
Illustrative field heuristic from Addy Osmani, December 2024; 70/30 is not an empirical task split.
  • The 70/30 split is an illustrative wall, not a measured ratio
  • Osmani observed the pattern especially among non-engineers using AI coding tools
  • The hard residue includes debugging, architecture, security, performance, and polish

Fast apparent progress can conceal how much production work remains. The residue overlaps the jagged frontier: generating a prototype and making it reliable can fall on different sides of the model's capability.

THE EAGER JUNIOR

AI is like having a very eager junior developer on your team. They can write code quickly, but they need constant supervision and correction.

Addy Osmani, “The 70% Problem,” December 4, 2024

The metaphor comes from Osmani's first-person field guide, not a controlled study; its value is diagnostic, not statistical.

THE MEASURED SPEEDUP

Copilot cut completion time on one specified task

Two timing bars showing one hour eleven minutes with Copilot and two hours forty-one minutes without, alongside the 55.8 percent speed estimate.The exhibit bounds the result to 95 professional developers completing one JavaScript HTTP-server task, with a 21 to 89 percent confidence interval around the estimated speed gain.ONE HTTP-SERVER TASKRANDOMIZED · N=95 PROFESSIONALS1H 11M2H 41MCOPILOTCONTROL55.8% FASTER95% CI: 21% TO 89%COMPLETION: 78% VS 70%P=.0017
Peng, Kalliamvakou, Cihon & Demirer, 2023; N=95 professional developers; one JavaScript HTTP-server task; completion 78% with Copilot versus 70% control; p=.0017.

The Peng et al. experiment used one JavaScript HTTP-server assignment scored for correctness and completeness; it did not measure Osmani's 70/30 wall or long-run maintenance.

  • 95 professional developers were randomly split between Copilot and control
  • Average time: 1h 11m with Copilot versus 2h 41m without
  • Estimated speed gain: 55.8%; 95% confidence interval 21% to 89%

One result measures speed. One heuristic warns what the stopwatch omits.

The experiment establishes a bounded causal speedup; Osmani's observation supplies a question for broader delivery work. Neither licenses calling every job 70% automated.

WHY THE FINISH RESISTS

The residue is heterogeneous, coupled, and expensive to verify

Three-band diagram of a hard production residue: heterogeneous cases, coupled repairs, and expensive verification.The figure avoids treating the residue as one homogeneous block and stages three reasons finishing can resist the speed of initial generation.THE FINISH IS NOT ONE BLOCKHETEROGENEOUSLOOK-ALIKE CASESDIFFERENT FIXESCOUPLEDLOCAL REPAIRDISTANT BREAKEXPENSIVE TO VERIFYCONTEXT · ADVERSARYEXPERT JUDGMENTFLUENCY DOES NOT REMOVETHESE COSTS
  • Heterogeneous: superficially similar cases can require different fixes
  • Coupled: a local repair can break another state or interface
  • Expensive to verify: correctness may require production context and expert judgment

This is where a relevant verifier may be absent and metacognitive demand rises: fluent output can hide uncertainty about the surrounding system.

THE FALSE ECONOMY

Automate the visible tasks, orphan the production obligation

A task ledger where routine generation transfers to AI while production obligations fall into an unowned column.The diagram shows that automating visible tasks does not transfer accountability for reliability, maintenance, and exceptional cases.AUTOMATE TASKS · NOT ACCOUNTABILITYSPECIALISTSEATROUTINE OUTPUTRELIABILITYMAINTENANCEEXCEPTIONSAIROUTINEOUTPUTOWNER: BLANKRELIABILITYMAINTENANCEEXCEPTIONSTHE TASK MOVEDTHE OBLIGATION DID NOT
  • Routine generation moves to the tool
  • Reliability, maintenance, and exceptions remain
  • If the specialist leaves, ownership can disappear before the work does

That is the false economy: the task list shrinks on paper while the residual becomes an unpriced liability and deskilling erodes the capacity to catch it.

COLLAPSE TASKS, NOT OWNERSHIP

Split the work before changing the seat

  1. 01Move repeatable tasks only with a relevant, tested check
  2. 02Name the person or on-call specialist for costly exceptions
  3. 03Schedule unaided drills so the owner can still diagnose failures

If no one can sign for the costly exception, the work has not vanished; the last-mile obligation has merely lost its nameplate.

A task ledger routing repeatable checked work to AI and costly exceptions to a named human owner with scheduled drills.The process separates task automation from responsibility and adds a practice cadence so rare-failure ownership remains operational.COLLAPSE TASKS · KEEP OWNERSHIPREPEATABLECOSTLYEXCEPTIONAI ROUTERELEVANTTESTED CHECKLOG RESULTSNAMED OWNERON-CALLSPECIALISTSIGNS FAILUREPRACTICE CADENCEUNAIDED DRILLPLANTED FAULT · PERIODIC REVIEWNAME + EVIDENCE + PRACTICE

WHO OWNS THE FINISH

Price the finish before celebrating the prototype

  • What remains after the happy path works?
  • What evidence proves it is production-ready?
  • Who owns failures, maintenance, and expiry?

Use the 70% problem as a prompt, not a percentage: expose the residue, name its evidence, and assign the verification owner before cutting capacity.

Read the transcript

01 · THE DEMO THAT DAZZLES

A founder describes an app in the morning. By lunch, the happy path works. The screen looks finished, the room lights up, and the old estimate suddenly feels absurd. Then production begins. A user arrives with an input nobody tried. A repair breaks another state. Authentication, accessibility, observability, recovery, and maintenance enter the room. The prototype was real. So is everything it did not exercise. The seventy percent problem names that change in terrain: fast apparent progress, followed by a finish that resists the same shortcut.

02 · THE SHAPE, NOT A MEASURE

Keep the number honest. Seventy and thirty are not an empirical partition of software work. Addy Osmani used seventy percent to describe a wall he had observed, especially when non-engineers used AI coding tools. A prototype can arrive remarkably quickly. Then debugging, architecture, security, performance, error handling, and polish demand knowledge the first pass did not supply. The ratio is illustrative. The useful claim is about shape: early progress can be rapid while the remaining work is disproportionately stubborn, uneven, and easy to underestimate.

03 · THE EAGER JUNIOR

Osmani published his field guide on December fourth, 2024. His comparison was direct. AI is like having a very eager junior developer on your team. They can write code quickly, but they need constant supervision and correction. That is an observation from practice, not a controlled result. Its value is diagnostic. If the tool makes implementation appear without making the operator better able to inspect, debug, and maintain it, the visible gain can arrive before the obligation that governs whether the software survives.

04 · THE MEASURED SPEEDUP

Now put a measured result beside the heuristic without pretending they are the same evidence. Peng, Kalliamvakou, Cihon, and Demirer randomly split ninety-five professional developers. Each was asked to build the same JavaScript HTTP server. GitHub Classroom tests scored correctness and completeness. Developers with Copilot averaged one hour and eleven minutes. The control group averaged two hours and forty-one minutes. The estimated speed gain was 55.8 percent, with a ninety-five percent confidence interval from twenty-one to eighty-nine percent. Completion was seventy-eight percent with Copilot and seventy percent without. That is a large, bounded result on one specified task. It did not measure long-run maintenance or a seventy-thirty split.

05 · TWO KINDS OF EVIDENCE

The turn is not that one source cancels the other. One experiment estimates a causal speed gain on a defined assignment. One field observation warns about work the assignment and stopwatch do not capture. The mistake is to collapse those claims into seventy percent of every job is automated. The experiment cannot establish that. The heuristic does not measure it. Held together honestly, they ask a better question: where did the observed speedup end, and what production obligation remained outside the test?

06 · WHY THE FINISH RESISTS

The finish is not one homogeneous block. First, cases are heterogeneous. Two inputs that look alike can need different handling. Second, repairs are coupled. A local fix can break a distant state, interface, or assumption. Third, verification can be expensive. Knowing that the software is correct may require production context, adversarial testing, and expert judgment. Those three properties make fluent output especially dangerous. It can look coherent while leaving the operator uncertain about the system around it. That is why the last stretch can resist the speed of the first.

07 · Advertisement · Bubble AI App Builder

Some ideas do not need another document before they become testable. With Bubble AI, you describe the app you want, and Bubble creates a working starting point: interface, data, and logic you can inspect. From there, you refine visually, connect AI models and services, and turn the first version into something real enough to use.

08 · THE FALSE ECONOMY

Now the staffing error becomes visible. A team lists the code and documents the tool can generate, declares the seat mostly automated, and removes the specialist. Routine production moves. Reliability, maintenance, and exceptional cases do not. Their ownership simply disappears from the task list. The remaining generalist inherits failures without the context or practice to diagnose them. That is not proof that the seat must remain unchanged. It is proof that automating visible tasks and transferring the production obligation are different acts. If no one accepts the second, the saving is an unpriced liability.

09 · COLLAPSE TASKS, NOT OWNERSHIP

Split the work before changing the seat. Move repeatable tasks when a relevant check has been tested on representative cases. Name the person, team, or on-call specialist who owns costly exceptions. Then protect that capacity. Schedule unaided drills, planted faults, or periodic reviews so the owner can still diagnose a failure when automation is unavailable or wrong. A role can change. A seat may even go. Responsibility cannot be deleted by moving routine output to a machine. It needs a name, evidence, and enough practice to remain real.

10 · WHO OWNS THE FINISH

Use the seventy percent problem as a prompt, not a percentage. Ask what remains after the happy path works. Ask what evidence proves the artifact is production-ready. Ask who owns failures, maintenance, and the date that evidence expires. If those fields are blank, the prototype has not removed the work. It has hidden the finish behind a persuasive first pass. Price that finish before celebrating the demo, and assign it before cutting the capacity that used to carry it.

01 / 10 · THE DEMO THAT DAZZLES0:00 / 6:52