Addy Osmani argues that automated coding systems still need human judgment at the points where product intent, architecture and ownership are decided. In a post on X, the Chrome engineering leader outlined a model for software factories in which agents generate code inside an environment of deterministic checks, while humans focus on decisions that automated systems cannot reliably settle.
Judgment belongs upstream
Osmani’s accompanying framework is headed “Where Human Judgment Goes,” with the subtitle “Generation is abundant. Judgment is scarce. So place it on purpose.”
Its first stage, “01 · Upstream,” places humans “before any code exists.” The listed decisions include:
- Product intent: the problem, users and non-goals
- System shape: boundaries, invariants and coupling
- The quality bar: what “good enough” means
The framework warns that “cheap implementation makes a mistake here very expensive.” That places responsibility for defining the system’s direction before an agent begins producing implementation details.
Osmani also cautions that human review should not be applied uniformly. Code review remains useful in a “lights-on factory,” he states, but it should concentrate on areas where automated back-pressure breaks down or where maintainability trade-offs require judgment.
Checks should produce evidence
The second stage, “02 · The Loop,” describes an agent operating inside “deterministic back-pressure.” The agent generates code, while the surrounding environment pushes back through checks such as:
- Types, tests, mutation testing and property checks
- Complexity, security and performance budgets
- Architecture rules and dependency policy
Osmani’s post lists security scanners, automated tests and linting for architecture rules among the checks that can run early and continuously. He also warns that the number of checks does not equal quality. Teams need to determine which checks provide a useful signal-to-noise ratio, then deliberately tighten or relax constraints.
A reply from a user highlighted a limitation for agent workflows that leave no diff behind. If the work exists only as a session on another server, the reviewer may receive little more than a transcript of what the agent clicked. Osmani responded that transcripts should not be treated as the primary evidence. Instead, agents should emit a record of their declared intent, the preconditions checked before acting and the assertions verified afterward.
The framework makes the same distinction more directly: “Evidence, not output. It all fires before a human ever looks.”
That approach also treats automated checks as fallible signals rather than final answers. One reply about mutation testing noted that it can expose tests that pass without verifying meaningful behavior. Another commenter described reviewing 168 statements identified by a tool: three revealed genuine test gaps, while 40 were attributed to a tool defect. The result was described as a source of leads, not definitive conclusions.
Review should be concentrated
The third stage, “03 · The Exits,” returns to humans “where it still changes things.” The highlighted review area covers changes with a large blast radius and weak checks, as well as subjective trade-offs and new abstractions.
A separate workflow illustration groups multiple GitHub pull requests into larger “Change” units. Those groups carry risk scores of 18, 54 and 86, labeled low, medium and high. Zohar Einy described a similar approach in a reply, with an agent scoring the risk of each grouped change and reviewers focusing on the riskiest ones.
The process ends with “ship,” alongside the qualification that someone owns what went out. Its closing statement is blunt: “‘The agent wrote it’ is not an engineering explanation.”
Tools are emerging around the handoff
Several replies connected Osmani’s model to tools intended to keep agents running with limited supervision while preserving points for human intervention.
Other replies pointed to verifiable coding-agent runtimes and tools for managing reusable context. The common thread was not removing humans entirely, but reducing routine supervision while preserving human involvement for design choices, quality assurance and understanding consequential changes.
Osmani’s framework closes with a simple principle: “judgment is the scarce resource.”
Source: Addy Osmani on X
