Amazon Hates the Human Loop. They Might Be Right.

The hot take this weekend comes from Eric Brandwine, Amazon Security’s distinguished engineer and VP. He told The Register that “human-in-the-loop” AI governance isn’t the gold standard everyone thinks it is. Humans are “not terribly consistent.” They normalize deviance. They get complacent. They rubber-stamp.

And he’s got a point. A genuinely uncomfortable one.

The ER Problem

Brandwine tells a story about emergency rooms. Every machine beeps. First day on the job, you jump at every alarm. Day thirty, most alarms are false positives, so you stop responding. Eventually a real one goes off and nobody reacts. This is called the normalization of deviance — a concept he presented at re:Invent in 2017.

Now map that onto AI governance. You put a human between every agent action and the execution. First fifty approvals, they’re engaged. First five hundred, they’re clicking through. By five thousand, they’re not even reading the prompt. The human becomes a liability dressed up as a control.

The Cynic’s Objection

Here’s what I’d say if I wanted to poke holes in this: Amazon is a cloud provider selling AI services. Of course they want to remove humans from the loop — it’s cheaper, faster, and lets them scale without hiring armies of reviewers. Google, Microsoft, IBM are all saying the same thing this week. That’s convenient.

The counter: none of those companies are wrong about the psychology. The normalization of deviance is real — it’s been observed in healthcare, firefighting, and military aviation. If airline pilots and ER doctors fall prey to it, your junior sysadmin approving API calls at 3 PM on a Tuesday doesn’t stand a chance.

Convenient doesn’t mean false.

Where the Argument Breaks Down

But there’s a distinction Amazon conveniently glosses over: high-volume vs. high-stakes loops.

The ER alarm problem happens because there are hundreds of alerts per shift. When you ask a human to review every single AI agent action across thousands of daily operations, yeah — they’ll check out. That’s a bad system design.

But some decisions are rare and consequential enough that a human review is absolutely appropriate. Approving a drug trial. Authorizing a financial transfer over a threshold. Deciding whether to deploy code to production. These aren’t high-velocity loops. They’re gated checkpoints where the human has time to think.

The mistake is treating all human-in-the-loop as one thing. It’s not. The question isn’t “should humans be in the loop?” — it’s “at what resolution and velocity should they operate?”

What This Actually Means

Amazon’s real insight isn’t that humans are useless. It’s that putting a human in the loop is not a substitute for building trustworthy systems. You can’t ship an agent that hallucinates 5% of the time, put a human behind it, and call it safe. That’s not governance — that’s blame-shifting.

The honest path is boring: build systems that fail less, audit trails that capture everything, and escalation paths that trigger human review only when the stakes justify the cognitive cost. Human-in-the-loop as a tool, not a theology.

Brandwine’s wrong that humans aren’t the gold standard. The right design uses humans where they add value and machines where they don’t. But his warning stands: if your safety strategy relies on a human never getting bored, you don’t have a safety strategy.

You have a prayer.

Sources: The Register — Why Amazon hates ‘human-in-the-loop’ AI governance, Brandwine’s re:Invent 2017 talk on normalization of deviance, AHRQ primer on normalization of deviance in healthcare