The human in the noose: On the special hell of marking an AI’s homework

Here’s a (seeming) paradox: If you hang out with programmers, you probably know some senior, experienced, talented coders who are delighted with AI and can’t stop talking about how they’re producing the best code of their lives.

At the same time, you probably also know some coders who are equally senior and equally talented who are demoralized and even terrified about how bad the AI code they’re shipping is: “Man, I work in avionics, and my advice to you is just don’t ever fly on an airplane again. We’re producing history’s gnarliest tech debt at a scale never seen.”

How to resolve this paradox? What to make of two groups of people – both equally reliable narrators of their own experience – who are using the same tools to do the same job and having totally opposite experiences?

The answer is to be found in the rich history of labor and automation conflicts, which stretches back to the Industrial Revolution and beyond. This history tells us that when automation is driven by labor, it’s deployed to improve quality, as workers find ways to use new tools to make better things. By contrast, when automation is driven by capital, it is deployed to increase throughput; that is, to make more things (which are often inferior).

But assets don’t sweat themselves. Automation is always paired with human labor, a person who uses the new tool to produce goods and services that are ultimately offered for sale. Automation is always a system that joins humans and machines, and in that system, the human is almost always the bottleneck. After all, machines usually beat us on stamina, strength and dexterity. To sweat an asset, you have to really sweat the humans around it.

This motif emerges whenever automation and labor come into conflict. Think of Charlie Chaplin caught in the machine’s gears in Modern Times, or Lucy and Ethel trying to stuff chocolate bon-bons into boxes at superhuman speed. The humans are the bottleneck, so the machines are set to run at the absolute limit of human speed and endurance, and when the human misses a step, the insensate, remorseless machine can’t slow down and let them catch up.

Of centaurs and reverse centaurs

This is where “centaurs” and “reverse centaurs” come in. In labor/automation theory, a “centaur” is a human assisted by a machine. The metaphor here is to the centaurs of Greek mythology, in which a reasoning human head rules over a horse’s strong and inexhaustible body. A machine-human centaur sees human intellect guiding a machine. You’re a centaur when you ride a bicycle (you even look like one!). You’re a centaur when you use a spell-checker, too. Being a centaur is great.

Maybe you’ve already anticipated the circumstances of a “reverse centaur”: when a machine directs the actions of a human. The human isn’t there to provide judgment or guidance; rather, the human is conscripted to serve as a peripheral to a machine, to do the tasks it can’t do for itself.

Here, then, is the answer to the paradox of the AI-assisted coders. The first group, the ones who are excited about the code they’re shipping using AI? They’re centaurs. They are deciding how and when to use AI. That’s labor-driven automation, and while workers aren’t infallible, skilled workers who know how to do their jobs are the best people to decide when and how to use a new tool.

The second group, the ones who are terrified about the tech debt they’re generating with AI? They’re not AI-assisted coders at all – they’re the human assistants to an AI. They’re working in shops where most of their colleagues have been fired, where they live every day in terror of losing their livelihoods, and where their jobs now consist of marking the AI’s homework at superhuman speed, for superhuman stretches. They’re called “humans in the loop,” but what they really are is accountability sinks and moral crumple-zones, the designated sacrificial lambs who will be offered up when the bad code that the AI slips past them wrecks something or kills someone.

The good news is, this distinction tells us how AI tools – a grab-bag of applied statistical inference tools that would be called “plugins” in the absence of the accompanying financial bubble – can be used to improve workers’ lives and the quality of their work. Just let workers decide how to use those tools. Create centaurs. In this regard, AI is a normal technology: one that can be deployed in ways that make things better, or worse. Being “anti-AI” (in the sense of being opposed to statistical inference techniques) is weird.

AI is a pathological economic phenomenon

But here’s the bad news. AI is an abnormal, pathological economic phenomenon. The sector has squandered trillions to develop this technology, despite a few mere tens of billions in revenue. It has lousy unit economics: Every new customer the AI industry attracts loses it money, and every time that customer uses AI, the industry loses (a lot!) more money, and every generation of AI technology loses more money than the previous generation.

Those coders who love their AI assistants? The AI companies are selling them $100 bills for a dollar apiece. When the industry tried to clean up its balance sheets (as a run-up to their IPOs) and tried charging $5 for their $100 bills, its customers loudly proclaimed that those C-notes were not worth a fin.

Anything that can’t go on forever eventually stops. The AI industry has created history’s most efficient money incinerators, and investors have been providing the fuel. They can try to shift that duty to us (say, by forcing their AI companies into our retirement accounts), but we don’t have enough to keep those fires going indefinitely.

The only way the AI companies can survive after they run out of speculators and suckers is to spend less on their services than they charge. Making AI cheaper is a mug’s game, since every time a better new AI model emerges, everyone switches to it, which means that staying in business means committing to endless rounds of new, more expensive development.

Reverse centaurs: Used up by the machine

The alternative is increasing revenues. The only way to do that is to fire workers and replace them with chatbots. That’s always been AI’s value proposition (for investors): We will replace trillions of dollars’ worth of workers with chatbots, and those workers’ former employers will split the savings with us.

Which is to say: The only way AI can succeed is to create as many reverse centaurs as possible. AI that helps workers do better things won’t turn trillions of dollars in losses into trillions of dollars in profits. To do that, you need to fire most workers and put the jobspocalpyse’s survivors in harness as mounts for their chatbot overlords.

A reverse centaur isn’t just used by a machine – we are used up by it. Amazon has built the most automated warehouses in human history, and those warehouses have the highest rate of injury in the sector. That’s not a coincidence, it’s a consequence. If you’re going to make a nine-figure bet on automation, you’re going to run those machines as fast as you can, and that cadence is limited by the speed and endurance of workers. Make someone work at the limit of their speed and endurance for hour after hour, and they will make a mistake – and end up impaled on a forklift.

If the AI companies have a future (a very, very, very big if), it lies in forcing us to mark their AI’s homework, and absorb the blame for its mistakes.