Credit: Copyright Julia Galdo and Cody Cloud (JUCO)/CC-BY 3.0

Last year we featured a lengthy interview with tech journalist/science fiction author Cory Doctorow about his book, Enshittification: Why Everything Suddenly Got Worse and What To Do About It. The prolific Doctorow is back with a provocative new book that serves as a follow-up of sorts, focusing on AI and related issues: The Reverse Centaur’s Guide to Life After AI.

Doctorow doesn’t actually enjoy talking about AI, but he’s constantly being asked to comment on it. “I made the tactical error of being sick of talking about AI,” Doctorow told Ars. “So I wrote a book about why I think it’s a dumb thing to keep asking people to talk about, and now I have to talk about it.” Reverse Centaur is Doctorow’s attempt to “sort out the bullshit from the material reality.”

In automation theory, per Doctorow, a “centaur” describes a human augmented with a technology, like machine learning, or even just driving a car, or using autocomplete. A reverse centaur “is a machine head on a human body, a person who is serving as a squishy meat appendage for an uncaring machine,” Doctorow said in a speech last December. He gave the example of an Amazon delivery driver, surrounded by AI cameras monitoring their driving, who essentially serves as a peripheral to the delivery van.

Being a centaur is generally viewed as a positive thing; few people relish being a reverse centaur. And yet the AI industry seems intent on using those tools to create more reverse centaurs. It’s one thing to incorporate AI tools in the medical field to help radiologists process X-ray images and spot possible tumors they might otherwise have missed. It’s quite another to fire nine out of 10 radiologists and let AI make the diagnoses, with the remaining radiologist solely responsible for checking the AI’s work—and, ultimately, taking the blame for any errors.

Doctorow is not virulently anti-AI; he uses AI tools regularly and sees potential in many of those tools as useful plugins or cool new apps. But he is nonetheless alarmed at all the hype surrounding AI, the enormous capital expenditures, the unrealistic expectations and self-serving messaging, and the potentially catastrophic economic consequences when the AI bubble inevitably pops.

“The bubble doesn’t want cheap useful things,” Doctorow said. “It wants expensive ‘disruptive’ things: big foundational models that lose billions of dollars every year. When the AI investment mania halts, most of the models are going to disappear, because it just won’t be economical to keep the data centers running. The collapse of the AI bubble is going to be ugly. Seven AI companies currently account for more than a third of the stock market, and they endlessly pass around the same $100 billion IOU. AI is the asbestos in the walls of our technological society, stuffed with wild abandon by a finance sector and tech monopolists run amok. We will be excavating it for a generation or more.”

Naturally Doctorow has some ideas about how to push back against the prevailing narrative of AI’s inevitability. Ars caught up with him to learn more.

Ars Technica: We touched briefly on AI last year when we chatted about your prior book. Reverse Centaur seems like a natural outgrowth of that.

Cory Doctorow: Enshittification is primarily a thesis about how firms in the absence of constraint get tilted to the bad, but it’s also a thesis about how the constraint of competition, when it falls away, produces all kinds of perverse outcomes. One of those perverse outcomes is that firms that have saturated their markets can no longer grow, and they have to find other markets. There’s a ticking bomb when you saturate your market because it’s only a matter of time until investors start to worry that you’re not a growth stock, you’re a mature stock. Mature stocks trade at a small fraction of the multiple that growth stocks do.

There’s an enormous amount of liquidity in growth stocks, which means that you can use growth stocks to grow. You can buy other companies with shares, and shares are an endogenous substance that you make on the premises by typing zeros into a spreadsheet. Firms with growth stocks can grow by typing zeros, whereas firms that are mature, they have to use money if they want to grow, and you’re not allowed to make money on the premises. If you do, the Treasury Department shows up and takes you away in handcuffs. So you can see why firms would be very anxious to maintain the perception that they have room for growth even after they have 90 percent market shares.

“The capital markets have the object permanence of a toddler, and they would lose a game of peekaboo if they were drafted to play in the league.”

That’s why those firms started promoting stories about how they were going to conquer imaginary markets. Imaginary markets have no agreed-upon valuation because you just made them up. Unless you can turn an imaginary market into a real market pretty quickly, you need to come up with another imaginary market and announce that this is the new imaginary market you’re going to conquer. It’s easier than you’d think because the capital markets have the object permanence of a toddler, and they would lose a game of peekaboo if they were drafted to play in the league. So you can say, “Oh, actually, it’s not metaverse. It’s crypto. It’s not crypto. It’s Web3. It’s not Web3. It’s something else.” And the markets will forgive you, provided you do it quickly enough.

But something different happened with AI. It is much, much bigger in terms of capitalization than anything we’ve ever seen—not just bigger than other tech bubbles, bigger than other bubbles. When I wrote the book, capital expenditure (CapEx) globally was $700 billion, now it’s $1.4 trillion. Meta wasted $60 billion on the metaverse. They spent $150 billion in the last three years on AI, and they say they’re going to spend another $150 billion this year.

So this is a much bigger bet, and it raises the question: If the material basis for this is creating a narrative so that you can continue to grow by dint of having a highly liquid growth stock, what’s the ideological basis? Why are people willing to make such a bigger bet? Some of it is that there’s more “there” there with AI. It’s real computer science. It was remarkable 10 years ago, when a couple of computer scientists and their grad students took some existing techniques, applied them in a new way, and got a very surprising result that turned out to not only produce dividends the first time around, but to have somewhat linear returns on investment, which is not usually the case.

There was a lot of low-hanging fruit in AI, although it’s tapering off now because, as they say in finance, anything that can’t go on forever has to stop. So we’re losing the end of that growth period in terms of returns to scale.

Ars Technica: Why do you think AI is so appealing to political and business leaders in particular? 

Cory Doctorow: It’s not just that it makes for a good demo. AI really appeals to a fantasy that I think all of us have to some extent, but that powerful people really have, of a world without people in it—because hell really is other people. You can’t get stuff done without other people helping you. You can’t have romance without a romantic partner. You can’t have social media without people to socialize with. You can’t play a board game, or do a startup, or build a bridge, or build a house, or do politics without other people. And other people stubbornly refuse to organize everything they do to make you happy.

Particularly if you’re rich and powerful, it’s very galling. So AI is very attractive. One of the reasons DOGE fired so many government workers was because it played into the fantasy that you can have a government without government employees. In the corporate sphere, it’s the fantasy of a business without workers, because every corporate leader is haunted by the secret fear that if they don’t show up for work, everything goes on just fine. But if the workers don’t show up, everything shuts down. Maybe they’re not really driving the car, maybe they’re strapped in the backseat with a toy steering wheel.

If that’s the case, AI will let them wire the toy steering wheel directly into the drivetrain. So you can have an amazing idea as a corporate visionary, and you don’t have to have any ego-shattering confrontations with people who know how to do things, who tell you you’re actually an idiot. You just type some stuff to the chatbot, and it shits out your product. If you combine those two things—the material necessity to have a growth narrative and the ideological attractiveness of a world without people—you get $1.4 trillion in CapEx for a sector that is turning over $50 billion a year and has to replace all of its assets every 24 to 30 months.

Ars Technica: You raised an interesting point recently on your blog: workers actually wanted earlier technological breakthroughs and often had to fight to get them into the workplace. With AI, people are more likely to feel that the technology is being shoved down our throats; some workers are even required to use it.

Cory Doctorow: I think that’s entirely right. One of the things that I’ve been attending to a lot lately is the difference between the bubbles that we had before and the bubble that we’re having now. People will say, “Oh, Amazon wasn’t profitable, and it became profitable. And the web wasn’t profitable, and it became profitable. The web was a bubble.” Of course the web was a bubble. You don’t get pets.com and all those Super Bowl ads without a bubble. But it is a very obvious error of logic to say, “Once, there was a thing that lost money and then it made money, therefore, if you are losing money, someday you’ll make money.”

“AI is the money-losingest thing our species has ever done. We have never lost as much money as we’ve lost on AI.”

The thing that made the web profitable was not that it was unprofitable, it was things like good unit economics, where every time someone started using the web, the web got less unprofitable. Every time a web user used the web again, the total profits generated went up. Every generation of web technology made the web more profitable. That’s the opposite of AI. Every AI customer loses money for the company, every use of AI by that customer loses money for the company, and every generation of AI loses more money than the last one. AI is the money-losingest thing our species has ever done. We have never lost as much money as we’ve lost on AI.

Another giant material difference is the social reception. If you look back to the business press of the aughts and the late ’90s, it’s full of hand-wringing editorials about how bosses will cope with workers who are smuggling in the web. You look at those same press outlets today, and it’s full of people saying, “What are we going to do about the fact that no one in the workplace wants to use AI?”—along with ads for firms that will spy on your workers for you so that you can punish the workers who refuse to use AI.

Ars Technica: AI nonetheless does have thoughtful, sensible defenders.

Cory Doctorow: One of the paradoxes that I try to explore in this book is the workers who are not fools, who are historic good, reliable narrators of their own experience, and who tell you that AI is making their lives better. The foundational idea of science fiction is that what the gadget does is less important than who it does it for and who it does it to. I call those people centaurs. They are workers who are assisted by technology and who decide how that technology is going to assist them. Whereas the workers who hate it are workers who are being asked to produce more with AI at the expense of quality, at a higher speed, at the expense of their own wellbeing, and who understand that they’re being recruited to be what Dan Davies calls accountability sinks—to take the blame when the AI screws up their job.

Once you put it that way, it’s very easy to see why some workers would say, “Oh yeah, I found a thing that AI is good for and I use it, and that’s fine. I’m even excited about it.” And why other workers would be like, “This is making me miserable.” It’s the difference between the words on the Greek temple, “Know thyself,” and your boss shining 16 cameras in your face and going, “I know you better than you do. And by the way, I think you could work an extra hour a day without breaking a sweat.”

Ars Technica: You make a point of emphasizing that you are not fundamentally anti-AI, despite sharply criticizing the industry. 

Cory Doctorow: I have many comrades who describe themselves as anti-AI, and I’ve had some very spirited, productive but heated debates with those people because I don’t think AI is exceptional. That means that I don’t think it’s exceptionally evil. The argument that it’s the fruit of the poisonous tree, that it was made by bad people in bad ways, so you shouldn’t use it—I think it’s very foolish. That is not the merit on which we judge technology.

You can talk about whether giving money to these companies is bad. I think it is. You can talk about whether the environmental impact of using foundation models is unsustainable and unsupportable. I think, by and large, it is. But that is not to say that statistical inference using convoluted deep neural networks is bad or—and this is where I get into many arguments—that scraping the web to train a convoluted neural network is bad. I think it’s fine. Scraping is good actually.

I think it’s very dangerous to say, “The way that we’re going to fix the problems we have with AI is to make it illegal to make a record of what’s on the Internet.” I think that’s catastrophic. That’s how we never again will know what was on CBS News before it turned into Chud News. Everything Nate Silver ever published on his website was just zeroed out by Disney. You can only see it at the Internet Archive because we scrape. It’s just bonkers to say, “It is theft to make transient copies of works, to analyze those transient copies, to publish the results of your analysis.”

Those are all socially beneficial activities, and we will all lose if we prohibit it—not least because the firms that creative workers are worried about it, the big media companies, are extremely capable of entering into arrangements with the Big Tech companies to license their corpuses to them in order to try and put us all out of a job. If we get the right to decide who can train an AI with our work, our bosses are just going to modify our contracts to say, “Great, you now must license that right to me. And it’s non-negotiable.” Failure to learn from that lesson is not tragedy. It is farce.

Rather than ask for a new copyright law, we could make a new labor law, because the only people who’ve ever beaten AI are the Hollywood screenwriters and actors. And the reason they were able to beat them is because uniquely, among workers in America, they are exempt from the Taft-Hartley Act‘s prohibition on what’s called sectoral bargaining, which is when all the workers in a sector bargain with all the employers in a sector. Now, there are so few workers in America who aren’t media workers, who care about copyright, that it rounds to zero, but every single worker in America would benefit from extending sectoral bargaining across the board.

Ars Technica: It could be catastrophic, economically speaking, when the AI bubble finally bursts. But you point out that there might very well be something useful left over when that happens. 

Cory Doctorow: I advise to go long on laser tag arenas because you can definitely turn a data center into one of those. There’s not much else you can do with them, unfortunately. A bubble is a way for insiders to pump, and then dump, some mania to the normy investors, to people who’ve been flushed into the capital markets because they’ve been denied a defined benefits pension and who are only really offered market-based pensions. That means you have to be the sucker at the table. You have to put your money into the market if you don’t want to die homeless and starving after you retire.

The dot-com bubble was very bad. It separated a lot of pension funds and ordinary investors from their money, but it left behind something very useful. In the early years of the aughts, there was, amidst the carnage, quite a liberating vibe where all the stupid money went home and you could get servers for pennies on the dollar. Everybody knew how to code. A generation of humanities undergraduates were induced to drop out of university and learn Python, Pearl, and HTML, and a lot of them were really creative. Your rent dropped by two-thirds in San Francisco. I bought six $1,200 Steelcase Leap chairs, still in the plastic wrap, from a failed dot-com guy on a sidewalk on 19th Street in the Mission District for $25-$50 each and used them as a dining room set for the next 10 years.

So there was a very productive residue that was left behind by the dot-com bubble. It gave rise to a more robust form of the web, Web 2.0, full of things that were more useful, more interesting, more thought-through, more creative, more innovative than the stuff that the bubble threw off in Web 1.0. There are other examples of bubbles that are less likely to throw off that residue. Around that time, we also had Enron. Enron produced nothing, although I do have a pad of Enron stationery that a friend in Austin bought at the bankruptcy auction and sent to me.

“We can distinguish between bubbles with productive residues and unproductive bubbles while still not saying that bubbles are good. Bubbles are bad and destructive.”

So we can distinguish between bubbles with productive residues and unproductive bubbles while still not saying that bubbles are good. Bubbles are bad and destructive. When the cryptocurrency bubble bursts, all that’s going to be left are shitty monkey JPEGs and worse Austrian economics. But when AI bursts, you’re going to be able to buy GPUs for pennies on the dollar. You’re going to have your pick of applied statisticians, many of whom are very creative and have interesting ideas for things you could build with AI, but are stuck building the things their bosses want to build. There are going to be these open source models that have barely been touched. Any time someone tries to optimize them, they find so many opportunities to make them run on lower-end and commodity hardware.

DeepSeek was a spin-out of a Chinese hedge fund; the fund gave them $6 million and said, “Go play with these open source models. See what you can squeeze out of them.” When they launched, their model was so good running on commodity hardware that the market did a mass sell off, $600 billion in 24 hours—the largest 24-hour decapitalization of any firm in the history of markets. If you’ve got cheap hardware, and you’ve got applied statisticians, you’ve got these open source models and you’ve got a technology that fundamentally is interesting, and has done useful things, and will do useful things in the future—that’s a better setup than one in which we’re all running around arguing about whether the word-guessing program is going to wake up, become God, and turn us into paperclips.

Ars Technica:  You also push back a little on the “AI is coming for your job” messaging.

Cory Doctorow: I think we have to distinguish between the AI doing your job, and the AI being incapable of doing your job, but your boss is such a sucker that he fires you and replaces you with the AI anyway. There’s infinite evidence for the second one. I think that there’s very little evidence for the first one, at least so far. A lot of the stories we’ve heard, when you interrogate them, just turn out to be nonsense. There’s a chapter in the book about how many of the demos for AI have just turned out to be people in India pretending to be robots.

The most egregious example was when Amazon announced that cashiers were now out of a job because now you could just walk into [an Amazon Go store], grab stuff off the shelf, and walk out again, and the AI knows what you took. There wasn’t an AI. It was three people in India watching each customer through a network of cameras in the ceiling trying to guess what you put in your bag.

I think there’s lots of things that skilled workers will ask AI to do that will help them do their jobs. There’s lots of things that skilled workers will ask AI to do that they’ll be wrong about and that won’t help them do their jobs. And there’s probably space at the margin to replace humans with AI, at least in some cases. But the idea that we’re at a “jobspocalypse” is such a self-serving narrative. If you’re trying to convince people that the way you’re going to turn $1.4 trillion in CapEx into more than $1.4 trillion in revenue is by convincing bosses to fire workers and replace them with chatbots, you have to have a story about how the chatbot can do anyone’s job.

Here’s a wager. If you ever have the opportunity to interview Dario Amodei or Sam Altman, I want you to ask them this. Someday, you will retire. Right now, I want you to make a binding decision. Will the thing that wipes your ass and takes care of you when you are too old and frail to take care of yourself be a person or an AI? We’re just going to use whatever it is that’s around at that time, and you get to choose. I think we should ask anyone who says they know how to fix things, would they themselves go to an old folk’s home run according to the principles they’re establishing?

Ars Technica: We are now starting to see news stories about how companies that invested in AI are suddenly getting hefty bills. 

Cory Doctorow: They’re getting the bill because the AI companies are trying to get out before they’re stuck holding the bag. They want to do IPOs, and to do IPOs, they need to clean up their balance sheet. So they’re like, “I bet these [companies] are pretty price insensitive. Let’s just jack it. Let’s go from a 90 percent subsidy to a 40 percent subsidy and more than double everyone’s prices. They’ll hang in there.” And then you get the CTO of Uber saying, “I’m not sure why we put AI in the business to begin with, and I really don’t know why we’d use it if it was $20,000 a seat. So, I don’t know that we are going to use AI anymore.”

This is quite a backfire. It actually shows you how insulated these people are from any sense of how their products are received, from what people think of them, from the actual fundamentals of real businesses that have to bring in more money than they spend. It’s weird. I’m hardly a captain of industry or one of the great champions of markets, but I do understand that, by and large, firms should bring in more money than they spend if they are to be an attractive investment prospect.

Ars Technica: We hear plenty about the negative aspects of AI. What do you like about it?

Cory Doctorow: I have a couple of local models on my computer, which is just a framework laptop running Ubuntu. It doesn’t even have a GPU. I use Whisper to transcribe audio. I will sometimes want to cite something I’ve heard in a podcast and not remember where I heard it. One time, I just threw the last 30 hours of audio I’d listened to at Whisper, and it shot out verbatim logs that were good enough that when I searched the full text, I could find it. And it gave me time codes so I could check the transcript. That’s amazing.

The idea that I might someday have a computer full of audio and video files with full text indexing is great. I could even imagine conversational interfaces to that: “Where’s the photo of my daughter at her birthday party where she’s dressed like a pirate?” AI doesn’t have to be 100 percent accurate for that to be useful. It doesn’t have to be free from false positives. It can just be okay. That stuff’s running on your own computer. It’s not burning down a rainforest. It’s not consuming the last three drops of potable water left in Nevada. There is a certain kind of person who is performatively horrified by AI: “But, but, but that’s energy you wouldn’t have used.” I’m like, “You have never said that about someone who turns cell shading on while playing an MMORPG.” Everything you do with your computer burns electricity.

I’ve been using another chatbot where I paste my daily blog post in and say, “Find my typos.” It finds a lot of the errors that are normally not caught by a regular spell checker: doubled up words, punctuation marks, or words that are actual words but are misspellings for other words. When you dial up the sensitivity to the point where it actually catches all of those, it also gets a lot of false positives. That’s fine for 1,500 to 3,000 words. I never feed it a book. On a 100,000-word manuscript, it’s going to give me thousands of false positives and it just won’t be useful. I treat this like a plugin to my word processor. It’s fine. Sometimes it’s good and sometimes it’s not.

I have a friend, Patrick Ball, who is the best programmer I know, and he founded and works at an NGO called the Human Rights Data Analysis Group (HRDAG). They’re one of the most important NGOs that no one’s ever heard of. What they do is very rigorous statistical extrapolations of information that’s not in the record, about wars, civil wars, coups, oppressive states, and they’re used for truth and reconciliation, human rights tribunals, war crimes trials. They’ve worked on every high-profile war crimes trial in the century.

Patrick is using a bunch of Copilots to write software to do a lot of special-purpose stuff. For example, they work with Innocence Project of New Orleans, which has exonerated a bunch of [wrongly convicted] people. They can go through all the arrest reports from the New Orleans PD and find the ones that have linguistic correlates that match successful exonerations. Then they give those to the lawyers who would otherwise just be starting alphabetically or chronologically, sorting to the top the ones that are most like the ones that led to a successful exoneration.

It’s not like they’re asking the chatbot to write a brief for them, but this is a hugely important function, and it is getting innocent people out of prison. You don’t want innocent people in prison. That should be the least controversial thing in the world. That’s just good, and the proof is in the pudding.