There’s No Escalator to AGI

The more powerful these systems get, the more we see their limits. Not their minds.

Here's my I don't want to work on the manuscript anymore and it should be Friday but it's not thought:

I think what's most encouraging about today’s AI discourse—thanks to all the pushback from the New Critics—is that we’re finally starting to blame human factors when we don’t like tech outcomes. You don’t see Nick Bostrom thundering about the imminent arrival of “superintelligence” anymore.

When the tech is weak, we can imagine it's just the beginning of something great—superintelligence, AGI, etc. But when it gets stronger and more convincing, its limits become more obvious, not less. The more powerful the illusion, the more the seams show. We raise hell, not about the machines, but about the humans who built them.

You get people like Emily Bender (the fire ant of the North) calling out Big Tech for stealing our agency, pushing these mind-like tools onto the public, and pretending we’re on the express elevator to AGI. Thankfully, we’re finally getting a kind of crabby humanism, sneaking in through the side door: complaints about power, politics, and goofballs like Sam Altman.

At last, this feels human again. This is a game played by flawed people, with flawed motives—and it has nothing to do with magical escalators to AGI or ASI. That was never real. What’s real is people pulling the wool over each other’s eyes, like they always have.

There’s no AGI. We have no idea how to build it. If it turns out to be just big-data prediction at scale, I’ll eat my hat. Data isn’t knowledge.

Erik J. Larson