There’s No Escalator to AGI
The more powerful these systems get, the more we see their limits. Not their minds.
Here's my I don't want to work on the manuscript anymore and it should be Friday but it's not thought:
I think what's most encouraging about today’s AI discourse—thanks to all the pushback from the New Critics—is that we’re finally starting to blame human factors when we don’t like tech outcomes. You don’t see Nick Bostrom thundering about the imminent arrival of “superintelligence” anymore.
When the tech is weak, we can imagine it's just the beginning of something great—superintelligence, AGI, etc. But when it gets stronger and more convincing, its limits become more obvious, not less. The more powerful the illusion, the more the seams show. We raise hell, not about the machines, but about the humans who built them.
You get people like Emily Bender (the fire ant of the North) calling out Big Tech for stealing our agency, pushing these mind-like tools onto the public, and pretending we’re on the express elevator to AGI. Thankfully, we’re finally getting a kind of crabby humanism, sneaking in through the side door: complaints about power, politics, and goofballs like Sam Altman.
At last, this feels human again. This is a game played by flawed people, with flawed motives—and it has nothing to do with magical escalators to AGI or ASI. That was never real. What’s real is people pulling the wool over each other’s eyes, like they always have.
There’s no AGI. We have no idea how to build it. If it turns out to be just big-data prediction at scale, I’ll eat my hat. Data isn’t knowledge.
Erik J. Larson
Wise words. No need to buy a paper hat, just in case.
After T.S. Eliot:
Where is the wisdom, lost in knowledge?
Where is the knowledge, lost in information?
Where is the information, lost in data?
I like the focus on the flawed people behind this. I'm reading two Sam Altman bios (or bio adjacent) and he is an intriguing character to say the least. But definitely a character and clearly a millennial. He is Steve Jobs in sheep's clothing. The thing is, I do think they believe it. The rest of us right now are along for the ride and the media laps it up and it's just good enough to cause significant disruption in the present. A perfect storm of challenging coverage given so many of the variables.