Our Cultural Moment

The lesson of Generative AI is that minds and machines are different

human excellence

Hi everyone,

The philosopher Immanuel Kant once said (and I paraphrase) that he wanted to prove to the layperson that he or she was correct, but in a way that only the expert could understand. There is, I think, something vaguely similar at work today watching the spectacle of discussion about AI. Generative AI seems to be demonstrating the commonsense notion that minds and machines are different even as the technology itself has become astronomically complex and its workings largely ineffable. I call this a cultural moment—the one we’re living in today, right now—because at long last we have a different vantage point to think about AI and technological change. For decades we’ve confidently mythologized the future of machines. Now we’re seeing that future play out. At issue is the premise that increased “intelligence” means the arrival of motivations and agency. That’s not what we see.

In the past, when AI didn’t work well, we were free to imagine fantastical futures—“Dreamy” or “Fearsome” AI. Today we see widespread adoption of Generative AI that convincingly mimics cognition but very poorly convinces us that it’s sprouting an internal state, a mind. Some say it’s still just a matter of time before the machines come alive. I say we can now see the writing on the wall.

From Games to Generative Models: A Changing Narrative

Once, AI couldn’t “think” at all. It solved specific problems well—primarily games. In the 2000s and early 2010s, systems like DeepMind’s AlphaGo dominated narrow tasks such as Go and chess, which humans like to think require exceptional intelligence. The systems performed impressively, and futurists speculated wildly about an imminent leap to Artificial General Intelligence (AGI), then “superintelligence,” as Nick Bostrom warned in his 2014 book Superintelligence: Paths, Strategies, Dangers.

But this era involved a sleight of hand. Advances in narrow tasks, like beating a chess grandmaster, were equated with progress toward minds capable of agency and motivation. Yet for all the ink spilled by the media, the public, and the pundits about the coming of superintelligent AI—meaning “smarter than us,” which implies first “smart like us”—the actual state of affairs was Alexa that spit out canned responses and self-driving cars that ran into school buses and slammed on the brakes for leaves blowing across the road. Suitably, it was the playing of fixed rule games that somehow buttressed the myth of a coming superintelligent AI that would take over and exterminate us if we weren’t hyper vigilant. But games are, of course, the flimsiest of evidence for anything of the sort. Such lacunae in the philosophy of AI were roundly ignored.

Fast forward to the 2020s. The adoption of large language models (LLMs) like GPT brought a cultural shift. My reaction evolved from “Wow, this sort of works,” to “What does this mean for AI?” My first realization: we’ve developed a system that, for all practical purposes, passes the Turing Test. My next realization: Holy sht*.

I’d previously critiqued machine learning systems as lacking a type of inference we humans use to explain plausible causes from effects and generally to form hypotheses about what we observe given what we already know in a way not tied to the past—to datasets, that is. That critique still holds—LLMs are prone to bizarre blind spots and hallucinations, and though they can simulate human types of inference, their occasional mindless insistence that, say, A and not-A are both true show that in the end, we’re dealing with a machine. The Emperor has no clothes. But the scale of the data used for training, and innovative advances in the architecture used to “see” this data in sequences of tokens, like the attention model that processes entire input sequences, moved the needle on what’s possible. LLMs don’t replicate human (language-based) intelligence, but they mimic much of it convincingly. Viewed from the vantage point of an AI scientist, it’s progress. Big progress.

A Flawed but Useful Machine

Yet these models remain deeply flawed in spite of years now of playing Whack-A-Mole with quirks and errors and sometimes just zany hallucinatory output. A broad consensus has been emerging for some time now that they lack understanding but simulate understanding well enough to be useful. How useful they are to us depends on how well we can anticipate and control for their errors. We know that LLMs are bound to err, as they replace probability for truth. There’s no particularly perspicuous scientific answer to questions about the errors, however. They just happen. We ask of LLMs: why are the errors so stubborn and weird? Why does interacting with them feel like conversing with a polite, occasionally brilliant, but maddeningly stubborn “intellect”? One moment, the system summarizes the Cambrian Explosion with elegance. The next, it insists my coffee cup is simultaneously in Texas and Indiana. What kind of intelligence is this?

As I see it, the release of ChatGPT two years ago became a new cultural moment. It’s possible now to throw out the old myths—or see them with fresh eyes—by reflecting on the nature of the New AI itself. In other words, Generative AI reveals we’re not on a path to AGI, but this insight emerges because the systems work. Observing LLMs in action shows us their boundaries. Yet much of the ballyhoo online by media and everyone else continues to repeat the old script about the march to AGI. We shouldn’t be surprised that technological myths seem impossible to extirpate, I suppose. As Thomas Rid put it in his Rise of the Machines: A Cybernetic History (I quote here at length):

Technology myths have the form of a firm promise: the cyborg will be built; machines that are more intelligent than humans will be invented; the singularity is coming; cyberspace will be free. The myth is underdetermined by fact, yet it purports to be as certain and as hard as empirical evidence can get, shielded from debate and contradiction. Faith dressed as science.

Second, mythologies are remarkable not for their content, but for their form. The basis of the myth appears as fully experienced, innocent, and indisputable reality: computers are becoming ever faster; machines are ever more networked; encryption is getting stronger. But at the same time the myth makes a leap, it adds a peculiar form to the meaning. And this form is always emotional. Myths are convincing because they appeal to deeply held beliefs, to hopes, and often to fears about the future of technology and its impact on society. These beliefs are informed by visions and projections, by popular culture, by art, by fiction and science fiction, by plays, films, and stories. But the myth often harks back to fiction clandestinely, without making the cultural inheritance explicit. Science fiction novels, for instance, inspired the 1990s national-security debate. And sometimes hard-nosed experts even wrote the fiction, to spell out dystopian visions of future conflict, freed from the unbearable shackles of fact. […]

The third and most crucial feature of cybernetic myths is that they transcend the present. Mythical narratives form a path between the past and the future to keep a community’s shared experiences in living memory. […]

For [Cybernetics/AI] myths … the more stable anchor point is always in the future or, to be more precise, in a shared yet vague imagination of the future—not to close and not too distant. The golden range seems to be about twenty years forward, close enough to extrapolate from the past, yet distant enough to dare brave new ideas for the future. The outcome is equally effective. The technological myth draws a clear line from the future into the past and sees the present as a dot on this line.

There’s a sense in which it will prove impossible to rid the world of techno-myth, and I remain unconvinced that the world would be better even if we could. We seem to need futurism and myth—if only they inspire us and not freeze us in false worries and pointless squabbles. But the stubborn insistence since the field’s inception in the 1950s that AI would pass humans in “intelligence” (I use quotations here as nearly everyone who uses it in the AI discussion, as the philosophers like to say, “helps themselves to it”) holds on to the present day. It’s not an embarrassment somehow that Rid’s twenty year insight keeps getting trotted out afresh with every passing decade—we were twenty years away in the 1970s, and in the 1990s.

One of the most famous prognostications from futurist Ray Kurzweil has the cross over point to AGI at 2029. Kurzweil still holds to this view, more or less (he’s starting to caveat), but the majority of futurists still follow the idea that it’s coming soon but not too soon, so to speak. Next decade or two, for sure. We see evidence of a mindless but potentially useful technology, and (shocker!) the myth still persist. I think we need to read the tea leaves differently given our new vantage point with modern AI. I think it’s finally time to make a change in the old calcified debates, and it seems that today unlike in prior times it just might work.

So, we’re having a cultural moment. Breath deeply. We’re having a cultural moment now because I think that for perhaps the first time, we can look at the behavior of AI itself—look at its successes, that is, not its failures—and realize that we’ve been mythologizing for want of a clearer picture of what machines actually do when they get “smart.” And the answer, when compared to the stylish “decide to take over the world” narrative of existential risk drama is… not much. Our most modern AI responds to prompts, in a manner eerily similar to the old school commands of parents for children to “speak only when spoken to.” How terrifying, our new mindless servile machines are. How embarrassing to the purveyors of Fearsome AI.

What do we make of this turn of event? Here’s a thought: it’s psychologically easier to speculate about something underperforms but still improves along some metric—in this case, mostly machine learning-based tasks from games to driving cars. We can then extrapolate an exponential curve that lands us fire-breathing super smart AI in a couple of decades (it’s like the Babadook but owned by Open AI and with a 20$ a month API access). But if we’re almost there, so to speak, but we’re still seeing the same mindless hallucinations and errors from the old crappy systems in the new stuff, it’s more difficult to maintain the mythological high ground. Seems like minds and machines really are different, and this seems to hold true even as we advance along the progress curve.

LLMs gave us converstaional AI, roughly what pioneer Alan Turing used as a benchmark for success in his now famous 1950s “Turing Test” paper. But the stubborn idiocy and lack of awareness or understanding intrinsic to machines, not minds, persists. Why? The smart money is that machines and minds really are different, just as certain philosophers have said all along. A corollary point here is that we ought to revise our myths, too.

Here, we might take, say, popular historian Nuval Harari's lead and call AI a route not to natural intelligence but to some “alien” intelligence. Though in this case, I don't mean "alien" as in fantastically clever and smarter than us. I mean rather some kind of intelligence that fundamentally lacks motivation and agency and engages us and the world rather like a mindless but increasingly useful calculator with an ever-expanding array of buttons to push. In short, I see today as a kind of cultural moment because the success of AI is also the unraveling of the old futuristic ideas about it. We might still ruin the world with technology, but the egregious equating of artificial with natural intelligence has been exposed, and new philosophical positions will no doubt emerge. Or, to put it better, an very old philosophical position about the differences between minds and machines seems increasingly vindicated. We ought to take this as a starting point now to build a healthy and productive relationship with our technology.

Our cultural moment reveals what earlier mythologizers couldn’t see when AI didn’t “work.” AI isn’t a mind, nor does it possess agency like organic intelligence. But it has the potential to amplify our own intelligence and usher in a new era of human excellence—if we have the clarity to recognize the opportunities before us.

Erik J. Larson