DeepMind Takes on Protein Folding
Done with games, the famous AI company is now taking on biology. But is it a breakthrough, or more hype?
Hi everyone,
Thank you for your continued support! It has been truly wonderful.
I’ve been thinking about the problem of protein folding and so decided to tackle it here. I hope it’s useful to your understanding, if not of protein folding, of the current efforts to use deep neural networks to “crack the code.” I hope you enjoy.
Go, DeepMind, Go
If you aren’t familiar with the company DeepMind, you haven’t been living on planet Earth. Now owned by Google, it’s most visible founder, Demis Hassabis, was named by Time as one of the top 100 most influential people in AI. He made his name playing games.
The ancient board game Go, in particular. In March 2016 DeepMind’s AlphaGo used deep neural networks and reinforcement learning to best Lee Sedol, one of the top Go players in the world. The win was heralded as a breakthrough in AI and a sign of the coming dominance of AI over human smarts. AI diletantes like Sam Harris trumpeted the victory as clear evidence of existential threat to humans, who would soon be outmatched in all areas of intelligence by systems like AlphaGo.
The ballyhoo has since vanished, as researchers discovered flaws in systems like AlphaGo and its successors that enable amateur human players to beat them. Hmm. As is typical of AI hype, limitations were quietly reported and hastily forgotten.
The New Game
No surprise, Hassabis and Google DeepMind have pivoted, and have a new game: predicting the structure of proteins. It’s an ongoing bugbear in biology—a true puzzle. Scientists know that the three-dimensional shape of proteins determines how and what they do in our bodies, but determining their precise shape—known as the protein folding problem—is time-consuming, prone to error, and still unmastered and largely uncharted. We still don’t know the shape of scores of proteins. They’re mind bogglingly complicated beasts, with amino acids folding into convoluted knots, and the entire structure bunching up in seemingly endless ways. Hassabis hopes to unravel these microscopic knots and contortions, and early reports of successes matching protein shapes determined by experimental lab work with results from the new “AlphaGo,” AlphaFold2, have led quickly to excitement and, predictably, bold pronouncements about revolutionizing yet another field with big data AI.
To be sure, understanding proteins better would be a boon for curing many diseases and perhaps spur new vaccines and drug therapies. Hassabis, for his part, calls Google DeepMind’s foray into the protein folding problem “the most complex thing we’ve ever done.” It is a noble endeavor, and given the embarrassing denouement to playing complicated entertainment games like Go, it’s a clever pivot for a company with a reputation for pushing the state of the art in AI.
Here’s the good news. It works. Sometimes. Sort of. AlphaFold2 can sometimes predict the structure of proteins that would take months of work in the lab, down to the atom in some cases. Relieving some of the labor intensive lab work is genuine cause for celebration. And the potential to help humankind is obvious. The system is now getting used for research on Covid-19 as well as perennial evils like cancer and drug resistant microbes. Add to this, DeepMind has set up a database containing the predictions of an initial 800,000 proteins, and expects to add millions more in the next year. The work has been spun off into a new company called Isomorphic Labs and encouragingly, it plans to work closely with actual biologists and other scientists and technicians who bring a wealth of knowledge and experience to the table.
Prediction is, well, prediction
Here’s the bad news. First, the obvious problem: they’re predictions. The latest ebullient rhetoric reminds me of the billion euro funded Human Brain Project and its promoting of “predictive neuroscience” techniques. Researchers at the Swiss-based project touted the prediction by AI systems of eight out of ten nerve cell connections, part of the project of mapping the so-called “connectome” of the hundred billion neurons in the human brain. Sounds great, until you realize that fully two out of ten connections will be wrong. Whatever results from that won’t be a human brain in silica. Try a complicated and non-functioning computer-generated mess.
Here’s another Very Big Problem. Proteins move. They change their shape in context, over time, based on a range of factors (many too complicated to delve into here). Why is this a Very Big Problem? Because machine learning—any machine learning, including deep neural networks—is stuck in time. You provide it with prior examples, called “training examples,” it does a bunch of (typically) matrix multiplication to get an optimal assortment of weights to all the millions or billions or trillions of variables (called “parameters”), and it kicks out a big, heavy, stuck to the prior data, model. This model is then used to predict new, unseen examples—like the shape of a protein. If the protein is changing depending on context (and it is) like domain-swapping, the model is not going to capture these changes unless it’s seen them before. In other words, the really hard part of the protein folding problem is divining their holistic activity in the body, subject to all manner of forces and bonds, again depending on complicated conditions, and a static predictor from “AI” won’t be capable of that. Researchers sometimes admit as much, but the typical cheerleading in recent popular press about AI revolutionizing the field forces you to read between the lines to get to the truth. (Should universities offer a course on detecting AI hype? Maybe so.) Much of the work to date has simply verified that the AlphaFold2 system predicts the same structure as proteins that have already been solved. What we want, of course, is some way to understand the protein over time, to determine it’s multifarious properties as it interacts in our bodies. What we want is a superintelligence that can beat the performance of laborious lab work, not just reproduce or mimic it. Where’s the revolution?
It’s still interesting research—better than mucking around with board games and calling it an existential threat to humanity—but I’m worried it’s another example of very smart researchers and scientists getting tethered to supercomputers instead of pushing the frontier of fundamental knowledge and research. Sure, systems like AlphaFold2 can help by at times speeding up the time-consuming process inherent in the protein folding problem, and this has already occurred in some cases, but the ongoing, almost blind obsession with “AI” may in the end slow the field in certain respects. Modeling complicated biological structures using sometimes-right predictive computation is not a revolution. It’s a technique. It’s a tool in a toolbox.
What’s the Track Record of Modeling?
Modeling and prediction using computers in scientific fields is rarely questioned. More scientific papers are published year after year, and patent applications have soared since the 20th century. But the papers say less and less, and the patents (this has been verified) impact the world less and less. In nuclear fusion research—I podcasted about this recently—an early “AI” system called LASNEX though off by, as the New York Times put it, “a factor of 10,000” predicting instabilities inherent in laser fusion still consumed gobs of scientific man hours and thought and attention. It was nearly always wrong (trying to determine the number of lasers required to zap some fusible material like deuterium), but moth-to-the-flame otherwise brilliant plasma and fusion scientists still fretted and fussed over its results, made changes to the code, and otherwise directed their research away from discovery (and understanding) to modeling. There are better predictive systems for fusion research now, but I suspect the same “modeling trap” will limit the usefulness of those systems as well.
We’re addicted to computer modeling, calling it “AI.” The models might provide some benefit in a larger research context, but we treat them as ends in themselves. Like human expert prediction, the models are always limited and frequently wrong, but we’re in denial. Few can admit we have a modeling problem.
Now we’re doing protein folding. Scientists can and no doubt will use AlphaFold2 and related systems to speed up experiments and gain insights into some of the problems they face. But the system is already oversold as a revolution and panacea in an area of science we don’t completely understand. It’s not. It’s a tool. Let’s hope that future AI systems can tackle the hard problems inherent in protein folding. If not, it’s more modeling, and less actual science. That’s an old trick, and one of these days we should see through it.
Erik J. Larson
Hi Eric,
Thanks! Good to hear from you. Okaaaay:
I'm not sure I can help much beyond what I wrote, but here's a stab: there was something like 800 years of meticulously collected astronomical data, observational data, available to mathematicians and astronomers by the 16th century, all used to support the Ptolemaic model--the Earth at the center, geocentric--until Copernicus came along and proposed a heliocentric model (we're still using it today :)). If Copernicus, instead of thinking about an entirely new idea--that the Earth might revolve around the Sun rather than vice versa--were to take all the Ptolemaic "big data" and use it to better predict the motions of the planets, solve retrograde motion problems with more accurate modeling of epicycles and equants, and so on, would all that modeling and predicting have helped? Nope. We'd likely still think the Earth was at the center of the cosmos, and the correct model would have to await some future Copernicus. Thankfully, Copernicus didn't have a supercomputer, so we got to the truth. Copernicus essentially ignored much of the data and reconceptualized the problem first. Something like this is at least in the vicinity of what might help think through the issues. AI folks (like me) are really bad at this, too. When AI enthusiasts talk about deep neural networks, say, they talk as if they're extending human capabilities point blank, and there's nothing lost. But there often is something lost--the absence of human insight, as the focus has shifted to downstream concerns. In general, I think we're--I include myself--sort of bad at "thinking about thinking." If we did that better, I suspect the distinction would be much more obvious. I hope this helped!
Two quotes from two different George’s come to mind:
“All models are wrong, but some are useful.” George E.P. Box
Why did you climb Mt. Everest? “Because it was there.” George Mallory