High Schoolers Reveal Limitations of AlphaFold for Predicting Mutational Effects

Researchers at Skoltech tested AlphaFold’s ability to predict how mutations impact protein stability. They found the AI’s predictions often contradicted experimental results, challenging claims that AlphaFold has mastered predicting protein physics. The work is published in PLOS One.

AlphaFold recently achieved a breakthrough in structural bioinformatics by predicting protein structures from amino acid sequences. However, the researchers show it falls short for another key challenge – predicting mutational effects.

The study authors argue AlphaFold does not fully capture the complex physics underlying how mutations alter proteins. While transformative for structural modeling, AlphaFold cannot be considered an all-encompassing solution for protein bioinformatics.

The findings came from a bioinformatics boot camp where high school students compared AlphaFold’s predictions to lab experiments on protein mutations. The hands-on experience revealed gaps between AlphaFold’s capabilities and the advanced physics of proteins.

While a huge advance for structural modeling, the researchers emphasize AlphaFold has limitations like any technology. The study exemplifies the continued role of human intuition and experimentation in evaluating and advancing AI for science.

This collaborative project between students and scientists shows the synergy between human and artificial intelligence required for pushing the boundaries of biological discovery.

Poster of the project Playing With AlphaFold2 at the School of Molecular and Theoretical Biology held by Skoltech online in 2021. Credit: Dmitry Ivankov/Skoltech

Researchers Test Limits of AlphaFold for Predicting Mutational Impacts

AlphaFold recently achieved a major milestone in structural biology by predicting protein structures from sequences. This led some to speculate the AI had mastered protein physics and could solve remaining challenges in the field.

Researchers at Skoltech tested these claims by assessing AlphaFold’s ability to predict how mutations impact protein stability. They found the AI’s predictions often contradicted experimental results, revealing limitations.

The researchers note AlphaFold’s creators never claimed it could handle tasks beyond structure prediction. However, some enthusiasts speculated the AI was poised to revolutionize structural biology.

To evaluate, the team had AlphaFold predict the effects of single mutations on proteins with known structures. High school students helped process and analyze the data. They found AlphaFold reliably failed for this important challenge of predicting mutational impacts.

While a breakthrough for modeling, the researchers emphasize AlphaFold does not fully capture the complex physics underlying proteins. The study demonstrates the continued need for human intuition and experimentation to evaluate and advance AI in science.

The project exemplified synergistic collaboration between students and scientists to critically test the limits of an exciting new technology. This guides appropriate expectations and future progress.

Researchers Explain Continued Need for Human Insight in Protein Science

The researchers emphasize their findings highlight key challenges remaining in protein bioinformatics that require human intuition. For example, predicting how mutations affect protein properties can guide engineering proteins for real-world applications like thermostable enzymes.

On a fundamental level, the study demonstrates AI alone cannot fully capture the complexity of protein physics. Key tasks like predicting protein-DNA complexes, binding energies, and designing novel proteins remain open grand challenges.

The authors argue AlphaFold succeeds by recognizing patterns in known structures, not by deeply learning physical principles. It struggles to differentiate highly similar structures that should have clear stability differences.

Previous works noted AlphaFold sometimes makes physically improbable predictions, likely reflecting biases in its training data. This further indicates the program does not rely on an innate understanding of protein physics.

The researchers stress AI is not a panacea that automates scientific discovery. Humans must guide technology with scientific insight and evaluate limitations. High school students contributed to the work by critically analyzing AlphaFold’s capabilities.

Ultimately, the project exemplifies how human creativity and curiosity remain essential to pushing the frontiers of knowledge. While transformational, AI is one tool among many for unraveling the complexities of biology.

Reference: “Using AlphaFold to predict the impact of single mutations on protein stability and function” by Marina A. Pak, Karina A. Markhieva, Mariia S. Novikova, Dmitry S. Petrov, Ilya S. Vorobyev, Ekaterina S. Maksimova, Fyodor A. Kondrashov and Dmitry N. Ivankov, 16 March 2023, PLOS One.
DOI: 10.1371/journal.pone.0282689