Skip to content

Researchers struggle with accuracy of AI technology used to create new drug candidates

Artificial intelligence (AI) has numerous applications in healthcare, from analyzing medical images to optimizing the execution of clinical trials and even facilitating drug discovery.

AlphaFold2, an artificial intelligence system that predicts protein structures, has made it possible for scientists to identify and conjure a nearly infinite number of drug candidates for the treatment of neuropsychiatric disorders. However, recent studies have cast doubt on AlphaFold2’s accuracy in modeling ligand binding sites, the areas of proteins where drugs bind and begin sending signals inside cells to cause a therapeutic effect, as well as possible side effects.

In a new paper, Bryan Roth, MD, PhD, Michael Hooker Distinguished Professor of Pharmacology and director of the NIMH Psychoactive Drug Screening Program at the University of North Carolina School of Medicine, and colleagues from UCSF, Stanford, and Harvard determined that AlphaFold2 can return accurate results for ligand-binding structures, even when the technology has nothing to rely on. Their results were published in Science.

“Our results suggest that AF2 structures may be useful for drug discovery,” said Roth, senior author who holds a joint appointment at the UNC Eshelman School of Pharmacy. “With a nearly infinite number of possibilities for creating drugs that hit their intended target to treat a disease, this type of AI tool can be invaluable.”

AlphaFold2 and prospective modeling

Just like weather forecasting or stock market prediction, AlphaFold2 works by mining a massive database of known proteins to create models of protein structures. You can then simulate how different molecular compounds (such as drug candidates) fit into the protein’s binding sites and produce the desired effects. Researchers can use the resulting combinations to better understand protein interactions and create new drug candidates.

To determine the accuracy of AlphaFold2, researchers had to compare the results of a retrospective study with those of a prospective study. One retrospective study involves researchers feeding the prediction software with compounds they already know bind to the receptor. While a prospective study requires researchers to use the technology as a fresh slate and then provide the AI ​​platform with information about compounds that may or may not interact with the receptor.

The researchers used two proteins, sigma-2 and 5-HT2A, for the study. These proteins, which belong to two different protein families, are important in cellular communication and have been implicated in neuropsychiatric conditions such as Alzheimer’s disease and schizophrenia. The serotonin 5-HT2A receptor is also the main target of psychedelic drugs that show promise for the treatment of a large number of neuropsychiatric disorders.

Roth and his colleagues selected these proteins because AlphaFold2 had no prior information about sigma-2 and 5-HT2A or the compounds that might bind to them. Essentially, the technology was given two proteins that it was not trained for, essentially giving the researchers a “blank slate.”

First, the researchers fed the AlphaFold system the protein structures of sigma-2 and 5-HT2A, creating a prediction model. The researchers then accessed physical models of the two proteins that were produced using complex microscopy and X-ray crystallography techniques. With the push of a button, up to 1.6 billion potential drugs were directed to the experimental models and AlphaFold2 models. Interestingly, each model had a different result for the drug candidate.

Successful success rates

Although the models have different results, they show great promise for drug discovery. The researchers determined that the proportion of compounds that actually altered protein activity for each of the models was around 50% and 20% for the sigma-2 and 5-HT2A receptors, respectively. A result greater than 5% is exceptional.

Of the hundreds of millions of potential combinations, 54% of drug-protein interactions using the AlphaFold2 sigma-2 protein models were successfully triggered by a bound drug candidate. The experimental model for sigma-2 produced similar results with a success rate of 51%.

“This work would be impossible without collaboration among several leading experts from UCSF, Stanford, Harvard and UNC-Chapel Hill,” Roth said. “In the future we will test whether these results could be applicable to other therapeutic targets and classes of targets.”