Explainable AI for Omics Data

The Role of Explainable AI in Transforming Omics Data Analysis for Healthcare

In the rapidly advancing field of biomedical research, understanding complex biological processes is key to unlocking new medical insights and improving patient outcomes. One of the most promising approaches to achieving this goal is the analysis of omics data, which encompasses genomics, transcriptomics, proteomics, and metabolomics. These data types hold vast potential for unraveling the complexities of biological systems. However, the sheer volume and complexity of omics data pose significant challenges, particularly when artificial intelligence (AI) models are used to analyze them. AI has demonstrated impressive performance in extracting meaningful patterns from large datasets, but the “black-box” nature of many AI models limits their interpretability. This lack of transparency is a particular concern in healthcare, where understanding the reasoning behind AI-driven decisions is essential. Explainable Artificial Intelligence (XAI) aims to address these challenges by making AI models more transparent, interpretable, and trustworthy.

The Need for Explainable AI in Omics Data Analysis

Omics data are often high-dimensional and multifaceted, which makes their analysis both complex and computationally intensive. AI models, such as deep neural networks (DNNs) and convolutional neural networks (CNNs), have shown great promise in analyzing these data. However, many of these models are difficult to interpret, making it challenging for healthcare professionals to trust the results. In clinical settings, decisions based on AI models can have life-altering consequences, so it is imperative that these models are not only accurate but also interpretable. This is where XAI comes into play.

XAI refers to a set of AI techniques that are designed to make machine learning models more transparent and understandable. There are two main approaches to XAI:

1. Interpretable Models: These models are inherently transparent and are designed to be understandable from the outset.
2. Post-hoc Explanations: These methods are applied to existing black-box models to make them more interpretable. Post-hoc explanations may include techniques like feature relevance analysis and visual representations of the model’s inner workings.

A Systematic Review of XAI in Omics Data

A recent study conducted a systematic review of 405 publications to evaluate how XAI has been applied to omics data analysis. This study used a comprehensive classification scheme that categorized the research based on AI methods, explainability techniques, and types of omics data analyzed. The study found that the number of publications on XAI for omics data has steadily increased, particularly after 2017, with a peak in 2022. The research spanned various fields, including biomedical sciences, engineering, and information technology. However, the literature remains fragmented, with most studies focusing on isolated aspects of the field.

Key Findings from the Study:

• Data Types: Gene expression and DNA sequence data were the most commonly analyzed omics data types. Multi-omics data, which integrates different types of omics data, is also gaining popularity.
• AI Methods: Neural networks were the most commonly used AI models, followed by decision trees, random forests, and other machine learning methods.
• Explainability Techniques: Feature relevance methods (such as SHAP values) and visual explanations were the most commonly used post-hoc methods for interpreting AI models. Transparent models and architecture modifications were also frequently used.
• Medical Applications: The largest medical fields where XAI is applied are medical research, oncology, and clinical laboratory sciences. XAI models have been particularly useful in these areas for diagnosing diseases and guiding treatment decisions.

Specific Applications of XAI in Omics Data

The study also highlighted specific applications of XAI in various omics fields:

• DNA Sequence Data: CNNs are commonly used to analyze DNA sequence data, with feature relevance and visual explanations aiding in the interpretation of these models.
• Gene Expression Data: Deep neural networks and rule mining are frequently used for analyzing gene expression data. SHAP values and other feature relevance methods are applied to explain the models.
• Multi-Omics Data: XAI techniques are used to integrate different types of omics data, such as combining CNNs with recurrent neural networks (RNNs) or using multiple models to compare results.
• Other Omics Data: XAI is also applied to RNA sequencing data, single-cell RNA sequencing data, proteomics data, and microbiomic data.

Future Directions for XAI in Omics Data

The study identified several key research directions to further advance the application of XAI in omics data analysis. These directions aim to address current gaps in the field and improve the interpretability of AI models. Some of the proposed research directions include:

1. Apply XAI-Supported Neural Networks to Omics Data: How can XAI-supported neural networks, such as transformers, be better utilized for analyzing omics data? What data preparations are needed for XAI-enhanced convolutional neural networks (CNNs)?
2. Post-Hoc Analysis for Transparent Models: How can post-hoc explanations improve the interpretability of transparent models in omics data analysis? How can visual explanations aid in the analysis?
3. Simplification and Local Explanations for Deep Learning Models: How can deep learning models be simplified for better transparency? What role do local explanations play in explaining these models?
4. Development of New Interpretable Models: What approaches can be used to modify non-transparent models into interpretable models, such as through architecture modifications?
5. Combining Different Explainability Methods: How can different XAI methods be combined to enhance the interpretability of AI-based omics data analysis?
6. Explanations by Example and Text: How can text explanations and explanations by example improve the understanding of omics data?
7. Explore Novel Omics Data: How can XAI be applied to emerging omics data types, such as single-cell RNA sequencing and microbiomic data?
8. Role of XAI in Decision-Making: How does XAI help biomedical experts gain insights and make informed decisions?

Challenges and Opportunities

While the potential of XAI in omics data analysis is immense, there are several challenges that need to be addressed:

• Lack of Standardized Methods: There is a need for standardized methods to evaluate the quality of XAI explanations, which would allow for better comparison of different approaches.
• Complexity of Omics Data: The high dimensionality and complexity of omics data make it difficult to develop simple, interpretable models.
• Interdisciplinary Collaboration: Bridging the gap between AI and biomedical fields is crucial to advancing XAI in omics. Collaboration between these fields will help address the challenges and unlock the full potential of XAI.

Conclusion

Explainable AI is revolutionizing the way we analyze omics data, especially in healthcare applications. By improving the transparency and interpretability of AI models, XAI fosters trust and enables healthcare professionals to make better-informed decisions. As research in this field progresses, it is clear that XAI will play a pivotal role in advancing biomedical research and improving patient outcomes. The future of XAI in omics is filled with exciting opportunities for new discoveries, and the proposed research directions will undoubtedly drive the next wave of innovation in this area.

In conclusion, the quest for clarity in AI-driven omics analysis is an ongoing journey that promises transformative advancements in our understanding of biology and medicine. As XAI continues to evolve, it will not only improve our ability to analyze complex biological data but also enhance the trustworthiness and effectiveness of AI in critical healthcare applications.

FAQ: Explainable AI for Omics Data

1. What is the primary goal of Explainable Artificial Intelligence (XAI) in the context of omics data analysis?

The primary goal of XAI in omics data analysis is to make the complex models used to analyze biological data more understandable to humans. While AI models can achieve high predictive performance, especially with high-dimensional omics data, they often function as “black boxes,” where their inner workings and decision-making processes are opaque. XAI aims to provide transparency by either creating models that are inherently interpretable or by adding post-hoc explanations to clarify how the models arrive at their conclusions. This is especially crucial in healthcare, where understanding the “why” behind an AI’s prediction is just as important as the prediction itself.

2. What are the two main paradigms that XAI offers to improve AI model understandability?

XAI provides two main approaches for improving the understandability of AI models:

• Interpretable (Transparent) Models: This approach focuses on creating AI models that are inherently easy to understand. This can be achieved by using simpler models, such as decision trees, rule-based systems or modifying the architecture of complex models, like neural networks, to include prior knowledge of the biological processes. These models are designed from the ground up to be transparent.
• Post-hoc Explanations: This approach involves taking an existing “black box” AI model and adding methods to explain its internal functions after the model has been built. This often includes techniques like feature relevance analysis (identifying the most important variables) and visual explanations (using visualizations to show what the model is focusing on).

3. What types of omics data are most frequently analyzed using XAI, and which AI methods are commonly applied to them?

The most frequently analyzed omics data types, using XAI are:

• Gene expression data, followed by DNA sequence data are commonly analyzed. These are often analyzed using various AI methods, including neural networks (particularly deep neural networks), multiple AI pipelines, and rule-mining approaches.
• Multi-omics data, which combines two or more types of omics data, is also an emerging area of research. For multi-omics analysis, a mix of methods including multiple AI pipelines, deep neural networks, and transformers is often used.
• Other data types such as RNA sequence, proteomic, SNP data, and microbiomic data are also used to a lesser extent, often with methods such as NNs, rule mining, and statistical approaches.

4. Which explainability methods are most often used in conjunction with neural networks (NNs) for omics data?

The most common explainability methods applied to neural networks (NNs) in omics data analysis are:

• Feature relevance analysis: This involves identifying which input features or variables (e.g., specific genes, proteins, or genomic sequences) have the greatest impact on the NN’s predictions. Techniques like SHAP (SHapley Additive exPlanations) are often used.
• Architecture modification: This aims to modify or add elements to the neural network’s architecture, such as knowledge graphs or constraints, to make it more transparent and interpretable.
• Visual explanations: These methods aim to visualize what the neural network is focusing on when analyzing omics data, often in the form of heatmaps, which is most frequently used with convolutional neural networks.

5. Why is feature relevance analysis a popular method for explaining AI models in omics data analysis?

Feature relevance analysis is popular for several reasons:

• Simplicity and Versatility: It can be applied to many different AI models, including complex ones like neural networks and it can be implemented for both transparent and non-transparent models.
• Biological Interpretability: By identifying the most influential features (e.g., genes, proteins), researchers can gain insights into the biological mechanisms underlying the model’s predictions. This makes it easier to interpret and make sense of the results.
• Prioritization for Further Investigation: Feature relevance analysis can be used to prioritize which variables are worthy of closer examination, such as for drug development or understanding cancer pathways.

6. Are “transparent” AI models enough, or do they still need additional explanation methods?

While transparent models are inherently easier to understand than black-box models, they can still become complex when dealing with large and high-dimensional omics data. Post-hoc explainability methods like feature relevance and visualization can improve the understandability of transparent models even further by reducing complexity. While transparent models like decision trees are very interpretable, complex transparent models like Bayesian networks or rule mining systems, can benefit from post-hoc methods.

7. Why are simplification and local explanation techniques underutilized in XAI for omics data, and why might this be a problem?

Simplification and local explanation techniques are currently underutilized because:

• Complexity: Implementing these methods can be challenging, as it requires an in-depth understanding of the AI method, the use case, and the underlying omics data.
• Less Established: Compared to feature relevance, simplification and local explanations are less established and may require more specialized expertise to develop.

However, they offer unique advantages:

• Simplified Networks Simplification aims to reduce model complexity to improve the transparency of omics data analysis, which may allow for higher biological interpretability of hidden layers in neural networks.
• Local Explanations Local explanations can be especially helpful for omics data with spatial aspects like imaging data, allowing researchers to investigate small sections at a time by highlighting important regions contributing to the AI’s predictions.

Their underutilization can be a problem as there are many cases in which the complexity of current models is an issue, and these methods can be especially helpful when other methods fall short.

8. What are some promising future research directions for XAI in omics data analysis, and what questions should they address?

Based on current research gaps, the following are considered important future research directions for XAI in omics:

• Applying XAI-supported neural networks: How can we apply XAI specifically to understand NNs, especially transformers, when analyzing omics data? What are the optimal ways to prepare data to use NNs in the omics setting? Which explanation methods are most helpful for using NNs to understand underlying biological processes?
• Adding post-hoc analysis to transparent models: How can explanations improve the transparency of already interpretable models? And how can visual explanations aid the analysis of omics data when using transparent AI models?
• Using simplification and local explanations for post-hoc analysis: How can neural networks be simplified to improve transparency in omics data? How can local explanations help explain deep learning models?
• Developing new interpretable models and adapting existing post-hoc methods: What measures are necessary to put post-hoc methods into practical healthcare settings? How can existing non-transparent models be modified to create novel interpretable models?
• Combining different explainability methods: How can multiple explainability methods be combined for better understanding of AI outputs?
• Applying explanations by example and text explanations: How can providing examples improve post-hoc analysis of omics data? How can text explanations improve analysis?
• Investigating XAI for novel omics data: How can XAI be used to analyze newer types of omics data?
• Exploring the role of XAI in decision-making processes: How does XAI affect experts’ understanding and decision-making in real world scenarios?

Glossary of Key Terms

• Artificial Intelligence (AI): The development of computer systems that can perform tasks that typically require human intelligence, such as learning, problem-solving, and decision-making.
• Explainable Artificial Intelligence (XAI): A subfield of AI that focuses on making the decision-making processes of AI models transparent and understandable to humans.
• Omics Data: High-dimensional biological data derived from various “-omics” fields, such as genomics (DNA), transcriptomics (RNA), proteomics (proteins), and metabolomics (metabolites).
• Machine Learning (ML): A type of AI that enables computer systems to learn from data without being explicitly programmed, often using algorithms to identify patterns and make predictions.
• Deep Learning (DL): A subset of machine learning that uses artificial neural networks with multiple layers (deep neural networks) to analyze complex data and learn hierarchical representations.
• Black Box Model: An AI model whose inner workings are opaque and not easily interpretable by humans, making it difficult to understand why the model makes specific predictions.
• Interpretable Model: An AI model whose decision-making process is inherently understandable and transparent to humans due to its structure and algorithms, often referred to as transparent models.
• Post-hoc Explanation: An explanation method applied after an AI model has made a prediction, aiming to clarify how the model arrived at its output through techniques like feature relevance or visual explanations.
• Feature Relevance: An XAI method that identifies which specific features or variables in the input data were most important in influencing the AI model’s prediction.
• Visual Explanation: An XAI method that uses visual representations (e.g., graphs, images, heatmaps) to explain the behavior of an AI model and the relevance of input features.
• Systematic Mapping Study: A type of literature review method that systematically identifies, categorizes, and synthesizes existing research on a specific topic to gain an overview of the field and identify research gaps.
• Neural Network (NN): A computer system modeled on the human brain, comprised of interconnected nodes that learn from data, often through multiple layers, enabling complex pattern recognition and prediction tasks.
• Convolutional Neural Network (CNN): A type of deep learning model especially well-suited for processing images and other grid-like data, often used in genomics to identify patterns in sequences.
• Transformer: A neural network architecture that uses attention mechanisms to handle sequential data, which has shown promise in various fields, including language processing and genomics.
• Rule Mining: A machine learning technique used to discover meaningful relationships and rules within datasets, often producing transparent models that highlight important patterns.
• Gradient Boosting: An ensemble learning method that combines multiple weak prediction models (usually decision trees) to create a strong predictor, with results that are often transparent and explainable.
• SHAP (SHapley Additive exPlanations): A method for calculating feature importance in a machine learning model, based on game theory, offering a framework to explain predictions and understand the contribution of individual features to an outcome.

Reference

Toussaint, P. A., Leiser, F., Thiebes, S., Schlesner, M., Brors, B., & Sunyaev, A. (2024). Explainable artificial intelligence for omics data: a systematic mapping study. Briefings in Bioinformatics, 25(1), bbad453.