Collection of Explainability/Interpretability of Deep Neural Networks from classics to recent models, mainly on deep neural networks. Please leavn me an issue if you catch something awful. I will add this slowly.
Full taxonomy by Manolis Kellis and lots of models fall into mainly two categories: interpreting the model or its decision. This repository will mainly focus on the model decision, in terms of interpreting the model
... when using the word "understanding", we refer to a functional understanding of the model
An interpretation is the mapping of an abstract concept (e.g. a predicted class) into a domain that the human can make sense of.
An explanation is the collection of features of the interpretable domain, that have contributed for a given example to produce a decision (e.g. classification or regression).
Gradients tells which pixels will be more/less likely to be cat rather than what pixels construct a cat. (Video)
-
Model-Agnostic
- LIME Ribeiro, Marco Tulio, Sameer Singh, and Carlos Guestrin. "" Why should i trust you?" Explaining the predictions of any classifier." Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 2016. Paper
- SHAP Lundberg, Scott M., and Su-In Lee. "A unified approach to interpreting model predictions." Advances in neural information processing systems 30 (2017). Paper
-
Relevance Propagation
-
Bach, Sebastian, et al. "On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation." PloS one 10.7 (2015): e0130140. Paper
-
Samek, Wojciech, et al. "Evaluating the visualization of what a deep neural network has learned." IEEE transactions on neural networks and learning systems 28.11 (2016): 2660-2673. Paper
-
Montavon, Grégoire, et al. "Explaining nonlinear classification decisions with deep taylor decomposition." Pattern recognition 65 (2017): 211-222. Paper Nice PDF
-
Montavon, Grégoire, Wojciech Samek, and Klaus-Robert Müller. "Methods for interpreting and understanding deep neural networks." Digital Signal Processing 73 (2018): 1-15. Paper
-
Ancona, Marco, et al. "Towards better understanding of gradient-based attribution methods for deep neural networks." arXiv preprint arXiv:1711.06104 (2017). Paper
-
Montavon, Grégoire, et al. "Layer-wise relevance propagation: an overview." Explainable AI: interpreting, explaining and visualizing deep learning (2019): 193-209. Paper
-
Kohlbrenner, Maximilian, et al. "Towards best practice in explaining neural network decisions with LRP." 2020 International Joint Conference on Neural Networks (IJCNN). IEEE, 2020. Paper
-
Samek, Wojciech, et al. "Explaining deep neural networks and beyond: A review of methods and applications." Proceedings of the IEEE 109.3 (2021): 247-278. Paper
-
Nice Tutorial Code GitLab
-
-
Transformer / ViT Explanation
- Chefer, Hila, Shir Gur, and Lior Wolf. "Transformer interpretability beyond attention visualization." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021. Paper
- Berglund, Sandor, Francisco Ferrari, and Daniel Morales Brotons. "LRP-based Method for Transformer Interpretability." ML Reproducibility Challenge 2021 Fall Blind Submission Paper
-
Survey
- Das, Arun, and Paul Rad. "Opportunities and challenges in explainable artificial intelligence (xai): A survey." arXiv preprint arXiv:2006.11371 (2020). Paper
-
Lec 05: Interpretability, Dimensionality Reduction by Prof. Manolis Kellis (from Spring 2021 6.874 MIT Computational Systems Biology: Deep Learning in the Life Sciences) Video Lecture Note
-
CVPR 2018 Tutorial: Interpreting and Explaining Deep Models in Computer Vision by Bolei Zhou (MIT) Part. 1, Part. 2 Documents
-
GCPR 2017 Tutorial: Interpretable Machine Learning by Wojciech Samek & Klaus-Robert Muller PDF
-
ICASSP 2017 Tutorial: Methods for Interpreting and Understanding Deep Neural Networks Wojciech Samek, Gregoire Montavon and Klaus-Robert Muller (No pdf/video found) ICASSP2017 Tutorials Tutorial Paper
- Just use captum Documentation Git Repo
- Tutorial on LRP GitLab
- Based on above Github
Fraunhofer.de XAI