1pha/yet-awesome-explainability

Collection of Explainability/Interpretability of Deep Neural Networks.

Awesome Exlainability

Collection of Explainability/Interpretability of Deep Neural Networks from classics to recent models, mainly on deep neural networks. Please leavn me an issue if you catch something awful. I will add this slowly.

Taxonomy

Full taxonomy by Manolis Kellis and lots of models fall into mainly two categories: interpreting the model or its decision. This repository will mainly focus on the model decision, in terms of interpreting the model

Quotes

... when using the word "understanding", we refer to a functional understanding of the model

An interpretation is the mapping of an abstract concept (e.g. a predicted class) into a domain that the human can make sense of.

An explanation is the collection of features of the interpretable domain, that have contributed for a given example to produce a decision (e.g. classification or regression).

Gradients tells which pixels will be more/less likely to be cat rather than what pixels construct a cat. (Video)

Papers

• Model-Agnostic

  • LIME Ribeiro, Marco Tulio, Sameer Singh, and Carlos Guestrin. "" Why should i trust you?" Explaining the predictions of any classifier." Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 2016. Paper
  • SHAP Lundberg, Scott M., and Su-In Lee. "A unified approach to interpreting model predictions." Advances in neural information processing systems 30 (2017). Paper

• Relevance Propagation

  • Bach, Sebastian, et al. "On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation." PloS one 10.7 (2015): e0130140. Paper

  • Samek, Wojciech, et al. "Evaluating the visualization of what a deep neural network has learned." IEEE transactions on neural networks and learning systems 28.11 (2016): 2660-2673. Paper

  • Montavon, Grégoire, et al. "Explaining nonlinear classification decisions with deep taylor decomposition." Pattern recognition 65 (2017): 211-222. Paper Nice PDF

  • Montavon, Grégoire, Wojciech Samek, and Klaus-Robert Müller. "Methods for interpreting and understanding deep neural networks." Digital Signal Processing 73 (2018): 1-15. Paper

  • Ancona, Marco, et al. "Towards better understanding of gradient-based attribution methods for deep neural networks." arXiv preprint arXiv:1711.06104 (2017). Paper

  • Montavon, Grégoire, et al. "Layer-wise relevance propagation: an overview." Explainable AI: interpreting, explaining and visualizing deep learning (2019): 193-209. Paper

  • Kohlbrenner, Maximilian, et al. "Towards best practice in explaining neural network decisions with LRP." 2020 International Joint Conference on Neural Networks (IJCNN). IEEE, 2020. Paper

  • Samek, Wojciech, et al. "Explaining deep neural networks and beyond: A review of methods and applications." Proceedings of the IEEE 109.3 (2021): 247-278. Paper

  • Nice Tutorial Code GitLab

• Transformer / ViT Explanation

  • Chefer, Hila, Shir Gur, and Lior Wolf. "Transformer interpretability beyond attention visualization." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021. Paper
  • Berglund, Sandor, Francisco Ferrari, and Daniel Morales Brotons. "LRP-based Method for Transformer Interpretability." ML Reproducibility Challenge 2021 Fall Blind Submission Paper

• Survey

  • Das, Arun, and Paul Rad. "Opportunities and challenges in explainable artificial intelligence (xai): A survey." arXiv preprint arXiv:2006.11371 (2020). Paper

Videos/Tutorials/Lectures

• Lec 05: Interpretability, Dimensionality Reduction by Prof. Manolis Kellis (from Spring 2021 6.874 MIT Computational Systems Biology: Deep Learning in the Life Sciences) Video Lecture Note

• CVPR 2018 Tutorial: Interpreting and Explaining Deep Models in Computer Vision by Bolei Zhou (MIT) Part. 1, Part. 2 Documents

• GCPR 2017 Tutorial: Interpretable Machine Learning by Wojciech Samek & Klaus-Robert Muller PDF

• ICASSP 2017 Tutorial: Methods for Interpreting and Understanding Deep Neural Networks Wojciech Samek, Gregoire Montavon and Klaus-Robert Muller (No pdf/video found) ICASSP2017 Tutorials Tutorial Paper

• cs231n Lecture 12: Visualizing and Understanding Video PDF

Codes

• Just use captum Documentation Git Repo
• Tutorial on LRP GitLab
  • Based on above Github

Institutes

Fraunhofer.de XAI