frankaging/bert-lrp

On Explaining Your Explanations of BERT: An Empirical Study with Sequence Classification

bert-lrp's Issues

Bug report

In train_helper.py line 612, I think the "train_features" should be "test_features" to load the segment_ids.

word-deletion-correct.tsv

Hi!
Thanks for sharing this amazing work!
A question : how do you calculate the dataframe in word-deletion-correct.tsv file you use in order to plot figure 1 in the paper?
Thanks!

Question about the implemented LRP rules

Thank you for this great work.
Maybe you didn't realize it, Your implemented LRP rules are actually quite different than that has been proposed in this area.... based on my test, your implementation actually beat XAI SOTA algorithm published recently on some datasets...

That's said... while they work... I don't understand the intuition .... I would appreciate it if you can share some of insights.

First, The Linear alpha-beta rule: It looks like you implemented a modified version of alpha-beta rule
https://github.com/frankaging/BERT_LRP/blob/57f44c1bd7c1e3886110402652690b75e4ac0f67/code/util/lrp.py#L24-L44

The original alpha beta rule is in the following form:
$$R_j = \sum_{k} \alpha \frac{(a_j w_{jk})^+ }{\sum (a_j w_{jk})^+ } + \beta \frac{(a_j w_{jk})^- }{\sum (a_j w_{jk})^- } $$.

And what you implemented is in the following form:
$$R_j = \sum_{k} \alpha \frac{a_j w_{jk}^+ }{\sum a_j w_{jk}^+ } + \beta \frac{a_j w_{jk}^- }{\sum a_j w_{jk}^- } $$.

So the original rule preallocate alpha and beta portion of relevance towards positive and negative of (weight* input), while you allocate alpha and beta portion towards (pos_weight * input) and (neg_weight * input).

I don't understand the intuition behind this rule, because the input of linear layer in bert may not be strictly positive... so I don't understand why do we purely allocate a fixed portion of relevance to inputs based on the sign of associated weights.

Also, looking at how you are dealing with other layers, it seems you are directly using relevance multiplying the gradient of the layer's output to the input, for example, you did the skip connection as following:
https://github.com/frankaging/BERT_LRP/blob/57f44c1bd7c1e3886110402652690b75e4ac0f67/code/model/BERT.py#L647-L650

Can you share some insights what does this represent? I haven't seen anything like this before.....

It seems to me that you have done a lot of different tricks (these two above, the way you normalize the relevance after linear layer and how you force to take absolute value of relevance at the end despite the relevance are signed) that are quite different than what has been published in this area. To be honest, some of them don't make too much sense... but since they performs well on NLP tasks I've been testing... there must be things I don't understand/worth exploring

Have you tried these rules on other types of transformer (like VIT)? I adapted these rules on VIT...the result is not as good.. but maybe I made some mistakes in the code.. not sure.

Thanks