Comments (1)
Hey Reduan,
I am aware of this and think this is not really a problem, since the tensor you get with each pass is a new instance, and will be garbage collected anyway as soon as all references are gone (remember, the hook is registered to the tensor in the forward pass, which is also when a new tensor instance is created each time).
Even if the tensor stays around, the hook is bound to its specific backward graph, which is actually expected to not change.
From removing the handle you would gain the ability to get the gradient instead when doing another backward pass with different grad_outputs when the graph is retained (as you are doing in your conditional lrp code).
I have been thinking how to control the tensor handles before, since this is needed for computing the gradient with respect to the modified gradient, without triggering the hook.
But I think removing the hook before computing the second order gradient is not the best approach, since then you lose the ability to retain the graph for computing the modified gradients on the same graph.
I thought about implement something like a context or a switch to temporarily deactivate the hook (like with a condition within the hook) without removing it in the future.
from zennit.
Related Issues (20)
- Support for padding layers HOT 2
- Core/Composite: Simplify Composite.context
- Docs: Build Tutorials on ReadTheDocs with trained weights
- How to add support to DenseNet from torchvision HOT 2
- Layer-wise LRP score HOT 6
- LinearAttention Module HOT 1
- error occurred when run tutorial on custom model HOT 7
- Composite: bug in BetaSmooth constructor? HOT 3
- Module with Multiple Inputs HOT 2
- Smooth MaxPool2D rule HOT 2
- Obtanining graident of LRP otput w.r.t. network parameters HOT 12
- Custom rules for vision transformer HOT 1
- Feature Request: Debugger HOT 2
- Bug: MultiheadAttention in types HOT 4
- Binary classification model with one output for output layer HOT 1
- Proper way of handling classifier heads in Transformers HOT 2
- Canonizers: BatchNorm loses bias relevance
- ResNet: Unstable Attributions HOT 3
- Neuralized K-Means: make k-means amenable to neural network explanations
- Model inference outside of composite context HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from zennit.