Comments (3)
danielway
commented on July 27, 2024
The arithmetic nodes are recreated on each iteration during the forward pass, so we don't need to worry about their gradients carrying over into future iterations. While the network lives through multiple iterations, the node tree created by invoking the network in a forward pass is only used for a single backward pass.
from nn-zero-to-hero.
zivanfi
commented on July 27, 2024
Ah, I see, thank you!
from nn-zero-to-hero.
IsaacDynamo
commented on July 27, 2024
Thanks! I had the same question, but this explains it.
from nn-zero-to-hero.
Related Issues (20)
- Numerical instability in Google Colab - Part 4 of Makemore HOT 6
- Not a real issue: Video links broken HOT 5
- intro to neural networks first HOT 3
- Shouldn't names.txt be added to this repo? HOT 1
- difference on calculating loss for count-matrix
- b HOT 1
- If I place more than 3 layers of FlattenConsecutive in makemore_part5_cnn1.ipynb, I get error: not enough values to unpack (expected 3, got 2)
- Zero to hero
- Suggestion: Time Series Forecasting and Transfer Learning
- Factorized code for dC in makemore_part4_backprop.ipynb HOT 1
- RuntimeError: Trying to backward through the graph a second time (or directly access saved tensors after they have already been freed). Saved intermediate values of the graph are freed when you call .backward() or autograd.grad(). Specify retain_graph=True if you need to backward through the graph a second time or if you need to access saved tensors after calling backward. HOT 1
- Discrepancy in makemore_part1_bigrams single sample draw HOT 2
- Error in makemore_part1_bigrams.ipynb HOT 1
- Missing L.backward() before constructing graph HOT 1
- TypeError: unsupported operand type(s) for +: 'int' and 'Value' HOT 1
- need Discussions :)
- BatchNorm1d in makemore part3 may have error
- makemore_part4_backprop dhpreact exact part is False. HOT 2
- Not bug, just curiosity HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nn-zero-to-hero.