Comments (3)
I actually don't think it's a good idea. All optimization we do makes sense only for scalar valued functions, and implicit initialization of a scalar "grad output" to 1 is sensible in this case, because it gives us correct gradient values. We also allow starting backward from an arbitrary location in the graph, as long as you provide gradient w.r.t. each element of the tensor, because it's also perfectly valid mathematically, but I'm not sure why should we interpret "backward from a tensor" as "backward from a sum for tensor elements".
@rbgirshick when do you think it would be useful?
from pytorch.
It's really not important. In the past I've occasionally debugged gradient computations given a known incoming gradient. Choosing all ones is usually a good choice for debugging since it's easy to understand. So this change would only save one line and an extra arg in a niche use case. Feel free to close without implementing.
from pytorch.
ok, closing this, as it'll be a niche use case, and adding a torch.ones(x.size()) wont be that bad and be more explicit.
from pytorch.
Related Issues (20)
- DISABLED test_binary_op_list_error_cases__foreach_clamp_max_cuda_float64 (__main__.TestForeachCUDA) HOT 1
- [Feature Request] Support `dtype` arg in `torch._foreach_norm` HOT 2
- Support auto_functionalized for None returns
- TOR901 lint is too aggressive HOT 1
- MPS backend thinks that small floats are less than zero
- DISABLED test_dynamic_shapes (__main__.TestCompiledAutograd) HOT 2
- DISABLED test_select_expanded_v (__main__.TestAutogradWithCompiledAutograd) HOT 2
- DISABLED test_foreach_matches_forloop_RAdam_cpu_float64 (__main__.TestOptimRenewedCPU) HOT 2
- DISABLED test_sdpa_backwards_cuda_bfloat16 (__main__.TestNestedTensorSubclassCUDA) HOT 1
- 404 on torch.inference_mode doc page
- ValueError: weight_norm of 'weight' not found in ParametrizedConvTranspose1d HOT 1
- 2.2.0+ regresses SDPA performance on Windows
- DISABLED test_setting_default_saved_variable_hooks_twice_should_use_inner (__main__.TestAutogradWithCompiledAutograd) HOT 2
- Inductor can not fuse cat with a pointwise
- [Inductor] Generate triton block pointers for discontiguous strided tensors HOT 1
- `torch.compile` fails with `jacfwd` when multiplying/dividing float and tensor
- [Distributed] P2P Operations on NCCL do not respect tag HOT 1
- Triton installation not found.
- Broken Docker Image on dockerhub
- PyTorch Distributed Load Updates or Returns `state_dict` HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pytorch.