Named tensors

[elixir-nx]

Named tensors

[seanmor5]

I've started messing around with named tensors. My opinion is that we should enforce names as a requirement. I think the benefits of enforcing name requirements outweigh the possible annoyance or verbosity in the code. I think it also avoids us having to make decisions about how to preserve names in instances where one tensor's names are explicitly defined and another's aren't.

For example, if somebody tries to do:

Nx.add(Nx.tensor([1, 2, 3]), Nx.tensor([1, 2, 3], names: [:data])

Then we either have to drop the name altogether and it may be difficult to try to find out where the name got dropped or changed if it needs to be used later on or we have to raise here and enforce ops like this to work only with tensors that are all named or all unnamed.

If we force people to always name their tensors, then we have:

Nx.add(Nx.tensor([1, 2, 3], names: [:batch]), Nx.tensor([1, 2, 3], names: [:data]))

And it's to clear to see this shouldn't work. Then with broadcasting we just align the names and broadcast the data. What are your thoughts?

[josevalim]

My concern with enforcing names is that we lose the ability for general algorithms that work on any shape - that's why I am a bit hesitant of enforcing them. I also think it will make playing with the library for discovery a bit more uncomfortable (think about rewriting all examples to include names...).

Luckily, for the cases where one has name and the other does not, we should preserve the names of the one with names. The bigger question is what to do when there is a broadcast and some of those names do not have shapes. We will probably need to allow some of the names to be nil.

[seanmor5]

I think then we should by default initialize all the names to nil unless the names of a dimension are given as a list atoms. Then handling broadcasting would be easy.

So :names would be an option in all of the tensor creation methods that defaults to a list of nil matching the rank of the tensor. If :names is given by the user, it must have the same length as the rank of the tensor.

As far as default names go, I don't think we should provide generic defaults because I think the "defaults" really come with your application.

Anybody who's working with images will have the dimensions:

:batch, :height, :width, :channels

And other domains would inevitably have their own sort of consistent naming conventions. And then the responsibility is on whoever is writing an algorithm to document what their chosen naming standard is.

[seanmor5]

Resolved with #138 and #139