Deprecating Tensorflow support?

[cohenjer]

Deprecating Tensorflow support?

[aarmey]

Actually, it looks like tnp.experimental_enable_numpy_behavior() makes everything work except for the contrib/tt methods and randomized CP. That's not too bad...

[cohenjer]

100% agreed with @aarmey that the main problems are indexing/updating elements in Tensorflow tensors. Having a verbose and error-prone syntax using update_index, while all other backends do not need this syntax, is painful.

For now #494 seems like the right move.

My concern with the numpy experimental behavior is that I have no idea how stable it is. But if the changes to existing functions are minor as you suggest, it may be worthwhile to do it anyways.

[aarmey]

One more bit of information I found—JAX still requires tensor.at[indices].set(values) syntax if you plan to trace your code for jit or autodiff. Consequently, I don't think we can remove update_index() just by deprecating TensorFlow.

I think with the new information here, I'd suggest:

1. Moving to using tnp.experimental_enable_numpy_behavior() with the TensorFlow version restricted with a check
2. Reenabling the tests that would then function
3. Deprecating mxnet (lots of issues with it)?

Alternatively we could deprecate TensorFlow with a link to an issue to collect feedback? If nobody is using it, then removing it would be easiest and best...

[JeanKossaifi]

I agree, I also prefer the last option.

[aarmey]

We in fact already use the NumPy API in the backend. I think the crux of the issue comes down to Tensorflow (1) not supporting mutation, except through the frustrating update_index interface and (2) not supporting indexing via a list. I agree that skipping tests for specific backends is not ideal; I worry that it is creating many situations in which users will find code that doesn't work for their backend.

Incidentally, we actually skip many more tests due to issues in mxnet. I would propose deprecating that before Tensorflow given its development has stopped.

As for Tensorflow, maybe it is possible to get some of the skipped tests working? In that case I would recommend keeping it. If it is not possible to resolve the cases where we have used list-based indexing, it would be good to ensure there are clear errors for users when they encounter methods using this.

[JeanKossaifi]

Sounds like there are little downsides to using the experimental feature of TF if we are anyway considering deprecating it.

I agree @aarmey I was also reading up on JAX and looks like it's best to keep the update_index - JIT and autodiff are some of the main reasons to use that backend.