Also expose `torch` native datapipes in `torchdata.datapipes`

[pytorch]

Also expose `torch` native datapipes in `torchdata.datapipes`

[ejguan]

We added a deprecation warning in PyTorch
https://github.com/pytorch/pytorch/blob/e3af4be96338d0645310e7cab47ce78509a6603e/torch/utils/data/datapipes/iter/tararchivereader.py#L37
We will remove them in the next release after 1.10. Before that, if any user imports DataPipe from torchdata, they should use the one in torchdata rather than the one in pytorch core.

For the list of duplicated DataPipes, we should not expose them from PyTorch core to torchdata but directly use the implementation from torchdata.

[pmeier]

I'll send a PR.

[ejguan]

That's a good idea to have a single source of Truth. And, we need to carefully take care of duplicated DataPipes like TarArchiveReader without overriding them from data repo.

Feel free to send a PR.

[pmeier]

That's a good idea to have a single source of Truth. And, we need to carefully take care of duplicated DataPipes like TarArchiveReader without overriding them from data repo.

The second part makes it sound that there is no single source of truth if torchdata.datapipes diverges from torch.utils.data.datapipes. Looking at TarArchiveReader, I found this

data/torchdata/datapipes/iter/util/tararchivereader.py

Lines 11 to 12 in 77c07bf

	# TODO(VitalyFedyunin): This file copy-pasted from the pytorch repo
	# nuke source class when repo is open-sourced

So what is the plan here? Remove everything from torch.utils.data.datapipes that is also in torchdata.datapipes and import and expose the rest?