datarevenue-berlin/drfs

drfs's Issues

Parent of RemotePath returns broken object

path = RemotePath('memory://root/dir/file.txt')
directory = path.parent
other = directory / 'test.txt'

Results in

Traceback (most recent call last):
  File "<input>", line 3, in <module>
  File "/Users/kayibal/miniconda3/envs/curation/lib/python3.8/pathlib.py", line 926, in __truediv__
    return self._make_child((key,))
  File "/Users/kayibal/code/datarevenue/drfs/drfs/path.py", line 126, in _make_child
    res._storage_options = self._storage_options
AttributeError: 'RemotePath' object has no attribute '_storage_options'

Can't specify root without any directory/bucket name

initializing structure like this creates weird paths:

structure = Structure("adl://")

which is valid for data lake storage seems not to be supported. Seems to be unique to ADL though. So not sure how we should handle it. For now I will just save everything into a predefined folder.

Examples module should not be pip installed

Black format this repo

We should also black format this code base asap and add a check for it in AWS CB.

RemotePath doesn't support colons in filename