cloudera/cloudera-airflow-plugins

cloudera-airflow-plugins's Issues

Tenacity breaking change

I'm currently testing the Cloudera Airlfow plugins on Airflow 2.3.2 which pins tenacity to tenacity>=6.2.

Currently there is an issue with the CDEHook RetryHandler I've discovered. The __call__ function does not correctly handle the updated versions of tenacity that call the retry handler with retrystate parameter. This causes the code to break and error with TypeError: __call__() got an unexpected keyword argument 'retry_state'.

Tenacity updated the iter function of BaseRetrying class.

This seems to block using the CDEOperator for Airflow 2.3.2

Can't install pathlib required dependency

Hello,

I'm having trouble installing pathlib dependency since the 24th of February 2022.
I'm using python 3.8, pip version 20.2.4, setuptools 60.9.3 and wheel 0.37.1.

The error that I have is :

ERROR: Command errored out with exit status 1:
command: /home/vsts/work/1/s/.tox/py38/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-gdfmd4bj/pathlib/setup.py'"'"'; file='"'"'/tmp/pip-install-gdfmd4bj/pathlib/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-mnmahwn7
cwd: /tmp/pip-install-gdfmd4bj/pathlib/
Complete output (19 lines):
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-install-gdfmd4bj/pathlib/setup.py", line 6, in
setup(
File "/home/vsts/work/1/s/.tox/py38/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 109, in setup
_setup_distribution = dist = klass(attrs)
File "/home/vsts/work/1/s/.tox/py38/lib/python3.8/site-packages/setuptools/dist.py", line 456, in init
for ep in metadata.entry_points(group='distutils.setup_keywords'):
File "/home/vsts/work/1/s/.tox/py38/lib/python3.8/site-packages/setuptools/_vendor/importlib_metadata/init.py", line 999, in entry_points
return SelectableGroups.load(eps).select(**params)
File "/home/vsts/work/1/s/.tox/py38/lib/python3.8/site-packages/setuptools/_vendor/importlib_metadata/init.py", line 449, in load
ordered = sorted(eps, key=by_group)
File "/home/vsts/work/1/s/.tox/py38/lib/python3.8/site-packages/setuptools/_vendor/importlib_metadata/init.py", line 997, in
dist.entry_points for dist in unique(distributions())
File "/home/vsts/work/1/s/.tox/py38/lib/python3.8/site-packages/setuptools/_vendor/importlib_metadata/init.py", line 609, in entry_points
return EntryPoints._from_text_for(self.read_text('entry_points.txt'), self)
File "/home/vsts/work/1/s/.tox/py38/lib/python3.8/site-packages/setuptools/_vendor/importlib_metadata/init.py", line 917, in read_text
return self._path.joinpath(filename).read_text(encoding='utf-8')
AttributeError: 'PosixPath' object has no attribute 'read_text'

Is seems pathlib is not installable anymore in my environment.
I got around the problem by manually installing all other dependencies excepting pathlib and installing pathlib2 to be able to use your cloudera-airflow-plugins.

Do you intent to switch to pathlib2 as it is being maintained and pathlib is not anymore ?

Thank you for your answer,

Regards,

Sami

Publish this plugin to a real Python repository

Many users using this plugin should be member of a (quite) big company.

This company is very likely to have a proxy.

Because of this proxy, the installation of this plugin in a local environment could be quite hard.

For example, in my environment, when I run pip install https://github.com/cloudera/cloudera-airflow-plugins/releases/download/v1.0.0/cloudera_airflow_provider-1.0.0-py3-none-any.whl, I fall into an error like

So it could be quite appreciate to upload this plugin to a « real » python repository (e.g. PyPI) to facilitate its integration in local environments.