eudat-b2handle / pyhandle Goto Github PK
View Code? Open in Web Editor NEWPython library for HANDLE system
Home Page: https://EUDAT-B2HANDLE.github.io/PYHANDLE/
License: Apache License 2.0
Python library for HANDLE system
Home Page: https://EUDAT-B2HANDLE.github.io/PYHANDLE/
License: Apache License 2.0
Dear pyhandle developers,
I encounter issues with the Python requirements in the setup.py python_requires='>=3.6.*<3.11'
when running pipenv install. The requirement is included in the Pipfile.lock, but not correctly and subsequently pyhandle is not installed due to: Ignoring pyhandle: markers 'python_version >= "3.6.*<3.11"' don't match your environment
whereas I'm using Python 3.10.8.
This requirement turns out to be not PEP440 compliant as it should be according to the description.
I think it should become something like this: python_requires='>=3.6, <3.11'
If a credentials JSON file specifies a 'handle_server_url' param, this will be overwritten in the REST client's init method, even if the init method's own handle_server_url param is not set. A simple "if handle_server_url != None" would probably help..?
When trying to install pyhandle 1.1.1 with pip2 it fails with the following error:
Yue are using pip version 8.1.2, however version 22.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting pyhandle
Downloading https://files.pythonhosted.org/packages/c6/c1/f787154f5f9f2ae242c40875f12bc39b9bd98161f286252e77c80982ccbb/pyhandle-1.1.1.tar.gz (76kB)
Complete output from command python setup.py egg_info:
/usr/lib64/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/usr/lib64/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'long_description_content_type'
warnings.warn(msg)
error in pyhandle setup command: 'install_requires' must be a string or list of strings containing valid project/version requirement specifiers
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-kpCUCs/pyhandle/
You are using pip version 8.1.2, however version 22.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
The command '/bin/sh -c pip3 install -r /app/B2SAFE-core/scripts/tests/requirements.txt && pip2 install PyMySQL==0.10.1 && pip2 install pyhandle' returned a non-zero code: 1
We are currently implementing the Handle system for our archives so we can request PID's. However, trying to pip install the package, I discovered that only Python 3.5 and lower are supported, while we are running more modern versions.
In your devel
branch there is support for at least Python 3.7 (Python 3.8 would be really nice). Is it possible to release a new version with these fixes? It would be really appreciated!
This is a bug that can occur when we try to modify a handle record (via REST client), but pass an empty dict of changes.
In that case, pyhandle will try to return a variable that is not assigned, here:
https://github.com/EUDAT-B2HANDLE/PYHANDLE/blob/master/pyhandle/client/resthandleclient.py#L602
When the variable nothingchanges
is True, the variable resp
never gets defined or assigned, so we run into an exception here:
https://github.com/EUDAT-B2HANDLE/PYHANDLE/blob/master/pyhandle/client/resthandleclient.py#L578
pidify_1 | self.__handle_client.modify_handle_value(handle, **record_kv)
pidify_1 | File "/usr/local/lib/python3.12/site-packages/pyhandle/client/resthandleclient.py", line 490, in modify_handle_value
pidify_1 | return self.__handle_modification(handle, ttl, add_if_not_exist, overwrite, **kvpairs)
pidify_1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pidify_1 | File "/usr/local/lib/python3.12/site-packages/pyhandle/client/resthandleclient.py", line 602, in __handle_modification
pidify_1 | return json.loads(decoded_response(resp))['handle']
pidify_1 | ^^^^
pidify_1 | UnboundLocalError: cannot access local variable 'resp' where it is not associated with a value
I introduced indices
to allow passing several handle record indices.
But now we allow passing various options which are appended to the HTTP GET
request (e.g. type
, which results in ?type=bla
being appended to the Handle Server URL). So why should we not let users directly use index
for indices?
In case of type
, we allow passing lists. So, type=['bla', 'bli']
will result in ?type=bla&type=bli
. We could do the same with index
.
Opinions?
If I instantiate pyhandle using JSON creds file, it needs me to specify the client inside it, and also in the script (PyHandleClient('rest')
):
creds = pyhandle.clientcredentials.PIDClientCredentials.load_from_JSON(credentials_file)
client = pyhandle.handleclient.PyHandleClient('rest').instantiate_with_credentials(creds, HTTPS_verify=False)
Not passing it in the script (PyHandleClient()
) fails, even if it is in the JSON file:
# client = pyhandle.handleclient.PyHandleClient().instantiate_with_credentials(creds, HTTPS_verify=False)
Traceback (most recent call last):
File "check_api_interaction_create.py", line 36, in <module>
client = pyhandle.handleclient.PyHandleClient().instantiate_with_credentials(
TypeError: __init__() missing 1 required positional argument: 'client'
If client not provided in JSON, but it's provided in the script (PyHandleClient('rest')
):
File "/foo/venv3/lib/python3.6/site-packages/pyhandle-1.0.4-py3.6.egg/pyhandle/clientcredentials.py", line 179, in __init__
raise CredentialsFormatError(msg=msg)
pyhandle.handleexceptions.CredentialsFormatError: Problem with credentials: Client not provided or empty.
If the client provided, but inconsistent - JSON says "db", init says "rest":
raise CredentialsFormatError(msg=msg)
pyhandle.handleexceptions.CredentialsFormatError: Problem with credentials: (['db_host', 'db_user', 'db_password', 'db_name']) are missing or empty.
Hi,
Are there any plans to support Python 3.7? Your setup.py indicates the following Python versions are supported: python_requires='>=2.6,!=3.0.,!=3.1.,!=3.2.,!=3.3.,!=3.4.*,<3.6',
Is there anything holding back support for newer Python versions?
I'm working on a python script to upload handle identifiers to SURFsara servers:
https://github.com/Rijksmuseum/handle_upload
At the moment I'm using Python 2.7, which works ok, but I would prefer to migrate to 3.7 at some time.
Example:
result = client.retrieve_handle_record_json("hdl:123/456")
This causes an exception because it does not match the expected syntax (index:prefix/suffix). While this is correct, use of the 'hdl:' prefix (or doi:) is sort of legitimate and is also accepted by the main proxies (hdl.handle.net and doi.org). Thus, all methods accepting Handle names should intercept and remove 'hdl:' and 'doi:' before trying an int conversion.
Currently README recommends running the tests via python setup.py test
, but this gives a deprecated warning (in python 3.6.8), so we should recommend a different command. See error message below.
(venv3) [m@localhost PYHANDLE]$ python setup.py test
/home/merret/work/repos_and_scripts/pyhandle/PYHANDLE/pyhandle/venv3/lib/python3.6/site-packages/setuptools/dist.py:493: UserWarning: Normalizing '1.0.5-dev' to '1.0.5.dev0'
warnings.warn(tmpl.format(**locals()))
running test
WARNING: Testing via this command is deprecated and will be removed in a future version. Users looking for a generic test entry point independent of test runner are encouraged to use tox.
/home/merret/work/repos_and_scripts/pyhandle/PYHANDLE/pyhandle/venv3/lib/python3.6/site-packages/setuptools/installer.py:30: SetuptoolsDeprecationWarning: setuptools.installer is deprecated. Requirements should be satisfied by a PEP 517 installer.
SetuptoolsDeprecationWarning,
The section in the docs here
https://eudat-b2safe.github.io/PYHANDLE/pyhandleclientrest.html#general-usage
explains to the first-time user how to get a PyHandle client instance. It is however not clear from where to import PyHandleClient.
E.g., just add a line
from pyhandle.handleclient import PyHandleClient
When I run a script using pyhandle that loads the credentials info from a JSON file that sits in a subdirectory, where also private key and certificate are sitting, I cannot load them using relative paths.
Files:
workingdir/
workingdir/script.py
workingdir/credentials/
workingdir/credentials/creds.json
workingdir/credentials/privkey.pem
workingdir/credentials/clientcert.pem
Contents of the creds.json
:
{
"private_key": "./privkey.pem",
"certificate_only": "./clientcert.pem",
"prefix": "21.xxx",
"handleowner": "200:0.NA/21.xxx",
"HTTPS_verify": "./xxx_bundle.crt",
"client": "rest",
"handle_server_url": "xxx"
}
# Paths of key+cert are relative to JSON file!
And I want to do this:
creds = pyhandle.clientcredentials.PIDClientCredentials.load_from_JSON('./credentials/creds.json')
client = pyhandle.handleclient.PyHandleClient('rest').instantiate_with_credentials(creds, HTTPS_verify=False)
But I get an error:
pyhandle.handleexceptions.CredentialsFormatError: Problem with credentials: The certificate file was not found at the specified path: ./clientcert.pemThe private key file was not found at the specified path: ./privkey.pem.
If I add the subdirectory to the JSON file:
{
"private_key": "./credentials/privkey.pem",
"certificate_only": "./credentials/clientcert.pem",
"prefix": "21.xxx",
"handleowner": "200:0.NA/21.xxx",
"HTTPS_verify": "./xxx_bundle.crt",
"client": "rest",
"handle_server_url": "xxx"
}
# Paths of key+cert are relative to script!
Then I also get an error:
pyhandle.handleexceptions.CredentialsFormatError: Problem with credentials: (certficate file): The file was not found at the specified path: /workingdir/credentials/credentials/clientcert.pem.
Now the subdirectory is added twice!
No way out!
Hi,
I tried to build an rpm from the code. So we can install an rpm. On Centos 6.8 (python 2.6) it gives:
$ python setup.py bdist_rpm
/usr/lib64/python2.6/distutils/dist.py:266: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
running bdist_rpm
running egg_info
creating pyhandle.egg-info
writing requirements to pyhandle.egg-info/requires.txt
writing pyhandle.egg-info/PKG-INFO
writing top-level names to pyhandle.egg-info/top_level.txt
writing dependency_links to pyhandle.egg-info/dependency_links.txt
writing manifest file 'pyhandle.egg-info/SOURCES.txt'
reading manifest file 'pyhandle.egg-info/SOURCES.txt'
writing manifest file 'pyhandle.egg-info/SOURCES.txt'
creating build
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/rpm
creating build/bdist.linux-x86_64/rpm/SOURCES
creating build/bdist.linux-x86_64/rpm/SPECS
creating build/bdist.linux-x86_64/rpm/BUILD
creating build/bdist.linux-x86_64/rpm/RPMS
creating build/bdist.linux-x86_64/rpm/SRPMS
writing 'build/bdist.linux-x86_64/rpm/SPECS/pyhandle.spec'
running sdist
warning: sdist: standard file not found: should have one of README, README.txt
warning: sdist: missing required meta-data: url
warning: sdist: missing meta-data: either (author and author_email) or (maintainer and maintainer_email) must be supplied
creating pyhandle-1.0.0
creating pyhandle-1.0.0/pyhandle
creating pyhandle-1.0.0/pyhandle.egg-info
creating pyhandle-1.0.0/pyhandle/tests
creating pyhandle-1.0.0/pyhandle/tests/testcases
making hard links in pyhandle-1.0.0...
hard linking setup.cfg -> pyhandle-1.0.0
hard linking setup.py -> pyhandle-1.0.0
hard linking pyhandle/__init__.py -> pyhandle-1.0.0/pyhandle
hard linking pyhandle/clientcredentials.py -> pyhandle-1.0.0/pyhandle
hard linking pyhandle/compatibility_helper.py -> pyhandle-1.0.0/pyhandle
hard linking pyhandle/dbhsexceptions.py -> pyhandle-1.0.0/pyhandle
hard linking pyhandle/handleclient.py -> pyhandle-1.0.0/pyhandle
hard linking pyhandle/handleexceptions.py -> pyhandle-1.0.0/pyhandle
hard linking pyhandle/handlesystemconnector.py -> pyhandle-1.0.0/pyhandle
hard linking pyhandle/hsresponses.py -> pyhandle-1.0.0/pyhandle
hard linking pyhandle/searcher.py -> pyhandle-1.0.0/pyhandle
hard linking pyhandle/utilhandle.py -> pyhandle-1.0.0/pyhandle
hard linking pyhandle.egg-info/PKG-INFO -> pyhandle-1.0.0/pyhandle.egg-info
hard linking pyhandle.egg-info/SOURCES.txt -> pyhandle-1.0.0/pyhandle.egg-info
hard linking pyhandle.egg-info/dependency_links.txt -> pyhandle-1.0.0/pyhandle.egg-info
hard linking pyhandle.egg-info/not-zip-safe -> pyhandle-1.0.0/pyhandle.egg-info
hard linking pyhandle.egg-info/requires.txt -> pyhandle-1.0.0/pyhandle.egg-info
hard linking pyhandle.egg-info/top_level.txt -> pyhandle-1.0.0/pyhandle.egg-info
hard linking pyhandle/tests/__init__.py -> pyhandle-1.0.0/pyhandle/tests
hard linking pyhandle/tests/main_test_script.py -> pyhandle-1.0.0/pyhandle/tests
hard linking pyhandle/tests/mockresponses.py -> pyhandle-1.0.0/pyhandle/tests
hard linking pyhandle/tests/test_script.py -> pyhandle-1.0.0/pyhandle/tests
hard linking pyhandle/tests/utilities.py -> pyhandle-1.0.0/pyhandle/tests
hard linking pyhandle/tests/testcases/__init__.py -> pyhandle-1.0.0/pyhandle/tests/testcases
hard linking pyhandle/tests/testcases/clientcredentials_unit_test.py -> pyhandle-1.0.0/pyhandle/tests/testcases
hard linking pyhandle/tests/testcases/handleclient_2_read_patched_unit_test.py -> pyhandle-1.0.0/pyhandle/tests/testcases
hard linking pyhandle/tests/testcases/handleclient_read_integration_test.py -> pyhandle-1.0.0/pyhandle/tests/testcases
hard linking pyhandle/tests/testcases/handleclient_read_patched_unit_test.py -> pyhandle-1.0.0/pyhandle/tests/testcases
hard linking pyhandle/tests/testcases/handleclient_search_integration_test.py -> pyhandle-1.0.0/pyhandle/tests/testcases
hard linking pyhandle/tests/testcases/handleclient_search_unit_test.py -> pyhandle-1.0.0/pyhandle/tests/testcases
hard linking pyhandle/tests/testcases/handleclient_unit_test.py -> pyhandle-1.0.0/pyhandle/tests/testcases
hard linking pyhandle/tests/testcases/handleclient_write_integration_test.py -> pyhandle-1.0.0/pyhandle/tests/testcases
hard linking pyhandle/tests/testcases/handleclient_write_patched_unit_test.py -> pyhandle-1.0.0/pyhandle/tests/testcases
hard linking pyhandle/tests/testcases/handleconnector_patched_unit_test.py -> pyhandle-1.0.0/pyhandle/tests/testcases
hard linking pyhandle/tests/testcases/handleconnector_unit_test.py -> pyhandle-1.0.0/pyhandle/tests/testcases
hard linking pyhandle/tests/testcases/utilconfig_unit_test.py -> pyhandle-1.0.0/pyhandle/tests/testcases
copying setup.cfg -> pyhandle-1.0.0
Writing pyhandle-1.0.0/setup.cfg
creating dist
tar -cf dist/pyhandle-1.0.0.tar pyhandle-1.0.0
gzip -f9 dist/pyhandle-1.0.0.tar
tar -cf dist/pyhandle-1.0.0.tar pyhandle-1.0.0
gzip -f9 dist/pyhandle-1.0.0.tar
removing 'pyhandle-1.0.0' (and everything under it)
copying dist/pyhandle-1.0.0.tar.gz -> build/bdist.linux-x86_64/rpm/SOURCES
building RPMs
rpmbuild -ba --define _topdir /home/robertv/git/PYHANDLE/build/bdist.linux-x86_64/rpm --clean build/bdist.linux-x86_64/rpm/SPECS/pyhandle.spec
Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.LsIirR
+ umask 022
+ cd /home/robertv/git/PYHANDLE/build/bdist.linux-x86_64/rpm/BUILD
+ LANG=C
+ export LANG
+ unset DISPLAY
+ cd /home/robertv/git/PYHANDLE/build/bdist.linux-x86_64/rpm/BUILD
+ rm -rf pyhandle-1.0.0
+ /bin/tar -xvvf -
+ /usr/bin/gzip -dc /home/<user>/git/PYHANDLE/build/bdist.linux-x86_64/rpm/SOURCES/pyhandle-1.0.0.tar.gz
...
-rw-rw-r-- <user>/<user> 774 2017-05-10 12:22 pyhandle-1.0.0/PKG-INFO
+ STATUS=0
+ '[' 0 -ne 0 ']'
+ cd pyhandle-1.0.0
+ /bin/chmod -Rf a+rX,u+w,g-w,o-w .
+ exit 0
Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.lckHdK
+ umask 022
+ cd /home/<user>/git/PYHANDLE/build/bdist.linux-x86_64/rpm/BUILD
+ cd pyhandle-1.0.0
+ LANG=C
+ export LANG
+ unset DISPLAY
+ python setup.py build
Traceback (most recent call last):
File "setup.py", line 67, in <module>
long_description=open('README.md').read()
IOError: [Errno 2] No such file or directory: 'README.md'
error: Bad exit status from /var/tmp/rpm-tmp.lckHdK (%build)
RPM build errors:
Bad exit status from /var/tmp/rpm-tmp.lckHdK (%build)
error: command 'rpmbuild' failed with exit status 1
What are we doing wrong?
The HandleReverseLookupServlet has a default limit for queries of 1000 (only returns max. 1000 Handles per query). This can be overridden with a limit URL parameter. The search_handles method should also support this parameter explicitly.
Current workaround is: introduce 'limit' as an allowed_search_key and then set it to e.g. "100000" (as string).
Seems to be a duplicate of handleclient.py but with less code. Is this a leftover from earlier implementation attempts?
../../../../../../../usr/local/lib/python3.10/site-packages/future-0.18.2-py3.10.egg/past/builtins/misc.py:45
/usr/local/lib/python3.10/site-packages/future-0.18.2-py3.10.egg/past/builtins/misc.py:45: DeprecationWarning: the imp module is deprecated in favour of importlib and slated for removal in Python 3.12; see the module's documentation for alternative uses
from imp import reload
../client/resthandleclient.py:157
/home/jenkins/workspace/B2HANDLE-ORG_PYHANDLE_PR-61@5/PYHANDLE/pyhandle/client/resthandleclient.py:157: DeprecationWarning: invalid escape sequence '*'
'''
../client/resthandleclient.py:174
/home/jenkins/workspace/B2HANDLE-ORG_PYHANDLE_PR-61@5/PYHANDLE/pyhandle/client/resthandleclient.py:174: DeprecationWarning: invalid escape sequence '*'
'''
../client/resthandleclient.py:203
/home/jenkins/workspace/B2HANDLE-ORG_PYHANDLE_PR-61@5/PYHANDLE/pyhandle/client/resthandleclient.py:203: DeprecationWarning: invalid escape sequence '*'
'''
../client/resthandleclient.py:227
/home/jenkins/workspace/B2HANDLE-ORG_PYHANDLE_PR-61@5/PYHANDLE/pyhandle/client/resthandleclient.py:227: DeprecationWarning: invalid escape sequence '*'
'''
pyhandle/tests/main_test_script.py::RESTHandleClientReadaccessFakedTestCase::test_get_value_from_handle_normal
/home/jenkins/workspace/B2HANDLE-ORG_PYHANDLE_PR-61@5/PYHANDLE/pyhandle/tests/testcases/handleclient_read_patched_unit_test.py:42: DeprecationWarning: Please use assertEqual instead.
self.assertEquals(val, 'val1',
pyhandle/tests/main_test_script.py::RESTHandleClientSearchNoAccessTestCase::test_create_revlookup_query_forbiddenkeys
/home/jenkins/workspace/B2HANDLE-ORG_PYHANDLE_PR-61@5/PYHANDLE/pyhandle/tests/testcases/handleclient_search_unit_test.py:51: DeprecationWarning: Please use assertRaisesRegex instead.
with self.assertRaisesRegexp(ReverseLookupException, 'Cannot search for key[.]*'):
pyhandle/tests/main_test_script.py::RESTHandleClientSearchNoAccessTestCase::test_create_revlookup_query_fulltext
/home/jenkins/workspace/B2HANDLE-ORG_PYHANDLE_PR-61@5/PYHANDLE/pyhandle/tests/testcases/handleclient_search_unit_test.py:47: DeprecationWarning: Please use assertRaisesRegex instead.
with self.assertRaisesRegexp(ReverseLookupException, 'Full-text search is not implemented yet[.]*'):
pyhandle/tests/main_test_script.py::RESTHandleClientSearchNoAccessTestCase::test_create_revlookup_query_noterms
/home/jenkins/workspace/B2HANDLE-ORG_PYHANDLE_PR-61@5/PYHANDLE/pyhandle/tests/testcases/handleclient_search_unit_test.py:55: DeprecationWarning: Please use assertRaisesRegex instead.
with self.assertRaisesRegexp(ReverseLookupException, 'No search terms have been specified[.]*'):
pyhandle/tests/main_test_script.py::RESTHandleClientSearchNoAccessTestCase::test_search_handle_for_forbiddenkeys
/home/jenkins/workspace/B2HANDLE-ORG_PYHANDLE_PR-61@5/PYHANDLE/pyhandle/tests/testcases/handleclient_search_unit_test.py:31: DeprecationWarning: Please use assertRaisesRegex instead.
with self.assertRaisesRegexp(ReverseLookupException, 'Cannot search for key[.]*'):
pyhandle/tests/main_test_script.py::RESTHandleClientSearchNoAccessTestCase::test_search_handle_for_fulltext
/home/jenkins/workspace/B2HANDLE-ORG_PYHANDLE_PR-61@5/PYHANDLE/pyhandle/tests/testcases/handleclient_search_unit_test.py:37: DeprecationWarning: Please use assertRaisesRegex instead.
with self.assertRaisesRegexp(ReverseLookupException, 'Full-text search is not implemented yet[.]*'):
pyhandle/tests/main_test_script.py::RESTHandleClientSearchNoAccessTestCase::test_search_handle_noterms
/home/jenkins/workspace/B2HANDLE-ORG_PYHANDLE_PR-61@5/PYHANDLE/pyhandle/tests/testcases/handleclient_search_unit_test.py:43: DeprecationWarning: Please use assertRaisesRegex instead.
with self.assertRaisesRegexp(ReverseLookupException, 'No search terms have been specified[.]*'):
pyhandle/tests/main_test_script.py::RESTHandleClientNoaccessTestCase::test_create_authentication_string
/home/jenkins/workspace/B2HANDLE-ORG_PYHANDLE_PR-61@5/PYHANDLE/pyhandle/tests/testcases/handleclient_unit_test.py:178: DeprecationWarning: Please use assertEqual instead.
self.assertEquals(expected, auth,
Logging in INFO level is way too verbose. I keep adding logging.getLogger('pyhandle').setLevel(logging.WARNING)
too all my code.
When using http_verify=True in python2, I get
(vi_3.3)[merret@centos]$ python modify_single_pid.py -v
2020-09-04 21:35:17,948 - __main__ - INFO - Will modify this handle: 21.14101/TESTTESTTEST
2020-09-04 21:35:17,948 - __main__ - INFO - Handle record will contain these values: {'TEST': 'TRUEE', 'SOMEFLAG': 'HIHI', 'CREATED': '20200904', 'DESCRIPTION': 'Apple pie', 'CONTACT': '[email protected]', 'DELETEME': 'DO IT'}
Traceback (most recent call last):
File "modify_single_pid.py", line 137, in <module>
creds, HTTPS_verify=https_verify)
File "/home/merret/vi_3.3/lib/python3.3/site-packages/pyhandle/handleclient.py", line 68, in instantiate_with_credentials
return self.handle_client.instantiate_with_credentials(credentials, **config)
File "/home/merret/vi_3.3/lib/python3.3/site-packages/pyhandle/client/resthandleclient.py", line 243, in instantiate_with_credentials
inst = RESTHandleClient(**key_value_pairs)
File "/home/merret/vi_3.3/lib/python3.3/site-packages/pyhandle/client/resthandleclient.py", line 110, in __init__
self.__handlesystemconnector = HandleSystemConnector(handleclient=self, **args)
File "/home/merret/vi_3.3/lib/python3.3/site-packages/pyhandle/handlesystemconnector.py", line 83, in __init__
self.__setup_for_writeaccess(args)
File "/home/merret/vi_3.3/lib/python3.3/site-packages/pyhandle/handlesystemconnector.py", line 164, in __setup_for_writeaccess
self.__setup_for_auth_by_user_and_pw()
File "/home/merret/vi_3.3/lib/python3.3/site-packages/pyhandle/handlesystemconnector.py", line 177, in __setup_for_auth_by_user_and_pw
self.check_if_username_exists(self.__username)
File "/home/merret/vi_3.3/lib/python3.3/site-packages/pyhandle/handlesystemconnector.py", line 463, in check_if_username_exists
resp = self.send_handle_get_request(handle)
File "/home/merret/vi_3.3/lib/python3.3/site-packages/pyhandle/handlesystemconnector.py", line 278, in send_handle_get_request
resp = self.__session.get(url, headers=head, verify=veri)
File "/home/merret/vi_3.3/lib/python3.3/site-packages/requests/sessions.py", line 543, in get
return self.request('GET', url, **kwargs)
File "/home/merret/vi_3.3/lib/python3.3/site-packages/requests/sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
File "/home/merret/vi_3.3/lib/python3.3/site-packages/requests/sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
File "/home/merret/vi_3.3/lib/python3.3/site-packages/requests/adapters.py", line 449, in send
timeout=timeout
File "/home/merret/vi_3.3/lib/python3.3/site-packages/urllib3/connectionpool.py", line 677, in urlopen
chunked=chunked,
File "/home/merret/vi_3.3/lib/python3.3/site-packages/urllib3/connectionpool.py", line 381, in _make_request
self._validate_conn(conn)
File "/home/merret/vi_3.3/lib/python3.3/site-packages/urllib3/connectionpool.py", line 978, in _validate_conn
conn.connect()
File "/home/merret/vi_3.3/lib/python3.3/site-packages/urllib3/connection.py", line 371, in connect
ssl_context=context,
File "/home/merret/vi_3.3/lib/python3.3/site-packages/urllib3/util/ssl_.py", line 350, in ssl_wrap_socket
context.load_verify_locations(ca_certs, ca_cert_dir, ca_cert_data)
TypeError: load_verify_locations() takes at most 2 arguments (3 given)
Using the same script in python2, all I get is this:
Traceback (most recent call last):
File "modify_single_pid.py", line 137, in <module>
creds, HTTPS_verify=https_verify)
File "/home/merret/work/githubrepos/ESGF/fixerscript/handletools/cmip6tools/venv2/lib/python2.7/site-packages/pyhandle/handleclient.py", line 68, in instantiate_with_credentials
return self.handle_client.instantiate_with_credentials(credentials, **config)
File "/home/merret/work/githubrepos/ESGF/fixerscript/handletools/cmip6tools/venv2/lib/python2.7/site-packages/pyhandle/client/resthandleclient.py", line 243, in instantiate_with_credentials
inst = RESTHandleClient(**key_value_pairs)
File "/home/merret/work/githubrepos/ESGF/fixerscript/handletools/cmip6tools/venv2/lib/python2.7/site-packages/pyhandle/client/resthandleclient.py", line 110, in __init__
self.__handlesystemconnector = HandleSystemConnector(handleclient=self, **args)
File "/home/merret/work/githubrepos/ESGF/fixerscript/handletools/cmip6tools/venv2/lib/python2.7/site-packages/pyhandle/handlesystemconnector.py", line 83, in __init__
self.__setup_for_writeaccess(args)
File "/home/merret/work/githubrepos/ESGF/fixerscript/handletools/cmip6tools/venv2/lib/python2.7/site-packages/pyhandle/handlesystemconnector.py", line 164, in __setup_for_writeaccess
self.__setup_for_auth_by_user_and_pw()
File "/home/merret/work/githubrepos/ESGF/fixerscript/handletools/cmip6tools/venv2/lib/python2.7/site-packages/pyhandle/handlesystemconnector.py", line 177, in __setup_for_auth_by_user_and_pw
self.check_if_username_exists(self.__username)
File "/home/merret/work/githubrepos/ESGF/fixerscript/handletools/cmip6tools/venv2/lib/python2.7/site-packages/pyhandle/handlesystemconnector.py", line 463, in check_if_username_exists
resp = self.send_handle_get_request(handle)
File "/home/merret/work/githubrepos/ESGF/fixerscript/handletools/cmip6tools/venv2/lib/python2.7/site-packages/pyhandle/handlesystemconnector.py", line 278, in send_handle_get_request
resp = self.__session.get(url, headers=head, verify=veri)
File "/home/merret/work/githubrepos/ESGF/fixerscript/handletools/cmip6tools/venv2/lib/python2.7/site-packages/requests/sessions.py", line 543, in get
return self.request('GET', url, **kwargs)
File "/home/merret/work/githubrepos/ESGF/fixerscript/handletools/cmip6tools/venv2/lib/python2.7/site-packages/requests/sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
File "/home/merret/work/githubrepos/ESGF/fixerscript/handletools/cmip6tools/venv2/lib/python2.7/site-packages/requests/sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
File "/home/merret/work/githubrepos/ESGF/fixerscript/handletools/cmip6tools/venv2/lib/python2.7/site-packages/requests/adapters.py", line 514, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='pid1.dkrz.de', port=8001): Max retries exceeded with url: /api/handles/21.14101/ADMINLIST (Caused by SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)'),))
modify_handle_value
(https://github.com/EUDAT-B2HANDLE/PYHANDLE/blob/master/pyhandle/client/resthandleclient.py#L389) does not return anything if successful.delete_handle_value
(https://github.com/EUDAT-B2HANDLE/PYHANDLE/blob/master/pyhandle/client/resthandleclient.py#L502) and delete_handle
register_handle_json
returns the handle: https://github.com/EUDAT-B2HANDLE/PYHANDLE/blob/master/pyhandle/client/resthandleclient.py#L600 , same for __handle_registering
and all those wrapping it.client = PyHandleClient("rest").instantiate_for_read_access(handle_server_url="https://handle.dkrz.de:8003")
INFO:pyhandle.client.resthandleclient:Instantiating RESTHandleClient at 2018-04-13_14:00
INFO:pyhandle.handlesystemconnector:Instantiating HandleSystemConnector
INFO:pyhandle.handlesystemconnector: - handle_server_url set to default: https://hdl.handle.net
INFO:pyhandle.handlesystemconnector: - url_extension_REST_API set to default: /api/handles/
INFO:pyhandle.handlesystemconnector: - https_verify set to default: True
INFO:pyhandle.searcher:Instantiating Searcher
INFO:pyhandle.searcher: - https_verify set to default: True
INFO:pyhandle.searcher: - allowed_search_keys set to default: ['URL', 'CHECKSUM']
INFO:pyhandle.searcher: - solrbaseurl: No default.
INFO:pyhandle.searcher: - reverselookup_url_extension set to default: /hrls/handles/
INFO:pyhandle.searcher: - reverselookup_username: Not specified. No default.
INFO:pyhandle.searcher: - reverselookup_password: Not specified. No default.
INFO:pyhandle.searcher:Reverse lookup not possible. Neither username nor password were provided.
INFO:pyhandle.searcher:Reverse lookup not possible. No URL for reverse lookup provided.
INFO:pyhandle.client.resthandleclient: - HS_ADMIN_permissions set to default: 011111110011
INFO:pyhandle.client.resthandleclient: - modify_HS_ADMIN set to default: False
INFO:pyhandle.client.resthandleclient: - handleowner: Will be set to default for each created handle separately.
INFO:pyhandle.client.resthandleclient:Instantiating RESTHandleClient at 2018-04-13_14:00
INFO:pyhandle.handlesystemconnector:Instantiating HandleSystemConnector
Traceback (most recent call last):
File "", line 1, in
File "/home/tobias/virtualenv/lib/python2.7/site-packages/pyhandle/handleclient.py", line 65, in instantiate_for_read_access
return self.handle_client.instantiate_for_read_access(config)
File "/home/tobias/virtualenv/lib/python2.7/site-packages/pyhandle/client/resthandleclient.py", line 168, in instantiate_for_read_access
inst = RESTHandleClient(handle_server_url, **config)
File "/home/tobias/virtualenv/lib/python2.7/site-packages/pyhandle/client/resthandleclient.py", line 110, in init
self.__handlesystemconnector = HandleSystemConnector(handleclient=self, **args)
File "/home/tobias/virtualenv/lib/python2.7/site-packages/pyhandle/handlesystemconnector.py", line 79, in init
self.__store_args_or_set_to_defaults(args, defaults)
File "/home/tobias/virtualenv/lib/python2.7/site-packages/pyhandle/handlesystemconnector.py", line 97, in __store_args_or_set_to_defaults
LOGGER.info(' - handle_server_url set to '+self.__handle_server_url)
TypeError: cannot concatenate 'str' and 'dict' objects
Sometimes I need the option ?auth
in GET requests to make sure the primary server is requested. I cannot pass that.
We get the following error when trying to install the latest version:
$ pip3 install --user pyhandle
Collecting pyhandle
Using cached https://files.pythonhosted.org/packages/4f/5e/e959a063a8746da34aa860700217398da6061e40e04dea4b366c0428500f/pyhandle-1.2.1-py2.py3-none-any.whl
Collecting pymysql==0.8.0; python_version < "3.11.0" (from pyhandle)
Using cached https://files.pythonhosted.org/packages/e5/07/c0f249aa0b7b0517b5843eeab689b9ccc6a6bb0536fc9d95e65901e6f2ac/PyMySQL-0.8.0-py2.py3-none-any.whl
Collecting requests==2.28.1; python_version >= "3.6.0" (from pyhandle)
Could not find a version that satisfies the requirement requests==2.28.1; python_version >= "3.6.0" (from pyhandle) (from versions: 0.2.0, 0.2.1, 0.2.2, 0.2.3, 0.2.4, 0.3.0, 0.3.1, 0.3.2, 0.3.3, 0.3.4, 0.4.0, 0.4.1, 0.5.0, 0.5.1, 0.6.0, 0.6.1, 0.6.2, 0.6.3, 0.6.4, 0.6.5, 0.6.6, 0.7.0, 0.7.1, 0.7.2, 0.7.3, 0.7.4, 0.7.5, 0.7.6, 0.8.0, 0.8.1, 0.8.2, 0.8.3, 0.8.4, 0.8.5, 0.8.6, 0.8.7, 0.8.8, 0.8.9, 0.9.0, 0.9.1, 0.9.2, 0.9.3, 0.10.0, 0.10.1, 0.10.2, 0.10.3, 0.10.4, 0.10.6, 0.10.7, 0.10.8, 0.11.1, 0.11.2, 0.12.0, 0.12.1, 0.13.0, 0.13.1, 0.13.2, 0.13.3, 0.13.4, 0.13.5, 0.13.6, 0.13.7, 0.13.8, 0.13.9, 0.14.0, 0.14.1, 0.14.2, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.1.0, 1.2.0, 1.2.1, 1.2.2, 1.2.3, 2.0.0, 2.0.1, 2.1.0, 2.2.0, 2.2.1, 2.3.0, 2.4.0, 2.4.1, 2.4.2, 2.4.3, 2.5.0, 2.5.1, 2.5.2, 2.5.3, 2.6.0, 2.6.1, 2.6.2, 2.7.0, 2.8.0, 2.8.1, 2.9.0, 2.9.1, 2.9.2, 2.10.0, 2.11.0, 2.11.1, 2.12.0, 2.12.1, 2.12.2, 2.12.3, 2.12.4, 2.12.5, 2.13.0, 2.14.0, 2.14.1, 2.14.2, 2.15.1, 2.16.0, 2.16.1, 2.16.2, 2.16.3, 2.16.4, 2.16.5, 2.17.0, 2.17.1, 2.17.2, 2.17.3, 2.18.0, 2.18.1, 2.18.2, 2.18.3, 2.18.4, 2.19.0, 2.19.1, 2.20.0, 2.20.1, 2.21.0, 2.22.0, 2.23.0, 2.24.0, 2.25.0, 2.25.1, 2.26.0, 2.27.0, 2.27.1)
No matching distribution found for requests==2.28.1; python_version >= "3.6.0" (from pyhandle)
A lower version installs:
$ pip3 install --user pyhandle==1.2.0
Collecting pyhandle==1.2.0
Downloading https://files.pythonhosted.org/packages/30/e0/6c07d215d8b13fb330e45eea9bd8b9e6d705af64a7e4211447faa44aa5ed/pyhandle-1.2.0.tar.gz (76kB)
100% |████████████████████████████████| 81kB 11.2MB/s
Collecting requests==2.23.0 (from pyhandle==1.2.0)
Downloading https://files.pythonhosted.org/packages/1a/70/1935c770cb3be6e3a8b78ced23d7e0f3b187f5cbfab4749523ed65d7c9b1/requests-2.23.0-py2.py3-none-any.whl (58kB)
100% |████████████████████████████████| 61kB 13.0MB/s
Collecting datetime (from pyhandle==1.2.0)
Using cached https://files.pythonhosted.org/packages/30/ff/03e2b71fbcdd6a084c2158cf2ecb0dbdde3cac0d974cf0815fcae79f6568/DateTime-5.0-py3-none-any.whl
Requirement already satisfied: future in /usr/lib/python3.6/site-packages (from pyhandle==1.2.0)
Requirement already satisfied: six in /usr/lib/python3.6/site-packages (from pyhandle==1.2.0)
Collecting pymysql==0.8.0 (from pyhandle==1.2.0)
Using cached https://files.pythonhosted.org/packages/e5/07/c0f249aa0b7b0517b5843eeab689b9ccc6a6bb0536fc9d95e65901e6f2ac/PyMySQL-0.8.0-py2.py3-none-any.whl
Requirement already satisfied: chardet<4,>=3.0.2 in /usr/lib/python3.6/site-packages (from requests==2.23.0->pyhandle==1.2.0)
Collecting certifi>=2017.4.17 (from requests==2.23.0->pyhandle==1.2.0)
Downloading https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl (158kB)
100% |████████████████████████████████| 163kB 8.7MB/s
Requirement already satisfied: idna<3,>=2.5 in /usr/lib/python3.6/site-packages (from requests==2.23.0->pyhandle==1.2.0)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/lib/python3.6/site-packages (from requests==2.23.0->pyhandle==1.2.0)
Collecting zope.interface (from datetime->pyhandle==1.2.0)
Downloading https://files.pythonhosted.org/packages/d0/75/c80af74b361cedd35bdfb45ade9c22320d3431722186ff8ab1614df80059/zope.interface-5.5.2-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (253kB)
100% |████████████████████████████████| 256kB 5.8MB/s
Requirement already satisfied: pytz in ./.local/lib/python3.6/site-packages (from datetime->pyhandle==1.2.0)
Requirement already satisfied: setuptools in /usr/lib/python3.6/site-packages (from zope.interface->datetime->pyhandle==1.2.0)
Installing collected packages: certifi, requests, zope.interface, datetime, pymysql, pyhandle
Running setup.py install for pyhandle ... done
Successfully installed certifi-2023.7.22 datetime-5.0 pyhandle-1.2.0 pymysql-0.8.0 requests-2.23.0 zope.interface-5.5.2
On this version of OS 3.6 is the default python version.
When I call the method check_handle_syntax(string)
with the handle hdl:21.140x/bla
I get this error message:
pyhandle.handleexceptions.HandleSyntaxError: Handle hdl:21.1410x/bla does not have expected syntax: Index is not an integer.
Expected: index:prefix/suffix
But hdl:21.140x/bla
is a perfectly valid handle. At least, the error message should be more helpful.
Because of the colon :
, the method assumes we're trying to pass a single field, i.e. with an index in front. That does not make sense to me, as we're checking the syntax of a handle.
Where?
Because the pyhandle package locks requests on version 2.23.0 (not higher, not lower), it is uninstallable together with some other packages in our project that demand newer versions of requests.
Is there a reason that setup.py locks it on this version?
We are currently migrating our servers and would like to upgrade our Python versions to one of the latest versions. Currently PyHandle is our bottleneck. Do you have any indication of when we can expect a new release that is compatible with Python 3.9 or higher?
One further issue may be that if private key authentication is to be used, it is logically nto required to provide a password in the credentials file. However, omitting a password will cause the credentials file read method to throw an error, complaining that without password, searchign will not be possible.
Suggested solution: modify credentials JSON read method so that possibility for searches is not checked if a specific flag parameter is given. Or make a subclass that is specific for the REST mode and only there the error will be thrown.
Text from #13
The PIDClientCredentials is a good class to conveniently cover all possible authentication cases and get the information either through function calls or from a JSON file.
The batch client has however its own methods. To use both clients transparently, the batch client should be able to work with PIDClientCredential instances as well (e.g. via a generic authenticate() method that takes a PIDClientCredentials instance as only parameter).
/usr/local/lib/python3.9/site-packages/pyhandle-1.0.5.dev0-py3.9.egg/pyhandle/handlesystemconnector.py:197: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if msg is not '':
/usr/local/lib/python3.9/site-packages/pyhandle-1.0.5.dev0-py3.9.egg/pyhandle/handlesystemconnector.py:498: SyntaxWarning: "is" with a literal. Did you mean "=="?
if action is 'GET':
/usr/local/lib/python3.9/site-packages/pyhandle-1.0.5.dev0-py3.9.egg/pyhandle/handlesystemconnector.py:502: SyntaxWarning: "is" with a literal. Did you mean "=="?
elif action is 'PUT' or action is 'DELETE':
/usr/local/lib/python3.9/site-packages/pyhandle-1.0.5.dev0-py3.9.egg/pyhandle/handlesystemconnector.py:502: SyntaxWarning: "is" with a literal. Did you mean "=="?
elif action is 'PUT' or action is 'DELETE':
/usr/local/lib/python3.9/site-packages/pyhandle-1.0.5.dev0-py3.9.egg/pyhandle/handlesystemconnector.py:510: SyntaxWarning: "is" with a literal. Did you mean "=="?
if action is 'PUT':
/usr/local/lib/python3.9/site-packages/pyhandle-1.0.5.dev0-py3.9.egg/pyhandle/client/dbhandleclient.py:297: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if query_result_as_dict is not 0:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.