egnyte/python-egnyte

Python client for the Egnyte Public API.

python-egnyte's Issues

Would you guys please publish the latest code to PyPI?

Hi there,

It looks like your code on PyPI is from last year. Would you mind uploading the latest version of your code (looks like you've already declared this code to be 0.6) to PyPI? Preferably including the code change I had pushed ;)

Thank you,
Paul

Make poll_delay in egnyte/events.py configurable

poll_delay is set to 1.0 by default and we want it to configurable and want custom polling like once ini 30 seconds or 45 seconds.

Making frequent calls every 1 second to events API sometimes leads to rate limiting by Egnyte API.

Expecting something like config file in parent directory or env variable where custom poll delay can be set.

API request limits

There is a very strict limit on the number of requests that can be performed sequentially (2 per app per second). This means if you want to do several operations in quick succession (i.e. generate something, see if it's been successful then access it) you are likely to hit this upper bound which will result in an insufficient permissions error.

It would be useful to have:

1. A distinct error type when API request limits have been hit.
2. Wrapping the requests interface to include some kind of 'last requests time' which imposes this limit automatically.

TypeError: 'RequestError' object is not iterable

python-egnyte/egnyte/exc.py

Line 18 in 42e58e0

for item in self:

Under some scenarios this line generates a fatal error when trying to generate the string of the error in a statement like this:

try:
    something
except ..... as e:
   err = str(e)

It is incredible that the Python SDK is no longer supported... We will migrate from Egnyte, that's for sure.

bulk_upload times out or 502s

Without fail, running a client.bulk_upload() times out at 30 seconds, even on small files. If I invoke with a longer timeout, I get a 502 from the server.

Code used:

import egnyte

class verboselogger(egnyte.client.ProgressCallbacks):
    force_newline = True
    def creating_directory(self, cloud_folder):
        sys.stderr.write('Creating %s\n' % (cloud_folder.path))
    def upload_start(self, local_path, cloud_file, size):
        sys.stderr.write('Starting Upload: %s (%d bytes)\n' % (cloud_file.path, size))
    def upload_finish(self, cloud_file):
        sys.stderr.write('Finished Upload: %s\n' % (cloud_file.path))

egnyte_path = '/Private/xxx/yyy/zzz'
client = egnyte.EgnyteClient({"domain": egnyte_domain, "access_token": egnyte_token})
pdf_dir = '/path/to/pdfs'

client.folder(egnyte_path).delete()
folder = client.folder(egnyte_path).create(ignore_if_exists=True)
client.bulk_upload(['%s/' % (pdf_dir)], egnyte_path, progress_callbacks=verboselogger())

Results:

Creating /Private/xxx/yyy/zzz
Creating /Private/xxx/yyy/zzz/
Starting Upload: /Private/xxx/yyy/zzz//_catalog.txt (1768 bytes)
Finished Upload: /Private/xxx/yyy/zzz//_catalog.txt
Starting Upload: /Private/xxx/yyy/zzz//index.pdf (1355115 bytes)
Traceback (most recent call last):
  File "./test_create.py", line 183, in <module>
    client.bulk_upload(['%s/' % (pdf_dir)], egnyte_path, progress_callbacks=verboselogger())
  File "/home/user/.local/lib/python2.7/site-packages/egnyte/client.py", line 128, in bulk_upload
    cloud_file.upload(fp, size, progress_callbacks.upload_progress)
  File "/home/user/.local/lib/python2.7/site-packages/egnyte/resources.py", line 97, in upload
    r = self._client.POST(url, data=chunk, headers={'Content-length': str(size)})
  File "/home/user/.local/lib/python2.7/site-packages/egnyte/base.py", line 78, in POST
    return self._retry(self._session.post, url, data=data, headers=headers, **kwargs)
  File "/home/user/.local/lib/python2.7/site-packages/egnyte/base.py", line 57, in _retry
    response = func(*args, **kwargs)
  File "/home/user/.local/lib/python2.7/site-packages/requests/sessions.py", line 581, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/home/user/.local/lib/python2.7/site-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/user/.local/lib/python2.7/site-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/home/user/.local/lib/python2.7/site-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='xxxxxx.egnyte.com', port=443): Read timed out. (read timeout=30)

When invoked with the following:

client = egnyte.EgnyteClient({"domain": egnyte_domain, "access_token": egnyte_token, "timeout": "120"})

the results are as below:

Creating /Private/xxx/yyy/zzz
Creating /Private/xxx/yyy/zzz/
Starting Upload: /Private/xxx/yyy/zzz//_catalog.txt (1768 bytes)
Finished Upload: /Private/xxx/yyy/zzz//_catalog.txt
Starting Upload: /Private/xxx/yyy/zzz//index.pdf (1355115 bytes)
Traceback (most recent call last):
  File "./test_create.py", line 183, in <module>
    client.bulk_upload(['%s/' % (pdf_dir)], egnyte_path, progress_callbacks=verboselogger())
  File "/home/user/.local/lib/python2.7/site-packages/egnyte/client.py", line 128, in bulk_upload
    cloud_file.upload(fp, size, progress_callbacks.upload_progress)
  File "/home/user/.local/lib/python2.7/site-packages/egnyte/resources.py", line 98, in upload
    exc.default.check_response(r)
  File "/home/user/.local/lib/python2.7/site-packages/egnyte/exc.py", line 168, in check_response
    raise error_type(*errors)
egnyte.exc.RequestError: <RequestError: {url: 'https://xxxxxx.egnyte.com/pubapi/v1/fs-content/Private/xxx/yyy/zzz//index.pdf'}, {http response: '<html><body><h1>502 Bad Gateway</h1>
The server returned an invalid or incomplete response.
</body></html>
'}, {http status: '502'}, {headers: '{'X-XSS-Protection': '1; mode=block', 'X-Content-Type-Options': 'nosniff', 'X-Egnyte-Request-Id': 'xxxxxxx:xxxxxxx:xxxx4_--_xxxxx|avl-www07.dc.egnyte.lan+https_l1_webui', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains', 'Cache-Control': 'no-cache', 'Content-Type': 'text/html'}'}>

Note that the failure does not always occur on the same file. Sometimes 1-2 files are uploaded fine. None of these are very large - 3MB at most.

The versions of the various requests modules are:

requests                 2.22.0
requests-file            1.4.3
requests-toolbelt        0.8.0

This is on a CentOS 7 system and (for now) the script is using python 2.7.

folder.create ignore_if_exists parameter is not respected

I haven't had time to dig into why, but when using

client.folder("/Shared/Temp/").create(ignore_if_exists=True)
An AttributeError or egnyte.exc.RequestError is thrown if the folder exists already.

From a quick glimpse it looks like the error is caused by this line not properly checking if the error that should be ignored is present in the request result:

python-egnyte/egnyte/exc.py

Line 174 in 68815e9

result = any((errors[:len(ignored)] == ignored) for ignored in self.ignored_errors)

API Call Limit Check

This is not code specific, but I am wondering if there is any functionality in the API to output the number of calls that have been performed relative to the rate limit imposed. It is still unclear to me if the 1000 call / day is a rolling limitation or resets at a specific time of day/night.

I could develop an in-house API counter but would get complex if calls are distributed between different servers and am hoping there is a simple solution here. If not no worries, just thought I would inquire.

FileOrFolder.link documentation issues

Comment for FileOrFolder.link says:

• accessibility: Determines how the link can be accessed ('Anyone', 'Password', 'Domain', 'Recipients')"

but I've had to use 'anyone' and 'domain'.
It also says that expiry_date and expiry_clicks are optional, but I was required to use one. (Ideally, both could be used, but that's probably not a python-egnyte issue).

Link to documentation broken

The link to the documentation in the "Full documentation" section of the readme leads to a 404 Site not found error.

Update Events Module to Version 2

Egnyte have recently updated the Events module to version 2 which also includes permissions change events by default. The codebase currently uses version 1 so misses the permission change events.

I can't see any other functional changes this has and should be a simple case of changing the version number in urls in events.py from v1 to v2. I've monkey patched this locally and have seen no issues so far.

Opaque error when the client exceeds the API limit.

Currently when the client hits the request limit, the HTTP 403 with the response body <h1>Developer Over Rate</h1>.

But the user gets an opaque message:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.7/site-packages/egnyte/resources.py", line 204, in list
    return self._get()
  File "/usr/local/lib/python3.7/site-packages/egnyte/resources.py", line 53, in _get
    json = exc.default.check_json_response(self._client.GET(self._url))
  File "/usr/local/lib/python3.7/site-packages/egnyte/exc.py", line 183, in check_json_response
    r = self.check_response(response, *ok_statuses)
  File "/usr/local/lib/python3.7/site-packages/egnyte/exc.py", line 168, in check_response
    raise error_type(*errors)
egnyte.exc.InsufficientPermissions: <exception str() failed>

The errors passed into the error constructor include the essential information, but something is missing when the error prints.

[{'url': 'https://celsiustx.egnyte.com/pubapi/v1/fs/Shared/Celsius%20Partners'}, {'http response': '<h1>Developer Over Rate</h1>'}, {'http status': 403}, {'headers': {'X-Mashery-Responder': 'avl-mashery-4-4-3-slave01', 'X-AccessToken-QPS-Allotted': '2', 'X-AccessToken-QPS-Current': '1', 'X-AccessToken-Quota-Allotted': '1000', 'X-AccessToken-Quota-Current': '1001', 'X-Quota-Reset': 'Saturday, April 10, 2021 12:00:00 AM GMT', 'Retry-After': '37342', 'X-Error-Detail-Header': 'Account Over Rate Limit', 'X-Mashery-Error-Code': 'ERR_403_DEVELOPER_OVER_RATE', 'Content-Type': 'text/xml', 'Date': 'Fri, 09 Apr 2021 13:37:39 GMT', 'Content-Length': '28', 'X-Content-Type-Options': 'nosniff', 'X-XSS-Protection': '1; mode=block', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains', 'X-Egnyte-Request-Id': '*********'}}]

Parameter "role" missing in update method of class User

The role of a user cannot be updated as the parameter 'role'is missing in the parameter list of the update method

Any way to search custom file/folder metadata?

That is all.

HTTP Documentation of groups.create doesn't reflect the required "members" attribute

Running
group = client.groups.create('groupname')
raises an error. After debugging, the content of the request is:

'{"formErrors":[],"inputErrors":{"members":[{"code":"MEMBERS_FIELD_IS_REQUIRED","msg":"Members field is required."}]}}'

Even though the HTTP documentation shows that members is optional.
This makes using the groups.create impossible without modifying the source code.

The code below is a workaround.

client._session.post(
    '{prfx}pubapi/v2/groups'.format(prfx=client._url_prefix),
    json = {'displayName' : 'test3','members' : [{"value": 1}]}
    )

Audit reports framework - add email suppression option and new report types

Recent updates to the audit reports framework mean that:

1. login, file and permissions reports now have the option to suppress report generation emails. This is not currently supported by the SDK.
2. New report types (user provisioning and group provisioning) have been added which are not yet supported by the SDK.

For FileOrFolder Set Value for Metadata Namespace, Get Metadata

It would be nice if this were possible using the File and Folder object:
https://developers.egnyte.com/docs/read/Metadata_API#Set-Values-for-a-Namespace

Similarly, it would be nice you could access the metadata from the File and Folder objects.