UnicodeDecodeError: 'charmap' codec can't decode byte

[vimeo]

UnicodeDecodeError: 'charmap' codec can't decode byte

[KlausStangl]

As a follow-up... I got above message with current python 3.4.
This Lib should be used in another product running Python 2.6.7
-> I had to change HTTP_METHODS from {} to [ ... ] in client.py to be able to install.
Everything seams fine. Only upload doesn't work. Even small files take some time and than you get the following message:

video_uri = v.upload('Z:/OUTPUT/test.mp4')
Traceback (most recent call last):
File "", line 1, in
File "C:\Program Files\Thinkbox\Deadline6\bin\lib\site-packages\vimeo\upload.py", line 22, in upload
return self._perform_upload(filename, ticket)
File "C:\Program Files\Thinkbox\Deadline6\bin\lib\site-packages\vimeo\upload.py", line 48, in _perform_upload
self._make_pass(target, f, size, last_byte)
File "C:\Program Files\Thinkbox\Deadline6\bin\lib\site-packages\vimeo\upload.py", line 88, in _make_pass
}, data=f)
File "C:\Program Files\Thinkbox\Deadline6\bin\lib\site-packages\vimeo\client.py", line 75, in caller
*_kwargs)
File "requests\api.py", line 110, in put
return request('put', url, data=data, *_kwargs)
File "requests\api.py", line 49, in request
response = session.request(method=method, url=url, *_kwargs)
File "requests\sessions.py", line 459, in request
resp = self.send(prep, *_send_kwargs)
File "requests\sessions.py", line 571, in send
r = adapter.send(request, **kwargs)
File "requests\adapters.py", line 409, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', error(10054, 'An existing connection was forcibly closed by the remote host'))

[KlausStangl]

I've added an output with counter... it seams like it hangs right at the first byte:

video_uri = v.upload('Z:/OUTPUT/test.mp4')
0
Traceback (most recent call last):
File "", line 1, in
File "vimeo\upload.py", line 22, in upload
return self._perform_upload(filename, ticket)
File "vimeo\upload.py", line 51, in _perform_upload
self._make_pass(target, f, size, last_byte)
File "vimeo\upload.py", line 91, in _make_pass
}, data=f)
File "vimeo\client.py", line 75, in caller
*_kwargs)
File "C:\Python26\lib\site-packages\requests\api.py", line 110, in put
return request('put', url, data=data, *_kwargs)
File "C:\Python26\lib\site-packages\requests\api.py", line 49, in request
response = session.request(method=method, url=url, *_kwargs)
File "C:\Python26\lib\site-packages\requests\sessions.py", line 459, in request
resp = self.send(prep, *_send_kwargs)
File "C:\Python26\lib\site-packages\requests\sessions.py", line 571, in send
r = adapter.send(request, **kwargs)
File "C:\Python26\lib\site-packages\requests\adapters.py", line 409, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', BadStatusLine('',))

[KlausStangl]

I tried to debug and found out that it doesn't return from caller() -> request_func(...). Digging deeper there seams to be a problem with/in requests -> conn.urlopen(...) which throws the ConnectionError.
I installed the latest Requests from today, maybe that's the problem?

[sclm]

It's possible that there's a bug if they've recently released a new version. My machine is a bit out of date, with 2.4.1 installed. Can you try 2.4.1 or 2.4.2 to see if that resolves your issue?

[KlausStangl]

Just installed requests-2.4.1 with current vimeo.py, only change are those {} -> [], doesn't work. Still get the Connection Error.
Is there something like "this combination should work for sure" (python / libs)?
I tried current python 3.4 without luck (first error above), 2.6.5, 2.6.6, 2.7.7... (connection error)
As written I've tried current requests lib, v2.4.1 and 2.3.0, there's not any other library needed to get this running?

[KlausStangl]

Here's an output before conn.urlopen. Simple print: print('conn.urlopen vorher '+url+' '+str(request.method)+' '+str(request.body)+' '+str(request.headers)+' '+str(timeout))
First "get" is ok, second "put" hangs:

/me/videos POST {"upgrade_to_1080": "false", "type": "streaming"} {'Content-Length': '49', 'Accept-Encoding': 'gzip, deflate', 'Accept': 'application/vnd.vimeo.*;version=3.2', 'User-Agent': 'pyvimeo 0.1; (http://developer.vimeo.com/api/docs)', 'Connection': 'keep-alive', 'Content-Type': 'application/json', 'Authorization': 'Bearer xxxxxxx'} Timeout(connect=1, read=30, total=None)

/upload?ticket_id=1ac5d5e4ab15669a2dfdc479e3468228&video_file_id=308902188&signature=0331e3ec958340fe8ef96e67c528a28d&v6=1&full_hd=0&redirect_url=https%3A%2F%2Fvimeo.com%2Fupload%2Fapi%3Fvideo_file_id%3D308902188%26app_id%3D46322%26ticket_id%3D1ac5d5e4ab15669a2dfdc479e3468228%26signature%3D5a8f5cd83e9f87989a308043241090bb41505ea7 PUT <open file 'Z:/OUTPUT/test.mp4', mode 'r' at 0x0000000002CFB4B0> {'Content-Length': '3317959', 'Accept-Encoding': 'gzip, deflate', 'Connection':
'keep-alive', 'Accept': 'application/vnd.vimeo.*;version=3.2', 'User-Agent': 'pyvimeo 0.1; (http://developer.vimeo.com/api/docs)', 'Content-Range': 'bytes: 0-3317959/3317959', 'Authorization': 'Bearer xxxxxxxx'} Timeout(connect=None, read=None, total=None)

[KlausStangl]

I followed the code and found out that it hangs in connectionpool.py from urllib3 when _make_request is executed by urlopen.
In _make_request -> httplib_response = conn.getresponse (...) is executed and it stops right there for a short timeout.
It jumps back into the exception part of urlopen in same file.
Don't really have a glue why getresponse doesn't work...

[sclm]

I'm going to see if I can replicate this today. Can you confirm that you're on Windows?

I haven't been able to replicate this on any platform I have easy access to, so I'm going to reach out to one of my fellow groups for a windows box.

[KlausStangl]

Hi!
Yes, I'm on Windows. Main python version I'll be using will be 2.6.7, but I got the same error on 2.7.8. Different error on 3.4 as seen above.
Meanwhile I found out that it changes when I use requests v2.4.1. I was able to start an upload but it hangs right at the first Byte.
Problem seams to be in connectionpool -> urlopen:
httplib_response = self._make_request(conn, method, url,
timeout=timeout,
body=body, headers=headers)
It hangs right there, than throws an exception. There's a new "video" on vimeo with error message.
Thanks for your help,
Klaus

[KlausStangl]

As an additional information to my last post, I tried the versions from Sep 15, 2014 at this point with Python 2.7.8 / 2.6.7, that's where at least a new empty video was created on vimeo!
I used current / master release for everything else!

[KlausStangl]

Here's what I do...

1. Install Python 2.6.6
2. c:\Python26\python.exe get-pip.py
3. c:\Python26\Scripts\pip.exe install pyvimeo
   Error: SyntaxError: ('invalid syntax', ('c:\Python26\Lib\site-packages\vimeo\client.py', 17, 27, " HTTP_METHODS = {'head', 'get', 'post', 'put', 'patch', 'options', 'delete'}\n"))
4. Change {} to [] in client.py
5. Start Python
6. import vimeo
7. v = vimeo.VimeoClient( token, key, secret ) # declared those before
8. about_me = v.get('/me')
9. print -> Data OK
10. video_uri = v.upload('Z:/OUTPUT/test.mp4')
    Takes some time than:
    Traceback (most recent call last):
    File "", line 1, in
    File "c:\Python26\lib\site-packages\vimeo\upload.py", line 22, in upload
    return self._perform_upload(filename, ticket)
    File "c:\Python26\lib\site-packages\vimeo\upload.py", line 48, in _perform_upload
    self._make_pass(target, f, size, last_byte)
    File "c:\Python26\lib\site-packages\vimeo\upload.py", line 88, in _make_pass
    }, data=f)
    File "c:\Python26\lib\site-packages\vimeo\client.py", line 75, in caller
    *_kwargs)
    File "c:\Python26\lib\site-packages\requests\api.py", line 105, in put
    return request('put', url, data=data, *_kwargs)
    File "c:\Python26\lib\site-packages\requests\api.py", line 49, in request
    return session.request(method=method, url=url, *_kwargs)
    File "c:\Python26\lib\site-packages\requests\sessions.py", line 457, in request
    resp = self.send(prep, *_send_kwargs)
    File "c:\Python26\lib\site-packages\requests\sessions.py", line 569, in send
    r = adapter.send(request, **kwargs)
    File "c:\Python26\lib\site-packages\requests\adapters.py", line 407, in send
    raise ConnectionError(err, request=request)
    requests.exceptions.ConnectionError: ('Connection aborted.', BadStatusLine('',))

[KlausStangl]

After further testing I found that if I set timeout=1 in self.put(...) instead of None I get a little further. A video is created but nothing gets uploaded.
Right at the first byte I get a Timeout Exception, after that exception urlopen / _make_request is executed twice and fails with:
Traceback (most recent call last):
File "", line 1, in
File "c:\Python26\lib\site-packages\vimeo\upload.py", line 22, in upload
return self._perform_upload(filename, ticket)
File "c:\Python26\lib\site-packages\vimeo\upload.py", line 62, in _perform_upload
assert finalized_resp.status_code == 201, "Failed to create the video."
AssertionError: Failed to create the video.

[KlausStangl]

Just tried it with a completely fresh windows 7 install, no FW or something else...

1. Installed python 2.7.8
2. O:\Python27\python.exe get-pip.py
3. O:\Python27\Scripts\pip.exe install pyvimeo
4. Start Python

import vimeo
token = "xxxxx"
key = "xxxxx"
secret = "xxxxx"
v = vimeo.VimeoClient( token, key, secret )
about_me = v.get('/me')
assert about_me.status_code == 200
about_me_dict = about_me.json()
print( ((about_me_dict.get("upload_quota")).get("space").get("free")) )
xxxxx
video_uri = v.upload('Z:/OUTPUT/test.mp4')
Traceback (most recent call last):
File "", line 1, in
File "O:\Python27\lib\site-packages\vimeo\upload.py", line 22, in upload
return self._perform_upload(filename, ticket)
File "O:\Python27\lib\site-packages\vimeo\upload.py", line 48, in _perform_upload
self._make_pass(target, f, size, last_byte)
File "O:\Python27\lib\site-packages\vimeo\upload.py", line 88, in _make_pass
}, data=f)
File "O:\Python27\lib\site-packages\vimeo\client.py", line 75, in caller
*_kwargs)
File "O:\Python27\lib\site-packages\requests\api.py", line 105, in put
return request('put', url, data=data, *_kwargs)
File "O:\Python27\lib\site-packages\requests\api.py", line 49, in request
return session.request(method=method, url=url, *_kwargs)
File "O:\Python27\lib\site-packages\requests\sessions.py", line 457, in request
resp = self.send(prep, *_send_kwargs)
File "O:\Python27\lib\site-packages\requests\sessions.py", line 569, in send
r = adapter.send(request, **kwargs)
File "O:\Python27\lib\site-packages\requests\adapters.py", line 407, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', error(10054, 'An existing connection was forcibly closed by the remote host'))

[KlausStangl]

The same with a fresh Windows / Python 2.6.6 Install

1. Installed python 2.7.8
2. O:\Python27\python.exe get-pip.py
3. O:\Python27\Scripts\pip.exe install pyvimeo
   Running setup.py install for pyvimeo
   SyntaxError: ('invalid syntax', ('O:\Python26\Lib\site-packages\vimeo\client.py', 17, 27, " HTTP_METHODS = {'head', 'get', 'post', 'put', 'patch', 'options', 'delete'}\n"))
   -> Exchanged {} against [] in client.py
4. Start Python

import vimeo
token = "xxxxx"
key = "xxxxx"
secret = "xxxxx"
v = vimeo.VimeoClient( token, key, secret )
about_me = v.get('/me')
assert about_me.status_code == 200
about_me_dict = about_me.json()
print( ((about_me_dict.get("upload_quota")).get("space").get("free")) )
xxxxx
video_uri = v.upload('Z:/OUTPUT/test.mp4')
Traceback (most recent call last):
File "", line 1, in
File "O:\Python26\lib\site-packages\vimeo\upload.py", line 22, in upload
return self._perform_upload(filename, ticket)
File "O:\Python26\lib\site-packages\vimeo\upload.py", line 48, in _perform_upload
self._make_pass(target, f, size, last_byte)
File "O:\Python26\lib\site-packages\vimeo\upload.py", line 88, in _make_pass
}, data=f)
File "O:\Python26\lib\site-packages\vimeo\client.py", line 75, in caller
*_kwargs)
File "O:\Python26\lib\site-packages\requests\api.py", line 105, in put
return request('put', url, data=data, *_kwargs)
File "O:\Python26\lib\site-packages\requests\api.py", line 49, in request
return session.request(method=method, url=url, *_kwargs)
File "O:\Python26\lib\site-packages\requests\sessions.py", line 457, in request
resp = self.send(prep, *_send_kwargs)
File "O:\Python26\lib\site-packages\requests\sessions.py", line 569, in send
r = adapter.send(request, **kwargs)
File "O:\Python26\lib\site-packages\requests\adapters.py", line 407, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', error(10054, 'An existing connection was forcibly closed by the remote host'))

[KlausStangl]

Further testing...
I added some debug outputs in connectionpool.py...
When uploading a file and the first byte should be uploaded, I get the following output from _make_request / connectionpool.py

1. first time URL with ticket and PUT (method)
2. second run URL with ticket and PUT
3. third URL with shorter ticket and DELETE
   The question is why conn.getresponse() doesn't get a response....
   Thanks for your help,
   Klaus

[KlausStangl]

Another test-run... this time uploaded via a virtual Debian and python 2.7.3 and had no problems at all!?
-Klaus

[sclm]

@KlausStangl I swear we are looking at this - I strongly suspect it's a Windows only issue within requests (and we don't have many of those machines lying around).

Is running this in Debian working well for you?

[KlausStangl]

Hi!
Thanks for coming back about the issue! Great to hear you're working on it! Really "funny" if I'm the only one using Windows :-)
Yes, Debian with Python works great. It's a good temporary solution for now.
If you need access to a Windows machine I can setup a VM with SSH/VNC if that helps?
Meanwhile I tried several different configurations (python / requests / urllib3), without luck. conn.getresponse() just doesn't get a response.
-Klaus

[greedo]

@KlausStangl we use the request library to make all of our http calls. I reached out to their team. They say the error is coming from the core python stdlib httplib. This error can also occur on *nix and Windows systems alike sporadically. I will continue my investigation further with the httplib team, however for now it may be best to catch the wrapped requests.exceptions.ConnectionError requests error and retry the call.

https://github.com/kennethreitz/requests/issues/2364