keep2share doesn't work with authentication; parse errors

[mcrapet]

keep2share doesn't work with authentication; parse errors

[Raziel-23]

@mcrapet Here grep_http_header_location doesn't work as expected. Here it is example based on above log:

#!/bin/bash
DEBUG=2
VERBOSE=3
source /usr/share/plowshare/core.sh

LOCATION='Location: http://prx-155.keep2share.cc/c320681128265/5ef0b79a42ce7/e36972741db6c?temp_url_sig=85c3edd8ba3da45a7e1d22a227e63d44c529e1d5b9e803169bad2e9da7991944505ef2a03a8ce3f41094e2441dd2ea75660ae09de6f614839b801041858efad8&temp_url_expires=1463641774&id=39fcffcc7cc7e&node_id=155&project=moneyplatform&user_id=551585&name=Test+file+50MB.zip'

echo "$LOCATION" | grep_http_header_location

It throws an error. When I remove [[:cntrl:]] from grep_http_header_location it works fine:

diff --git a/src/core.sh b/src/core.sh
index ce392f2..b22f04c 100644
--- a/src/core.sh
+++ b/src/core.sh
@@ -684,7 +684,7 @@ match_json_true() {
 #        or -I/--head flag)
 # stdout: result
 grep_http_header_location() {
-    parse_all '^[Ll]ocation:' 'n:[[:space:]]\+\(.*\)[[:cntrl:]]$'
+    parse_all '^[Ll]ocation:' 'n:[[:space:]]\+\(.*\)$'
 }
 grep_http_header_location_quiet() {
     parse_all '^[Ll]ocation:' 'n:[[:space:]]\+\(.*\)[[:cntrl:]]$' 2>/dev/null

Just to be clear I don't know if [[:cntrl:]] is important.

[mcrapet]

[[:cntrl:]] is portable way to match 0x0D byte.
According to HTTP protocol each header line must end with \x0D\x0A.

We could eventually workaround this in module source.

[Raziel-23]

It is strange, I just download this file as anonymous and save a log. It downloads just fine and in the log in Location at the end it contains this byte. I'm confused. Maybe this is a server issue, I don't know.

[Raziel-23]

@mcrapet Maybe this is related to characters encoding? Maybe it eats carriage return characters.

@rickt Please open your raw log (not edited) in vim editor and check if you have ^M character at the end of the Location line. Maybe when you edited your log your editor ate those characters, I just want be sure. Normally in a http header you should have those '^M' characters at the end of line.

[mcrapet]

We support keep2share premium account only.
Free account doesn't work here (there's no Location line at all)

We must add this detection.

[Raziel-23]

@mcrapet I know how distinguish an account type on keep2share, thanks to @dpeddi. See here and here. They set account_expires field for premium accounts (for free accounts this field is always false). If you want to I can do this and refactor login function with new return code, plus add support for free accounts download from API.

[mcrapet]

@Raziel-23 Go ahead, you know this module better than me, you'll be faster.

[rickt]

@rickt Please open your raw log (not edited) in vim editor and check if you have ^M character at the end of the Location line.

done, there is no ^M. i checked via several methods (cat -v, od -c, etc) to be absolutely sure. definitely no ^M.

to be double and triple sure (i am nothing, if not thorough) i just re-ran everything, captured std(out|err) and scp'd the actual logfile to my own box, no editing: http://fix8r.com/tmp/59044_log.txt.

(edit: edited out private info only)

[Raziel-23]

@rickt Your account on keep2share is premium? For this module download from free accounts is not supported yet (I will add this). Here is how a log looks for me for anonymous download:

This is very strange for me.

[rickt]

@rickt Your account on keep2share is premium?
yes, its a premium account.

[Raziel-23]

@rickt I have not idea where the problem can be, CC @mcrapet

[Raziel-23]

@rickt I have some more questions, I just want to narrow down this problem. Please execute this script on your computer:

#!/bin/bash
DEBUG=2
VERBOSE=3
source /usr/share/plowshare/core.sh
curl -I http://www.cnn.com/ | grep_http_header_location 

Note: /usr/share/plowshare/core.sh is one of the plowshare core file, on your system it may be in a different location.

Please tell me if it prints an error or it prints a new location. The problem maybe either a keep2share server send a malformed http header for premium accounts, or the problem is on your system (either with curl, maybe older version, or with ~/.curlrc configuration file, or with something else).

[mcrapet]

@Raziel-23 : DEBUG=2 is useless. I know I did mistake first. The proper variable is DEBUG.
If you want to fix free account, I can test it.

[Raziel-23]

@mcrapet Ok, here it is:
Raziel-23@2edfadd
Please test it. Feel free to change this commit if you want. keep2share is one of the most popular hoster, so don't hesitate to change if you want. All crucial changes are listed in a commit message.

I lately have less free time to code, I'm sorry you had to wait for this.

[Raziel-23]

@mcrapet I harden more the code, here it is a final version:
Raziel-23@7228d7b

[Raziel-23]

@mcrapet and a final final version :D :
Raziel-23@a827cba
I added check for private links. Here are test links (without a file for premium accounts only, because from free account I cannot set this flag):

http://k2s.cc/file/b7ff84370e658/Test_file_5MiB.zip
http://k2s.cc/file/01d8df62c4de5/Test_file_5MiB_private.zip
http://k2s.cc/file/c1672cfa4f357/Test_file_5MiB_removed.zip

[mcrapet]

This is just fine for me! 👍