HTTP (including HTTPS and HTTP/2) support for the Cro library for building distributed systems in Raku.
Home Page: https://cro.services/
License: Artistic License 2.0
This is part of the Cro libraries for implementing services and distributed systems in Raku. See the Cro website for further information and documentation.
Hi,
I found that tomcat had a bug about cookie processing [0].
CookieString cannot parse cookies that legacy tomcat (i.e., tomcat which contains a bug of [0]) generates.
The bug is as follows (c.f., [0]):
When I set a cookie named "cookie_name" with value "value" and path "/", the header that's generated by Tomcat 8.0.28 looks like:
Set-Cookie: cookie_name=value; Path=/
With 8.5.11, it looks like:
Set-Cookie: cookie_name=value;path=/
Note the missing space after the semicolon and the change from "Path" to "path".
CookieString is strictly based on RFC 6265 [1] and requires SP after semicolon, so it cannot accept the latter one (i.e., "Set-Cookie: cookie_name=value;path=/").
I don't have a concrete idea but I think Cro::HTTP should provide a way to process such little-bit violated cookies.
Cheers,
Having deployed a cro site, I found that the SSLLabs score for the site was negatively affected by the default cipher set in cro-http.
I solved on my site by setting the cipher set manually in service.p6
I have a potential pull request to make the mozilla recommendations the default here: lancew@f7c6e4c
Raising an issue before pull request as I have not yet run tests and wanted to get opinions before pushing a pull request.
Lance
# Failed test 'Got a push promise when they are disabled!'
# at t/http2-push-promise.t line 61
# Failed test 'Got zero push promises when they are disabled'
# at t/http2-push-promise.t line 65
# expected: '0'
# got: '1'
# Looks like you failed 2 tests of 7
t/http2-push-promise.t .........
Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/7 subtests
Is there a way to insert some sort of middleware into Cro::HTTP::Client after constructing the request but before transmissions? I'd like to implement AWS request signatures, but doing so requires working with a completed request. I have implemented something like this here:
https://github.com/zostay/Amazon-DynamoDB/blob/master/lib/Amazon/DynamoDB.pm6#L764
But it seems like this would be much nicer to do in the pipeline. This would need to go into the pipeline after the body has been serialized, but before transmission. I don't see a place where I can do that.
Currently when I install Cro::HTTP the tests fail with
# Failed test 'Correct responses logged'
# at t/http-log-file.t line 58
# expected: '[OK] 200 / - 127.0.0.1
# [OK] 200 /route - 127.0.0.1
# [OK] 200 /route - 127.0.0.1
# '
# got: '[OK] 200 / - ::1
# [OK] 200 /route - ::1
# [OK] 200 /route - ::1
# '
# Failed test 'Error responses logged'
# at t/http-log-file.t line 59
# expected: '[ERROR] 500 /error - 127.0.0.1
# '
# got: '[ERROR] 500 /error - ::1
# '
# Looks like you failed 2 tests of 2
I've created a new issue because I didn't want to hijack the other issue like i did already, sorry.
Based on the queston on StackOverflow I've changed my exampel to
use Cro::HTTP::Router;
use Cro::HTTP::Server;
use Cro::HTTP::Auth::Basic;
class MyUser does Cro::HTTP::Auth {
has $.username;
}
subset LoggedInUser of MyUser where { .username.defined }
class MyBasicAuth does Cro::HTTP::Auth::Basic[MyUser, "username"] {
method authenticate(Str $user, Str $pass --> Bool) {
say "authentication called";
my $success = $user eq 'admin' && $pass eq 'secret';
# forbidden unless $success;
return $success;
}
}
sub routes()
{
my %storage;
my $fake-uuid = 0;
route {
before MyBasicAuth.new;
delegate <*> => route {
get -> LoggedInUser $user, 'api', $uuid {
content 'application/json', %storage{$uuid};
}
post -> LoggedInUser $user, 'api' {
request-body -> %json-object {
$fake-uuid++;
%storage{$fake-uuid} = %json-object;
created "api/$fake-uuid", 'application/json', %json-object;
}
}
}
}
}
my Cro::Service $app = Cro::HTTP::Server.new:
:host<localhost>, :port<10000>,
application => routes;
$app.start;
react whenever signal(SIGINT) { $app.stop; exit }
Authentication seems to work at a first glance:
> curl --fail http://localhost:10000/api -H 'Content-Type: application/json' -d '{"random": "data"}'
curl: (22) The requested URL returned error: 401 Unauthorized
> curl -v --fail http://admin:secret@localhost:10000/api -H 'Content-Type: application/json' -d '{"random": "data"}'
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 10000 (#0)
* Server auth using Basic with user 'admin'
> POST /api HTTP/1.1
> Host: localhost:10000
> Authorization: Basic YWRtaW46c2VjcmV0
> User-Agent: curl/7.54.0
> Accept: */*
> Content-Type: application/json
> Content-Length: 18
>
* upload completely sent off: 18 out of 18 bytes
< HTTP/1.1 201 Created
< Location: api/2
< Content-type: application/json
< Content-length: 18
<
* Connection #0 to host localhost left intact
{"random": "data"}%
> curl --fail http://admin:secret@localhost:10000/api/2
{"random": "data"}%
But GET on the same API end point without Authorization headed leads to 405 instead of 401.
> curl -v --fail http://localhost:10000/api/2
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 10000 (#0)
> GET /api/2 HTTP/1.1
> Host: localhost:10000
> User-Agent: curl/7.54.0
> Accept: */*
>
* The requested URL returned error: 405 Method Not Allowed
* stopped the pause stream!
* Closing connection 0
curl: (22) The requested URL returned error: 405 Method Not Allowed
Hey guys, nice work with Cro! :D
So, this is how a successful test run looks.
===========8<----------------
$ cd cro-http
$ zef test .
===> Testing: Cro::HTTP:ver('0.7')
Sudden error
in block at t/http-client.t line 71
in code at /home/sjn/src/cro-http/lib/Cro/HTTP/Router.pm6 (Cro::HTTP::Router) line 79
No more
in block at t/http-log-file.t line 19
in code at /home/sjn/src/cro-http/lib/Cro/HTTP/Router.pm6 (Cro::HTTP::Router) line 79
===> Testing [OK] for Cro::HTTP:ver('0.7')
------------------------>8===============
It looks like something went wrong, but nothing did.
Would it be possible to make the output look less "error-y"? At the moment, it's an attention-waster for newbies like myself. :)
See this workaround and provide a better way.
$ perl6 -Ilib t/http2-client.t
ok 1 - # SKIP No ALPN support
1..1
Some exceptions were thrown in END blocks:
X::AdHoc: Cannot look up attributes in a Cro::Service type object
in method stop at C:\rakudo\share\perl6\site\sources\1B7E3505A9D9EFF8CA35709763578AFDA3C6C355 (Cro::Service) line 21
in code at t/http2-client.t line 51
I'm on Arch linux, that is, the system is up to date. And I have this exception:
When installing a dependency, I get this exception when installing Cro::Router:
===> Testing: IO::Socket::Async::SSL:ver<0.7.0>
Cannot locate symbol 'sk_num' in native library 'libssl.so'
in method setup at /usr/share/perl6/sources/24DD121B5B4774C04A7084827BFAD92199756E03 (NativeCall) line 291
in method CALL-ME at /usr/share/perl6/sources/24DD121B5B4774C04A7084827BFAD92199756E03 (NativeCall) line 570
in block at /home/me123/.zef/store/IO-Socket-Async-SSL-0.7.0.tar.gz/IO-Socket-Async-SSL-0.7.0/lib/IO/Socket/Async/SSL.pm6 (IO::Socket::Async::SSL) line 436
in block at /home/me123/.zef/store/IO-Socket-Async-SSL-0.7.0.tar.gz/IO-Socket-Async-SSL-0.7.0/lib/IO/Socket/Async/SSL.pm6 (IO::Socket::Async::SSL) line 401
in block <unit> at t/client-server.t line 171
===> Testing [FAIL]: IO::Socket::Async::SSL:ver<0.7.0>
Aborting due to test failure: IO::Socket::Async::SSL:ver<0.7.0> (use --force-test to override)
in code at /usr/share/perl6/vendor/sources/7926F4F3ED4C81AA5DA2A54C8AE1E03D03424CCE (Zef::Client) line 375
in method test at /usr/share/perl6/vendor/sources/7926F4F3ED4C81AA5DA2A54C8AE1E03D03424CCE (Zef::Client) line 355
in code at /usr/share/perl6/vendor/sources/7926F4F3ED4C81AA5DA2A54C8AE1E03D03424CCE (Zef::Client) line 531
in sub at /usr/share/perl6/vendor/sources/7926F4F3ED4C81AA5DA2A54C8AE1E03D03424CCE (Zef::Client) line 528
in method install at /usr/share/perl6/vendor/sources/7926F4F3ED4C81AA5DA2A54C8AE1E03D03424CCE (Zef::Client) line 623
in sub MAIN at /usr/share/perl6/vendor/sources/E4784A2A0FA00D16808817186E95FE74BEF3FE2D (Zef::CLI) line 152
in block <unit> at /usr/share/perl6/vendor/resources/3065D08F5332CA244672D7F8A05B603F92BB8A7D line 3
in sub MAIN at /usr/share/perl6/vendor/bin/zef line 2
in block <unit> at /usr/share/perl6/vendor/bin/zef line 2
Is there a workaround for it? At the moment I only want to install everything and be able to run it.
When we have a delegate, the target has its prefix stripped for the purposes of the transform we delegate it to. The logger currently uses that target, but it should use the original full target instead, otherwise the logs are somewhat confusing.
Here is the detailed output from t/http-cookiejar.t
ok 1 - Empty cookie jar contents returns empty list
ok 2 - Empty cookie jar contents with uri returns empty list
not ok 3 - Two cookies were added
# Failed test 'Two cookies were added'
# at t/http-cookiejar.t line 20
# expected: '3'
# got: '2'
not ok 4 - Cookie addition is neutral
# Failed test 'Cookie addition is neutral'
# at t/http-cookiejar.t line 23
# expected: '3'
# got: '2'
not ok 5 - Cookie with bad domain was not added
# Failed test 'Cookie with bad domain was not added'
# at t/http-cookiejar.t line 28
# expected: '4'
# got: '3'
ok 6 - Uri-based check
not ok 7 - Good cookies are here
# Failed test 'Good cookies are here'
# at t/http-cookiejar.t line 31
# expected: '4'
# got: '3'
not ok 8 - Clear for absent url leaves jar untouched
# Failed test 'Clear for absent url leaves jar untouched'
# at t/http-cookiejar.t line 34
# expected: '4'
# got: '3'
not ok 9 - Clear for absent url with existing cookie name leaves jar untouched
# Failed test 'Clear for absent url with existing cookie name leaves jar untouched'
# at t/http-cookiejar.t line 37
# expected: '4'
# got: '3'
not ok 10 - One cookie was removed
# Failed test 'One cookie was removed'
# at t/http-cookiejar.t line 40
# expected: '3'
# got: '2'
ok 11 - All cookies from correct domain were removed
ok 12 - Call to clear clears cookie jar
ok 13 - Cookie with duration was added successfully
ok 14 - Creation time is preserved during cookie update
ok 15 - Cookie was deleted on negative-time cookie addition
ok 16 - Cookies are added for sub-domains
ok 17 - Rejected for incorrect sub-domain
ok 18 - Header was added
ok 19 - Setted string is correct
1..19
# Looks like you failed 7 tests of 19
I am currently running:
This is Rakudo version 2018.10-78-g8b93ec9f5 built on MoarVM version 2018.10-72-g4fbaeee5f
implementing Perl 6.d.
use Cro::HTTP::Router;
use Cro::HTTP::Server;
use Cro::HTTP::Auth::Basic;
class MyUser does Cro::HTTP::Auth {
has $.username;
}
subset LoggedInUser of MyUser where { .username.defined }
class MyBasicAuth does Cro::HTTP::Auth::Basic[MyUser, "username"] {
method authenticate(Str $user, Str $pass --> Bool) {
say "authentication called";
forbidden unless $user eq 'admin' && $pass eq 'secret';
return $user eq 'admin' && $pass eq 'secret';
}
}
sub routes()
{
my %storage;
route {
before MyBasicAuth.new;
post -> LoggedInUser $user, 'api' {
request-body -> %json-object {
my $uuid = 'fake';
%storage{$uuid} = %json-object;
content 'application/json', %json-object
}
}
}
}
my Cro::Service $app = Cro::HTTP::Server.new:
:host<localhost>, :port<10000>,
application => routes;
$app.start;
react whenever signal(SIGINT) { $app.stop; exit }
and with curl --fail -H "Content-Type: application/json" -d '{"string":"hi"}' http://admin:foo@localhost:10000/api you will get:
Can only use 'content' inside of a request handler
in sub set-status at /home/koto/Work/cro/cro-http/lib/Cro/HTTP/Router.pm6 (Cro::HTTP::Router) line 846
in sub forbidden at /home/koto/Work/cro/cro-http/lib/Cro/HTTP/Router.pm6 (Cro::HTTP::Router) line 823
in method authenticate at perl6.pl6 line 14
in method process-auth at /home/koto/Work/cro/cro-http/lib/Cro/HTTP/Auth/Basic.pm6 (Cro::HTTP::Auth::Basic) line 26
so we need to pass $*REQUEST in process of Basic middleware.
I suspect this issue is the same for all authentication types we have, so those want a test too.
This would be a general Cro feature I think. Sometimes I just want to enable a trace mode so that I can see what Cro is doing. My current use case is that I would like to run a curl with the same headers, URI, etc that Cro is sending, so I can tweak them until the other server stops complaining.
I am sure there would be a discussion on how best to do this, and I wasn't sure how best to fit it in, so I haven't got a patch or anything that makes a start on this.
Take the following simple server with one POST route:
use Cro::HTTP::Router;
use Cro::HTTP::Server;
my $server = Cro::HTTP::Server.new(
:1337port,
:http<1.1>,
tls => %(
certificate-file => 'resources/fake-tls/server-crt.pem',
private-key-file => 'resources/fake-tls/server-key.pem',
),
application => route {
post -> {
request-body -> %params {
content 'text/plain', %params.perl;
}
};
},
);
$server.start;
react {
whenever signal(SIGINT) {
$server.stop;
done;
}
}
Running the following curl is all good:
$ curl --http1.1 -kd foo=bar https://localhost:1337
Cro::HTTP::Body::WWWFormUrlEncoded.new
However if I change :http<1.1> to :http<1.1 2> and then re-run the command I get:
$ curl --http1.1 -kd foo=bar https://localhost:1337
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to localhost:1337
And the finally if I ask curl to use h2 I get no output from curl:
$ curl --http2 -kd foo=bar https://localhost:1337
But I get the following warning from the server:
$ perl6 server.p6
An operation first awaited:
in sub request-body at /home/jraspass/.rakudup/2018.09/install/share/perl6/site/sources/F08078CCFE814F501DC10B2C366442F8CDADDF16 (Cro::HTTP::Router) line 689
in block at service.p6 line 13
in block at /home/jraspass/.rakudup/2018.09/install/share/perl6/site/sources/F08078CCFE814F501DC10B2C366442F8CDADDF16 (Cro::HTTP::Router) line 130
Died with the exception:
Can not decode a utf-8 buffer as if it were ascii
in method parse at /home/jraspass/.rakudup/2018.09/install/share/perl6/site/sources/DE40AE13F16DFE4C9CB8BC6D1D5598444E245797 (Cro::HTTP::BodyParsers) line 49
in method body at /home/jraspass/.rakudup/2018.09/install/share/perl6/site/sources/557D6C932894CB1ADE0F83C0596851F9212C2A67 (Cro::MessageWithBody) line 77
in sub request-body at /home/jraspass/.rakudup/2018.09/install/share/perl6/site/sources/F08078CCFE814F501DC10B2C366442F8CDADDF16 (Cro::HTTP::Router) line 689
in block at service.p6 line 13
in block at /home/jraspass/.rakudup/2018.09/install/share/perl6/site/sources/F08078CCFE814F501DC10B2C366442F8CDADDF16 (Cro::HTTP::Router) line 130
Can someone explain to me why this would be a bad thing to do? If not, would you accept a PR to basically do:
push @!cookies: $newCookie
I don't have an openssl that supports ALPN; this test dies instead of being skipped.
$ perl6 --version
This is Rakudo version 2017.08-79-g4b02b8aad built on MoarVM version 2017.08.1-103-g41e1b7ce
implementing Perl 6.c.
$ PERL6LIB=lib perl6 t/http-server.t
ok 1 - Service does Cro::Service
ok 2 - Server not listening until started
ok 3 - Can start service
ok 4 - Can connect once service is started
ok 5 - Got a response from the server
ok 6 - Response has 200 status in it
ok 7 - Response contains expected body
ok 8 - Can stop service
ok 9 - Server not listening after stopped
HTTP/2 is requested, but ALPN is not supported
in method new at /Users/wcoleda001/tmp/cro-http/lib/Cro/HTTP/Server.pm6 (Cro::HTTP::Server) line 96
in block <unit> at t/http-server.t line 76
The documentation indicates that there is a patch function that handles the PATCH HTTP method, but we didn't yet implement it. Let's add that, as well as adding it here and also adding support in Cro::HTTP::Client. Plus let's update the docs for Cro::HTTP::Client to mentioned that.
The docs speculate that you should be able to do destructuring of a request body:
post -> 'product' {
request-body -> (:$name!, :$description!, :$price!) {
# Do stuff here...
}
}
And indeed that should work well with JSON (I think we have a test, but it'd be wise to check). It won't, however, work yet with forms posted from the browser, which is a pity.
Thankfully, an object can choose how it destructures, so we can implement this. I'd implement #12 first. And then we can just have method Capture() { Capture.new(hash => self.hashed) } or some such. Plus some tests to make sure it works out.
I want to use the cro client to fetch a rss xml from a server which uses a self signed certificate. Its in our work network so I do not care about man-in-the-middle attach and so on.
The error
comes from IO::Socket::Async::SSL module and according to its docu I have to give the insecure => True parameter. But I did not find a way to give that to cro::HTTP::Client.
Client.new(secure => False) did not what I want.
I did not think it is a bug. But the docu did not help with that issue.
Thanks for helping!
Wolfgang
Following the documentation at https://cro.services/docs/reference/cro-http-client, I set out with a simple script to test some REST API interactions against https://vial-http.appspot.com/
#!/usr/bin/env perl6
my $resp = await Cro::HTTP::Client.get('https://vial-http.appspot.com');
say $resp;
However, this hangs almost indefinitely and the issue is reproducible every time.
Trying out a few different other URLs, I notice this seems to be correlated with HTTPS URLs (with the problem seemingly prevalent with those associated with google? but not always).
while read -r url; do ( set -x; timeout 4 perl6 ./cro.p6 "$url" &>/dev/null ); echo $?; done < urls
+ timeout 4 perl6 ./cro.p6 https://addons.mozilla.org/en-GB/firefox/
0
+ timeout 4 perl6 ./cro.p6 'https://isbnsearch.org/search?s=Think+Perl+6'
1
+ timeout 4 perl6 ./cro.p6 https://cro.services/docs/reference/cro-http-client
0
+ timeout 4 perl6 ./cro.p6 'https://github.com/croservices/cro-http/issues?utf8=%E2%9C%93&q=is%3Aissue+https'
0
+ timeout 4 perl6 ./cro.p6 https://ipapi.co/
0
+ timeout 4 perl6 ./cro.p6 https://www.yahoo.com/
124
+ timeout 4 perl6 ./cro.p6 'https://twitter.com/?lang=en'
1
+ timeout 4 perl6 ./cro.p6 https://www.facebook.com/
0
+ timeout 4 perl6 ./cro.p6 https://en.wikipedia.org/wiki/Facebook
124
+ timeout 4 perl6 ./cro.p6 'https://www.instagram.com/?hl=en'
0
+ timeout 4 perl6 ./cro.p6 https://google.co.uk/ [196/16140]
124
+ timeout 4 perl6 ./cro.p6 http://vial-http.appspot.com/
0
+ timeout 4 perl6 ./cro.p6 https://vial-http.appspot.com/
124
+ timeout 4 perl6 ./cro.p6 http://google.com/
0
+ timeout 4 perl6 ./cro.p6 https://google.com/
124
+ timeout 4 perl6 ./cro.p6 http://google.com.au/
0
+ timeout 4 perl6 ./cro.p6 https://google.com.au/
124
+ timeout 4 perl6 ./cro.p6 https://cloud.google.com/
124
+ timeout 4 perl6 ./cro.p6 https://www.google.com/gmail/
124
+ timeout 4 perl6 ./cro.p6 https://accounts.google.com/ServiceLogin
124
+ timeout 4 perl6 ./cro.p6 https://itunes.apple.com/us/
0
+ timeout 4 perl6 ./cro.p6 https://www.blog.google/products
124
+ timeout 4 perl6 ./cro.p6 https://zapier.com/apps/
124
+ timeout 4 perl6 ./cro.p6 https://www.dropbox.com/
0
+ timeout 4 perl6 ./cro.p6 https://passwords.google.com/
124
+ timeout 4 perl6 ./cro.p6 https://images.google.com/
124
+ timeout 4 perl6 ./cro.p6 https://google.com/settings/ads
124
+ timeout 4 perl6 ./cro.p6 https://firebase.google.com/
124
+ timeout 4 perl6 ./cro.p6 https://apps.google.com/
124
+ timeout 4 perl6 ./cro.p6 https://mail.google.com/
124
+ timeout 4 perl6 ./cro.p6 https://gmail.com/
124
+ timeout 4 perl6 ./cro.p6 https://myaccount.google.com/security
124
+ timeout 4 perl6 ./cro.p6 https://translate.google.com/
124
+ timeout 4 perl6 ./cro.p6 https://books.google.com/
124
+ timeout 4 perl6 ./cro.p6 https://fonts.google.com/
124
+ timeout 4 perl6 ./cro.p6 https://domains.google/
124
+ timeout 4 perl6 ./cro.p6 https://support.google.com/legal
124
+ timeout 4 perl6 ./cro.p6 http://history.google.com/
124
$ cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux buster/sid"
NAME="Debian GNU/Linux"
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
$ perl6 -v
This is Rakudo Star version 2018.04.1 built on MoarVM version 2018.04.1
implementing Perl 6.c.
$ zef info Cro::HTTP::Client
- Info for: Cro::HTTP::Client
- Identity: Cro::HTTP:ver<0.7.6.1>
- Recommended By: /usr/share/perl6/site
- Installed: Yes
Description: Asynchronous HTTP, both client and server side. Includes HTTP/2.0 support.
License: Artistic-2.0
Source-url: https://github.com/croservices/cro-http.git
Provides: 47 modules
Depends: 11 items
$ dpkg -l | grep -i ssl
ii libcrypt-openssl-bignum-perl 0.09-1 amd64 Perl module to access OpenSSL multiprecision integer arithmetic libraries
ii libcrypt-openssl-random-perl 0.11-1+b5 amd64 module to access the OpenSSL pseudo-random number generator
ii libcrypt-openssl-rsa-perl 0.30-1 amd64 module for RSA encryption using OpenSSL
ii libcurl4:amd64 7.60.0-2 amd64 easy-to-use client-side URL transfer library (OpenSSL flavour)
ii libflac8:amd64 1.3.2-3 amd64 Free Lossless Audio Codec - runtime C library
ii libio-socket-ssl-perl 2.058-1 all Perl module implementing object oriented interface to SSL sockets
ii libnet-ssleay-perl 1.85-1 amd64 Perl module for Secure Sockets Layer (SSL)
ii libssl-dev:amd64 1.1.0h-4 amd64 Secure Sockets Layer toolkit - development files
ii libssl1.0.0:amd64 1.0.1t-1+deb8u6 amd64 Secure Sockets Layer toolkit - shared libraries
ii libssl1.0.2:amd64 1.0.2o-1 amd64 Secure Sockets Layer toolkit - shared libraries
ii libssl1.1:amd64 1.1.0h-4 amd64 Secure Sockets Layer toolkit - shared libraries
ii libwavpack1:amd64 5.1.0-3 amd64 audio codec (lossy and lossless) - library
ii libzstd1:amd64 1.3.4+dfsg-3 amd64 fast lossless compression algorithm
ii openfortivpn 1.7.1-1 amd64 Fortinet client for PPP+SSL VPN tunnel services
ii openssl 1.1.0h-4 amd64 Secure Sockets Layer toolkit - cryptographic utility
ii perl-openssl-defaults:amd64 3 amd64 version compatibility baseline for Perl OpenSSL packages
ii python-certifi 2018.4.16-1 all root certificates for validating SSL certs and verifying TLS hosts
ii python3-certifi 2018.4.16-1 all root certificates for validating SSL certs and verifying TLS hosts (python3)
ii ssl-cert 1.0.39 all simple debconf wrapper for OpenSSL
This curl sample works fine (althought you will receive 40x because of wrong credentials):
curl -i -X POST
-A "sadjlksajdksajdlksjd"
-H "Accept: application/json"
-H "Content-Type: application/json"
-d '{
"username":"YouUserName",
"password":"YourPassWord"
}' https://account-api-test.icann.org/api/authenticate
Also in LWP all is fine:
use LWP::Simple;
my $host = 'https://account-api-test.icann.org/api/authenticate';
my %headers = ( 'Content-Type' => 'application/json' );
my $content = '{"username":"YouUserName","password":"YourPassWord"}';
my $html = LWP::Simple.post($host, %headers, $content);
But Cro gives an 50* error:
my $resp = await Cro::HTTP::Client.post('https://account-api-test.icann.org/api/authenticate/',
content-type => 'application/json',
headers => [Accept => 'application/json'],
body => { :username('YouUserName'), :password('YourPassWord') }
);
Here is a trace:
[TRACE(anon 1)] Cro::ConnectionConditional EMIT TCP Message
50 4f 53 54 20 2f 61 70 69 2f 61 75 74 68 65 6e POST /api/authen
74 69 63 61 74 65 2f 20 48 54 54 50 2f 31 2e 31 ticate/ HTTP/1.1
0d 0a 48 6f 73 74 3a 20 61 63 63 6f 75 6e 74 2d ..Host: account-
61 70 69 2d 74 65 73 74 2e 69 63 61 6e 6e 2e 6f api-test.icann.o
72 67 0d 0a 63 6f 6e 74 65 6e 74 2d 74 79 70 65 rg..content-type
3a 20 61 70 70 6c 69 63 61 74 69 6f 6e 2f 6a 73 : application/js
6f 6e 0d 0a 41 63 63 65 70 74 3a 20 61 70 70 6c on..Accept: appl
69 63 61 74 69 6f 6e 2f 6a 73 6f 6e 0d 0a 43 6f ication/json..Co
6e 6e 65 63 74 69 6f 6e 3a 20 63 6c 6f 73 65 0d nnection: close.
0a 43 6f 6e 74 65 6e 74 2d 6c 65 6e 67 74 68 3a .Content-length:
20 35 35 0d 0a 0d 0a 55....
[TRACE(anon 1)] Cro::ConnectionConditional EMIT TCP Message
7b 22 75 73 65 72 6e 61 6d 65 22 3a 20 22 59 6f {"username": "Yo
75 72 55 73 65 72 4e 61 6d 65 22 2c 22 70 61 73 urUserName","pas
73 77 6f 72 64 22 3a 20 22 59 6f 75 72 50 61 73 sword": "YourPas
73 57 6f 72 64 22 7d sWord"}
[TRACE(anon 1)] Cro::TLS::Connector EMIT TCP Message
48 54 54 50 2f 31 2e 31 20 33 30 31 20 4d 6f 76 HTTP/1.1 301 Mov
65 64 20 50 65 72 6d 61 6e 65 6e 74 6c 79 0d 0a ed Permanently..
44 61 74 65 3a 20 54 75 65 2c 20 31 31 20 44 65 Date: Tue, 11 De
63 20 32 30 31 38 20 31 37 3a 34 36 3a 35 35 20 c 2018 17:46:55
47 4d 54 0d 0a 53 65 72 76 65 72 3a 20 41 70 61 GMT..Server: Apa
63 68 65 0d 0a 4c 6f 63 61 74 69 6f 6e 3a 20 68 che..Location: h
74 74 70 3a 2f 2f 6d 61 69 6e 74 65 6e 61 6e 63 ttp://maintenanc
65 2e 69 63 61 6e 6e 2e 6f 72 67 2f 61 70 69 2f e.icann.org/api/
61 75 74 68 65 6e 74 69 63 61 74 65 2f 0d 0a 43 authenticate/..C
61 63 68 65 2d 43 6f 6e 74 72 6f 6c 3a 20 6d 61 ache-Control: ma
78 2d 61 67 65 3d 36 30 0d 0a 45 78 70 69 72 65 x-age=60..Expire
73 3a 20 54 75 65 2c 20 31 31 20 44 65 63 20 32 s: Tue, 11 Dec 2
30 31 38 20 31 37 3a 34 37 3a 35 35 20 47 4d 54 018 17:47:55 GMT
0d 0a 43 6f 6e 74 65 6e 74 2d 4c 65 6e 67 74 68 ..Content-Length
3a 20 32 35 34 0d 0a 43 6f 6e 6e 65 63 74 69 6f : 254..Connectio
6e 3a 20 63 6c 6f 73 65 0d 0a 43 6f 6e 74 65 6e n: close..Conten
74 2d 54 79 70 65 3a 20 74 65 78 74 2f 68 74 6d t-Type: text/htm
6c 3b 20 63 68 61 72 73 65 74 3d 69 73 6f 2d 38 l; charset=iso-8
38 35 39 2d 31 0d 0a 53 74 72 69 63 74 2d 54 72 859-1..Strict-Tr
61 6e 73 70 6f 72 74 2d 53 65 63 75 72 69 74 79 ansport-Security
3a 20 6d 61 78 2d 61 67 65 3d 34 38 32 31 31 32 : max-age=482112
30 30 3b 20 70 72 65 6c 6f 61 64 0d 0a 0d 0a 3c 00; preload....<
21 44 4f 43 54 59 50 45 20 48 54 4d 4c 20 50 55 !DOCTYPE HTML PU
42 4c 49 43 20 22 2d 2f 2f 49 45 54 46 2f 2f 44 BLIC "-//IETF//D
54 44 20 48 54 4d 4c 20 32 2e 30 2f 2f 45 4e 22 TD HTML 2.0//EN"
3e 0a 3c 68 74 6d 6c 3e 3c 68 65 61 64 3e 0a 3c >..<
74 69 74 6c 65 3e 33 30 31 20 4d 6f 76 65 64 20 title>301 Moved
50 65 72 6d 61 6e 65 6e 74 6c 79 3c 2f 74 69 74 Permanently</tit
6c 65 3e 0a 3c 2f 68 65 61 64 3e 3c 62 6f 64 79 le>.<body
3e 0a 3c 68 31 3e 4d 6f 76 65 64 20 50 65 72 6d >.
54 68 65 20 64 6f 63 75 6d 65 6e 74 20 68 61 73 The document has
20 6d 6f 76 65 64 20 3c 61 20 68 72 65 66 3d 22 moved <a href="
[93 bytes not displayed]
[TRACE(anon 1)] Cro::ConnectionConditional EMIT HTTP Response
HTTP/1.1 301 Moved Permanently
Date: Tue, 11 Dec 2018 17:46:55 GMT
Server: Apache
Location: http://maintenance.icann.org/api/authenticate/
Cache-Control: max-age=60
Expires: Tue, 11 Dec 2018 17:47:55 GMT
Content-Length: 254
Connection: close
Content-Type: text/html; charset=iso-8859-1
Strict-Transport-Security: max-age=48211200; preload
[TRACE(anon 1)] Cro::ConnectionConditional DONE
[TRACE(anon 1)] Cro::TLS::Connector DONE
[TRACE(anon 1)] Cro::ConnectionConditional DONE
[TRACE(anon 2)] Cro::HTTP::RequestSerializer EMIT TCP Message
50 4f 53 54 20 2f 61 70 69 2f 61 75 74 68 65 6e POST /api/authen
74 69 63 61 74 65 2f 20 48 54 54 50 2f 31 2e 31 ticate/ HTTP/1.1
0d 0a 48 6f 73 74 3a 20 6d 61 69 6e 74 65 6e 61 ..Host: maintena
6e 63 65 2e 69 63 61 6e 6e 2e 6f 72 67 0d 0a 63 nce.icann.org..c
6f 6e 74 65 6e 74 2d 74 79 70 65 3a 20 61 70 70 ontent-type: app
6c 69 63 61 74 69 6f 6e 2f 6a 73 6f 6e 0d 0a 41 lication/json..A
63 63 65 70 74 3a 20 61 70 70 6c 69 63 61 74 69 ccept: applicati
6f 6e 2f 6a 73 6f 6e 0d 0a 43 6f 6e 6e 65 63 74 on/json..Connect
69 6f 6e 3a 20 63 6c 6f 73 65 0d 0a 43 6f 6e 74 ion: close..Cont
65 6e 74 2d 6c 65 6e 67 74 68 3a 20 35 35 0d 0a ent-length: 55..
0d 0a ..
[TRACE(anon 2)] Cro::HTTP::RequestSerializer EMIT TCP Message
7b 22 75 73 65 72 6e 61 6d 65 22 3a 20 22 59 6f {"username": "Yo
75 72 55 73 65 72 4e 61 6d 65 22 2c 22 70 61 73 urUserName","pas
73 77 6f 72 64 22 3a 20 22 59 6f 75 72 50 61 73 sword": "YourPas
73 57 6f 72 64 22 7d sWord"}
[TRACE(anon 2)] Cro::TCP::Connector EMIT TCP Message
48 54 54 50 2f 31 2e 31 20 35 30 33 20 53 65 72 HTTP/1.1 503 Ser
76 69 63 65 20 55 6e 61 76 61 69 6c 61 62 6c 65 vice Unavailable
0d 0a 44 61 74 65 3a 20 54 75 65 2c 20 31 31 20 ..Date: Tue, 11
44 65 63 20 32 30 31 38 20 31 37 3a 34 36 3a 35 Dec 2018 17:46:5
37 20 47 4d 54 0d 0a 53 65 72 76 65 72 3a 20 41 7 GMT..Server: A
70 61 63 68 65 0d 0a 52 65 74 72 79 2d 41 66 74 pache..Retry-Aft
65 72 3a 20 33 30 0d 0a 43 6f 6e 74 65 6e 74 2d er: 30..Content-
4c 61 6e 67 75 61 67 65 3a 20 65 6e 2c 20 61 72 Language: en, ar
2c 20 66 72 2c 20 65 73 2c 20 72 75 2c 20 7a 68 , fr, es, ru, zh
0d 0a 4c 61 73 74 2d 4d 6f 64 69 66 69 65 64 3a ..Last-Modified:
20 54 75 65 2c 20 31 37 20 4d 61 72 20 32 30 31 Tue, 17 Mar 201
35 20 31 37 3a 31 37 3a 31 37 20 47 4d 54 0d 0a 5 17:17:17 GMT..
45 54 61 67 3a 20 22 38 36 63 2d 35 31 31 37 66 ETag: "86c-5117f
32 33 37 35 65 31 34 30 22 0d 0a 41 63 63 65 70 2375e140"..Accep
74 2d 52 61 6e 67 65 73 3a 20 62 79 74 65 73 0d t-Ranges: bytes.
0a 43 6f 6e 74 65 6e 74 2d 4c 65 6e 67 74 68 3a .Content-Length:
20 32 31 35 36 0d 0a 58 2d 46 72 61 6d 65 2d 4f 2156..X-Frame-O
70 74 69 6f 6e 73 3a 20 53 41 4d 45 4f 52 49 47 ptions: SAMEORIG
49 4e 0d 0a 52 65 66 65 72 72 65 72 2d 50 6f 6c IN..Referrer-Pol
69 63 79 3a 20 6f 72 69 67 69 6e 2d 77 68 65 6e icy: origin-when
2d 63 72 6f 73 73 2d 6f 72 69 67 69 6e 0d 0a 43 -cross-origin..C
6f 6e 6e 65 63 74 69 6f 6e 3a 20 63 6c 6f 73 65 onnection: close
0d 0a 43 6f 6e 74 65 6e 74 2d 54 79 70 65 3a 20 ..Content-Type:
74 65 78 74 2f 68 74 6d 6c 3b 20 63 68 61 72 73 text/html; chars
65 74 3d 55 54 46 2d 38 0d 0a 0d 0a et=UTF-8....
[TRACE(anon 2)] Cro::HTTP::ResponseParser EMIT HTTP Response
HTTP/1.1 503 Service Unavailable
Date: Tue, 11 Dec 2018 17:46:57 GMT
Server: Apache
Retry-After: 30
Content-Language: en, ar, fr, es, ru, zh
Last-Modified: Tue, 17 Mar 2015 17:17:17 GMT
ETag: "86c-5117f2375e140"
Accept-Ranges: bytes
Content-Length: 2156
X-Frame-Options: SAMEORIGIN
Referrer-Policy: origin-when-cross-origin
Connection: close
Content-Type: text/html; charset=UTF-8
[TRACE(anon 2)] Cro::HTTP::RequestSerializer DONE
Unexpected error: Server responded with 503 Service Unavailable //
Cro::HTTP::Response.new(request => Cro::HTTP::Request, status => 503, body-parser-selector => Cro::HTTP::BodyParserSelector::ResponseDefault, body-serializer-selector => Cro::HTTP::BodySerializerSelector::ResponseDefault, http-version => "1.1", http2-stream-id => Int)
This is the code that fails:
#!/usr/bin/env perl6
use v6.d;
use Cro::HTTP::Client;
use Cro::HTTP::Response;
sub MAIN {
my Str $userid = join '', do gather for ^18 { take ('a'..'z').pick };
my Str $challstr = '4%7C' ~ join '', do gather for ^128 { take 256.rand.floor.base(16).lc };
my Str $query = "act=getassertion&userid=$userid&challstr=$challstr";
my Cro::HTTP::Response $response = await Cro::HTTP::Client.get:
"https://play.pokemonshowdown.com/~~showdown/action.php?$query";
my Str $assertion = await $response.body-text;
fail 'Login server gave an HTTP response (???)' if $assertion.starts-with: '<';
fail 'This username is registered' if $assertion eq ';';
fail $assertion.substr: 2 if $assertion.starts-with: ';;';
say "Received the assertion: $assertion";
}When I run it, it runs for about 30 seconds until MoarVM panics:
bastille% CRO_TRACE=1 perl6 test.p6
[TRACE(anon 1)] Cro::ConnectionConditional EMIT TCP Message
50 52 49 20 2a 20 48 54 54 50 2f 32 2e 30 0d 0a PRI * HTTP/2.0..
0d 0a 53 4d 0d 0a 0d 0a ..SM....
[TRACE(anon 1)] Cro::ConnectionConditional EMIT TCP Message
00 00 24 04 00 00 00 00 00 00 01 00 00 10 00 00 ..$.............
02 00 00 00 00 00 03 00 00 00 64 00 04 00 00 ff ..........d.....
ff 00 05 00 00 40 00 00 06 00 00 03 e8 .....@.......
[TRACE(anon 1)] Cro::ConnectionConditional EMIT TCP Message
00 01 66 01 05 00 00 00 01 82 87 44 7f c7 01 2f ..f........D.../
7e 7e 73 68 6f 77 64 6f 77 6e 2f 61 63 74 69 6f ~~showdown/actio
6e 2e 70 68 70 3f 61 63 74 3d 67 65 74 61 73 73 n.php?act=getass
65 72 74 69 6f 6e 26 75 73 65 72 69 64 3d 77 70 ertion&userid=wp
77 6b 7a 69 6c 62 76 78 7a 72 67 6d 79 75 6e 70 wkzilbvxzrgmyunp
26 63 68 61 6c 6c 73 74 72 3d 34 25 37 43 39 39 &challstr=4%7C99
35 36 61 63 61 36 34 38 32 32 34 65 33 31 61 36 56aca648224e31a6
39 30 61 63 32 61 32 63 62 63 63 33 62 64 33 62 90ac2a2cbcc3bd3b
35 31 31 38 36 37 37 35 64 31 32 33 38 36 32 61 51186775d123862a
38 62 64 62 66 64 32 36 37 39 64 62 61 66 39 36 8bdbfd2679dbaf96
31 35 37 35 39 65 39 61 65 38 61 64 36 33 30 38 15759e9ae8ad6308
37 63 64 32 61 34 37 64 33 63 35 66 61 36 30 66 7cd2a47d3c5fa60f
32 37 38 36 64 62 65 32 66 62 66 32 37 65 61 38 2786dbe2fbf27ea8
64 35 32 61 31 62 39 37 34 39 31 33 38 63 39 64 d52a1b9749138c9d
31 64 61 39 34 33 37 61 64 37 61 37 64 39 31 34 1da9437ad7a7d914
36 34 35 34 35 33 30 32 65 31 37 64 66 65 32 32 64545302e17dfe22
64 33 35 66 38 62 34 63 34 64 32 37 63 61 32 37 d35f8b4c4d27ca27
66 34 65 62 33 62 65 64 65 66 61 39 39 39 39 30 f4eb3bedefa99990
66 34 32 38 39 32 31 38 65 33 35 37 36 62 63 38 f4289218e3576bc8
63 64 37 65 64 39 30 34 34 65 63 35 32 64 37 34 cd7ed9044ec52d74
65 34 35 63 35 63 63 37 30 31 64 34 33 33 32 35 e45c5cc701d43325
66 39 30 39 66 66 18 70 6c 61 79 2e 70 6f 6b 65 f909ff.play.poke
6d 6f 6e 73 68 6f 77 64 6f 77 6e 2e 63 6f 6d monshowdown.com
[TRACE(anon 1)] Cro::TLS::Connector EMIT TCP Message
00 00 12 04 00 00 00 00 00 00 03 00 00 01 00 00 ................
04 00 01 00 00 00 05 00 ff ff ff 00 00 04 08 00 ................
00 00 00 00 7f ff 00 00 ........
[TRACE(anon 1)] Cro::ConnectionConditional EMIT TCP Message
00 00 24 04 00 00 00 00 00 00 01 00 00 10 00 00 ..$.............
02 00 00 00 00 00 03 00 00 00 64 00 04 00 00 ff ..........d.....
ff 00 05 00 ff ff ff 00 06 00 00 03 e8 .............
[TRACE(anon 1)] Cro::ConnectionConditional EMIT TCP Message
00 00 00 04 01 00 00 00 00 .........
[TRACE(anon 1)] Cro::TLS::Connector EMIT TCP Message
00 00 00 04 01 00 00 00 00 00 00 00 04 01 00 00 ................
00 00 ..
[TRACE(anon 1)] Cro::TLS::Connector EMIT TCP Message
00 01 7c 01 04 00 00 00 01 3f e1 1f 88 61 96 e4 ..|......?...a..
59 3e 94 03 8a 68 1d 8a 08 01 7d 40 bd 71 b7 ae Y>...h....}@.q..
01 95 31 68 df 5f 92 49 7c a5 8a e8 19 aa fb 50 ..1h._.I|......P
93 8e c4 15 30 5a 99 56 7b 77 e5 8a 22 4b 25 a6 ....0Z.V{w.."K%.
92 09 19 78 2f 46 f3 81 74 0c b4 e3 25 0c 8d 88 ...x/F..t...%...
b2 46 e1 65 14 ac b5 24 62 0b 6d 85 e7 de 71 e6 .F.e...$b.m...q.
7e d4 2f 9a cd 61 51 06 f9 ed fa 50 0d ad a0 76 ~./..aQ....P...v
2c 20 28 17 ae 36 f5 c0 32 a6 2d 1b fe d4 ac 69 , (..6..2.-....i
9e 06 3e d4 90 f4 8c d5 40 bd 67 ea 5a 4f 52 27 ..>[email protected]'
3f 89 0f e2 a5 c8 7a 7e d4 c6 94 d7 aa aa 3d 7f ?.....z~......=.
77 ed 41 a4 81 51 5e 54 57 a5 6d c6 d9 0c 72 39 w.A..Q^TW.m...r9
21 02 f8 a4 95 f7 5d 01 01 2b 21 6d b0 bc fb ce !.....]..+!m....
3c cf da 85 f3 59 ac 2a 20 df 69 7e 94 03 2b 68 <....Y.* .i~..+h
1d 8b 08 02 02 81 7a e3 6f 5c 03 2a 62 d1 bf ed ......z.o\.*b...
4d 03 f2 b4 33 16 06 42 cb 8c 88 00 fb 52 b1 a6 M...3..B.....R..
78 18 fb 52 43 d2 33 55 05 67 ea 5a 4f 52 27 3f x..RC.3U.g.ZOR'?
89 0f e2 a5 c8 7a 7e d4 c6 94 d7 aa aa 3d 7f 7b .....z~......=.{
8b 84 84 2d 69 5b 05 44 3c 86 aa 6f 40 87 2f 9a ...-i[.D<..o@./.
ca 44 ac 44 ff bf a4 7e 56 1c c5 81 c0 34 f0 01 .D.D...~V....4..
f4 a5 85 ac f6 25 6b 6c 34 1f cc e9 4d 68 b8 c3 .....%kl4...Mh..
16 16 b3 d8 95 ad b0 cb 92 83 db 24 b4 0e c2 ae ...........$....
43 d2 c1 24 a9 62 4c 66 23 28 c8 7a 98 2f 9a ca C..$.bLf#(.z./..
44 ac 44 ff 3f 76 87 25 07 b6 49 68 1d 85 40 85 D.D.?v.%..Ih..@.
24 ab 58 3f 5f 8f 6a 36 5c 7a 50 b4 d3 ed 48 f8 $.X?_.j6\zP...H.
84 d2 d8 39 6d 00 05 5a 00 00 00 00 00 01 39 39 ...9m..Z......99
35 36 61 63 61 36 34 38 32 32 34 65 33 31 61 36 56aca648224e31a6
39 30 61 63 32 61 32 63 62 63 63 33 62 64 33 62 90ac2a2cbcc3bd3b
35 31 31 38 36 37 37 35 64 31 32 33 38 36 32 61 51186775d123862a
38 62 64 62 66 64 32 36 37 39 64 62 61 66 39 36 8bdbfd2679dbaf96
31 35 37 35 39 65 39 61 65 38 61 64 36 33 30 38 15759e9ae8ad6308
37 63 64 32 61 34 37 64 33 63 35 66 61 36 30 66 7cd2a47d3c5fa60f
32 37 38 36 64 62 65 32 66 62 66 32 37 65 61 38 2786dbe2fbf27ea8
64 35 32 61 31 62 39 37 34 39 31 33 38 63 39 64 d52a1b9749138c9d
[857 bytes not displayed]
MoarVM panic: Memory allocation failed; could not allocate 1467064320 bytes
According to top, it was trying to allocate around 3GB of memory. This makes it sound like there's a massive memory leak somewhere. Forcing the request to use HTTP/1.1 doesn't cause this to happen.
6.d.PREVIEW must be deprecated as of now but is still used by Cro::HTTP::Client.
use Cro::HTTP::Router;
use Cro::HTTP::Server;
use Cro::HTTP::Auth::Basic;
class MyUser does Cro::HTTP::Auth {
has $.username;
}
subset LoggedInUser of MyUser where { .username.defined }
class MyBasicAuth does Cro::HTTP::Auth::Basic[MyUser, "username"] {
method authenticate(Str $user, Str $pass --> Bool) {
say "authentication called";
forbidden unless $user eq 'admin' && $pass eq 'secret';
return $user eq 'admin' && $pass eq 'secret';
}
}
sub routes()
{
my %storage;
route {
before MyBasicAuth.new;
post -> LoggedInUser $user, 'api' {
request-body -> %json-object {
my $uuid = 'fake';
%storage{$uuid} = %json-object;
content 'application/json', %json-object
}
}
}
}
my Cro::Service $app = Cro::HTTP::Server.new:
:host<localhost>, :port<10000>,
application => routes;
$app.start;
react whenever signal(SIGINT) { $app.stop; exit }
and then curl --fail -H "Content-Type: application/x-www-form-urlencoded" -d '{"string":"hi"}' http://admin:secret@localhost:10000/api, so you will get:
An operation first awaited:
in sub request-body at /home/koto/Work/cro/cro-http/lib/Cro/HTTP/Router.pm6 (Cro::HTTP::Router) line 716
in block at perl6.pl6 line 24
in block at /home/koto/Work/cro/cro-http/lib/Cro/HTTP/Router.pm6 (Cro::HTTP::Router) line 130
Died with the exception:
Cannot unbox a type object (Nil) to int.
in sub decode-payload-to-pairs at /home/koto/Work/cro/cro-http/lib/Cro/HTTP/BodyParsers.pm6 (Cro::HTTP::BodyParsers) line 62
in method parse at /home/koto/Work/cro/cro-http/lib/Cro/HTTP/BodyParsers.pm6 (Cro::HTTP::BodyParsers) line 50
in method body at /home/koto/.perl6/sources/557D6C932894CB1ADE0F83C0596851F9212C2A67 (Cro::MessageWithBody) line 77
in sub request-body at /home/koto/Work/cro/cro-http/lib/Cro/HTTP/Router.pm6 (Cro::HTTP::Router) line 716
in block at perl6.pl6 line 24
in block at /home/koto/Work/cro/cro-http/lib/Cro/HTTP/Router.pm6 (Cro::HTTP::Router) line 130
Needs to be traced and fixed with a test.
I have the following post request captured from using CRO_TRACE=1
[TRACE(anon 1)] RequestSerializerExtension EMIT HTTP Request
POST /oauth/token HTTP/1.1
Host: login.eveonline.com
User-agent: WebService::EveOnline v0.5.0 (rakudo)
content-type: application/x-www-form-urlencoded
Authorization: Basic *<basic auth hash>*
This reqeust has the following payload:
[TRACE(anon 1)] Cro::ConnectionConditional EMIT TCP Message
00 00 64 00 00 00 00 00 01 63 6f 64 65 3d 65 6d ..d......code=em
72 66 46 6f 46 4b 71 37 6b 49 6c 37 44 4e 46 64 rfFoFKq7kIl7DNFd
38 6e 64 53 6a 30 77 73 41 5a 4a 4a 31 6e 5f 64 8ndSj0wsAZJJ1n_d
6b 69 37 6b 47 62 4f 71 61 6f 52 74 5a 57 41 6e ki7kGbOqaoRtZWAn
4f 5a 43 45 6e 46 62 59 7a 36 73 62 45 6b 30 26 OZCEnFbYz6sbEk0&
67 72 61 6e 74 5f 74 79 70 65 3d 61 75 74 68 6f grant_type=autho
72 69 7a 61 74 69 6f 6e 5f 63 6f 64 65 rization_code
The requet fails because the Content-Length header is not present. Is there something special I need to do to get CRO::HTTP::Client to send it?
Thanks
I saw this error in a log:
Type check failed in assignment to $!value; expected CookieValue but got Str ("WP+Cookie+check;")
in submethod BUILD at /opt/rakudo-pkg/share/perl6/site/sources/37FBFD9B566A62619AF26B1921EC347685B7E3A3 (Cro::HTTP::Cookie) line 119
in method unpack-cookie at /opt/rakudo-pkg/share/perl6/site/precomp/C7DF6F1622E3BEB7004AFFCAD661AC4004A7D493/5E/5E2DF5A33A94880673FB1EADC4F4FF29F2A2D5FF line 1
in method cookie-value at /opt/rakudo-pkg/share/perl6/site/sources/5E2DF5A33A94880673FB1EADC4F4FF29F2A2D5FF (Cro::HTTP::Request) line 140
in method existing-session at /opt/rakudo-pkg/share/perl6/site/sources/317C734B0B40A37D6FD7736A4DB5EAD2B6D55925 (Cro::HTTP::Session::InMemory) line 95
in block at /opt/rakudo-pkg/share/perl6/site/sources/317C734B0B40A37D6FD7736A4DB5EAD2B6D55925 (Cro::HTTP::Session::InMemory) line 89
Maybe CookieValue is too tight?
Hi !
I was wondering if there was a way to perform a request with Cro::HTTP::Client using a specific version of the IP protocol. I am used to curl and being able to do:
curl -4 http://example.com
curl -6 http://example.com
to force using one or the other.
The use case here in particular is to recover an apparent IPs from a web service, both IPv4 and IPv6.
Thanks for a great tool!
Keep up the good work.
Regards,
Mark.
From this line:
https://github.com/shuppet/p6-api-discord/blob/getting-stuff/lib/API/Discord/User.pm6#L40
Via this line:
https://github.com/shuppet/p6-api-discord/blob/getting-stuff/lib/API/Discord/HTTPResource.pm6#L55
We get Accept-Encoding: gzip:
https://gist.github.com/Altreus/da26124597b0c8795ff7b858b925adad
Can not get current uri of Cro::HTTP::Client after doing a GET method.
Could you guys add this feature support . :)
Whenever I try to access the headers used in a response, I try using the following code:
$resp.request.headers().gist.sayI get the following error:
Cannot look up attributes in a Cro::HTTP::Request type object
in method headers at /home/cbwood/Projects/rakudobrew/moar-master/install/share/perl6/site/sources/92FAFA2680EF5E9D02868F366DD1A69EC6268609 (Cro::HTTP::Message) line 10
Is the request object not getting passed back when the response is created?
perl6 -MCro::HTTP::Client -e 'my $client = Cro::HTTP::Client.new;my $url = "https://cs85.salesforce.com/services/data/v40.0/query/?q=SELECT+id,RecordTypeId,Name+FROM+Account+WHERE+RecordTypeId+IN+%28%27012b0000000UTlhAAG%27,%27012b0000000UTlfAAG%27,%27012b0000000UTliAAG%27%29+AND+ParentId=%27%27";my $resp = await $client.get: $url, :http<1.1>, auth => { bearer => q[token] };say $resp'
[TRACE(anon 1)] RequestSerializerExtension EMIT Cro::HTTP::Request.new(body-parser-selector => Cro::HTTP::BodyParserSelector::RequestDefault, body-serializer-selector => Cro::HTTP::BodySerializerSelector::RequestDefault, connection => Any, method => "GET", target => "/services/data/v40.0/query/", http-version => Str, http2-stream-id => Int)
[TRACE(anon 1)] Cro::HTTP::RequestSerializer EMIT Cro::TCP::Message.new(data => Blob[uint8].new(71,69,84,32,47,115,101,114,118,105,99,101,115,47,100,97,116,97,47,118,52,48,46,48,47,113,117,101,114,121,47,32,72,84,84,80,47,49,46,49,13,10,72,111,115,116,58,32,99,115,56,53,46,115,97,108,101,115,102,111,114,99,101,46,99,111,109,13,10), connection => Any)
Note I've stripped the Auth header off the request trace as I don't see how it can be an issue.
Decoded that's
GET /services/data/v40.0/query/ HTTP/1.1
Host: cs85.salesforce.com
So it looks like it's stripping off the query.
Cro::Uri.new(origin => "https://cs85.salesforce.com/services/data/v40.0/query/?q=SELECT+id,RecordTypeId,Name+FROM+Account+WHERE+RecordTypeId+IN+(012b0000000UTlhAAG,012b0000000UTlfAAG,012b0000000UTliAAG)+AND+ParentId=", scheme => "https", authority => "cs85.salesforce.com", userinfo => Str, host => "cs85.salesforce.com", host-class => Host::RegName, port => Int, path => "/services/data/v40.0/query/", query => "q=SELECT+id,RecordTypeId,Name+FROM+Account+WHERE+RecordTypeId+IN+(012b0000000UTlhAAG,012b0000000UTlfAAG,012b0000000UTliAAG)+AND+ParentId=", fragment => Str)
That's what I get when I run Cro::Uri.parse on the URL.
I'm wondering if it's this private sub?
sub construct-url($path) {
my $pos = $parsed-url.Str.index('/', 8);
$parsed-url.Str.comb[0..$pos-1].join ~ $path;
}
Possibly that needs to be able to accept a query too? (I'm getting soo over my head in the code right now though).
TravisCI: https://travis-ci.org/croservices/cro-http/jobs/280881015#L1590
My machine:
===> Testing: Cro::HTTP:ver<0.7>
Tried to get the result of a broken Promise
in block <unit> at t/http-client.t line 452
Original exception:
Tried to get
the result of a broken Promise
in block at /Users/cal/.zef/store/cro-http.git/4a3dba9ebfd37af099f360b733d25cd70b7219fb/lib/Cro/HTTP/Client.pm6 (Cro::HTTP::Client) line 473
Original exception:
Tried to get the result of a broken Promise
in block at /Users/cal/.zef/store/cro-core.git/52f6af5ef195ea4ac1bbedf12fca891289400923/lib/Cro.pm6 (Cro) line 346
Original exception:
Tried to get the result of a broken Promise
in block at /Users/cal/.zef/store/cro-tls.git/e090f2487e9a0ed1e6d752c55da8f9cda5ea2d7a/lib/Cro/TLS.pm6 (Cro::TLS) line 106
Original exception:
Cannot locate symbol 'SSL_CTX_set_alpn_protos' in native library 'libssl.dylib'
in method setup
at /Users/cal/local/share/perl6/sources/24DD121B5B4774C04A7084827BFAD92199756E03 (NativeCall) line 320
in method CALL-ME at /Users/cal/local/share/perl6/sources/24DD121B5B4774C04A7084827BFAD92199756E03 (NativeCall) line 424
in block at /Users/cal/.zef/store/p6-io-socket-async-ssl.git/15a75527c501f83c8a181f12809e1b4de73a0d1e/lib/IO/Socket/Async/SSL.pm6 (IO::Socket::Async::SSL) line 176
in block at /Users/cal/.zef/store/p6-io-socket-async-ssl.git/15a75527c501f83c8a181f12809e1b4de73a0d1e/lib/IO/Socket/Async/SSL.pm6 (IO::Socket::Async::SSL) line 166
===> Testing [FAIL]: Cro::HTTP:ver<0.7>
Perl6 version:
% perl6 -v
This is Rakudo version 2017.09-19-g9190a3b80 built on MoarVM version 2017.09.1
implementing Perl 6.c.
Our current Multipart class does not yet do the Associative role and implement AT-KEY and EXISTS-KEY.
We should do something like we do for WWWFormUrlEncoded (earlier in that file) - that is, build up an internal hash keyed on name on the first call to AT-KEY or EXISTS-KEY. It should return the Part with that name, or a Cro::HTTP::MultiValue containing the Part objects.
Finally, so you can mostly pretend a WWWFormUrlEncoded and a Multipart are about the same, then I suggest we add a Stringy and Str methods to Part that delegate to body-text.
We should have tests for these also.
If there is an error in a request that causes an exception, it would be very helpful if the request that caused it were also available in the exception object.
I'm not sure whether this is a bug in cro-http, or cro itself (or even MoarVM).
Consider the following script:
use Cro::HTTP::Client;
await [1..10].map({Cro::HTTP::Client.get("http://google.com").then({say "Got it"})});
On my system (Mac OS X 10.12.6, "This is Rakudo version 2017.07 built on MoarVM version 2017.07" from latest brew install perl6) it terminates promptly after printing "Got it" 10 times, as expected. However, changing the 10 to 50 results in only a couple of "Got it"'s getting printed (sometimes zero, always less than 10), and then no progress whatsoever — I get a deadlock, or something that really looks like it: no CPU is being consumed.
Using only one Cro::HTTP::Client object like below, makes little difference: (I seem to only get immediate deadlock, not a handful of successful queries as above)
use Cro::HTTP::Client;
my $client = Cro::HTTP::Client.new;
await [1..50].map({$client.get("http://google.com").then({say "Got it"})});
Error message follows:
No such method 'e' for invocant of type 'Slip'
in sub static-resource at /home/cbwood/Projects/cro-http/lib/Cro/HTTP/Router.pm6 (Cro::HTTP::Router) line 1064
in block at t/http-router.t line 1938
in block at /home/cbwood/Projects/cro-http/lib/Cro/HTTP/Router.pm6 (Cro::HTTP::Router) line 130=
Really like the design of Cro with its signature powered route dispatching etc.
This isn't really a bug - more a newbie would-like-to-have. I'm writing a RESTful API client - based on Cro::HTTP::Client. The service is returning 403 responses - probably due to a problem on my side. It would be great if I could toggle a TRACE of the HTTP exchange with an environment variable so I can see what is going on (e.g., MOJO_USERAGENT_DEBUG).
I can see there is the ability to trace the server parts of cro - I wonder if there is something for the client-side?
Thanks again for Cro.
Something like:
use Cro::HTTP::BodySerializers;
use YAML;
class Cro::HTTP::BodySerializer::YAML does Cro::HTTP::BodySerializer {
method is-applicable(Cro::HTTP::Message $message, $body --> Bool)
{
with $message.content-type {
(.type eq 'text' && .subtype eq 'x-yaml') &&
($body ~~ Map || $body ~~ List)
}
else {
False
}
}
method serialize(Cro::HTTP::Message $message, $body --> Supply)
{
my $yaml = yaml.dump($body).encode('utf-8');
self!set-content-length($message, $yaml.bytes);
supply { emit $yaml }
}
}
Note, I used YAML instead of YAMLish... I realize the dependency hell this leads to, so I suspect you won't want it in this repository. (Could make a Cro::HTTP::BodySerializer::YAMLish too..)
Where is the right place for this? Could you put it in a separate croservices repository? Should I simply release it separately to ecosystem through CPAN myself?
Tried to put an emojum in a URI.
in method panic at /home/altreus/rakudo/rakudo-star-2018.04/install/share/perl6/site/sources/2324DD93FE1564E732D8C179A4652788D7AC67A6 (Cro::Uri) line 173
in regex URI at /home/altreus/rakudo/rakudo-star-2018.04/install/share/perl6/site/sources/2324DD93FE1564E732D8C179A4652788D7AC67A6 (Cro::Uri) line 27
in regex TOP at /home/altreus/rakudo/rakudo-star-2018.04/install/share/perl6/site/sources/2324DD93FE1564E732D8C179A4652788D7AC67A6 (Cro::Uri) line 23
in method parse at /home/altreus/rakudo/rakudo-star-2018.04/install/share/perl6/site/sources/2324DD93FE1564E732D8C179A4652788D7AC67A6 (Cro::Uri) line 313
in method request at /home/altreus/rakudo/rakudo-star-2018.04/install/share/perl6/site/sources/BD9BD084131C6E7D2BF4FE09570E5EBE604DB8FA (Cro::HTTP::Client) line 327
in method post at /home/altreus/rakudo/rakudo-star-2018.04/install/share/perl6/site/sources/BD9BD084131C6E7D2BF4FE09570E5EBE604DB8FA (Cro::HTTP::Client) line 298
in method send at /home/altreus/src/p6-api-discord/lib/API/Discord/HTTP.pm6 (API::Discord::HTTP) line 68
in method create at /home/altreus/src/p6-api-discord/lib/API/Discord/HTTP.pm6 (API::Discord::HTTP) line 111
in method add-reaction at /home/altreus/src/p6-api-discord/lib/API/Discord/Message.pm6 (API::Discord::Message) line 127
in block at examples/autostar.p6 line 12
in block at /home/altreus/src/p6-api-discord/lib/API/Discord.pm6 (API::Discord) line 172
in method handle-opcode at /home/altreus/src/p6-api-discord/lib/API/Discord/Connection.pm6 (API::Discord::Connection) line 143
in block at /home/altreus/src/p6-api-discord/lib/API/Discord/Connection.pm6 (API::Discord::Connection) line 126
Associated code exists in https://github.com/shuppet/p6-api-discord/
Cro::HTTP::Auth::WebToken::Bearer is using the "Auth" header rather than the "Authorization" header. Is there a reason for this?
I have a cookie set by other programs (javascript actually) which I want to import via Cro.
I find myself doing this:
get -> 'something', :$JSON! is cookie
{
my $decoded = decode-percents($JSON);
my %session = from-json($decoded);
}
I grabbed decode-percents from Cro::URI.
Cro::HTTP::Cookie currently restricts CookieValue to a certain set of values instead of encoding/decoding it the same as is done with URI segments.
Sending content over a couple MB causes "operation canceled". The amount of data shipped seems to vary from run to run so I expect some buffer is filling up. Of course, I'm not exactly sure where it happens as I'm not getting a backtrace and "operation canceled" doesn't exist in the cro source.
This is on the croservices/cro-http:0.7.6 container within kubernetes 1.10.5.
use Cro::HTTP::Router;
sub routes() is export {
route {
get -> 'sample.txt' {
content 'text/plain', 'A' x 10_000_000;
}
}
}
I found the problem with binary data (images) but it seems to happen with text too.
The 2 files out and err seems to be completly empty.
$ perl6 -Ilib t/http-log-file.t
not ok 1 - Correct responses logged
# Failed test 'Correct responses logged'
# at t/http-log-file.t line 68
# expected a match with: /'[OK] 200 / - ' ('127.0.0.1' || '::1') \n
# '[OK] 200 /route - ' ('127.0.0.1' || '::1') \n
# '[OK] 200 /route - ' ('127.0.0.1' || '::1') \n
# '[OK] 200 /special/first - ' ('127.0.0.1' || '::1') \n/
# got: ""
not ok 2 - Error responses logged
# Failed test 'Error responses logged'
# at t/http-log-file.t line 74
# expected a match with: /'[ERROR] 500 /error - ' ('127.0.0.1' || '::1') \n/
# got: ""
1..2
# Looks like you failed 2 tests of 2
I couldn't figure out how to do this easily but I might put more effort into it if no one can see an obvious way of doing it quickly.
Basically, I would like to handle 4xx errors from webservices that have codes outside of the standard set: Twitter and Discord have different rate-limiting status codes (420 and 429 respectively), but I'm having trouble creating a role that correctly extends request.
It would be far easier if I could supply a method that was able to retry, but I don't know enough Perl6 yet to refactor request such that it can still .emit properly.
So I will be experimenting with my own patches for academic purposes but I beseech thee to write it properly for the good of the future :)
cro-http/lib/Cro/HTTP/Router.pm6
Lines 942 to 971 in 8084e8e
Perhaps it should be IO() $base?
zef install --/test cro
===> Searching for: cro
No candidates found matching identity: cro
We were looking into a separate issue and noticed that we got a POST request via HTTP/1.1 even when the client object was constructed with http => 2.
It seems most likely to be a tracer issue, according to sena_kun on IRC
13:30:11 <sena_kun> oh, this is probably just a mistake of tracer.
13:30:29 <sena_kun> `PRI * HTTP/2.0` is being sent.
I'm hand-rolling the http-signature spec and its been working until it connected to an http/2 server. The value of the header field has been lowercased, when the spec says the key should be lowercased. Repeating the same request with :http<1.1> fixes the problem.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.