swlib / saber Goto Github PK
View Code? Open in Web Editor NEW⚔️ Saber, PHP异步协程HTTP客户端 | PHP Coroutine HTTP client - Swoole Humanization Library
Home Page: https://packagist.org/packages/swlib/saber
License: Apache License 2.0
⚔️ Saber, PHP异步协程HTTP客户端 | PHP Coroutine HTTP client - Swoole Humanization Library
Home Page: https://packagist.org/packages/swlib/saber
License: Apache License 2.0
例如这个链接,返回的是302跳转,然后用此工具获取到的内容的空的,报404错误
而用CURL 设置 重定向跟踪可以正确得到内容
请问在本工具里需要怎么做设置呢?
PSR的RequestInterface没有规定实现exec方法,所以我设计组装一个http客户端无关的request的时候,我肯定不能绑定和客户端强相关的exec方法到Request上,因为每个客户端的异常类型、处理逻辑都不相同。
我设计一个composer组件,在组装请求部分,返回了个psr对象,本意是guzzle或者saber等支持PSR标准的HTTP客户端都可以按psr标准把这个对象代表的请求发送出去,现在Guzzle可以做到($guzzleClient->send($PSRrequest)
)而saber因为把PSR相关的处理逻辑绑定到他自定义的Request上,导致这样的设计没法实施。
Line 698 in 2388361
$responses = Swlib\SaberGM::requests($urls, [
'exception_report' => 0,
'retry_time' => 3,
'retry' => function (Swlib\Saber\Request $request) {
$uri = $request->getUri();
echo date('Y-m-d H:i:s') . " DNS hijack check {$uri} re-requesting...\n";
},
]);
[root@localhost urlcheckerclient]# php check_dns.php
2018-11-28 06:01:15 DNS hijack check http://www.mantoub.com/ re-requesting...
2018-11-28 06:01:15 DNS hijack check http://www.mantoub.com/ re-requesting...
2018-11-28 06:01:18 DNS hijack check http://www.mantoub.com/ re-requesting...
PHP Warning: DOMDocument::loadHTML(): Empty string supplied as input in /vagrant_data/urlcheckerclient/vendor/swlib/util/src/DataParser.php on line 160
2018-11-28 06:01:24 DNS hijack check http://www.mantoub.com/ re-requesting...
2018-11-28 06:01:34 DNS hijack check http://www.mantoub.com/ re-requesting...
2018-11-28 06:01:40 DNS hijack check http://www.mantoub.com/ re-requesting...
PHP Warning: DOMDocument::loadHTML(): Empty string supplied as input in /vagrant_data/urlcheckerclient/vendor/swlib/util/src/DataParser.php on line 160
swoole 4.4.0 版本,使用saber报错Class Swoole\Buffer is deprecated, it will be removed in v4.5.0。
file:vendor/swlib/http/src/SwooleBuffer.php line 19
请问适配4.4以上版本的saber什么时候会有呢
测试代码:
<?php
require('vendor/autoload.php');
go(function() {
$saber = Swlib\Saber::create([
'base_uri' => 'http://127.0.0.1',
'headers' => [
'Host' => 'example.com',
'X-Test' => 'test'
]
]);
echo $saber->get('/host.php');
echo "\n-------------------\n";
$saber = Swlib\Saber::create([
'base_uri' => 'http://127.0.0.1'
]);
$saber->setOptions([
'headers' => [
'Host' => 'example.com',
'X-Test' => 'test'
]
]);
echo $saber->get('/host.php');
});
服务器输出内容是:(打印了$_SERVER):
HTTP/1.1 200 OK
Server: nginx
Date: Wed, 03 Oct 2018 02:57:14 GMT
Content-Type: text/html; charset=UTF-8
Transfer-Encoding: chunked
Connection: keep-alive
Vary: Accept-Encoding
Content-Encoding: gzip
Array
(
[TEMP] => /tmp
[TMPDIR] => /tmp
[TMP] => /tmp
[PATH] => /usr/local/bin:/usr/bin:/bin
[USER] => www
[HOME] => /home/www
[HTTP_X_TEST] => test
[HTTP_CONTENT_TYPE] => application/x-www-form-urlencoded
[HTTP_USER_AGENT] => Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36
[HTTP_ACCEPT_ENCODING] => gzip
[HTTP_ACCEPT] => text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
[HTTP_HOST] => 127.0.0.1
[HTTP_CONNECTION] => keep-alive
[REDIRECT_STATUS] => 200
[SERVER_NAME] => _
[SERVER_PORT] => 80
[SERVER_ADDR] => 127.0.0.1
[REMOTE_PORT] => 46126
[REMOTE_ADDR] => 127.0.0.1
[SERVER_SOFTWARE] => nginx/1.12.2
[GATEWAY_INTERFACE] => CGI/1.1
[REQUEST_SCHEME] => http
[SERVER_PROTOCOL] => HTTP/1.1
[DOCUMENT_ROOT] => /web/wwwroot/default
[DOCUMENT_URI] => /host.php
[REQUEST_URI] => /host.php
[SCRIPT_NAME] => /host.php
[CONTENT_LENGTH] =>
[CONTENT_TYPE] => application/x-www-form-urlencoded
[REQUEST_METHOD] => GET
[QUERY_STRING] =>
[SCRIPT_FILENAME] => /web/wwwroot/default/host.php
[FCGI_ROLE] => RESPONDER
[PHP_SELF] => /host.php
[REQUEST_TIME_FLOAT] => 1538535434.4496
[REQUEST_TIME] => 1538535434
)
-------------------
HTTP/1.1 200 OK
Server: nginx
Date: Wed, 03 Oct 2018 02:57:14 GMT
Content-Type: text/html; charset=UTF-8
Transfer-Encoding: chunked
Connection: keep-alive
Vary: Accept-Encoding
Content-Encoding: gzip
Array
(
[TEMP] => /tmp
[TMPDIR] => /tmp
[TMP] => /tmp
[PATH] => /usr/local/bin:/usr/bin:/bin
[USER] => www
[HOME] => /home/www
[HTTP_X_TEST] => test
[HTTP_CONTENT_TYPE] => application/x-www-form-urlencoded
[HTTP_USER_AGENT] => Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36
[HTTP_ACCEPT_ENCODING] => gzip
[HTTP_ACCEPT] => text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
[HTTP_HOST] => 127.0.0.1
[HTTP_CONNECTION] => keep-alive
[REDIRECT_STATUS] => 200
[SERVER_NAME] => _
[SERVER_PORT] => 80
[SERVER_ADDR] => 127.0.0.1
[REMOTE_PORT] => 46128
[REMOTE_ADDR] => 127.0.0.1
[SERVER_SOFTWARE] => nginx/1.12.2
[GATEWAY_INTERFACE] => CGI/1.1
[REQUEST_SCHEME] => http
[SERVER_PROTOCOL] => HTTP/1.1
[DOCUMENT_ROOT] => /web/wwwroot/default
[DOCUMENT_URI] => /host.php
[REQUEST_URI] => /host.php
[SCRIPT_NAME] => /host.php
[CONTENT_LENGTH] =>
[CONTENT_TYPE] => application/x-www-form-urlencoded
[REQUEST_METHOD] => GET
[QUERY_STRING] =>
[SCRIPT_FILENAME] => /web/wwwroot/default/host.php
[FCGI_ROLE] => RESPONDER
[PHP_SELF] => /host.php
[REQUEST_TIME_FLOAT] => 1538535434.4521
[REQUEST_TIME] => 1538535434
)
可以看到,X-Test头是成功了的,Host头修改无效
在docker-compose或者k8s等有内置dns的情况下,host的域名一般不会写".xxx",而是直接用服务名,这时候过滤器会报错。
先post再get,竟然还保持post时的请求主体数据,建议get默认情况下不带任何请求主体数据。
另外,getParsedJsonObject(bool $reParse = false): object 在php7.1是不支持的。建议判断版本<=7.1时定义成getParsedJsonObject(bool $reParse = false): \stdClass
带了get参数去升级为websocket失败, 使用js来连接是成功的
$websocket = \Swlib\SaberGM::websocket('ws://127.0.0.1:2023/CloseOrderSercive');
while (true) {
echo $websocket->recv(1) . "\n";
$websocket->push("hello");
co::sleep(1);
}
I keep getting this error on the latest version of this library (with latest swoole)
$res = $saber->get(
$uri,
[
'timeout' => 60,
'redirect' => false
]
);
HTTP -1 Unknown: Connect timeout! the server is not listening on the port or the network is missing!
go(function () use ($host, $port, $poolSize, $queueKey) {
$pool = new DotCloud\RedisPool($host, $port, $poolSize);
while (true) {
$redis = $pool->get();
if ($redis->lLen($queueKey) > 0) {
$data = $redis->rPop($queueKey);
if (empty($data)) {
$pool->put($redis);
continue;
}
// ......
// redis 里取出数据[ ['uri'=> ''], ['uri'=> ''] ],使用requests并发请求
$responses = Swlib\SaberGM::requests($urls);
// ......
}
$pool->put($redis);
}
});
# php --ri swoole
swoole
swoole support => enabled
Version => 4.2.9
Author => Swoole Group[email: [email protected]]
coroutine => enabled
debug => enabled
trace_log => enabled
epoll => enabled
eventfd => enabled
signalfd => enabled
cpu_affinity => enabled
spinlock => enabled
rwlock => enabled
sockets => enabled
openssl => OpenSSL 1.0.2k-fips 26 Jan 2017
http2 => 1.31.1
pcre => enabled
zlib => enabled
mutex_timedlock => enabled
pthread_barrier => enabled
futex => enabled
mysqlnd => enabled
async_redis => enabled
Directive => Local Value => Master Value
swoole.enable_coroutine => On => On
swoole.aio_thread_num => 2 => 2
swoole.display_errors => On => On
swoole.use_namespace => On => On
swoole.use_shortname => On => On
swoole.fast_serialize => Off => Off
swoole.unixsock_buffer_size => 8388608 => 8388608
# cat /var/log/messages | grep swoole
Dec 3 11:06:01 izwz99ilp8qh025s8ak1y9z kernel: php[12589]: segfault at 8 ip 00007fdd0544409b sp 00007fdd04f9fed0 error 4 in swoole.so[7fdd053c9000+132000]
Dec 3 13:40:32 izwz99ilp8qh025s8ak1y9z kernel: php[16214]: segfault at 8 ip 00007fd42a84409b sp 00007fd42a39fed0 error 4 in swoole.so[7fd42a7c9000+132000]
# uname -a
Linux izwz99ilp8qh025s8ak1y9z 3.10.0-693.2.2.el7.x86_64 #1 SMP Tue Sep 12 22:26:13 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
# php -v
PHP 7.1.24 (cli) (built: Nov 7 2018 18:08:20) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.1.0, Copyright (c) 1998-2018 Zend Technologies
with Zend OPcache v7.1.24, Copyright (c) 1999-2018, by Zend Technologies
# gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/4.8.5/lto-wrapper
Target: x86_64-redhat-linux
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-bootstrap --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-linker-hash-style=gnu --enable-languages=c,c++,objc,obj-c++,java,fortran,ada,go,lto --enable-plugin --enable-initfini-array --disable-libgcj --with-isl=/builddir/build/BUILD/gcc-4.8.5-20150702/obj-x86_64-redhat-linux/isl-install --with-cloog=/builddir/build/BUILD/gcc-4.8.5-20150702/obj-x86_64-redhat-linux/cloog-install --enable-gnu-indirect-function --with-tune=generic --with-arch_32=x86-64 --build=x86_64-redhat-linux
Thread model: posix
gcc version 4.8.5 20150623 (Red Hat 4.8.5-28) (GCC)
require_once './vendor/autoload.php';
go(function () {
$websocket = \Swlib\SaberGM::websocket('127.0.0.1:9093'); //这里想给后面加上token和uid咋加呢?
});
PHP Fatal error: Uncaught Swlib\Http\Exception\ConnectException: Connection is forcibly cut off by the remote server in /home/ihipop/Dev/workspace/composer/saber/src/Request.php:641
Stack trace:
#0 /home/ihipop/Dev/workspace/composer/taobao-top/src/client/Adapter/SaberAdapter.php(39): Swlib\Saber\Request->recv()
#1 {main}
thrown in /home/ihipop/Dev/workspace/composer/saber/src/Request.php on line 641
v1.0.6
use pool=>true
<?php
\Swoole\Runtime::enableCoroutine();
require __DIR__ . '/vendor/autoload.php';
go(function () {
$saber = Swlib\Saber::create([
'use_pool' => true,
'keep_alive' => true,
]);
$i = 1;
while (true) {
try{
// $result = $saber->psr()->withMethod('GET')->withUri(new \Swlib\Http\Uri('http://gw.api.taobao.com/router/rest'))->exec()->recv();
$result = $saber->get('http://gw.api.taobao.com/router/rest');
echo $result->statusCode .'=>'.$i. "\n";
}catch (\Throwable $e){
echo $e->__toString();
break;
}finally{
$i++;
}
}
echo $i;
});
如题,哈哈
`SSL/TLS证书
$client->set(array(
'ssl_cert_file' => $your_ssl_cert_file_path,
'ssl_key_file' => $your_ssl_key_file_path,
));
`
swoole官方文档上有双向认证的配置,在saber中是怎么操作?
对于http状态码为304的请求,会一直超时,用curl试了是可以的
如果使用这种写法:array: [$object, 'method_name'], 在配置拦截器时应该这样:'before' => [[$object, 'method_name']]
swlib\util\src\DataParser.php 第94行
public static function stringToJsonArray(string $var): array
{
//无法解析被urlencode后的数据
var_dump($var);//%7b%22jlunit%22%3a%22%22%2c%22jsunit%22%3a%22%22%2c%22sgunit%22%3a%22%22%2c%22serialno%22%3a%2244b16204daaf41c3bed192681d70e5e5%22%2c%22errorcode%22%3a%220%22%2c%22errormessage%22%3a%22%22%7d
return ($var = json_decode($var, true)) === false ? [] : $var;
}
�请问如何�自动带着cookie请求,模拟一个用户登录
(base) MBP:blive chen$ php artisan test
[2019-07-03 19:49:48 @20747.0] ERROR (PHP Fatal Error: 10001):
Swoole\Coroutine\Http\Client::upgrade: API must be called in the coroutine
Stack trace:
#0 Swoole\Coroutine\Http\Client->upgrade() called at [/Users/chen/wwwroot/blive/vendor/swlib/saber/src/WebSocket.php:40]
#1 Swlib\Saber\WebSocket->__construct() called at [/Users/chen/wwwroot/blive/vendor/swlib/saber/src/Saber.php:308]
#2 Swlib\Saber->upgrade() called at [/Users/chen/wwwroot/blive/vendor/swlib/saber/src/Saber.php:134]
#3 Swlib\Saber::websocket() called at [/Users/chen/wwwroot/blive/vendor/swlib/saber/src/SaberGM.php:101]
#4 Swlib\SaberGM::websocket() called at [/Users/chen/wwwroot/blive/app/Console/Commands/test.php:43]
#5 App\Console\Commands\test->handle()
#6 call_user_func_array() called at [/Users/chen/wwwroot/blive/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php:32]
#7 Illuminate\Container\BoundMethod::Illuminate\Container\{closure}() called at [/Users/chen/wwwroot/blive/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php:90]
#8 Illuminate\Container\BoundMethod::callBoundMethod() called at [/Users/chen/wwwroot/blive/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php:34]
#9 Illuminate\Container\BoundMethod::call() called at [/Users/chen/wwwroot/blive/vendor/laravel/framework/src/Illuminate/Container/Container.php:576]
#10 Illuminate\Container\Container->call() called at [/Users/chen/wwwroot/blive/vendor/laravel/framework/src/Illuminate/Console/Command.php:183]
#11 Illuminate\Console\Command->execute() called at [/Users/chen/wwwroot/blive/vendor/symfony/console/Command/Command.php:255]
#12 Symfony\Component\Console\Command\Command->run() called at [/Users/chen/wwwroot/blive/vendor/laravel/framework/src/Illuminate/Console/Command.php:170]
#13 Illuminate\Console\Command->run() called at [/Users/chen/wwwroot/blive/vendor/symfony/console/Application.php:921]
#14 Symfony\Component\Console\Application->doRunCommand() called at [/Users/chen/wwwroot/blive/vendor/symfony/console/Application.php:273]
#15 Symfony\Component\Console\Application->doRun() called at [/Users/chen/wwwroot/blive/vendor/symfony/console/Application.php:149]
#16 Symfony\Component\Console\Application->run() called at [/Users/chen/wwwroot/blive/vendor/laravel/framework/src/Illuminate/Console/Application.php:90]
#17 Illuminate\Console\Application->run() called at [/Users/chen/wwwroot/blive/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php:133]
#18 Illuminate\Foundation\Console\Kernel->handle() called at [/Users/chen/wwwroot/blive/artisan:37]
(base) MBP:blive chen$
正在下载图片http://xxxxx/school/images/10~12/2_01.jpg
正在下载图片http://fxxxx/school/images/10~12/2_02.jpg
正在下载图片http://xxxx/school/images/10~12/2_03.jpg
正在下载图片http://fxxxxschool/images/10~12/2_04.jpg
terminate called after throwing an instance of 'std::bad_alloc'
what(): std::bad_alloc
以下 10 个请求 串发和并发的例子,为啥并发时间还是这么慢啊?
$responses = SaberGM::requests([
['uri' => 'https://www.qq.com'],
['uri' => 'https://www.baidu.com'],
['uri' => 'https://news.qq.com'],
['uri' => 'https://baike.baidu.com'],
['uri' => 'https://new.qq.com/ch/finance/'],
['uri' => 'https://zhidao.baidu.com'],
['uri' => 'https://new.qq.com/ch/auto/'],
['uri' => 'https://pan.baidu.com'],
['uri' => 'https://sports.qq.com'],
['uri' => 'https://www.hao123.com']
]);
$t1 = microtime(true);
SaberGM::get('https://www.qq.com');
SaberGM::get('https://www.baidu.com');
SaberGM::get('https://news.qq.com');
SaberGM::get('https://baike.baidu.com');
SaberGM::get('https://new.qq.com/ch/finance/');
SaberGM::get('https://zhidao.baidu.com');
SaberGM::get('https://new.qq.com/ch/auto/');
SaberGM::get('https://pan.baidu.com');
SaberGM::get('https://sports.qq.com');
SaberGM::get('https://www.hao123.com');
$t2 = microtime(true);
$runt = $t2-$t1;
$result = "multi-requests [ {$responses->success_num} ok, {$responses->error_num} error ]:\n" ."consuming-time: {$responses->time}s\nrunt: $runt";
// multi-requests [ 10 ok, 0 error ]:
// consuming-time: 0.95005702972412s
// runt: 1.6822638511658
测试脚本:
go(function () {
$pool = Saber::create([
'base_uri' => 'http://www.qq.com',
'use_pool' => 10
]);
var_dump($pool->get('/')->success);
});
错误信息:
Fatal error: Uncaught InvalidArgumentException: Host should not be empty! in /web/wwwroot/test/vendor/swlib/saber/src/Request.php:118
Stack trace:
#0 /web/wwwroot/test/vendor/swlib/saber/src/Request.php(152): Swlib\Saber\Request->getConnectionTarget()
#1 /web/wwwroot/test/vendor/swlib/saber/src/Saber.php(446): Swlib\Saber\Request->withPool(10)
#2 /web/wwwroot/test/vendor/swlib/saber/src/Saber.php(378): Swlib\Saber::transOptionsToRequest(Array, Object(Swlib\Saber\Request))
#3 /web/wwwroot/test/vendor/swlib/saber/src/Saber.php(145): Swlib\Saber->setOptions(Array, Object(Swlib\Saber\Request))
#4 /web/wwwroot/test/vendor/swlib/saber/src/Saber.php(108): Swlib\Saber->__construct(Array)
#5 /web/wwwroot/test/test.php(14): Swlib\Saber::create(Array)
#6 {main}
thrown in /web/wwwroot/test/vendor/swlib/saber/src/Request.php on line 118
use_pool不为数字时则正常,简单看了一下是因为Request.php里面处理不一样:
if (is_numeric($this->use_pool)) {
// limit max num
ClientPool::getInstance()->setMaxEx($this->getConnectionTarget(), $bool_or_max_size);
}
[
'before' => function (Swlib\Saber\Request $request) {
echo "\n=====================设置Cookies====================\n";
},
'after' => function (Swlib\Saber\Response $response) {
echo "\n=====================获取Cookies====================\n";
},
]
我的想法是在以上拦截器里写,但不知如何写这块的代码。看文档说要去看Saber\Http,但没找到相关代码和文档
请求了次数多了以后就出现这样的问题,已经开启连接池
文档上没说明到底多大的限制 https://wiki.swoole.com/wiki/page/246.html
hp --ri swoole
swoole
Swoole => enabled
Author => Swoole Team <[email protected]>
Version => 4.3.1
Built => Apr 12 2019 18:22:24
coroutine => enabled
epoll => enabled
eventfd => enabled
signalfd => enabled
cpu_affinity => enabled
spinlock => enabled
rwlock => enabled
sockets => enabled
openssl => OpenSSL 1.1.0h 27 Mar 2018
http2 => enabled
pcre => enabled
zlib => enabled
mutex_timedlock => enabled
pthread_barrier => enabled
futex => enabled
mysqlnd => enabled
async_redis => enabled
coroutine_postgresql => enabled
Directive => Local Value => Master Value
swoole.enable_coroutine => On => On
swoole.display_errors => On => On
swoole.use_shortname => On => On
swoole.unixsock_buffer_size => 8388608 => 8388608
我看composer里是70,源码里也没啥不兼容70的地方,这个库简直php爬虫工程师的核弹啊
Fatal error: Uncaught Swlib\Http\Exception\ConnectException: Request timeout! the server hasn't responded over the timeout setting(5s)! in /var/www/html/swoole/vendor/swlib/saber/src/Request.php:641
使用代码
$http = new swoole_http_server("0.0.0.0", 9510);
$http->on("start", function ($server) use ($urls) {
echo "Swoole http server is started at http://127.0.0.1:9501\n";
for ($i = 0; $i < count($urls); $i++) {
$url = $urls[$i]['url'];
Swoole\Timer::tick(1000, function () use ($url) {
go(function () use($url) {
echo SaberGM::get($url, [
])->getBody();
});
});
}
});
請問有辦法不帶header嗎?
因為目前看起來都會自帶
/src/Request.php
的633
行,应为-3,项目中写成了3,部分代码:
public function recv()
{
retry_recv:
if (self::STATUS_WAITING !== $this->_status) {
throw new \BadMethodCallException('You can\'t recv because client is not in waiting stat.');
}
$this->client->recv($this->getTimeout());
$this->_status = self::STATUS_NONE;
$this->_time = microtime(true) - $this->_start_time;
$is_report = $this->getExceptionReport() & HttpExceptionMask::E_CONNECT;
$statusCode = $this->client->statusCode;
$errCode = $this->client->errCode;
if ($statusCode < 0 || $errCode !== 0) {
if ($is_report) {
if ($statusCode === -1) {
$message = 'Connect timeout! the server is not listening on the port or the network is missing!';
} elseif ($statusCode === -2) {
$timeout = $this->getTimeout();
$message = "Request timeout! the server hasn't responded over the timeout setting({$timeout}s)!";
} elseif ($statusCode === 3) { //这里写错啦!
$message = 'Connection is forcibly cut off by the remote server';
} else {
$message = "Linux Code {$errCode}: " . swoole_strerror($errCode);
}
$exception = new ConnectException($this, $statusCode, $message);
$ret = $this->callInterceptor('exception', $exception);
if (!$ret) {
$this->tryToRevertClientToPool(true);
throw $exception;
}
} else {
// Exception is no longer triggered after an exception is ignored
$this->setExceptionReport(HttpExceptionMask::E_NONE);
}
}
目前在用saber做clickhouse的包,由于ck的并发能力较低,适合大批量数据一次写入,默认的socket_buffer_size不够用了,或者在swoole底层可以通过co::set设置全局的配置,然后单个客户端的配置可以覆盖全局配置?
HTTP代理是时候怎么使用密码
主要是guzzle的api已经大规模使用了,如果直接在guzzle上跑协程可以很快很兼容
require __DIR__ . '/../vendor/autoload.php';
go(function () {
$tidList = [
'4824528997',
'5309603784',
'6042765344',
];
foreach ($tidList as $tid) {
$requests[] = [
'uri' => 'https://tieba.baidu.com/p/' . $tid . '?fid=52',
];
}
$res = \Swlib\SaberGM::requests($requests, [
'max_co' => 3,
]);
});
RT……最小化代码……
报错是这个:
RequestQueue.php#L28
Stack trace:
#0 /mnt/hgfs/144/wwwroot/dev_resources/vendor/swlib/saber/src/RequestQueue.php(118): Swlib\Saber\RequestQueue->enqueue(Object(Swlib\Saber\Request))
#1 /mnt/hgfs/144/wwwroot/dev_resources/vendor/swlib/saber/src/Saber.php(278): Swlib\Saber\RequestQueue->recv()
#2 /mnt/hgfs/144/wwwroot/dev_resources/vendor/swlib/saber/src/SaberGM.php(90): Swlib\Saber->requests(Array, Array)
#3 /mnt/hgfs/144/wwwroot/dev_resources/php/zzz_swooleCurl.php(23): Swlib\SaberGM::requests(Array, Array)
#4 {main}
thrown in /mnt/hgfs/144/wwwroot/dev_resources/vendor/swlib/saber/src/RequestQueue.php on line 28
如果 $tidList 只有两个, max_co 参数无论如何设定都不会产生错误
如果 $tidList 有三个,而且 max_co 大于等于3,就会产生这个报错……
目前还没有仔细去查怎么解决这个问题……不过这个似乎……可以算个bug吧………………
环境
ubuntu 16.04
php 7.1.7
swoole 4.2.13
swlib/saber v1.0.6
ContentType::MULTIPART
\Swlib\Util\DataParser::toMultipartString($options['data'], $boundary);
\Swlib\Util\DataParser::__callStatic line 80
\Swlib\Util\DataParser::arrayToMultipartString
请求在swoole作为网关分发请求到业务服务的场景下,如何使用saber来避免高并发场景下出现大量TIME_WAIT的现象?
连接池的方式我不是很理解,如果被请求方的webserver不支持HTTP KeepAlive的话,依然是短连接,那连接池里面的实例再每次发起新的请求时不是还要重新建立TCP吗?
下载一个重定向的链接文件失败
go(function(){
SaberGM::download(
'https://github.com/swlib/saber/archive/v1.0.6.tar.gz',
'./v1.0.6.tar.gz'
);
});
结果
You are being redirected.
代码是这样的
$chan = new \Swoole\Coroutine\Channel(1);
go(function () use($chan) {
$uri = 'http://10.40.32.1:8800';
$content = \Swlib\SaberGM::post($uri);
$chan->push($content);
});
$res = [];
$chanNum = 1;
while ($chanNum > 0) {
$item = $chan->pop();
$res[] = $item;
$chanNum--;
}
print_r($res);
但是我通过断点,看到阻塞发生在了
swlib/saber/src/Request.php $this->client->recv($this->getTimeout()); 这一行代码.
我通过tcpdump 抓包发现
17:34:06.867207 IP 4f86ed14dc50.40110 > bogon.8800: Flags [S], seq 623124665, win 29200, options [mss 1460,sackOK,TS val 2069769 ecr 0,nop,wscale 7], length 0
17:34:06.868292 IP bogon.8800 > 4f86ed14dc50.40110: Flags [S.], seq 452901758, ack 623124666, win 65535, options [mss 1460,wscale 2,eol], length 0
17:34:06.868325 IP 4f86ed14dc50.40110 > bogon.8800: Flags [.], ack 1, win 229, length 0
17:34:06.869648 IP 4f86ed14dc50.40110 > bogon.8800: Flags [F.], seq 1, ack 1, win 229, length 0
17:34:06.869798 IP bogon.8800 > 4f86ed14dc50.40110: Flags [.], ack 2, win 65535, length 0
17:34:06.872652 IP bogon.8800 > 4f86ed14dc50.40110: Flags [F.], seq 1, ack 2, win 65535, length 0
17:34:06.872667 IP 4f86ed14dc50.40110 > bogon.8800: Flags [.], ack 2, win 229, length 0
在tcp建立连接之后,客户端主动断开连接FIN. 并没有把链接交到应用层http.服务器web server都没有访问日志,这是什么原因呢? 难道是swoole底层问题?
首先确实是服务提供方的返回body有问题,但是DataParser::stringToJsonArray()这块是否考虑健壮一点。
Return value of Swlib\\Util\\DataParser::stringToJsonArray() must be of the type array, null returned
代码是这样的:
SaberGM::psr([
'uri' => 'https://httpbin.org/ip',
'ssl_verify_peer' => true,
'ssl_host_name' => 'httpbin.org',
'proxy'=>'http://192.162.133.40:47163',
])->withCAFile()->getProxy();
如果加了代理会报上面的错误,去掉代理正常访问。用guzzleHttp试过代理ip正常可用
Swlib\Http\SwooleBuffer 转字符串,就是乱码
如果响应的字节比较少,就不是乱码,比如get('http://myip.ipip.net/')
如果响应的字节多,就是乱码,比如get('http://www.myip.cn/')
不知道是不是 swoole 的问题,文档里也有人反映 https://wiki.swoole.com/wiki/page/582.html
环境
CentOS 7.3 + docker(alpine 3.8)
php 7.1.23,装了zlib
swoole 4.3.1
swlib/saber v1.0.6
731724ac8bad:~# php --ri swoole
swoole
Swoole => enabled
Author => Swoole Team [email protected]
Version => 4.3.1
Built => Mar 20 2019 12:52:44
coroutine => enabled
epoll => enabled
eventfd => enabled
signalfd => enabled
spinlock => enabled
rwlock => enabled
sockets => enabled
openssl => LibreSSL 2.7.4
http2 => enabled
mutex_timedlock => enabled
pthread_barrier => enabled
mysqlnd => enabled
async_redis => enabled
coroutine_postgresql => enabled
Directive => Local Value => Master Value
swoole.enable_coroutine => On => On
swoole.display_errors => On => On
swoole.use_shortname => On => On
swoole.unixsock_buffer_size => 8388608 => 8388608
731724ac8bad:~#
Web socket Demo https://html5demos.com/web-socket/
\Swoole\Runtime::enableCoroutine(true);
go(function(){
$websocket = \Swlib\SaberGM::websocket('wss://remy-ws.glitch.me:443');
while (true) {
echo $websocket->recv(-1) . "\n";
$websocket->push("hello");
co::sleep(1);
}
});
$ php src/client2.php
PHP Warning: Swoole\Coroutine\Http\Client::recv(): you should not use recv without defer. in /home/ihipop/Dev/workspace/***/vendor/swlib/saber/src/WebSocket.php on line 65
PHP Warning: Swoole\Coroutine\Http\Client::push(): websocket handshake failed, cannot push data. in /home/ihipop/Dev/workspace/***/vendor/swlib/saber/src/WebSocket.php on line 72
Does the coroutine http client of swoole support wss:// (Websocket Secure)
?
Should $websocke->client->setDefer();
be call automaticly at \Swlib\Saber\WebSocket::__construct
我想Post带一个header头X-Requested-With: XMLHttpRequest
$this->saber = Saber::create([
'base_uri' => $this->baseurl,
'use_pool' => true
]);
// 调用部分
// 循环从队列中取连接 请求 我的理解 这必然是协程的 使用了 连接池不是这么用的吗?
while ($this->pageQueue->count()) {
$res = $this->saber->get($this->pageQueue->dequeue());
}
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.