Node.js implementation of a proxy server (think Squid) with support for SSL, authentication and upstream proxy chaining.
Home Page: https://www.npmjs.com/package/proxy-chain
License: Apache License 2.0
Inbound request is correctly forwarded by proxy-chain to upstream HTTP proxy,
but the HTTP client (node request here for testing) doesn't get the response body until
the proxy-chain server is stopped (manually).
proxy-chain shows
Received response from target (200)
HTTP client still waits for response.
Hello.
I have an issue with breaking connection, that I don't want to continue. Let's say, I want to use proxy-chain as a load-balancer for my other proxies. But I want to select node depending on username and password. If password is incorrect or i have no addresses/nodes available - I want to reject this connection and set custom error status. For now I can only throw new Error(...) from customResponseFunction to break this connection. Can I do it more graceful, with setting statusCode and message? Thank you.
Hi,
I ran into an error where sometimes when running a server, it fails to secure a socket connection. Here's the stack trace:
{ Error: Client network socket disconnected before secure TLS connection was established
at TLSSocket.onConnectEnd (_tls_wrap.js:1088:19)
at Object.onceWrapper (events.js:277:13)
at TLSSocket.emit (events.js:194:15)
at endReadableNT (_stream_readable.js:1103:12)
at process._tickCallback (internal/process/next_tick.js:63:19)
code: 'ECONNRESET',
path: undefined,
host: null,
port: null,
localAddress: undefined }
Killing and starting the server over again fixes it (does not re-occur), but it happens in about 1 in 3 runs, and happens for every request. This is the trace on the server side:
Server[80]: 3 | Using HandlerTunnelChain
Server[80]: 3 | Connecting to upstream proxy...
Server[80]: 3 | Target socket assigned
Server[80]: 3 | Connected to upstream proxy
Server[80]: 3 | Target socket ended
Server[80]: 3 | Target socket closed
Server[80]: 3 | Source socket ended
Server[80]: 3 | Closing handler
Connection 3 closed
{ srcTxBytes: 203,
srcRxBytes: 139,
trgTxBytes: 143,
trgRxBytes: 205 }
Here's the request I'm making (which goes to an upstream proxy provider):
`
const userAgent = new UserAgent();
const userAgentString = userAgent.toString();
var agent = new HttpsProxyAgent(proxy_url);
request(
{
url: "https://example.com",
agent: agent,
method: 'get',
gzip: true,
headers: {
'User-Agent': userAgentString,
}
},
function(err, res, body) {
console.log(err)
}
)
`
Is there any to use a socks proxy for upstreamProxyUrl? I couldn't find any way to achieve this in the documents.
It seems that proxy-chain is using HTTP CONNECT for https addresses. I'm looking for a way to bypass the proxy for static assets (img, js, css, font). Since most of the websites are under an SSL certificate, not sure if it's possible to do so.
I saw that https://github.com/luminati-io/luminati-proxy is able to do it but limited to its own proxy.
Any idea?
Hey guys great project.
I am looking at either running multiple instances of proxy-chain or alternately finding a way to listen on multiple ports (but specify different upstream proxy based on the port a request lands on).
My question is:
Which do you think makes more sense?
I didn't see anything in the docs about setting a single instance to listen on more than one port so I guess if that isn't a possibility then I will fall back and spin multiple instances which may make more sense anyway?
Hi,
I'm getting the following response from my proxy-chain server:
{ Error: unable to verify the first certificate at TLSSocket.onConnectSecure (_tls_wrap.js:1051:34) at TLSSocket.emit (events.js:189:13) at TLSSocket._finishInit (_tls_wrap.js:633:8) code: 'UNABLE_TO_VERIFY_LEAF_S IGNATURE' }
I have it behind a GCP load balancer - when connecting to it via an instance directly, it runs perfectly fine. However when it's behind a load balancer, something with the intermediate certificate isn't set correctly.
I've seen solutions to this problem using something like:
var rootCas = require('ssl-root-cas/latest').create(); require('https').globalAgent.options.ca = rootCas;
Is it possible to configure proxy-chain like this? I can't find any references to https in the source.
Thanks!
Fresh install in a clean project with a copy/paste of the exact demo script from the readme,. Happens with the upstreamProxyUrl set to a valid proxy or null.
Executing the following curl command:
curl -v -x http://bob:[email protected]:8000 https://ifconfig.co/json
works fine. The proxy responds with the correct response (200 + data).
When I try sending an invalid password, the client receives the correct response (407). The issue is that the server crashes when this occurs (see logs for the node process below).
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 8000 (#0)
* Establish HTTP proxy tunnel to ifconfig.co:443
* Proxy auth using Basic with user 'bob'
> CONNECT ifconfig.co:443 HTTP/1.1
> Host: ifconfig.co:443
> Proxy-Authorization: Basic Ym9iOlRvcHNlY3JldA==
> User-Agent: curl/7.54.0
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 407 Proxy Authentication Required
< Content-Type: text/html; charset=utf-8
< Proxy-Authenticate: Basic realm="ProxyChain"
* Authentication problem. Ignoring this.
< Server: ProxyChain
< Connection: close
< Content-Length: 27
<
* Received HTTP code 407 from proxy after CONNECT
* Closing connection 0
curl: (56) Received HTTP code 407 from proxy after CONNECT
Logs from node process:
ProxyServer[8000]: Listening...
Proxy server is listening on port 8000
ProxyServer[8000]: 1 | !!! Handling CONNECT ifconfig.co:443 HTTP/1.1
ProxyServer[8000]: 1 | Using HandlerTunnelDirect
ProxyServer[8000]: 1 | Connecting to target ifconfig.co:443
ProxyServer[8000]: 1 | Target socket assigned
ProxyServer[8000]: 1 | Connected
ProxyServer[8000]: 1 | Source socket ended
ProxyServer[8000]: 1 | Closing handler
Connection 1 closed
{ srcTxBytes: 4684,
srcRxBytes: 713,
trgTxBytes: 550,
trgRxBytes: 4589 }
ProxyServer[8000]: 1 | !!! Closed and removed from server
ProxyServer[8000]: 2 | !!! Handling CONNECT ifconfig.co:443 HTTP/1.1
ProxyServer[8000]: Request failed (status 407): Proxy credentials required.
events.js:174
throw er; // Unhandled 'error' event
^
Error: read ECONNRESET
at TCP.onStreamRead (internal/stream_base_commons.js:111:27)
Emitted 'error' event at:
at emitErrorNT (internal/streams/destroy.js:91:8)
at emitErrorAndCloseNT (internal/streams/destroy.js:59:3)
at process._tickCallback (internal/process/next_tick.js:63:19)
Process finished with exit code 1
Is it within the scope of this library to eventually support SOCKS5 proxies?
I had a look through your code and the tests suggest it's not supported currently.
using closeAnonymizedProxy without callback and using 'await' hangs and never closes the server.
version : latest
repro: example with anonymized proxy util function + add closeAnonymizedProxy at the end(before or after browser.close)
System : windows
Otherwise, the package is not usable from Electron. See apify/crawlee#522
The module is only needed for isInvalidHeader() in https://github.com/apifytech/proxy-chain/blob/master/src/tools.js#L2
Double-check that the HTTP header validation is compatible with Node.js' header validation (they change HTTP parser in version 12 I think, which has slightly different behavior), perhaps in unit tests we could test against Node's internal methods too, just to be sure.
Is it possible to switch the upstream proxy host without restarting the server? I'm currently using the anonymizeProxy helper like so:
const proxyChain = require('proxy-chain');
const user = '...';
const pass = '...';
const vpnHost = process.argv[2];
const oldProxyUrl = `http://${user}:${pass}@${vpnHost}:80`;
const newProxyUrl = proxyChain.anonymizeProxy(oldProxyUrl);
function cleanup(url) {
proxyChain.closeAnonymizedProxy(url);
}
newProxyUrl.then(url => {
console.log(url);
process.on('SIGINT', () => cleanup(url));
process.on('SIGTERM', () => cleanup(url));
});
But I have to restart the process to change the host. It would be nice if I could send a signal to the process and it would switch the upstream proxy without changing its own port.
I think one way to solve this might be to allow passing a desired port to anonymizeProxy.
Edit: I hacked around this by modifying PORT_SELECTION_CONFIG:
const proxyChainTools = require('proxy-chain/build/tools');
...
const vpnPort = process.argv[3];
...
// HACK: Force the proxy to choose a port by narrowing the min/max in config.
if (vpnPort) {
proxyChainTools.PORT_SELECTION_CONFIG.FROM = parseInt(vpnPort, 10);
proxyChainTools.PORT_SELECTION_CONFIG.TO = parseInt(vpnPort, 10);
}
Now I can quickly restart the node process to switch the host and keep the same port 🤷♂️
When I am sending request to HTTPS page through proxy-chain with prepareRequestFunction there is no useful info about request for identification.
For example if I am opening one HTTPS website two times in a row I want it to be passed through different proxies. But next time when I will open this website two times I need it to use the same proxies.
Mar 07 16:38:50 ip-172-31-62-12 [email protected] TypeError: Cannot read property '_httpMessage' of null
at ServerResponse.detachSocket (_http_server.js:223:17)
at HandlerTunnelChain.onTrgRequestConnect (/home/apifier/proxy/node_modules/proxy-chain/build/handler_tunnel_chain.js:91:30)
at Object.onceWrapper (events.js:418:26)
at ClientRequest.emit (events.js:311:20)
at Socket.socketOnData (_http_client.js:510:11)
at Socket.emit (events.js:311:20)
at addChunk (_stream_readable.js:294:12)
at readableAddChunk (_stream_readable.js:275:11)
at Socket.Readable.push (_stream_readable.js:209:10)
at TCP.onStreamRead (internal/stream_base_commons.js:186:23)
Hi !
I mounted the simplest instance of proxy-chain on an Amazon EC2 instance running with pm2, but when I came back today I found the proxy was down due to a memory leak with this error:
(node:26700) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 close listeners added to [Socket]. Use emitter.setMaxListeners() to increase limit
Do you have any idea of what is happening ?
Thanks !
What I'm trying to do is:
throw new ProxyChain.RequestError('Only Bob can use this proxy!', 400);
So when I use http, works fine, I receive an error I sent, but when I use https, I get -111 TUNNEL_CONNECTION_FAILED.
I kinda get why it might happen, but what do I do if I want receive my error, instead of -111?
Hi,
Can you elaborate at all on what these numbers represent:
{
srcTxBytes: Number,
srcRxBytes: Number,
trgTxBytes: Number,
trgRxBytes: Number,
}
For example, I tried doing a test with postman by doing a GET request to the README of your repository, but the size of request and response did not match up with any combination of the metrics above. Please help! This is a great tool but I'm having trouble measuring the traffic.
Thanks.
I need to modify the request header in the prepareRequestFunction callback. Can I do this?
const credentials = require(".././credentials");
const proxyChain = require("proxy-chain");
async function createSimpleProxy() {
const proxyHttpPort = 15000;
const proxyIp = `myip`;
const oldProxyUrl = `http://${credentials.username}:${
credentials.pw
}@${proxyIp}:${proxyHttpPort}`;
const newProxyUrl = await proxyChain.anonymizeProxy(oldProxyUrl);
console.log(newProxyUrl);
return newProxyUrl;
}
exports.createSimpleProxy = createSimpleProxy;
its being created properly getting http://127.0.0.1:26563
but after some time im getting
Unhandled rejection Error: listen EACCES: permission denied 0.0.0.0:49785
at Server.setupListenHandle [as _listen2] (net.js:1260:19)
at listenInCluster (net.js:1325:12)
at Server.listen (net.js:1412:7)
no process is running at this port and im using admin rights
Can I proxy requests to tor instances?
I'm thinking about using tor-request
Null trgSocket breaks Apify Actor run (see attached log for more details).
2020-02-04T14:32:40.741Z /home/myuser/node_modules/proxy-chain/build/handler_tunnel_chain.js:110
2020-02-04T14:32:40.743Z this.trgSocket.pipe(this.srcSocket);
2020-02-04T14:32:40.745Z ^
2020-02-04T14:32:40.747Z
2020-02-04T14:32:40.749Z TypeError: Cannot read property 'pipe' of null
2020-02-04T14:32:40.751Z at HandlerTunnelChain.onTrgRequestConnect (/home/myuser/node_modules/proxy-chain/build/handler_tunnel_chain.js:110:28)
2020-02-04T14:32:40.753Z at Object.onceWrapper (events.js:313:26)
2020-02-04T14:32:40.755Z at ClientRequest.emit (events.js:223:5)
2020-02-04T14:32:40.758Z at Socket.socketOnData (_http_client.js:490:11)
2020-02-04T14:32:40.760Z at Socket.emit (events.js:223:5)
2020-02-04T14:32:40.762Z at addChunk (_stream_readable.js:309:12)
2020-02-04T14:32:40.764Z at readableAddChunk (_stream_readable.js:290:11)
2020-02-04T14:32:40.766Z at Socket.Readable.push (_stream_readable.js:224:10)
2020-02-04T14:32:40.768Z at TCP.onStreamRead (internal/stream_base_commons.js:181:23)
On Apify platform, Node.js v12.14.1, NPM version 6.13.4.
ive noticed with some of the tools I use, basic authentication does not work - the username and password are null in the prepareRequestFunction. I think they must be getting lost somehow in the CONNECT event - any idea what might be causing this?
Hi, I wonder if there is a possibility to skip a request.
OS: MacOS
Version: 0.1.26
Problem: Method proxyChain.anonymizeProxy(oldProxyUrl) fails cause cannot pick the port.
It happens over and over again:
portastic:find Result reached the maximum of 1 ports, returning... +0ms
portastic:test Trying to test port 39565 +0ms
portastic:test TCP server on port 39565 closed +0ms
portastic:test Port 39565 was free +0ms
portastic:find Result reached the maximum of 1 ports, returning... +0ms
portastic:test Trying to test port 39566 +0ms
portastic:test TCP server on port 39566 closed +1ms
portastic:test Port 39566 was free +0ms
portastic:find Result reached the maximum of 1 ports, returning... +0ms
Do you have any guesses of the roots of this problem?
If request have multiple headers with same name, proxy-chain will leave only one (last). Example:
const http = require('http');
const ProxyChain = require('proxy-chain');
const server = http.createServer((req, res) => {
res.writeHead(200, [['Set-Cookie', 'foo=bar; path=/; HttpOnly'], ['Set-Cookie', 'bar=foo; path=/; HttpOnly']])
res.end();
}).listen(3000);
const proxyServer = (new ProxyChain.Server()).listen(8000);
Without proxy: curl 127.0.0.1:3000 -vvv
> GET / HTTP/1.1
> Host: 127.0.0.1:3000
> User-Agent: curl/7.55.1
> Accept: */*
>
< HTTP/1.1 200 OK
< Set-Cookie: foo=bar; path=/; HttpOnly
< Set-Cookie: bar=foo; path=/; HttpOnly
< Date: Mon, 18 Dec 2017 09:18:05 GMT
< Connection: keep-alive
< Transfer-Encoding: chunked
With proxy: curl -x 127.0.0.1:8000 127.0.0.1:3000 -vvv
> GET http://127.0.0.1:3000/ HTTP/1.1
> Host: 127.0.0.1:3000
> User-Agent: curl/7.55.1
> Accept: */*
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 OK
< Set-Cookie: bar=foo; path=/; HttpOnly
< Date: Mon, 18 Dec 2017 09:17:49 GMT
< Connection: keep-alive
< Transfer-Encoding: chunked
It seems that IPV6 is not supported now.
ProxyServer[8883]: 9 | !!! Closed and removed from server ProxyServer[8883]: 10 | !!! Handling CONNECT [2a00:1450:4014:80c::200a]:443 HTTP/1.1 ProxyServer[8883]: Request failed (status 400): Target "[2a00:1450:4014:80c::200a]:443" could not be parsed.
Is it possible to choose which ip to use on machine with multiple ips with this?
I get Error: net::ERR_SSL_PROTOCOL_ERROR when trying to access https site via http socket
puppeteer.launch({
headless: true,
args: [
`--window-size=${ua.data.viewportWidth},${ua.data.viewportHeight + windowFrame}`,
`--proxy-server=${await proxyChain.anonymizeProxy(getRandomProxy())}`,
]
}).then(async browser => {
const page = await browser.newPage();
await page.goto('https://some-site.com/'), {waitUntil: 'networkidle0'};
})
It seems that error is happening when site has Let'sEncrypt's cert, accessing sites with comodo certs (e.g. https://www.myip.com/) works
I would like to use proxy-chain with multiple upstreamProxyUrl. Does someone would advise on the best way to dynamically update upstreamProxyUrl ?
Thanks
Simon
Thank you for this useful module.
Is it possible to retry with another target if request failed with first one ?
Subject says it all.
I send a request to the proxy chain server with some specific header that i don't want to send to the upstream proxy.
I use that header to generate the upstream url in the prepareRequestFunction so i dont need to restart the server to change proxy and i already know how to fetch it but i want to remove it once i have generated the upstream url so it wont be send to the upstream proxy.
I have tried something like this but the header is still sent:
...
prepareRequestFunction: ({
request,
username,
password,
hostname,
port,
isHttp,
}) => {
let upstreamProxyUrl;
if (request.headers['proxy-to-use'] === undefined) throw new ProxyChain.RequestError("No proxy-to-use request header",400);
upstreamProxyUrl = `http://${_username}:${_password}@${request.headers['proxy-to-use']}`;
delete request.headers['proxy-to-use'];
return {
upstreamProxyUrl
};
}
...How can i change some property of the request object before sending the request to the upstream proxy?
Otherwise the proxy can crash with this error:
RangeError: Invalid status code: 0
at ServerResponse.writeHead (_http_server.js:194:11)
at HandlerForward.onTrgResponse (/home/bob/proxy/node_modules/proxy-chain/build/handler_forward.js:158:30)
at emitOne (events.js:116:13)
at ClientRequest.emit (events.js:211:7)
at HTTPParser.parserOnIncomingClient [as onIncoming] (_http_client.js:543:21)
at HTTPParser.parserOnHeadersComplete (_http_common.js:112:17)
at Socket.socketOnData (_http_client.js:440:20)
at emitOne (events.js:116:13)
at Socket.emit (events.js:211:7)
at addChunk (_stream_readable.js:263:12)
at readableAddChunk (_stream_readable.js:250:11)
at Socket.Readable.push (_stream_readable.js:208:10)
at TCP.onread (net.js:597:20)
Error: write after end
at write_ (_http_outgoing.js:622:15)
at ServerResponse.write (_http_outgoing.js:617:10)
at IncomingMessage.ondata (_stream_readable.js:639:20)
at emitOne (events.js:116:13)
at IncomingMessage.emit (events.js:211:7)
at IncomingMessage.Readable.read (_stream_readable.js:475:10)
at flow (_stream_readable.js:846:34)
at ServerResponse.<anonymous> (_stream_readable.js:707:7)
at emitNone (events.js:106:13)
at ServerResponse.emit (events.js:208:7)
at Socket.ondrain (internal/http.js:20:44)
at emitNone (events.js:111:20)
at Socket.emit (events.js:208:7)
at onwriteDrain (_stream_writable.js:474:12)
at afterWrite (_stream_writable.js:462:5)
at onwrite (_stream_writable.js:455:7)
at WriteWrap.afterWrite [as oncomplete] (net.js:876:12)
The stack trace is below. Also, it seems that ProxyChain server is emitting connectionClosed event for this connection, although prepareRequestFunction is never called for the connection.
Aug 17 02:43:32 ip-172-31-92-139 [email protected] ERROR {"level":"ERROR","msg":"Unexpected error in ProxyChain server","url":"http://www.acmetools.com/shop/tools/router-tables","exception":{"name":"TypeError","message":"The header content contains invalid characters","stack":"TypeError: The header content contains invalid characters\n at validateHeader (_http_outgoing.js:494:11)\n at ClientRequest.setHeader (_http_outgoing.js:498:3)\n at new ClientRequest (_http_client.js:173:14)\n at Object.request (http.js:38:10)\n at HandlerForward.run (/home/apifier/proxy/node_modules/proxy-chain/build/handler_forward.js:125:46)\n at Server.handlerRun (/home/apifier/proxy/node_modules/proxy-chain/build/server.js:411:21)\n at /home/apifier/proxy/node_modules/proxy-chain/build/server.js:208:24\n at tryCatcher (/home/apifier/proxy/node_modules/bluebird/js/release/util.js:16:23)\n at Promise._settlePromiseFromHandler (/home/apifier/proxy/node_modules/bluebird/js/release/promise.js:512:31)\n at Promise._settlePromise (/home/apifier/proxy/node_modules/bluebird/js/release/promise.js:569:18)\n at Promise._settlePromise0 (/home/apifier/proxy/node_modules/bluebird/js/release/promise.js:614:10)\n at Promise._settlePromises (/home/apifier/proxy/node_modules/bluebird/js/release/promise.js:693:18)\n at Async._drainQueue (/home/apifier/proxy/node_modules/bluebird/js/release/async.js:133:16)\n at Async._drainQueues (/home/apifier/proxy/node_modules/bluebird/js/release/async.js:143:10)\n at Immediate.Async.drainQueues (/home/apifier/proxy/node_modules/bluebird/js/release/async.js:17:14)\n at runCallback (timers.js:810:20)\n at tryOnImmediate (timers.js:768:5)\n at processImmediate [as _immediateCallback] (timers.js:745:5)"}}
Currently if no password is supplied, the ':' char is not added, causing the base64 of the username/password to be invalid
Hi,
I found this library very useful even though puppeter nowadays supports page.authenticate() but for some of my proxies I don't get authentication popup so only this library is the last hope or solutions like Squid. So thanks a lot.
Meanwhile I need a bit extend it and I'm curious now how to make it in a better way, because I have proxies that are limits me with concurrent connections, let's say 5 but browser for sure generate a waaaay more. I need something like a global throttler but not a network throttler, throttler for amount of active connections. If we reached limit then all further connections are coming into queue and awaiting of competition one of the processes that was queued already.
I checked the code a bit and basically my needs would solve a simple Promise inside onConnect callback. But to add it is not that trivial. Maybe you would have a better idea?
Thanks for any feedback.
Im using jest to test my code and Im getting a warning in tests that are using proxy-chain.
Jest did not exit one second after the test run has completed.
This usually means that there are asynchronous operations that weren't stopped in your tests. Consider running Jest with `--detectOpenHandles` to troubleshoot this issue.
It turned out that the following code is enought to get this message
const proxyChain = require('proxy-chain')
test('local proxy', async () => {
const lp = await proxyChain.anonymizeProxy(proxy);
await proxyChain.closeAnonymizedProxy(lp, true);
});A temporary solution for this problem is:
afterAll(async () => {
await new Promise(res => setTimeout(res, 5000));
});and jest's warning is gone.
It only supports web sockets via HTTP CONNECT method (used e.g. with SSL). The unit tests (testWsCall()) only test for that too. We should add full support for HTTP UPGRADE method in the main server and handle this correctly even with plain HTTP.
I'm using this with Capybara / Selenium WebDriver and notice sporadic ERR_TUNNEL_CONNECTION_FAILED errors when trying to load a page. Refreshing doesn't help. Some attempts work, some don't. I don't see any other error messages so I'm not really sure how to debug this. The only variable that changes between attempts is the upstream proxy hostname.
Versions:
chromedriver: ChromeDriver 2.36.540469 (1881fd7f8641508feb5166b7cae561d87723cfa8)
chrome: Version 68.0.3440.106 (Official Build) (64-bit)
capybara: 3.6.0
capybara-webkit: 1.15.0
proxy-chain: 0.2.4
Running the ps command, here's how the process looks that Capybara spawns:
/Applications/Google Chrome.app/Contents/MacOS/Google Chrome --disable-background-networking --disable-client-side-phishing-detection --disable-default-apps --disable-hang-monitor --disable-popup-blocking --disable-prompt-on-repost --disable-sync --disable-web-resources --enable-automation --enable-logging --force-fieldtrials=SiteIsolationExtensions/Control --ignore-certificate-errors --load-extension=/var/folders/kl/y1jrp7zs55sbx075jjjl3p280000gn/T/.org.chromium.Chromium.PogJ9o/internal --log-level=0 --metrics-recording-only --no-first-run --password-store=basic --proxy-server=http://127.0.0.1:34700 --remote-debugging-port=12736 --test-type=webdriver --use-mock-keychain --user-data-dir=/var/folders/kl/y1jrp7zs55sbx075jjjl3p280000gn/T/.org.chromium.Chromium.XKkbMD --window-size=1440,900 data:,
Version: 4.3
2020-03-10T04:44:10.266Z /home/myuser/node_modules/proxy-chain/build/handler_tunnel_chain.js:111
2020-03-10T04:44:10.276Z this.trgSocket.pipe(this.srcSocket);
2020-03-10T04:44:10.278Z ^
2020-03-10T04:44:10.279Z
2020-03-10T04:44:10.281Z TypeError: Cannot read property 'pipe' of null
2020-03-10T04:44:10.283Z at HandlerTunnelChain.onTrgRequestConnect (/home/myuser/node_modules/proxy-chain/build/handler_tunnel_chain.js:111:28)
2020-03-10T04:44:10.285Z at Object.onceWrapper (events.js:418:26)
2020-03-10T04:44:10.287Z at ClientRequest.emit (events.js:311:20)
2020-03-10T04:44:10.289Z at Socket.socketOnData (_http_client.js:510:11)
2020-03-10T04:44:10.291Z at Socket.emit (events.js:311:20)
2020-03-10T04:44:10.293Z at addChunk (_stream_readable.js:294:12)
2020-03-10T04:44:10.294Z at readableAddChunk (_stream_readable.js:275:11)
2020-03-10T04:44:10.296Z at Socket.Readable.push (_stream_readable.js:209:10)
2020-03-10T04:44:10.298Z at TCP.onStreamRead (internal/stream_base_commons.js:186:23)
Kindly share comprehensive code example to work from NodeJs behind corporate proxy. It will be very helpful, as the present documentation does not speak much about how to use this nice utility.
I'm using this awesome project to proxy certains requests and I noticed that keep alive doesn't work. Is there a way to make it work?
Some test connections are not closed and the tests fail.
Hi I'm using your proxy-chain module for some it works very well with my previous proxy with the anonymize function but I have a new one : ProxyMesh,
I need to send some headers only to the proxy and not to the final website, how can I achieve this ?
need to send this header "X-ProxyMesh-Not-IP"
also I need to get some headers from the proxy after the first call to "connect" on the proxy
I need to get the "X-ProxyMesh-IP"
find more info here: https://docs.proxymesh.com/article/7-request-response-headers#https
whole documentation : https://docs.proxymesh.com/article/7-request-response-headers
I suppose I need to use the prepareRequestFunction but I could not get it working !
I need this on HTTPS final sites !! the proxy is http
Hi,
First of all, awesome work. I really like the idea of this project.
After inspection I saw that a couple of hop-by-hop headers such as
const HOP_BY_HOP_HEADERS = [
'Connection',
'Keep-Alive',
'Proxy-Authenticate',
'Proxy-Authorization',
'TE',
'Trailers',
'Transfer-Encoding',
'Upgrade',
];are stripped from the forwarded http request that is passed to the upstream proxy.
This essentially means that there is no way to establish a Web Socket Connection when using this proxy. Web Sockets are initiated with the Connection and Upgrade header. When those headers are stripped, a targeted website could simply block the crawling/scraping based on the fact that the client is unable to establish a WebSocket connection.
This is just an example. The proxy could be detected via many other ways.
Wouldn't it be possible to use an transparent proxy on TCP/IP level (socks) that forwards the packet stream?
I am aware that this would mean that http/s proxies cannot be used anymore, but at least it would eliminate the detection methods...
Any ideas?
Hi,
I'm consistently getting this error sometime after "target socket assigned" and I have no idea why the error handling isn't catching it. This is via CONNECT through the handler base/handler chain workflow
3ks project!
Can u add some callback function for request and response? And intercept request body or intercept response body
Having an issue where inbound requests are being received by proxy-chain, when submitting multiple requests to the upstream proxy there is a massive delay. This only happens when using about 100 - 200 requests. Using lower requests i can get response delay down to a 1-3 seconds but anymore and delay turns to about 10 - 50 seconds. Tried using local proxy-chains http proxy to rule out any rate limits set by proxy provider, issue still exists.
Script otherwise is very powerful.
The handler_tunnel_chain.js file creates a CONNECT request to the real proxy and sets the desired target in the path parmater of the request options.
As far as I know the standard it to set it in the 'host' header...
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.