nsimmons/koa-better-http-proxy

Proxy middleware for Koa. Based on villadora/express-http-proxy

koa-better-http-proxy's Issues

Handling WebSocket Upgrade Request

Hello, I'm using koa-better-http-proxy, and the following code to support websocket upgrade:

/**
 * Create a WebSocket server.
 * @param server Server
 */
createWebSocketServer(server: http.Server | https.Server | http2.Http2SecureServer) {
	server.on('upgrade', (req: http.IncomingMessage, sock: net.Socket, head: Buffer) => {
		server.emit('request', req, new UpgradableResponse(sock, head));
	});
	const createContext = this.app.createContext;
	this.app.createContext = (req: http.IncomingMessage, res: http.ServerResponse | UpgradableResponse) => {
		const ctx: Koa.Context = createContext.call(this.app, req, res);
		if (res instanceof UpgradableResponse) {
			ctx.ws = true;
			ctx.upgrade = () => res.upgrade(req);
		}
		return ctx;
	};
}

The UpgradeableReponse provides a upgrade method which returns a ws.WebSocket instance. Then the method is copied in the overridden createContext to be made accessible in ctx. ctx.ws is also set to true to indicate this connection is upgradable.

However, I can't figure out how to implement this in koa-better-http-proxy and act like nginx's proxy_pass.

Proxy returns wrong response on high traffic

We are using koa-better-http-proxy to proxy another API for getting a user data. We have already checked that we were sending the correct request headers in every request. But at some point, there are few responses that the email data from the response and the email from our session didn't match like the logs below:

LOG: proxyReqOptDecorator { 
    transactionId: '0a8f92ca-5b77-4e8f-b3c0-03c079c05bde',
    emailFromSession: '[email protected]' 
}
LOG: proxyReqOptDecorator { 
    transactionId: '59cb6a30-d6d3-4475-a941-d0d655b369e7',
    emailFromSession: '[email protected]' 
}
LOG: userResDecorator { 
    transactionId: '0a8f92ca-5b77-4e8f-b3c0-03c079c05bde',
    emailFromResponse: '[email protected]' 
}
LOG: userResDecorator { 
    transactionId: '59cb6a30-d6d3-4475-a941-d0d655b369e7',
    emailFromResponse: '[email protected]' 
}

We have already asked the developers of the API we are proxying about it and they have told us that they haven't received requests in these cases. So we're wondering how we are still able to get data as a response.

Our dependencies are:

"koa": "^2.5.0",
"koa-better-http-proxy": "^0.2.4"

can not proxy https request

proxy http request, normal

but proxy https request ,eg
curl --insecure -v -x 'https://100.95.7.219:10115' https://www.google.com

• About to connect() to proxy 100.95.7.219 port 10115 (#0)
• Trying 100.95.7.219... connected
• Connected to 100.95.7.219 (100.95.7.219) port 10115 (#0)
• Establish HTTP proxy tunnel to www.baidu.com:443

CONNECT www.baidu.com:443 HTTP/1.1
Host: www.baidu.com:443
User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2
Proxy-Connection: Keep-Alive

• Proxy CONNECT aborted
• Closing connection #0
  curl: (56) Proxy CONNECT aborted

it seems that the proxy server can not establish connect with the target server
i don't know wherther it is caused by ssl certificate
but i can curl https://google.com in the proxy server ,
so i don't know what's the problem

if the proxy has some content body ,the proxy has never returned.

if the proxy has some content body, the proxy has never returned in postman.

app.use(
  proxy('https://xx.com', {
    proxyReqPathResolver: function (ctx: { url: any }) {
      const path = ctx.url.split('/').filter((e: string) => {
        return e !== '';
      });
      return `some url`;
    },
  }),
);

it will be ok if i delete the then(next) in the source code or not to carry the content body in the proxy
like this

    return buildProxyReq(container)
      .then(resolveProxyHost)
      .then(decorateProxyReqOpts)
      .then(resolveProxyReqPath)
      .then(decorateProxyReqBody)
      .then(prepareProxyReq)
      .then(sendProxyRequest)
      .then(copyProxyResHeadersToUserRes)
      .then(decorateUserRes)
      .then(sendUserRes)

how can i fix it?
thanks

How to set up multiple host names

Hi, I don't know if it is right to use Koajs as a reverse proxy but my goal is to proxy different external URLs(hosts) to different ports on my server. Could you provide one example?

How can I pass a table of hosts with their forward ports instead of just one host as the example below:
app.use(proxy('www.google.com'));

Thank you!

Can not proxy multipart request by default

By default, my multipart POST request (upload a file) gets lost, I don't see the parts of my request from the handler.

However, if I set the parseReqBody = false, then I can receive the multipart request from my handler correctly. But !!! the normal post like application/json get lost!

In brief, I can't let it proxy both multipart and application/json request at the same time.

Any ideas?

gzip error

[2018-09-06T02:53:18.863] [FATAL] mapping - { Error: unexpected end of file
at Zlib.zlibOnError [as onerror] (zlib.js:153:17)
at processChunkSync (zlib.js:481:12)
at zlibBufferSync (zlib.js:139:12)
at Object.gunzipSync (zlib.js:695:14)
at /root/mockingbirds/65003/mockingbird/node_modules/koa-better-http-proxy/app/steps/decorateUserRes.js:12:46
at decorateProxyResBody (/root/mockingbirds/65003/mockingbird/node_modules/koa-better-http-proxy/app/steps/decorateUserRes.js:44:22)
at run (/root/mockingbirds/65003/mockingbird/node_modules/core-js/modules/es6.promise.js:75:22)
at /root/mockingbirds/65003/mockingbird/node_modules/core-js/modules/es6.promise.js:92:30
at flush (/root/mockingbirds/65003/mockingbird/node_modules/core-js/modules/_microtask.js:18:9)
at args.(anonymous function) (/root/.nvm/versions/node/v10.1.0/lib/node_modules/pm2/node_modules/event-loop-inspector/index.js:133:29) errno: -5, code: 'Z_BUF_ERROR' }

When proxying a request with data gziped , error occurs

userOptions.headers is polluted by previous requests headers. Use decorator instead to add headers

If you are reusing the returned proxy function and you add headers to the proxy userOptions, the headers object will get polluted here. The next request will also get the additional headers.

Freezing userOptions.headers before passing it into proxy(url, userOptions) illicits this stack:

TypeError: Cannot add property client-ip, object is not extensible
      at extend (/Users/user/example/node_modules/koa-better-http-proxy/lib/requestOptions.js:15:17)
      at reqHeaders (/Users/user/example/node_modules/koa-better-http-proxy/lib/requestOptions.js:58:13)
      at Object.createRequestOptions [as create] (/Users/user/example/node_modules/koa-better-http-proxy/lib/requestOptions.js:74:14)
      at buildProxyReq (/Users/user/example/node_modules/koa-better-http-proxy/app/steps/buildProxyReq.js:11:41)
      at /Users/user/example/node_modules/koa-better-http-proxy/index.js:29:12
      at _default (/Users/user/example/build/server/main.js:902:23)
      at /Users/user/example/node_modules/koa-mount/index.js:58:11
      at dispatch (/Users/user/example/node_modules/koa-compose/index.js:42:32)
      at ProxyMiddleware (/Users/user/example/build/server/main.js:1054:13)
      at dispatch (/Users/user/example/node_modules/koa-compose/index.js:42:32)

Using the proxyReqOptDecorator instead of adding the headers to the userOptions doesn't seem to have this issue. Reusing proxy() may have other sideaffects as well. Just fyi

Support for unix socket

Would it be possible to add support for unix socket?

Question about defaultProxyReqPathResolver

I looked into the defaultProxyReqPathResolver and I am wondering why you use url.parse to parse the current requested url to extract the path. The koa documentation shows there is a ctx.request. path which already contains the path. Therefor we could save the expensive task of parsing a url. Or am I missing something that requires the url parsing here?

What should I use width koa-mount ?

const proxy = require('koa-better-http-proxy');
const Koa = require('koa');
const http = require('http');
const mount = require('koa-mount');
 var app = new Koa();
var proxyApp = proxy('127.0.0.1:8080');
var path = 'test1'
app.use(
        mount(path, proxyApp)
    )
        server = http.createServer(app.callback());
        server.listen(1234);

I want to be in port 1234 opening an proxy adopt port 1234/test1/ come to visit 127.0.0.1:8080
127.0.0.1:8080 has web
I succeeded
but All static resources All failed ...
They request like 127.0.0.1:1234/js/main.js
however correct url is like 127.0.0.1:1234/test1/js/main.js

I think of it 301 ...is work but so ugly...and when some request is post the lost...
What should I do?

How should I set the dynamic host and port?

I want to use the host and other config from ctx, how did I do?

proxy(host, {
    https: https,
    port: port
  }
)

ContentDecodingError while proxying requests to Github

Hello, I'm trying to proxy requests to Github but I'm always getting this error on the serverside:

http: error: ContentDecodingError: ('Received response with content-encoding: gzip, but failed to decode it.', error('Error -3 while decompressing: incorrect header check',))

Do you have any ideas how to solve it?

Log outcoming proxy request

Is there a way to log the details of outbound request?

How to deal with proxyReq.on 'error'?

problem

when i use koa-better-http-proxy to proxy a request，sometimes the target service is down，and i got an error like below

Error: connect ECONNREFUSED 10.9.25.65:4000
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1144:16) {
  errno: 'ECONNREFUSED',
  code: 'ECONNREFUSED',
  syscall: 'connect',
  address: '10.9.25.65',
  port: 4000
}

the ip has been changed for it is secret

and i want to catch this error by your proxy， so i can do something more

but i have found nothing about this in your document, so i went to see your code, and found you have deal with this in your code, but not provide an api to developer

what i have done

below is the code, which can deal with connect error
see koa-better-http-proxy/app/steps/sendProxyRequest.js

proxyReq.on('error', function(err) {
  // reject(error);
    if (err.code === 'ECONNRESET') {
      ctx.set('X-Timout-Reason', 'koa-better-http-proxy timed out your request after ' + timeoutDuration + 'ms.');
      ctx.set('Content-Type', 'text/plain');
      ctx.status = 504;
      resolve(Container);
    } else {
      reject(err);
    }
  });

i do find that it can catch the error，but it seems that you did not provide an option to throw this out

ERR_OUT_OF_RANGE when proxy response is bigger than 4.2GB

When a file bigger than 4.2GB is served by the proxy following error is thrown:

node:internal/validators:79
      throw new ERR_OUT_OF_RANGE(name, `>= ${min} && <= ${max}`, value);
      ^

RangeError [ERR_OUT_OF_RANGE]: The value of "length" is out of range. It must be >= 0 && <= 4294967296. Received 5_368_709_120
    at validateOffset (node:buffer:113:3)
    at Function.concat (node:buffer:550:5)
    at IncomingMessage.<anonymous> (/Users/thonda/programovani/WORK/other/my-proxy/node_modules/koa-better-http-proxy/app/steps/sendProxyRequest.js:18:42)
    at IncomingMessage.emit (node:events:539:35)
    at endReadableNT (node:internal/streams/readable:1345:12)
    at processTicksAndRejections (node:internal/process/task_queues:83:21) {
  code: 'ERR_OUT_OF_RANGE'
}

How to reproduce

1. create a file that's bigger than 4.2GB
   for example using mkfile command on MacOS

mkfile -n 5g testbigfile.csv

2. create server.js file that will serve the big file

const express = require('express');
const path = require('path');
const app = express();
const port = 3000;
app.get('*', async (req, res) => {
  const options = {
    root: path.join(__dirname)
  };
  const fileName = 'testbigfile.csv';
  res.sendFile(fileName, options, function (err) {
      if (err) {
          console.error(err);
      } else {
          console.log('Sent:', fileName);
      }
  });
});
app.listen(port, () => {
  console.log(`Example app listening on port ${port}`)
});

3. create proxy.js file

var proxy = require('koa-better-http-proxy');
var Koa = require('koa');
var app = new Koa();
app.use(proxy('http://localhost:3000'));
app.listen(4000);

4. start the servers: node proxy.js and node server.js
5. issue a GET request to the proxy `http://localhost:4000
   Error is produced by proxy.js

proxyReqOptDecorator documentation example is wrong

The Documentation for proxyReqOptDecorator states that you can use to rewrite the target path. But this is wrong as the proxyReqPathResolver is called AFTER proxyReqOptDecorator and by default overwrites the path that was set within the proxyReqOptDecorator. I think either the order of the two lifecycles should be switched or the documentation example should be adjusted to not state that you can change the path as part of that lifecycle.

Equivalent to proxyErrorHandler

How can I do error handling via proxyErrorHandler? As far as I can see there is simply no error handling facility in this module, is that true?

代理请求，偶现响应504

时而正常，时而报错，为什么会走到error里面？

Usage with mount?

I am attempting something like this:

const proxy = require('koa-better-http-proxy')
const mount = require('koa-mount')

const metricsServer = proxy('localhost:9461', { 'X-Special-Header': 'true' })

export default (app) => {
  app.use(
    mount('/metrics', metricsServer)
  )
}

But instead i'm getting raw HTML back for the server on 9460 (specifically my 404 page) - do you have any insights about how this could be used with koa-mount?

Configurable timeout, limit options

It'd be handy if we could set those two options using - as an alternative - a function exposing the ctx object.

That way, for example, the timeout and limit would have been defined by the request, i.e the payload.

'preserveHostHdr' is not exposed in the user options

The option preserveHostHdr is not assigned in the function resolveOptions but used in the function reqHeaders to determine if the proxy should bypass the Host param in the HTTP request header.

Which makes me cannot proxy requests from different domains to the same backend service.

The following patch will fix the issue:

--- lib/resolveOptions.js   2021-04-23 15:26:31.249257700 +0800
+++ lib/resolveOptions.js   2021-04-23 15:26:33.412777400 +0800        
@@ -15,6 +15,7 @@
   options = options || {};

   return {
+    preserveHostHdr: options.preserveHostHdr,
     proxyReqPathResolver:  options.proxyReqPathResolver,
     proxyReqOptDecorator: options.proxyReqOptDecorator,
     proxyReqBodyDecorator: options.proxyReqBodyDecorator,

Missing `Agent` type

In types.d.ts, the Agent type is not defined/imported. It was introduced in 0.2.6 and still present in 0.2.7.

This issue causes the following TypeScript error:

node_modules/koa-better-http-proxy/types.d.ts:8:13 - error TS2304: Cannot find name 'Agent'.
8     agent?: Agent,
              ~~~~~

Cannot proxy URL's starting with a specific path to a host.

In the express version of this repository, https://github.com/villadora/express-http-proxy#example, it's possible to do this: app.use('/proxy', proxy('www.google.com')); In Koa, this is not possible. I've set up a koa-router to handle this, but it's not working as expected.

Let's say I set up a GET route to "/proxy" with the second parameter being the proxy to said host. That URL will now be active, but it actually points to (using the above example) "google.com/proxy", not "google.com/" as one would expect a proxy to work, and indeed does work in the express example.

Can the behavior that exists in the express example be replicated using the Koa version of this tool?

Only proxy api and small file，because It‘s not pipeline stream type request

In the case of large files, monitor concat after req ('data '), and the processing speed and physical resource occupation are too large

Old version of Winston includes vulnerabilities

Hi there

tldr; Actually it seems winston is not used, I think we should remove it.

I have the following Dependabot alert;

Prototype Pollution in async
Upgrade async to version 2.6.4 or later

The old version of winston in package.json is the issue;

$ npm ls async
└─┬ [email protected]
  └─┬ [email protected]
    └── [email protected]