Does this work with res.send()?

[expressjs]

Does this work with res.send()?

[dougwilson]

Can you post code I can use to reproduce? I tried it out and it works just fine with Express 4.12.3

[dougwilson]

Most likely your request is not greater than 1kb (please read the docs, requests lower than 1kb are not compressed by default since it's a waste and can actually make them larger. See https://github.com/expressjs/compression#threshold for how to alter this). When you use res.send() the content-length header is sent; when you use res.write + res.end and don't write out a content-length header yourself, this module does not know if your request is going to be over 1kb and will simply always compress.

[CMCDragonkai]

Yea that makes sense. But it does mean there's a bit of difference between the 2 usages. Would be great if that was documented.

[dougwilson]

Please feel free to submit a PR with the documentation you desire :)

[dougwilson]

But TL;DR if you didn't already know there was a 1kb limit from the documentation, I'm not sure how you would have read the documentation regarding this behavior before asking, either... I'm always happy to answer questions, but the longer the documentation becomes, the less likely people are to read it.

[CMCDragonkai]

Is it not possible to always calculate the Content-Length even if I'm using res.write(data); res.end()?

Right now I have observed that res.send(data) has an automatic content length. res.end(data) has an automatic content length. The only one that doesn't is res.write(data); res.end(). Isn't there some way of accumulating the write chunks and incrementally counting it?

I'm currently writing a middleware that is facing this problem of non-determinism depending on how the user decides to send out content (res.send vs res.end vs res.write&res.end).

[dougwilson]

There isn't unless we buffer up the data in memory. We have to know if the response is going to be compressed before we can write even the first chunk to the client. res.write is the "I know what I'm doing and want chunks". This is just how Node.js works and there is nothing that can change here. You can always create a fork of this module that buffer's chunked responses in memory, though it'll use a lot of memory and mean people can no longer stream large files from disk.

Just think how you're supposed to send a 4GB file: you call res.write for each chunk such that you don't have to read 4GB of data into memory at all once and crash your process. If this module were to buffer everything to calculate the size, then people's servers are going to crash. There is an absolutely standard way to indicate the size to this module: simply set the content-length response header...

[dougwilson]

This is, in fact, literally the reason Express offers the res.send method: because Node.js gives you very low-level things and you need to do a lot of work yourself to get things write. If all you wanted to do was write out a string, res.send from Express will do all the "boring" work for you like setting the status code, the ETag header, the Content-Length header, the Content-Type header and more. If you're not using that method, you pretty much need to implement that all yourself. You can always file a bug report with Node.js itself asking for automatic Content-Length calculation, and then it would be there for this module to read, but it's not something that will ever be added to this module.

[CMCDragonkai]

Just a question regarding the Content-Length: Is it the length of the gzipped data OR is it the length of the data before gzipping?

[dougwilson]

You set it to the length of the content you are writing to res; this module will manipulate the header for you. That's what is known as "separate of concerns" in computer science :) When you add the Content-Length header, you are only concerns with what you know: the length of the content you are writing. Something else (like this module) that transforms the response will handle updating any appropriate headers.

[CMCDragonkai]

Yes but I want to know if my Content-Length is going the changed by your middleware? Can you give me a direct answer? I specifically want to know about the Content-Length that will be in the HTTP response header.

[dougwilson]

We remove the Content-Length header and send your response with Transfer-Encoding: chunked instead.

[CMCDragonkai]

Ok thank you.

[dougwilson]

Can you give me a direct answer?

I literally directly answered your question and you threw in a second question. You asked:

Is it the length of the gzipped data OR is it the length of the data before gzipping?

I answered:

When you add the Content-Length header, you are only concerns with what you know: the length of the content you are writing.

This thread is locked, I'm sorry.