microsoft/iis.compression

Compression schemes for Microsoft IIS

Microsoft IIS Compression

IIS Compression is a bundle of two IIS compression scheme providers iisbrotli.dll and iiszlib.dll that enable IIS to compress HTTP response contents. iisbrotli.dll supports Brotli compression, while iiszlib.dll supports both Gzip and Deflate compression.

An IIS compression scheme provider is a pluggable extension of IIS StaticCompressionModule and DynamicCompressionModule - essentially a dynamic link library (dll) that implements the compression algorithm and exposes the IIS HTTP Compression API. IIS StaticCompressionModule and DynamicCompressionModule load the registered compression scheme providers into worker process at runtime, and leverage them to perform compression for static files and dynamically-generated contents, respectively.

Documentation

• IIS Compression Overview
• Using IIS Compression

Installation

Download the Microsoft IIS Compression release from the following locations:

• Microsoft IIS Compression (x86) here.

• Microsoft IIS Compression (x64) here.

The IIS Compression installer registers iisbrotli.dll as the br (Brotli) compression scheme provider in applicationHost.config. It also replaces the default gzip compression scheme provider gzip.dll with iiszlib.dll. A sample httpCompression section in applicationHost.config is shown below:

<httpCompression directory="%SystemDrive%\inetpub\temp\IIS Temporary Compressed Files">
    <scheme name="br" dll="%ProgramFiles%\IIS\IIS Compression\iisbrotli.dll" />
    <scheme name="gzip" dll="%ProgramFiles%\IIS\IIS Compression\iiszlib.dll" />
    <dynamicTypes>
        <add mimeType="text/*" enabled="true" />
        <add mimeType="message/*" enabled="true" />
        <add mimeType="application/x-javascript" enabled="true" />
        <add mimeType="application/javascript" enabled="true" />
        <add mimeType="*/*" enabled="false" />
    </dynamicTypes>
    <staticTypes>
        <add mimeType="text/*" enabled="true" />
        <add mimeType="message/*" enabled="true" />
        <add mimeType="application/javascript" enabled="true" />
        <add mimeType="application/atom+xml" enabled="true" />
        <add mimeType="application/xaml+xml" enabled="true" />
        <add mimeType="image/svg+xml" enabled="true" />
        <add mimeType="*/*" enabled="false" />
    </staticTypes>
</httpCompression>

Install Visual Studio

1. Install Visual Studio 2017
2. You need to install these components additionally after(or while) installing Desktop development with C++

• Wix Toolset Visual Studio 2017 Extension
• Visual C++ ATL (x86/x64) with Spectre Mitigations
• VC++ 2017 version 15.9 v14.16 latest v141 tools

Build

1. Clone this project with submodules.

git clone --recurse-submodules https://github.com/Microsoft/IIS.Compression.git

2. Build the code using the build.cmd (the location of msbuild.exe needs to be added into PATH environment variable).

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

iis.compression's Issues

Support to Azure Web Apps (or confirm compiled binary redistribution license)

@shirhatti

I would like to make this exciting module available for Azure Web Apps. I can not install msi with Web Apps, but I can distribute dll with the mechanism called "Site Extension".

I can also create Site Extension. However, regarding redistribution of the compiled binary, I need to confirm if the license is OK.

system.webServer/httpCompression settings don't apply after the first tune

system.webServer/httpCompression settings don't apply after the first tune.
I want to change the dynamic compression level, but the new setting doesn't influence the final size of the transferred data.

Environment:
WinOS 2016 ver1607
IIS 10.0.14393.0
additionally installed https://download.microsoft.com/download/6/1/C/61CC0718-ED0E-4351-BC54-46495EBF5CC3/iiscompression_amd64.msi

I have added site, downloaded mentioned msi-package, enabled static and dynamic compression, setting up the level of compression, added types (application/json and application/json; charset=utf-8), add URLrewrite rule for the brotli.

After this action, brotli compression applying for the site successfully, but if I want to change or disable the compression it doesn't work. It looks like system.webServer/httpCompression doesn't work at all. I noticed that web.config changed but settings don't influence it all.

Only compression section works and provides the ability to disable compression

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <!-- To customize the asp.net core module uncomment and edit the following section. 
  For more info see https://go.microsoft.com/fwlink/?linkid=838655 -->
  <system.webServer>
    <modules>
      <remove name="WebDAVModule" />
    </modules>
    <handlers>
      <remove name="WebDAV" />
      <remove name="aspNetCore" />
      <add name="aspNetCore" path="*" verb="*" modules="AspNetCoreModule" resourceType="Unspecified" />
    </handlers>
    <aspNetCore processPath="dotnet" arguments=".\GoodStreamWebSpa_BigClients.dll" stdoutLogEnabled="false" stdoutLogFile=".\logs\stdout" requestTimeout="01:00:00" />
        <httpCompression>
            <remove name="gzip" />
            <remove name="br" />
            <scheme name="br" doDynamicCompression="true" dll="%ProgramFiles%\IIS\IIS Compression\iisbrotli.dll" dynamicCompressionLevel="2" />
            <scheme name="gzip" doStaticCompression="true" doDynamicCompression="true" dll="%ProgramFiles%\IIS\IIS Compression\iiszlib.dll" dynamicCompressionLevel="9" />
            <dynamicTypes>
                <add mimeType="application/json" enabled="true" />
                <add mimeType="application/json; charset=utf-8" enabled="true" />
            </dynamicTypes>
            <staticTypes>
                <add mimeType="application/json" enabled="true" />
                <add mimeType="application/json; charset=utf-8" enabled="true" />
            </staticTypes>
        </httpCompression>
        <rewrite>
            <rules>
                <rule name="Prioritize Brotli">
                    <match url="(.*)" />
                    <conditions>
                        <add input="{HTTP_ACCEPT_ENCODING}" pattern="\bbr(?!;q=0)\b" />
                    </conditions>
                    <serverVariables>
                        <set name="HTTP_ACCEPT_ENCODING" value="br" />
                    </serverVariables>
                    <action type="None" />
                </rule>
            </rules>
        </rewrite>
        <urlCompression doDynamicCompression="true" />
  </system.webServer>
</configuration>

Add minimum system requirements to the docs and wiki page

I noticed neither the Microsoft docs nor github wiki page list what the minimum requirements are.

• Is there a minimum IIS 10 version and OS version?
• Will it work on IIS 10 on Windows Server 2016 or only 2019?

Also the github wiki page has a obsolete link to mdsn for IIS HTTP Compression API, where should this link to instead?
https://msdn.microsoft.com/en-us/library/dd692872.aspx

IIS always use deflate

I've posted this question on stackoverflow, as well: https://stackoverflow.com/questions/54936824/brotli-compression-in-iis

I've been trying to setup Static Compression for an ASP.NET application hosted on IIS 10.

Snippet from applicationHost.config:

    <httpCompression directory="%SystemDrive%\inetpub\temp\IIS Temporary Compressed Files"
           minFileSizeForComp="2700" 
           staticCompressionIgnoreHitFrequency="true">
        <scheme name="br" dll="%ProgramFiles%\IIS\IIS Compression\iisbrotli.dll" />
        <scheme name="gzip" dll="%ProgramFiles%\IIS\IIS Compression\iiszlib.dll" />
        <staticTypes>
            <add mimeType="text/*" enabled="true" />
            <add mimeType="message/*" enabled="true" />
            <add mimeType="application/javascript" enabled="true" />
            <add mimeType="application/atom+xml" enabled="true" />
            <add mimeType="application/xaml+xml" enabled="true" />
            <add mimeType="image/svg+xml" enabled="true" />
            <add mimeType="*/*" enabled="false" />
        </staticTypes>
    </httpCompression>

Snippet from application's web.config:

<system.webServer>
  <staticContent>
     <remove fileExtension=".otf" />
     <remove fileExtension=".swz" />
     <remove fileExtension=".js" />
     <mimeMap fileExtension=".otf" mimeType="application/octet-stream" />
     <mimeMap fileExtension=".swz" mimeType="application/x-swz" />
     <mimeMap fileExtension=".js" mimeType="application/javascript" />
  </staticContent>
  <urlCompression doStaticCompression="true" />
</system.webserver>

However, IIS always use deflate, even if the request solely demand br encoding:

$ curl -s -o /dev/null -ikv -H "Accept-Encoding: br" -H "Content-Type: application/javascript" https://server/path/js_file.js

< HTTP/2 200 
< cache-control: public
< content-type: application/javascript
< content-encoding: deflate
< server: Microsoft-IIS/10.0
< date: Fri, 01 Mar 2019 00:43:30 GMT

Enabling Failed Request Tracing doesn't reveal much either. This is how the trace rule in IIS is configured (from application's web.config):

<tracing>
    <traceFailedRequests>
        <add path="/path/*">                 
            <traceAreas>
                <add provider="WWW Server" areas="Compression" verbosity="Verbose" />
            </traceAreas>
            <failureDefinitions statusCodes="200" />
        </add>
    </traceFailedRequests>
</tracing>

In the corresponding trace file, I don't see anything like STATIC_COMPRESSION_SUCCESS or STATIC_COMPRESSION_NOT_SUCCESS. In fact, the only relevant entry appears to be:

GENERAL_SET_RESPONSE_HEADER 

HeaderName
Content-Encoding 

HeaderValue
deflate 

Replace
false 

It's worth mentioning that a) Dynamic Content Compression module isn't installed on this server b) the SSL certificate is self-signed (I don't think it matters).

I can't tell whether this is caused by IIS's config or has something to do with the bundle.

content-encoding: gzip instead of brotli

I use IIS 10 14393 and I try to replace gzip compression with brotli.
I installed brotli on my server following the instruction from

applicationHost.config:

<httpCompression directory="%SystemDrive%\inetpub\temp\IIS Temporary Compressed Files">
            <scheme name="br" dll="%ProgramFiles%\IIS\IIS Compression\iisbrotli.dll" />
            <!-- <scheme name="gzip" dll="%ProgramFiles%\IIS\IIS Compression\iiszlib.dll" /> -->
            <staticTypes>
                <add mimeType="text/*" enabled="true" />
                <add mimeType="message/*" enabled="true" />
                <add mimeType="application/javascript" enabled="true" />
                <add mimeType="application/json" enabled="true" />
                <add mimeType="application/atom+xml" enabled="true" />
                <add mimeType="application/xaml+xml" enabled="true" />
                <add mimeType="*/*" enabled="false" />
            </staticTypes>
            <dynamicTypes>
                <add mimeType="text/*" enabled="true" />
                <add mimeType="message/*" enabled="true" />
                <add mimeType="application/javascript" enabled="true" />
                <add mimeType="application/atom+xml" enabled="true" />
                <add mimeType="application/xaml+xml" enabled="true" />
                <add mimeType="application/json" enabled="true" />
                <add mimeType="*/*" enabled="false" />
            </dynamicTypes>
        </httpCompression>

But I still get content-encoding: gzip on the response header instead of br.

The PR validation pipeline isn't working.

Also the PRs need to be gated with PR validation + at least 1 approval from IIS-Contributor.

Assigned to Baris first.

Update README.md with links to the official docs

@shirhatti

Since the official docs of IIS Compression have been out, could you help update README.md by adding the links:

https://docs.microsoft.com/en-us/iis/extensions/iis-compression/iis-compression-overview
https://docs.microsoft.com/en-us/iis/extensions/iis-compression/using-iis-compression

Move installer project to GitHub

Move IIS.Compression installer project into this repo

This repo is missing important files

There are important files that Microsoft projects should all have that are not present in this repository. A pull request has been opened to add the missing file(s). When the pr is merged this issue will be closed automatically.

Microsoft teams can learn more about this effort and share feedback within the open source guidance available internally.

Merge this pull request

In what state is this in?

Read me didn't quite explain if this is good to run in production or at some point in the future?

Hoping for some clarification and thank you for your efforts with this.

Crashes

I am getting some crashes when some extra compression settings are set. I literally only have a 1-line HTML file on the site and it crashes while trying to process it. It's not trying to process some complex file or anything.

<scheme name="br" dll="%ProgramFiles%\IIS\IIS Compression\iisbrotli.dll" dynamicCompressionLevel="10" staticCompressionLevel="10" />
<scheme name="gzip" dll="%ProgramFiles%\IIS\IIS Compression\iiszlib.dll" dynamicCompressionLevel="10" staticCompressionLevel="10" />

Here is what shows in Event Viewer:

Faulting application name: w3wp.exe, version: 10.0.14393.0, time stamp: 0x57899b8a
Faulting module name: iiszlib.dll, version: 1.0.9.0, time stamp: 0x5af2408d
Exception code: 0xc0000005
Fault offset: 0x00000000000025bb
Faulting process id: 0xeb8
Faulting application start time: 0x01d4376e563981e7
Faulting application path: c:\windows\system32\inetsrv\w3wp.exe
Faulting module path: C:\Program Files\IIS\IIS Compression\iiszlib.dll
Report Id: 6482ddcb-7c2d-4fba-8480-ee1a07381949
Faulting package full name: 
Faulting package-relative application ID: 

And here is what is shown in FREB:

Cannot run nuget restore - this repo uses private sources?

Does this repo use private sources?

AADSTS50020: User account '[email protected]' from identity provider 'live.com' does not exist in tenant 'Microsoft' and cannot access the application 'd5a56ea4-7369-46b8-a538-c370805301bf'(Azure Artifacts) in that tenant. The account needs to be added as an external user in the tenant first. Sign out and sign in again with a different Azure Active Directory user account.

Win Server 2019 : Brotli compression InProcess extreme memory usage

A long thread has been detailed here where it was found that the Microsoft IIS Compression module that enabled brotli compression causes a runaway memory usage issue.

This occurs with a netcore app (tested on 2.0, 3.0 and 3.1) running with InProcess enabled, so through w3wp.exe and is observed on Server 2019.

I used fiddler to find the difference is in the requests, between IE and Edge Chromium.
Edge: Accept-Encoding: gzip, deflate, br
IE: Accept-Encoding: gzip, deflate
So, brotli (iisbrotli.dll), IIS Compression Module is the problem and it is working fine after uninstalling it.

Need to remove agentPoolDemandTeam: 'IISAdmin'

We have to remove the below line to use all machines in the machine pool

agentPoolDemandTeam: 'IISAdmin'

https://github.com/microsoft/IIS.Compression/blob/master/.pipelines/iis.compression.build.official.yml#L17

New release has version nr lower that the previous

Previous version was version numbered as being: iiscompression_amd64.msi (v1.0.9)
New version is version numbered as iiscompression_amd64.msi (1.0.03219)

This is confusing :-/

Upgrade brotli to 1.0.9: integer overflow flaw

https://github.com/google/brotli#security-note

Version 1.0.9 contains a fix to "integer overflow" problem.

IIS.Compression is currently using 1.0.7 (like a lot of other vulnerable projects).
Integer overflow should be an emergency critical update.

Someone should probably do a PSA, because it looks like an endemic problem. Many projects appear to be copying the same 1.0.7 patches even though it has a known integer overflow, and who knows if someone will pull off a 0-day hat trick and release a worm.

Upload the module to Chocolatey

I am not sure if I am able to do this myself to comply with the legal right to distribute.

But I would like to be able to have the IIS Compression module available via Chocolatey so I can run modules like the Ansible Chocolatey module.

Make repo public

• Add README with build instructions
• Run APIScan
• Run Policheck
• Add ThirdPartyNotices.txt
• Add Codeowners file. See example

Final Step

• Make public

Installed but upon browser network insepction of https site, supported compressible files still compressed as gzip

As per above, this was installed on IIS 8.5 from iis.net/downloads/microsoft/iis-compression but after installing and iisreset, the files covered by the compression scheme still compressed via gzip.

Application.config shows the compression scheme installed as per installer action but it did not state the desired compression level for static or dynamic. Am I missing something?

I have tried saucecontrol version of brotli compression scheme but it requires URL rewrite rule on HTTP_ACCEPT_ENCODING to force prefer brotli over gzip. I would like to avoid installing that on assumption that this updated MSFT IIS compression scheme implementation would automagically do the job.

It would be helpful if you could enlighten what I've missed.

Best regards,
David