the-ccsn/traefik-plugin-rewritebody

traefik-plugin-rewritebody's Issues

Bug Report: Plugin Fails with Brotli Compressed JavaScript Files

Describe the bug
Traefik fails to serve the replaced page when the response is a Brotli-compressed JavaScript file (whether it is text/javascript or application/javascript). The transformation is visible in the Docker logs, and the replacement works well when the response encoding is gzip.

Relevant Config
Static Config:

providers:
  docker: {}
  file:
    directory: "/etc/traefik/dynamic"
    watch: true
log:
  filePath: "/logs/traefik.log"
  maxAge: 1
  level: DEBUG    
entryPoints:
  web:
    address: :80
    http:
      redirections:
        entryPoint:
          to: websecure
          scheme: https
  websecure:
    address: ":443"
api:
  dashboard: true

experimental:
  plugins:
    traefik-plugin-rewritebody:
      moduleName: "github.com/the-ccsn/traefik-plugin-rewritebody"
      version: "v0.1.3"  

Dynamic config:

http:
  middlewares:
    rewrite-to-githubassets:
      headers:
        customRequestHeaders:
          Host: "github.githubassets.com"
    my-rewrite-body:
      plugin:
        traefik-plugin-rewritebody:
          # Keep Last-Modified header returned by the HTTP service.
          # By default, the Last-Modified header is removed.
          lastModified: true

          # Rewrites all "foo" occurences by "bar"
          rewrites:
            - regex: "global-create-menu"
              replacement: "testReplacement"

          # logLevel is optional, defaults to Info level.
          # Available logLevels: (Trace: -2, Debug: -1, Info: 0, Warning: 1, Error: 2)
          logLevel: -2

          # monitoring is optional, defaults to below configuration
          # monitoring configuration limits the HTTP queries that are checked for regex replacement.
          monitoring:
            # methods is a string list. Options are standard HTTP Methods. Entries MUST be ALL CAPS
            # For a list of options: https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods
            methods:
              - GET
              - POST
            # types is a string list. Options are HTTP Content Types. Entries should match standard formatting
            # For a list of options: https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types
            # Wildcards(*) are not supported!
            types:
              - text/html
              - text/css
              - text/xml
              - text/javascript
              - application/x-www-form-urlencoded
              - application/json
              - application/javascript

  routers:
    dashboard:
      rule: Host(`traefik.docker.localhost`)
      service: api@internal
      entryPoints:
        - websecure
        - web
      tls:
        domains:
          - main: "docker.localhost"
            sans:
              - "*.docker.localhost"
          - main: "domain.local"
            sans:
              - "*.domain.local"

    git-asset:
      rule: Host(`githubassets.docker.localhost`)
      service: githubassets
      entryPoints:
        - websecure
        - web
      middlewares:
        - rewrite-to-githubassets
        - my-rewrite-body
      tls:
        domains:
          - main: "docker.localhost"
            sans:
              - "*.docker.localhost"

  services:
    githubassets:
      loadBalancer:
        servers:
          - url: "https://github.githubassets.com/"
        passHostHeader: true
tls:
  certificates:
    - certFile: "/etc/certs/local-cert.pem"
      keyFile: "/etc/certs/local-key.pem"

Expected behavior
The Brotli-compressed JavaScript file should be served with the replaced content as specified in the regex replacement.

Actual behavior
The Brotli-compressed JavaScript file is not served correctly. Instead, the response body is empty. The transformation appears to be successful in the Docker logs.

Logs
Traefik log file (Level: DEBUG):

2024-07-24T18:50:57Z DBG github.com/traefik/traefik/v3/pkg/server/service/loadbalancer/wrr/wrr.go:196 > Service selected by WRR: 8bc98d0a1f3fcd91
2024-07-24T18:50:57Z ERR fmt/print.go:305 > plugins-storage/sources/gop-1717797873/src/github.com/the-ccsn/traefik-plugin-rewritebody/vendor/github.com/andybalholm/brotli/hash.go:118:9: panic: github.com/andybalholm/brotli.backwardReferencePenaltyUsingLastDistance(...) module=github.com/the-ccsn/traefik-plugin-rewritebody plugin=plugin-traefik-plugin-rewritebody runtime=
2024-07-24T18:50:57Z ERR fmt/print.go:305 > plugins-storage/sources/gop-1717797873/src/github.com/the-ccsn/traefik-plugin-rewritebody/vendor/github.com/andybalholm/brotli/h5.go:115:21: panic: github.com/andybalholm/brotli.FindLongestMatch(...) module=github.com/the-ccsn/traefik-plugin-rewritebody plugin=plugin-traefik-plugin-rewritebody runtime=
2024-07-24T18:50:57Z ERR fmt/print.go:305 > plugins-storage/sources/gop-1717797873/src/github.com/the-ccsn/traefik-plugin-rewritebody/vendor/github.com/andybalholm/brotli/backward_references.go:41:49: panic: github.com/andybalholm/brotli.createBackwardReferences(...) module=github.com/the-ccsn/traefik-plugin-rewritebody plugin=plugin-traefik-plugin-rewritebody runtime=
2024-07-24T18:50:57Z ERR fmt/print.go:305 > plugins-storage/sources/gop-1717797873/src/github.com/the-ccsn/traefik-plugin-rewritebody/vendor/github.com/andybalholm/brotli/encode.go:752:21: panic: github.com/andybalholm/brotli.encodeData(...) module=github.com/the-ccsn/traefik-plugin-rewritebody plugin=plugin-traefik-plugin-rewritebody runtime=
2024-07-24T18:50:57Z ERR fmt/print.go:305 > plugins-storage/sources/gop-1717797873/src/github.com/the-ccsn/traefik-plugin-rewritebody/vendor/github.com/andybalholm/brotli/encode.go:1135:6: panic: github.com/andybalholm/brotli.encoderCompressStream(...) module=github.com/the-ccsn/traefik-plugin-rewritebody plugin=plugin-traefik-plugin-rewritebody runtime=
2024-07-24T18:50:57Z ERR fmt/print.go:305 > plugins-storage/sources/gop-1717797873/src/github.com/the-ccsn/traefik-plugin-rewritebody/vendor/github.com/andybalholm/brotli/writer.go:70:5: panic: github.com/andybalholm/brotli.writeChunk(...) module=github.com/the-ccsn/traefik-plugin-rewritebody plugin=plugin-traefik-plugin-rewritebody runtime=
2024-07-24T18:50:57Z ERR fmt/print.go:305 > plugins-storage/sources/gop-1717797873/src/github.com/the-ccsn/traefik-plugin-rewritebody/vendor/github.com/andybalholm/brotli/writer.go:106:12: panic: github.com/andybalholm/brotli.Close(...) module=github.com/the-ccsn/traefik-plugin-rewritebody plugin=plugin-traefik-plugin-rewritebody runtime=
2024-07-24T18:50:57Z ERR fmt/print.go:305 > plugins-storage/sources/gop-1717797873/src/github.com/the-ccsn/traefik-plugin-rewritebody/compressutil/compressutil.go:106:10: panic: github.com/the-ccsn/traefik-plugin-rewritebody/compressutil.compressWithBrotli(...) module=github.com/the-ccsn/traefik-plugin-rewritebody plugin=plugin-traefik-plugin-rewritebody runtime=
2024-07-24T18:50:57Z ERR fmt/print.go:305 > plugins-storage/sources/gop-1717797873/src/github.com/the-ccsn/traefik-plugin-rewritebody/compressutil/compressutil.go:52:9: panic: github.com/the-ccsn/traefik-plugin-rewritebody/compressutil.Encode(...) module=github.com/the-ccsn/traefik-plugin-rewritebody plugin=plugin-traefik-plugin-rewritebody runtime=
2024-07-24T18:50:57Z ERR fmt/print.go:305 > plugins-storage/sources/gop-1717797873/src/github.com/the-ccsn/traefik-plugin-rewritebody/httputil/response_writer.go:92:18: panic: github.com/the-ccsn/traefik-plugin-rewritebody/httputil.SetContent(...) module=github.com/the-ccsn/traefik-plugin-rewritebody plugin=plugin-traefik-plugin-rewritebody runtime=

Docker's console log:

2024-07-25 04:50:57 reverse-proxy-1  | Rewrite-Body | DEBUG2024/07/24 18:50:57 restricted.go:50: Starting supported request: &{GET /assets/global-create-menu-cd6a29cd32f9.js HTTP/2.0 2 0 map[Accept:[text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7; Accept-Encoding: gzip, deflate, br, zstd;] Accept-Encoding:[deflate, gzip, br, zstd] User-Agent:[curl/7.81.0] X-Forwarded-Host:[githubassets.docker.localhost] X-Forwarded-Port:[443] X-Forwarded-Proto:[https] X-Forwarded-Server:[3f5e8853da66] X-Real-Ip:[172.24.0.1]] 0xc019118480 <nil> 0 [] false github.githubassets.com map[] map[] <nil> map[] 172.24.0.1:47904 /assets/global-create-menu-cd6a29cd32f9.js 0xc000bd69a0 <nil> <nil> 0xc0132f8b10 <nil> [] map[]}
2024-07-25 04:50:57 reverse-proxy-1  | Rewrite-Body | DEBUG2024/07/24 18:50:57 restricted.go:50: Response body: "use strict";(globalThis.webpackChunk=globalThis.webpackChunk||[]).push([["global-create-menu"],{90119:(e,_,s)=>{var o=s(657),r=s(79315);(0,o.t)("global-create-menu",{Component:r.G})}},e=>{var _=_=>e(e.s=_);e.O(0,["react-lib","vendors-node_modules_github_mini-throttle_dist_index_js-node_modules_primer_octicons-react_di-b40d97","vendors-node_modules_primer_react_lib-esm_Box_Box_js","vendors-node_modules_primer_react_lib-esm_Button_Button_js","vendors-node_modules_primer_react_lib-esm_TooltipV2_Tooltip_js","vendors-node_modules_primer_behaviors_dist_esm_focus-zone_js","vendors-node_modules_clsx_dist_clsx_m_js-node_modules_primer_react_node_modules_primer_octico-c56103","vendors-node_modules_primer_react_lib-esm_ActionList_index_js","vendors-node_modules_primer_react_lib-esm_ActionMenu_ActionMenu_js-node_modules_primer_react_-5b2420","vendors-node_modules_primer_react_lib-esm_FeatureFlags_FeatureFlags_js-node_modules_primer_re-d6531f","ui_packages_react-core_create-browser-history_ts-ui_packages_safe-storage_safe-storage_ts-ui_-682c2c","ui_packages_react-core_register-partial_ts-ui_packages_global-create-menu_GlobalCreateMenu_tsx"],()=>_(90119)),e.O()}]);
2024-07-25 04:50:57 reverse-proxy-1  | //# sourceMappingURL=global-create-menu-ef686f2af525.js.map
2024-07-25 04:50:57 reverse-proxy-1  | Rewrite-Body | DEBUG2024/07/24 18:50:57 restricted.go:50: Transformed body: "use strict";(globalThis.webpackChunk=globalThis.webpackChunk||[]).push([["testReplacement"],{90119:(e,_,s)=>{var o=s(657),r=s(79315);(0,o.t)("testReplacement",{Component:r.G})}},e=>{var _=_=>e(e.s=_);e.O(0,["react-lib","vendors-node_modules_github_mini-throttle_dist_index_js-node_modules_primer_octicons-react_di-b40d97","vendors-node_modules_primer_react_lib-esm_Box_Box_js","vendors-node_modules_primer_react_lib-esm_Button_Button_js","vendors-node_modules_primer_react_lib-esm_TooltipV2_Tooltip_js","vendors-node_modules_primer_behaviors_dist_esm_focus-zone_js","vendors-node_modules_clsx_dist_clsx_m_js-node_modules_primer_react_node_modules_primer_octico-c56103","vendors-node_modules_primer_react_lib-esm_ActionList_index_js","vendors-node_modules_primer_react_lib-esm_ActionMenu_ActionMenu_js-node_modules_primer_react_-5b2420","vendors-node_modules_primer_react_lib-esm_FeatureFlags_FeatureFlags_js-node_modules_primer_re-d6531f","ui_packages_react-core_create-browser-history_ts-ui_packages_safe-storage_safe-storage_ts-ui_-682c2c","ui_packages_react-core_register-partial_ts-ui_packages_testReplacement_GlobalCreateMenu_tsx"],()=>_(90119)),e.O()}]);
2024-07-25 04:50:57 reverse-proxy-1  | //# sourceMappingURL=testReplacement-ef686f2af525.js.map
2024-07-25 04:50:57 reverse-proxy-1  | Rewrite-Body | WARNING2024/07/24 18:50:57 restricted.go:50: Recovered from: reflect: call of reflect.Value.Uint on int Value

Reproduction Steps
The error occurs both when using a browser and when using the following curl command:

curl --compressed \
  --header "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7" \
  --header "Accept-Encoding: gzip, deflate, br, zstd" \
  --insecure \
  -v \
  https://githubassets.docker.localhost/assets/global-create-menu-cd6a29cd32f9.js

According to my config file, It should replace text "global-create-menu" with "testReplacement".
The file's originally url is https://github.githubassets.com/assets/global-create-menu-cd6a29cd32f9.js. Although the file path is obfuscated, but it can be reproduced when accessing other file.

Server (please complete the following information):
- OS: Ubuntu 22.04.3 LTS
- Traefik Version: 3.1
- Plugin Version: v0.1.3

Additional context
The cert I used here was self-signed. I want to set up traefik as a reverse proxy to an external website.
The replacement works well when the response encoding is gzip. The issue seems to be specifically with Brotli encoding.

[Traefik Plugin Catalog] Plugin Analyzer has detected a problem.

The plugin was not imported into Traefik Plugin Catalog.

Cause:

failed to run the plugin with Yaegi: the load of the plugin takes too much time(10s), or an error, inside the plugin, occurs during the load: context deadline exceeded

Traefik Plugin Analyzer will restart when you will close this issue.

If you believe there is a problem with the Analyzer or this issue is the result of a false positive, please fill an issue on piceus repository.

[Traefik Plugin Catalog] Plugin Analyzer has detected a problem.

The plugin was not imported into Traefik Plugin Catalog.

Cause:

failed to run the plugin with Yaegi: the load of the plugin takes too much time(10s), or an error, inside the plugin, occurs during the load: context deadline exceeded

Traefik Plugin Analyzer will restart when you will close this issue.

If you believe there is a problem with the Analyzer or this issue is the result of a false positive, please fill an issue on piceus repository.

Feature: Add a RequestHost token

Is your feature request related to a problem? Please describe.
My webapp cannot use relative paths for static assets, so I am using this great plugin to rewrite urls for static assets in the html response body. I can generally just remove the scheme://host, leaving just the relative path, however it would be useful to be able to simply replace the hostname in the absolute path.

e.g. abc.com is the site, accessed via dev.xyz.com. If abc.com/abc.css is in the body, i want it to become dev.xyz.com/abc.css (rather than just /abc.css)

Describe the solution you'd like
I'm also using this plugin to rewrite headers, and it has a mechanism that allows you to use the RequestHost as a token in the replacement. https://github.com/jamesmcroft/traefik-plugin-rewrite-response-headers

e.g. something like

regex: "(https?://)abc\.com(/.*)?"
replacement: "$1{RequestHost}$2"

I've been poking around in the code of this plugin, but I'm new to traefik and Go, so not quite sure how to make something like this happen.

Is there any chance you could add a mechanism like this? Or point me in the right direction of creating a PR myself?

Thanks!

[Traefik Plugin Catalog] Plugin Analyzer has detected a problem.

The plugin was not imported into Traefik Plugin Catalog.

Cause:

failed to run the plugin with Yaegi: the load of the plugin takes too much time(10s), or an error, inside the plugin, occurs during the load: context deadline exceeded

Traefik Plugin Analyzer will restart when you will close this issue.

If you believe there is a problem with the Analyzer or this issue is the result of a false positive, please fill an issue on piceus repository.

[Traefik Plugin Catalog] Plugin Analyzer has detected a problem.

The plugin was not imported into Traefik Plugin Catalog.

Cause:

failed to run the plugin with Yaegi: the load of the plugin takes too much time(10s), or an error, inside the plugin, occurs during the load: context deadline exceeded

Traefik Plugin Analyzer will restart when you will close this issue.

If you believe there is a problem with the Analyzer or this issue is the result of a false positive, please fill an issue on piceus repository.