sjtug/caddy2-filter

Replace response body in caddy2

caddy2-filter's Issues

Hijack failed on protocol switch

Please see https://caddy.community/t/hijack-failed-on-protocol-switch/15635 and let me know if this is a problem with this plugin?

Efficient text replacement

Although it doesn't support regex, my understanding is that this package in conjunction with x/text/transform can efficiently perform text substitutions in streams: https://github.com/icholy/replace - it does not require buffering the entire body in memory.

This could be handy for production use cases requiring literal replacements in the response body.

caddy2-filter is incompatible with caddy v2.6.3

When I tried to build with caddy v2.6.3, it raised an panic.

2023/02/09 08:50:50 [INFO] exec (timeout=0s): /home/w0n9/golang/go/bin/go get -d -v github.com/sjtug/caddy2-filter github.com/caddyserver/caddy/[email protected] 
go: downloading github.com/sjtug/caddy2-filter v0.0.0-20220728031114-2540c7e290e0
panic: internal error: can't find reason for requirement on github.com/google/[email protected]

goroutine 1 [running]:
cmd/go/internal/modget.(*resolver).updateBuildList.func1({{0xc000a1af90?, 0xc001328420?}, {0xc0001ac6f0?, 0xc000f35470?}})
        /usr/local/go/src/cmd/go/internal/modget/get.go:1760 +0x114
cmd/go/internal/modget.(*resolver).updateBuildList(0xc0003ce000, {0xb2d350, 0xc000188000}, {0x0, 0x0, 0x0})
        /usr/local/go/src/cmd/go/internal/modget/get.go:1765 +0x597
cmd/go/internal/modget.(*resolver).applyUpgrades(0xc0003ce000, {0xb2d350, 0xc000188000}, {0x0?, 0x2?, 0x472445?})
        /usr/local/go/src/cmd/go/internal/modget/get.go:1312 +0x105
cmd/go/internal/modget.runGet({0xb2d350, 0xc000188000}, 0xc0001b8450?, {0xc00019e160, 0x2, 0x2})
        /usr/local/go/src/cmd/go/internal/modget/get.go:351 +0x45e
main.invoke(0xe27fa0, {0xc00019e130, 0x5, 0x5})
        /usr/local/go/src/cmd/go/main.go:225 +0x34e
main.main()
        /usr/local/go/src/cmd/go/main.go:179 +0x7d1
2023/02/09 08:50:53 [FATAL] exit status 2

How to reproduce this error

xcaddy build v2.6.3 --with github.com/sjtug/caddy2-filter

Caddyfile doesn't seem to be working properly

I implement a reverse proxy normally and I need to change the page content to my reverse proxy address, the configuration should be correct, but caddy won't work.

And it prompts this error below

root@debian10:~# ./caddy run -config /etc/caddy/Caddyfile -watch
2021/03/03 04:16:23.387 INFO    using provided configuration    {"config_file": "/etc/caddy/Caddyfile", "config_adapter": ""}
run: adapting config using caddyfile: /etc/caddy/Caddyfile:3: unrecognized directive: content_type

Here is my full caddyfile

filter {

  content_type text/html
  search_pattern images.original.com
  replacement images.other.com

}

www.other.com:80 {
  log
  encode gzip
  reverse_proxy * https://www.original.com {
    # 请求头Host设置
    header_up Host www.original.com
    # 请求头transparent设置
    header_up X-Real-IP {http.request.remote.host}
    header_up X-Forwarded-For {http.request.remote.host}
    header_down Set-Cookie www.original.com www.other.com
  }
}

My caddy version: v2.4.0-beta.1 h1:Ed/tIaN3p6z8M3pEiXWJL/T8JmCqV62FrSJCHKquW/I=

caddy list-modules

root@debian10:~# ./caddy list-modules
admin.api.load
admin.api.metrics
caddy.adapters.caddyfile
caddy.config_loaders.http
caddy.listeners.tls
caddy.logging.encoders.console
caddy.logging.encoders.filter
caddy.logging.encoders.filter.delete
caddy.logging.encoders.filter.ip_mask
caddy.logging.encoders.json
caddy.logging.encoders.single_field
caddy.logging.writers.discard
caddy.logging.writers.file
caddy.logging.writers.net
caddy.logging.writers.stderr
caddy.logging.writers.stdout
caddy.storage.file_system
http
http.authentication.hashes.bcrypt
http.authentication.hashes.scrypt
http.authentication.providers.http_basic
http.encoders.gzip
http.encoders.zstd
http.handlers.acme_server
http.handlers.authentication
http.handlers.encode
http.handlers.error
http.handlers.file_server
http.handlers.headers
http.handlers.map
http.handlers.metrics
http.handlers.push
http.handlers.request_body
http.handlers.reverse_proxy
http.handlers.rewrite
http.handlers.static_response
http.handlers.subroute
http.handlers.templates
http.handlers.vars
http.matchers.expression
http.matchers.file
http.matchers.header
http.matchers.header_regexp
http.matchers.host
http.matchers.method
http.matchers.not
http.matchers.path
http.matchers.path_regexp
http.matchers.protocol
http.matchers.query
http.matchers.remote_ip
http.matchers.vars
http.matchers.vars_regexp
http.reverse_proxy.selection_policies.cookie
http.reverse_proxy.selection_policies.first
http.reverse_proxy.selection_policies.header
http.reverse_proxy.selection_policies.ip_hash
http.reverse_proxy.selection_policies.least_conn
http.reverse_proxy.selection_policies.random
http.reverse_proxy.selection_policies.random_choose
http.reverse_proxy.selection_policies.round_robin
http.reverse_proxy.selection_policies.uri_hash
http.reverse_proxy.transport.fastcgi
http.reverse_proxy.transport.http
pki
tls
tls.certificates.automate
tls.certificates.load_files
tls.certificates.load_folders
tls.certificates.load_pem
tls.handshake_match.sni
tls.issuance.acme
tls.issuance.internal
tls.issuance.zerossl
tls.stek.distributed
tls.stek.standard

  Standard modules: 76

dns.providers.cloudflare
dns.providers.route53
http.handlers.filter
http.handlers.replace_response

  Non-standard modules: 4

  Unknown modules: 0

Can I split long regex into multiple lines?

Since there is a lot to type for each replacement, I am combining many regex into one. But this resulting in a very long line. Can I somehow split a regex into multiple lines in this module? Something like:

filter {
    search_pattern "(todelete1\
|todelete2\
|todelete3)"
    replacement ""
    content_type html
}

Thank you.

Use Caddy's request matchers, directive ordering

I saw you're using path as a matcher inside of the filter plugin. This isn't necessary, because you can already use Caddy's built-in request matchers for this.

When you register your Caddyfile directive with RegisterHandlerDirective, it allows a matcher token to be the first argument to your directive, like most other Caddyfile directives. This means you can do this:

filter /foo/* {
	...
}

If you want to use regexp patterns, then you'd do this:

@pattern {
	path_regexp ^/(foo|bar).*
}
filter @pattern {
	...
}

I would've suggested also getting rid of content_type but I think that looks at the response headers which isn't something you can do with request matching (obviously). So that one's fine to leave as-is.

Also, you should mention that users should set an order for the directive in their global options to use it, because otherwise they'll encounter an error when trying to use it (at least when outside of a route block). See https://caddyserver.com/docs/caddyfile/options, the directive ordering is here: https://caddyserver.com/docs/caddyfile/directives#directive-order

I'd recommend something like:

{
	order filter after encode
}

This will make your directive get handled just before the response is encoded (gzipped, etc) and after the response is templated.

Bug: Filter doesn't work on complicated HTTP responses?

Hi,

So I was testing this plugin out with a simple Caddyfile example:

www.MyRandomDomain.net {
    reverse_proxy {
        to https://www.lipsum.com
        header_up Host "www.lipsum.com"
    }
    filter {
        content_type .*
        search_pattern "Lorem Ipsum"
        replacement "New Name"
    }
}

In this, I expected all instances of "Lorem Ipsum" on the page to be turned into "New Name". However, nothing seems to happen. Is this a bug, or is there some usage of this plugin that is incorrect?

Is it possible to check the replaced content?

Hi! I would like to check if the {http.request.orig_uri.query} contains the expected regex.

If my url is: https://testcaddy.com/file.xml?test=12341234123412341234123412341234, then the pattern "test" should be replaced by test=12341234123412341234123412341234 (it is a GUID).

filter {
        path file\.xml
        search_pattern "test"
        replacement {http.request.orig_uri.query}
        content_type /^test=\b[a-zA-Z0-9]{32}\b$/
    }

I tried it but it not works...could you help me? Thanks!

Official download page

Is it possible that it is on the official page?