pteich/caddy-tlsconsul

🔒 Consul K/V storage for Caddy Web Server / Certmagic TLS data

caddy-tlsconsul's Issues

storage module 'caddy.storage.tlsconsul' is not a Caddyfile unmarshaler

Got the docker image and added the configuration option as suggested in the readme. But I get storage module 'caddy.storage.tlsconsul' is not a Caddyfile unmarshaler. Any suggestions?

Thanks

[Question] Is external volumes still needed when using this ?

Caddy requires write access to two locations: a data directory, and a configuration directory.
While it's not necessary to persist the files stored in the configuration directory, it can be convenient. 
However, it's very important to persist the data directory.

The above snippet is from caddy docker readme.

Does this plugin make both these storage requirements redundant ?

panic on `ConsulStorage{}.ConsulClient` being nil

Getting this panic error after upgrading certmagic, dns provider, storage, etc. modules. Not sure if it matters, but I switched from dnsimple to cloudflare dns provider as well.

certmagic.DefaultACME.Agreed = true
certmagic.DefaultACME.Email = email
certmagic.DefaultACME.DisableHTTPChallenge = true
certmagic.DefaultACME.DisableTLSALPNChallenge = true
certmagic.DefaultACME.CA = certmagic.LetsEncryptProductionCA
// config := dnsimple.NewDefaultConfig()
// config.AccessToken = accessToken
// dnsimpleProvider, err := dnsimple.NewDNSProviderConfig(config)
certmagic.DefaultACME.DNS01Solver = &certmagic.DNS01Solver{
	DNSProvider: &cloudflare.Provider{
		APIToken: accessToken,
	},
}
certmagic.Default.Storage = storageconsul.New()
magic := certmagic.NewDefault()
err := magic.ManageSync(domains)

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xa5e54d]

goroutine 25 [running]:

github.com/hashicorp/consul/api.(*Client).newRequest(0x0, 0x1024cdf, 0x3, 0xc000274000, 0x97, 0xc000274000)
	/Users/gurpartap/go/pkg/mod/github.com/hashicorp/consul/[email protected]/api.go:848 +0x6d

github.com/hashicorp/consul/api.(*KV).getInternal(0xc000262ba0, 0xc00003e240, 0x90, 0x0, 0xc000262c10, 0xe87d60, 0xc000036690, 0xc000036690, 0x87)
	/Users/gurpartap/go/pkg/mod/github.com/hashicorp/consul/[email protected]/kv.go:126 +0x105

github.com/hashicorp/consul/api.(*KV).Get(0xc000262ba0, 0xc00003e240, 0x90, 0xc000262c10, 0x0, 0x0, 0x0, 0x0)
	/Users/gurpartap/go/pkg/mod/github.com/hashicorp/consul/[email protected]/kv.go:65 +0xa5

github.com/pteich/caddy-tlsconsul.ConsulStorage.Load(0x0, 0x0, 0x0, 0x0, 0xc000220c60, 0x0, 0x0, 0x0, 0x0, 0xa, ...)
	/Users/gurpartap/go/pkg/mod/github.com/pteich/[email protected]/storage.go:127 +0x12d

github.com/caddyserver/certmagic.(*Config).loadCertResource(0xc00020bd60, 0x1178428, 0xc000172400, 0x104dab6, 0x27, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/Users/gurpartap/go/pkg/mod/github.com/caddyserver/[email protected]/crypto.go:239 +0x296

github.com/caddyserver/certmagic.(*Config).loadCertResourceAnyIssuer(0xc00020bd60, 0x104dab6, 0x27, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/Users/gurpartap/go/pkg/mod/github.com/caddyserver/[email protected]/crypto.go:171 +0xc1d

github.com/caddyserver/certmagic.(*Config).loadManagedCertificate(0xc00020bd60, 0x104dab6, 0x27, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/Users/gurpartap/go/pkg/mod/github.com/caddyserver/[email protected]/certificates.go:123 +0xc9

github.com/caddyserver/certmagic.(*Config).CacheManagedCertificate(0xc00020bd60, 0x104dab6, 0x27, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/Users/gurpartap/go/pkg/mod/github.com/caddyserver/[email protected]/certificates.go:110 +0xc9

github.com/caddyserver/certmagic.(*Config).manageOne(0xc00020bd60, 0x1189ee8, 0xc000042030, 0x104dab6, 0x27, 0xc000263e00, 0xa8ee05, 0xc0000a00b0)
	/Users/gurpartap/go/pkg/mod/github.com/caddyserver/[email protected]/config.go:323 +0xbe

github.com/caddyserver/certmagic.(*Config).manageAll(0xc00020bd60, 0x1189ee8, 0xc000042030, 0xc00009d0a0, 0x2, 0x2, 0x0, 0x0, 0x0)
	/Users/gurpartap/go/pkg/mod/github.com/caddyserver/[email protected]/config.go:312 +0x1f1

github.com/caddyserver/certmagic.(*Config).ManageSync(...)
	/Users/gurpartap/go/pkg/mod/github.com/caddyserver/[email protected]/config.go:251

[Question] I was wondering if this plugin integrates with CDP ?

Thanks for this great module, I was wondering if it integrates with CDP, I'm planning to use both in a Docker-Swarm mode environment, but I'm finding it hard to put things together, there are no pointers, I will appreciate any assistance.

Caddyfile configuration is ignored

It seems that when defining the module's configuration in the Caddyfile it gets ignored.
Tested on swarm using caddy 2.1-beta.1 with tlsconsul 1.2.0.

Here's my stack :

version: '3.7'

services:
  caddy_server:
    image: (latest beta 2.1 image built with tls consul 1.2.0)
    command: run --config /etc/caddy/Caddyfile --adapter caddyfile
    ports:
      - 80:80
      - 443:443
    networks:
      - consul
    configs:
      - source: caddyfile-consul-test
        target: /etc/caddy/Caddyfile

configs:
  caddyfile-consul-test:
    external: true

networks:
  consul:
    external: true

Caddyfile :

{
	email [email protected]
	storage consul {
    		address "consul-server:8500"
		token "consul-access-token"
		timeout 10
		prefix "caddytls"
		value_prefix "myprefix"
		aes_key "consultls-1234567890-caddytls-32"
		tls_enabled "false"
		tls_insecure "true"
	}
}
:80 {
	root * /usr/share/caddy
	file_server
}

Log :

{"level":"info","ts":1592393077.5028267,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
{"level":"info","ts":1592393077.5079827,"logger":"caddy.storage.consul","msg":"TLS storage is using Consul at "}
run: loading initial config: loading new config: loading storage module: loading module 'consul': provision caddy.storage.consul: unable to ping Consul: Get "http://127.0.0.1:8500/v1/agent/self": dial tcp 127.0.0.1:8500: connect: connection refused

It logs a blank address (TLS storage is using Consul at ) and defaults to 127.0.0.1:8500 despite the Caddyfile's consul address pointing to consul-server:8500.
It works fine using environment variables.

Caddy's import path has changed

Caddy's import path (and Go module name) has changed from

github.com/mholt/caddy

to

github.com/caddyserver/caddy

Unfortunately, Go modules are not yet mature enough to handle a change like this (see https://golang.org/issue/26904 - "haven't implemented that part yet" but high on priority list for Go 1.14) which caught me off-guard. Using Go module's replace feature didn't act the way I expected, either. Caddy now fails to build with plugins until they update their import paths.

I've hacked a fix into the build server, so downloading Caddy with your plugin from our website should continue working without any changes on your part, for now. However, please take a moment and update your import paths, and do a new deploy on the website, because the workaround involves ignoring module checksums and performing a delicate recursive search-and-replace.

I'm terribly sorry about this. I did a number of tests and dry-runs to ensure the change would be smooth, but apparently some unknown combination of GOPATH, Go modules' lack of maturity, and other hidden variables in the system or environment must have covered up something I missed.

This bash script should make it easy (run it from your project's top-level directory):

find . -name '*.go' | while read -r f; do
	sed -i.bak 's/\/mholt\/caddy/\/caddyserver\/caddy/g' $f && rm $f.bak
done

We use this script in the build server as part of the temporary workaround.

Let me know if you have any questions! Sorry again for the inconvenience.

Is it possible Decrypt/decompress certificates

it seems that the consul storage extension for caddy is gzipping or encrypting the certs in consul.

Is it possible to get the plain certs/ out of consul?

admin.backbiosis.com.crt
admin.backbiosis.com.json
admin.backbiosis.com.key

i tried to unzip the files but get this error:

./consul kv get caddy/certificates/acme-staging-v02.api.letsencrypt.org-directory/admin.backbiosis.com/admin.backbisosis.com.crt | gunzip -dc > cert.crt

gzip: stdin: not in gzip format`

i reissued the certs a couple of times just to make sure the certs are broken. but still cannot decompress/decrypt the files

the ssl endpoint works with the certs tho.

nil pointer access

I'm having a bit of trouble getting this to work, and more specifically, debugging my issues -- incoming connections just get reset, or terminated permaturely, and I'm finding it hard to get any information on what's going on.

Please advise how to debug, are there obscure logging options to turn on, or any other means of introspection? I'm happy to insert Printf lines into caddy, if that's what I'll have to do :)

project targets invalid Caddy version and doesn't compile anymore on 2.0.0

the go.mod points to a 2.0.1 version of Caddy but it's nowhere to be found in Caddy's releases. I assume it got skipped and merged into the current 2.1 beta perhaps ?

Anyway, compiling against the latest stable version of Caddy (2.0.0) results in a failed compilation because your go.mod requires certmagic 0.11.1 but Caddy 2.0.0 is not compatible with this version yet according to this issue.

I simply used the official docker builder image to compile it :

FROM caddy:2.0.0-builder AS builder
RUN caddy-builder github.com/pteich/caddy-tlsconsul@9b3eabb7bc8272133c982dbdb82073b31b48bf69

FROM caddy:2.0.0
COPY --from=builder /usr/bin/caddy /usr/bin/caddy
ENTRYPOINT ["/usr/bin/caddy"]

And ended up with :

[...]
go: downloading github.com/opencontainers/go-digest v1.0.0-rc1
go: downloading github.com/Microsoft/go-winio v0.4.14
go: downloading github.com/konsorten/go-windows-terminal-sequences v1.0.2
# github.com/caddyserver/caddy/v2/modules/caddytls/distributedstek
/src/caddy/modules/caddytls/distributedstek/distributedstek.go:144:16: not enough arguments in call to s.storage.Lock
	have (string)
	want (context.Context, string)
The command '/bin/sh -c caddy-builder github.com/pteich/caddy-tlsconsul@9b3eabb7bc8272133c982dbdb82073b31b48bf69' returned a non-zero code: 2

I'm not quite sure what happened with caddy 2.0.1 but maybe it would be better to stick to the latest released stable version of Caddy for new releases of TLSConsul. Or at least tag new builds targeting in-development or beta versions of Caddy differently like 1.2.3-beta or something ?

On a more positive note : Good job on the UnmarshalCaddyfile support, i'm looking forward to using it 👍

fatal error: concurrent map read and map write

One of our caddy instances was renewing ~5 certificates and we got this error in the log:

{"level":"info","ts":1624549442.7298193,"logger":"tls.renew","msg":"acquiring lock","identifier":"dom1.eu"}
{"level":"info","ts":1624549442.745405,"logger":"tls.renew","msg":"acquiring lock","identifier":"dom2.eu"}
{"level":"info","ts":1624549442.7781668,"logger":"tls.renew","msg":"acquiring lock","identifier":"dom3.eu"}
{"level":"info","ts":1624549442.7857873,"logger":"tls.renew","msg":"acquiring lock","identifier":"dom4.eu"}
{"level":"info","ts":1624549442.7933903,"logger":"tls.renew","msg":"acquiring lock","identifier":"dom5.eu"}
{"level":"info","ts":1624549443.8955014,"logger":"tls","msg":"served key authentication certificate","server_name":"dom2.eu","challenge":"tls-alpn-01","remote":"18.197.97.115:13872","distributed":true}
{"level":"info","ts":1624549444.227819,"logger":"tls","msg":"served key authentication certificate","server_name":"dom1.eu","challenge":"tls-alpn-01","remote":"18.197.97.115:13964","distributed":true}
{"level":"info","ts":1624549444.2303605,"logger":"tls","msg":"served key authentication certificate","server_name":"dom3.eu","challenge":"tls-alpn-01","remote":"3.120.130.29:35008","distributed":true}
{"level":"info","ts":1624549444.2698698,"logger":"tls","msg":"served key authentication certificate","server_name":"dom2.eu","challenge":"tls-alpn-01","remote":"66.133.109.36:30662","distributed":true}
{"level":"info","ts":1624549444.3561459,"logger":"tls","msg":"served key authentication certificate","server_name":"dom5.eu","challenge":"tls-alpn-01","remote":"18.197.97.115:13990","distributed":true}
{"level":"info","ts":1624549444.4526017,"logger":"tls","msg":"served key authentication certificate","server_name":"dom2.eu","challenge":"tls-alpn-01","remote":"18.116.86.117:42258","distributed":true}
{"level":"info","ts":1624549444.5201306,"logger":"tls","msg":"served key authentication certificate","server_name":"dom4.eu","challenge":"tls-alpn-01","remote":"18.197.97.115:14026","distributed":true}
{"level":"info","ts":1624549444.5312564,"logger":"tls","msg":"served key authentication certificate","server_name":"dom2.eu","challenge":"tls-alpn-01","remote":"34.221.186.243:63866","distributed":true}
{"level":"info","ts":1624549445.189281,"logger":"tls","msg":"served key authentication certificate","server_name":"dom1.eu","challenge":"tls-alpn-01","remote":"18.116.86.117:42442","distributed":true}
{"level":"info","ts":1624549445.2701666,"logger":"tls","msg":"served key authentication certificate","server_name":"dom3.eu","challenge":"tls-alpn-01","remote":"3.142.122.14:16940","distributed":true}
{"level":"info","ts":1624549445.3078008,"logger":"tls","msg":"served key authentication certificate","server_name":"dom4.eu","challenge":"tls-alpn-01","remote":"3.142.122.14:16962","distributed":true}
{"level":"info","ts":1624549445.4730475,"logger":"tls","msg":"served key authentication certificate","server_name":"dom3.eu","challenge":"tls-alpn-01","remote":"66.133.109.36:30916","distributed":true}
{"level":"info","ts":1624549445.698484,"logger":"tls.renew","msg":"lock acquired","identifier":"dom2.eu"}
{"level":"info","ts":1624549445.7244895,"logger":"tls.renew","msg":"certificate appears to have been renewed already","identifier":"dom2.eu","remaining":7772397.275513487}
{"level":"info","ts":1624549445.72452,"logger":"tls.renew","msg":"releasing lock","identifier":"dom2.eu"}

fatal error: concurrent map read and map write

goroutine 942994 [running]:
runtime.throw(0x18a199d, 0x21)
        runtime/panic.go:1117 +0x72 fp=0xc000067d20 sp=0xc000067cf0 pc=0x438652
runtime.mapaccess2_faststr(0x1641de0, 0xc004ab5e90, 0xc005faaae0, 0x1c, 0xc0017b5980, 0xc005024118)
        runtime/map_faststr.go:116 +0x4a5 fp=0xc000067d90 sp=0xc000067d20 pc=0x414505
github.com/pteich/caddy-tlsconsul.ConsulStorage.Unlock(0x0, 0x0, 0xc0043410e0, 0xc001aa2688, 0xc004ab5e90, 0xc004bfb920, 0x19, 0x0, 0x0, 0xa, ...)
        github.com/pteich/[email protected]/storage.go:87 +0x66 fp=0xc000067e70 sp=0xc000067d90 pc=0x14f70a6
github.com/pteich/caddy-tlsconsul.ConsulStorage.Lock.func1(0xc0050240c0, 0xc0015d2120, 0xc005faaae0, 0x1c)
        github.com/pteich/[email protected]/storage.go:75 +0xb8 fp=0xc000067fc0 sp=0xc000067e70 pc=0x14fa3d8
runtime.goexit()
        runtime/asm_amd64.s:1371 +0x1 fp=0xc000067fc8 sp=0xc000067fc0 pc=0x4728a1
created by github.com/pteich/caddy-tlsconsul.ConsulStorage.Lock
        github.com/pteich/[email protected]/storage.go:73 +0x68e

goroutine 1 [select (no cases), 335 minutes]:
github.com/caddyserver/caddy/v2/cmd.cmdRun(0xc00003a240, 0x0, 0x0, 0x0)
        github.com/caddyserver/caddy/[email protected]/cmd/commandfuncs.go:276 +0x1395
github.com/caddyserver/caddy/v2/cmd.Main()
        github.com/caddyserver/caddy/[email protected]/cmd/main.go:85 +0x25b
main.main()
        caddy/main.go:15 +0x25

goroutine 9 [select, 335 minutes]:
github.com/caddyserver/certmagic.(*RingBufferRateLimiter).permit(0xc000097180)
        github.com/caddyserver/[email protected]/ratelimiter.go:216 +0xb2
github.com/caddyserver/certmagic.(*RingBufferRateLimiter).loop(0xc000097180)
        github.com/caddyserver/[email protected]/ratelimiter.go:89 +0xa8
created by github.com/caddyserver/certmagic.NewRateLimiter
        github.com/caddyserver/[email protected]/ratelimiter.go:45 +0x148

goroutine 49 [chan receive, 335 minutes]:
github.com/caddyserver/caddy/v2.trapSignalsCrossPlatform.func1()
        github.com/caddyserver/caddy/[email protected]/sigtrap.go:42 +0x129
created by github.com/caddyserver/caddy/v2.trapSignalsCrossPlatform
        github.com/caddyserver/caddy/[email protected]/sigtrap.go:37 +0x35

goroutine 50 [chan receive, 335 minutes]:
github.com/caddyserver/caddy/v2.trapSignalsPosix.func1()
        github.com/caddyserver/caddy/[email protected]/sigtrap_posix.go:34 +0x139
created by github.com/caddyserver/caddy/v2.trapSignalsPosix

The whole caddy process died.
Any hint of what could have gone wrong ?

Migrate to new Storage interface

Just a few hours ago, @mholt merged his changed to pull caddytls out into its own repo called certmagic. Along with that comes a new Storage interface which can be used to plug into certmagic. Check it out here: https://github.com/mholt/certmagic/blob/master/storage.go

Note that it looks like the storage option was commented out along with those changes though https://github.com/mholt/caddy/blob/e0f1a02/caddytls/setup.go#L256, but that's probably just temporary.

NATS

Nats now has a KV that is designed to replace consul.

https://nats.io/blog/kv-cli/

Build error with caddy 2.5.0

Hi @pteich

I see a build error using caddy 2.5.0, would be happy if you could look into it. Many thanks.

2022/05/03 10:00:05 [INFO] exec (timeout=0s): /usr/local/go/bin/go build -o /usr/bin/caddy -ldflags -w -s -trimpath 
# github.com/pteich/caddy-tlsconsul
/go/pkg/mod/github.com/pteich/[email protected]/module.go:47:9: cannot use cs (variable of type *ConsulStorage) as type certmagic.Storage in return statement:
	*ConsulStorage does not implement certmagic.Storage (wrong type for Delete method)
		have Delete(key string) error
		want Delete(ctx context.Context, key string) error
/go/pkg/mod/github.com/pteich/[email protected]/storage.go:157:25: undefined: certmagic.ErrNotExist
/go/pkg/mod/github.com/pteich/[email protected]/storage.go:161:25: undefined: certmagic.ErrNotExist
/go/pkg/mod/github.com/pteich/[email protected]/storage.go:179:20: undefined: certmagic.ErrNotExist
/go/pkg/mod/github.com/pteich/[email protected]/storage.go:183:20: undefined: certmagic.ErrNotExist
/go/pkg/mod/github.com/pteich/[email protected]/storage.go:212:31: undefined: certmagic.ErrNotExist
/go/pkg/mod/github.com/pteich/[email protected]/storage.go:216:31: undefined: certmagic.ErrNotExist
/go/pkg/mod/github.com/pteich/[email protected]/storage.go:254:41: undefined: certmagic.ErrNotExist

Empty ClusterPluginConstructor in plugin registration

I was excited to find this plugin and tried it out.
As this plugin is currently not available via the official downloader I registered it in caddy/caddymain/main.go and compiled caddy.
Once I started caddy with activated consul storage plugin it crashes during startup:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x8241dc]
goroutine 1 [running]:
github.com/mholt/caddy/caddytls.NewConfig(0xc420120300, 0xc42017f750, 0xc420036cf0, 0x24)
        /home/mswart/go/src/github.com/mholt/caddy/caddytls/config.go:117 +0x58c
github.com/mholt/caddy/caddyhttp/httpserver.(*httpContext).InspectServerBlocks(0xc420216150, 0x7ffccdf69e20, 0x14, 0xc42019b660, 0x1, 0x1, 0xc42019b660, 0x1, 0x1, 0x0, ...)
        /home/mswart/go/src/github.com/mholt/caddy/caddyhttp/httpserver/plugin.go:193 +0x5a8
github.com/mholt/caddy.ValidateAndExecuteDirectives(0xdb2de0, 0xc4201fa2c0, 0xc420120300, 0xc420120300, 0x1, 0xc42000c5f0)
        /home/mswart/go/src/github.com/mholt/caddy/caddy.go:609 +0x2c6
github.com/mholt/caddy.startWithListenerFds(0xdb2de0, 0xc4201fa2c0, 0xc420120300, 0x0, 0x0, 0x0)
        /home/mswart/go/src/github.com/mholt/caddy/caddy.go:517 +0x126
github.com/mholt/caddy.Start(0xdb2de0, 0xc4201fa2c0, 0xdb2de0, 0xc4201fa2c0, 0x0)
        /home/mswart/go/src/github.com/mholt/caddy/caddy.go:474 +0xf6
github.com/mholt/caddy/caddy/caddymain.Run()
        /home/mswart/go/src/github.com/mholt/caddy/caddy/caddymain/run.go:184 +0x404
main.main()
        src/github.com/mholt/caddy/caddy/main.go:27 +0x27

The null pointer appears to be the result of calling RegisterClusterPlugin with an uninitialized variable and not a function.
Experiments to use NewConsulStorage or some variation of such a function were unsuccessful (go type/interface mismatch during compilation).

How do I compile and use this plugin correctly? Thx.

"failed to release lock"

Is this the expected behavior for thus plugin?