Add Vault support

[schemahero]

Add Vault support

[pauldthomson]

Ok cool. I'll just create a branch/fork and start poking about. Dealing with merge conflicts shouldn't be too bad, it seems adapting to change is the norm at the moment 😛

[marccampbell]

This completely makes sense, and is a great idea.

I'd be pretty open to either a flexible alternative auth, or just this one. I do think that if we want to build a more generic abstraction, we should at least design for (at least) 2 other auth systems that could be used to make sure we are building the API properly.

That said, SchemaHero is currently working to ship a v1alpha3, so the API is pretty fluid still and can be changed in future versions without too much effort.

Regarding the CRD spec change:

Agreed, the Database CRD feels like it needs an extension to support this. How about just adding a vault specific key for now, that can be used to get the secrets? Would that be possible with the existing connection string URI would would the connection string need to be optional also?

In the code, the Database object is always available when creating a plan or execution, so this should be pretty straight forward to getting the vault config and injecting it.

Finally, the v1alpha3 (master) right now is still somewhat early with hopes to have a reasonably stable release of it ready at the end of March (planned for the original Kubecon EU). If you want to proceed with this (which would be awesome), just be warned that there's not a lot of refactoring needed here, but there are some rough areas of the new plan/apply approach. We need to create issues for these and get them documented (i'll work on this)

[pauldthomson]

After an initial spike/POC of this, I'm working on tidying up etc. I was originally thinking of just allowing for the addition of the native Vault annotations (https://www.hashicorp.com/blog/dynamic-database-credentials-with-vault-and-kubernetes/#injecting-secrets-into-kubernetes-deployments) on the Database CRD and they would be copied down onto the Pods that are created for plan/migrate.

However, it makes sense to have the template (https://github.com/hashicorp/consul-template#secret) to inject creds specified in a format that makes sense to SchemaHero, i.e. in the form of a URI (example here). This annotation is linked to the vault.hashicorp.com/agent-inject-secret-{name} annotation in that the name portion needs to match. So perhaps it makes sense that both of these are pre-defined?

This really just leaves 2 of the annotations, one is to enable the Vault agent injector, and one to define which vault role to use for the sidecar. We need a way to trigger the Vault functionality, and specifying a Vault role per Database will allow for granular permissions by users, which is probably desirable.

So, we could expose these 2 annotations only in the Database CRD, or extend the ValueFrom struct to contain a VaultSecret type, that could capture Vault specifics (path to secret, role to use).

Any input would be much appreciated :)