Comments (9)
Here is my current Terraform-based workaround using a provisioner: I declare a module sensitive-value
containing the following:
variable "path" {
description = "The path to store the value at."
}
variable "value" {
description = "The value to json-encode and store. Sensitive fields are like $${ENV_VAR}."
type = "map"
}
variable "triggers" {
description = "Triggers for refreshing the sensitive value."
type = "map"
default = {}
}
resource "null_resource" "sensitive_value" {
triggers = "${var.triggers}"
provisioner "local-exec" {
command = <<EOT
envsubst <<EOTFJSON | vault write ${var.path} -
${jsonencode(var.value)}
EOTFJSON
EOT
}
}
It is used like this:
module "okta_duo_access" {
source = "../sensitive-value"
path = "auth/${vault_auth_backend.okta.id}/duo/access"
value = {
host = "redacted_subdomain.duosecurity.com"
ikey = "redacted_ikey"
skey = "$${DUO_SKEY}"
}
}
Where DUO_SKEY
is the environment variable holding the secret.
This works pretty well. The key advantage of doing this in Terraform itself is that you get the ordering guarantees (e.g. ${vault_auth_backend.okta.id}
makes sure this runs after the Okta backend is mounted).
It would still be great to have this directly supported by the provider, though.
EDIT: Updated example code to better handle arbitrary JSON values (in particular, the single quotes that can crop up in DB connection configs).
from terraform-provider-vault.
Hi guys, is there an update on this topic? Seems weird to me that there's no common idion for Vault + Terraform keeping secrets out of the state files - the only official approach is to encrypt the remote data (i.e. S3 backend with CMKs).
from terraform-provider-vault.
Hi @jgiles! Thanks for opening an issue. I must admit I'm a little confused. What it sounds like is you're looking for Terraform to make an API call, and then not keep any of the results around in state. Is that right? If that's the case, what does Terraform provide that using a provisioner that calls out to a vault
CLI doesn't provide? If Terraform isn't keeping any state around, I don't really see what benefit it's providing. Can you elaborate a bit on what you're hoping to achieve?
from terraform-provider-vault.
Sure!
(Note: Having written the below and then reading your comment again, it occurs to me that you might have meant I could use a trick with Terraform provisioners to accomplish this. If so, what would that look like?)
The vast majority of our Vault configuration data is non-sensitive: We are comfortable making visible to everyone with source access information like what people are in which groups, what policies are applied to which groups, how we have our internal CA configured, which IAM roles can get access to which secrets. We want to check in all this configuration and do change management through a "commit code, deploy" pattern.
However, there are a few pieces of information we do NOT want to check in: Okta/Duo API keys, AWS keys with the ability to generate temporary IAM roles, database credentials with root or near-root authority.
A provisioner calling out the the Vault CLI is our current solution: We do something similar to https://www.hashicorp.com/blog/codifying-vault-policies-and-configuration.html, checking in a mix of JSON files and HCL policy docs and running a Bash script calling vault write
or vault policy write
as appropriate. The aforementioned "sensitive" fields are subbed in from environment variables with an envsubst
preprocessing step.
Terraform offers a number of improvements over this arrangement:
- Proper handling of removed/deleted resources
- No needless overwrites of unchanged resources (some Vault resources do not tolerate overwrites)
- Confirmation of the configuration delta at deployment time
- Significantly improved configuration authoring environment: Variables, commenting, more concise and forgiving syntax, sanely re-usable modules
- Less brittle/hacky/error-prone than rolling our own provisioner code (in Bash or otherwise)
For all of these reasons, we want to Terraform the entirety of our Vault configuration. Except, there are a few fields we can't tolerate ending up in Terraform's saved state. For things like the AWS and DB creds, we want to create them, put them in Vault, and destroy all other copies.
Hence this proposal: Add the option to exclude selected Vault resource fields from persistence in Terraform state. It's hashicorp/terraform#15797 scoped down to just Vault rules.
For now, I can use our current process for the resources with sensitive pieces and Terraform for the rest. But that's not a great arrangement long-term; this seems like a problem worth solving in the Terraform tooling.
from terraform-provider-vault.
Yes, it would be amazing to be able to mark certain fields to either not be stored in state or to be encrypted. Seems like a perfect application for Vault Transit. That level of integration would probably have to happen in Terraform itself, I suppose. But in any case, yes, there are often situations (like the Vault LDAP auth provider or the various secret backends) where I would like to configure the backends with Terraform and benefit from keeping the state of all of the fields... except the one field that's secret. I don't want a management token for Consul stored in my TF state, although I would like to be able to configure Consul secret providers with Terraform. etc etc.
I'm not sure if this is the right ticket to have made this rant but i figure any noise helps the team notice.
from terraform-provider-vault.
For reference, the suggestion I was making was to use local-exec
to run a Vault command:
resource "null_resource" "vault_write" {
provisioner "local-exec" {
inline = [
"vault write ......",
]
}
}
Not keeping the results around in state means Terraform can't really do anything about diffing, ensuring the value is set, or anything like that, which basically breaks most of Terraform's functionality. At that point, you're essentially just running a CLI command, so why not just do that? If you want to make it happen when you call terraform apply
, the above is a way to make that happen.
Most of the benefits of Terraform you mention, namely:
- Proper handling of removed/deleted resources
- No needless overwrites of unchanged resources (some Vault resources do not tolerate overwrites)
- Confirmation of the configuration delta at deployment time
All rely on state. They can't function without the value being in state.
I know the Terraform core team is interested in encrypting an entire state file or just some fields of it, but as far as I know it's not on the immediate roadmap.
from terraform-provider-vault.
@paddycarver I think you're missing the point. We do want TF to handle the state for these secrets and keep track of things. But currently, doing so means we have to let the secrets be written to the tfstate file unencrypted. There is no option to encrypt certain fields at rest and let Terraform decrypt them in memory at runtime to do the comparisons.
from terraform-provider-vault.
I think you're missing the point.
I'm almost certain I am, because I remain very confused! :) I think all the talk of avoiding storing a value, keeping things out of state, and not persisting things had me confused.
My stance on encrypting the values is still that waiting for Terraform to encrypt fields marked as sensitive is a better approach than trying to reimplement encryption on each and every sensitive field ourselves, but I'm willing to be persuaded otherwise.
from terraform-provider-vault.
@paddycarver How do you safely capture the output of the command executed in the local-exec
provisioner? One option would be to redirect the output to disk, but then the secret gets leaked into the file.
My use case is something similar to:
provider "vault" {}
data "vault_generic_secret" "newrelic" {
path = "secret/newrelic"
# proposed flag
persist_to_state = false
}
provider "newrelic" {
api_key = data.vault_generic_secret.newrelic.data["newrelic_api_key"]
}
# Other New Relic resources
# ...
If the code looks something like this when using a provisioner, that would be great. And I would agree there is not so much value in a new flag to avoid persisting the secret to the state file. But, so far it seems there is no way to achieve anything similar to this. Do you know any?
from terraform-provider-vault.
Related Issues (20)
- [Enhancement]: vault_azure_secret_backend_role support for EntraID Roles
- [Bug]: vault_jwt_auth_backend tune block always updates unless config matches Vault server defaults
- [Enhancement]: Allow setting num_uses and ttl for vault_approle_auth_backend_role_secret_id
- [Enhancement]: `signature_bits ` field missing on some resources
- [Enhancement]: Allow setting not_after for vault_pki_secret_backend_root_cert
- [Bug]: Error 404: Unsupported Path for IdentityOidcRole API Request when using email as name HOT 1
- [Bug]: Wrong creation of intermediate CA certificate with `max_path_length = 0`
- [Bug]: vault_nomad_secret_backend can't be initialized without token
- [Bug]: vault_jwt_auth_backend sends request to false API path HOT 2
- [Docs]: MFA type is not documented HOT 1
- [Enhancement]: Allow updating `vault_kv_secret_v2 metadata` with `disable_read=true`
- [Bug]: `add_address_to_env` does not affect `local-exec` provisioners
- [Enhancement]: data source for getting Vault server address
- [Bug]: vault_ssh_secret_backend_ca gets replaced after provider upgrade HOT 2
- [Docs]: Update docs to include 'address' Argument for TFC/E Secrets backend
- [Enhancement]: Add persist_app parameter to vault_azure_secret_backend_role resource
- [Bug]: Snapshot agent local_max_space 0 is not allowed
- [Bug]: resource vault_identity_oidc_provider doesn't support import
- [Bug]: auth_login_userpass ignores password_file parameter
- Support database dynamic credential_type & credential_config block
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from terraform-provider-vault.