hashicorp/vault-plugin-secrets-kv

Vault Plugin: Key-Value Secrets Backend

This is a standalone backend plugin for use with Hashicorp Vault. This plugin provides Key-Value functionality to Vault.

Please note: We take Vault's security and our users' trust very seriously. If you believe you have found a security issue in Vault, please responsibly disclose by contacting us at [email protected].

Quick Links

- Vault Website: https://www.vaultproject.io
- KV Docs: https://www.vaultproject.io/docs/secrets/kv/index.html
- Main Project Github: https://www.github.com/hashicorp/vault

Getting Started

This is a Vault plugin and is meant to work with Vault. This guide assumes you have already installed Vault and have a basic understanding of how Vault works.

Otherwise, first read this guide on how to get started with Vault.

To learn specifically about how plugins work, see documentation on Vault plugins.

Usage

Please see documentation for the plugin on the Vault website.

This plugin is currently built into Vault and by default is accessed at kv. To enable this in a running Vault server:

$ vault secrets enable kv
Success! Enabled the kv secrets engine at: kv/

Additionally starting with Vault 0.10 this backend is by default mounted at secret/.

Developing

If you wish to work on this plugin, you'll first need Go installed on your machine (version 1.10+ is required).

For local dev first make sure Go is properly installed, including setting up a GOPATH. Next, clone this repository into $GOPATH/src/github.com/hashicorp/vault-plugin-secrets-kv. You can then download any required build tools by bootstrapping your environment:

$ make bootstrap

To compile a development version of this plugin, run make or make dev. This will put the plugin binary in the bin and $GOPATH/bin folders. dev mode will only generate the binary for your platform and is faster:

$ make
$ make dev

Once you've done that, there are two approaches to testing your new plugin version in Vault. You can add a temporary replace declaration in your local Vault checkout's go.mod (above the require declarations), such as:

replace github.com/hashicorp/vault-plugin-secrets-kv => /path/to/your/project/vault-plugin-secrets-kv

Alternatively, you could go through the plugin process. To do this, put the plugin binary into a location of your choice. This directory will be specified as the plugin_directory in the Vault config used to start the server.

...
plugin_directory = "path/to/plugin/directory"
...

Start a Vault server with this config file:

$ vault server -config=path/to/config.json ...
...

Once the server is started, register the plugin in the Vault server's plugin catalog:

$ vault plugin register \
        -sha256=<expected SHA256 Hex value of the plugin binary> \
        -command="vault-plugin-secrets-kv" \
        secret \
        kv

Note you should generate a new sha256 checksum if you have made changes to the plugin. Example using openssl:

openssl dgst -sha256 $GOPATH/vault-plugin-secrets-kv
...
SHA256(.../go/bin/vault-plugin-secrets-kv)= 896c13c0f5305daed381952a128322e02bc28a57d0c862a78cbc2ea66e8c6fa1

Enable the auth plugin backend using the secrets enable plugin command:

$ vault secrets enable -plugin-name='kv' plugin
...

Successfully enabled 'plugin' at 'kv'!

Tests

If you are developing this plugin and want to verify it is still functioning (and you haven't broken anything else), we recommend running the tests.

To run the tests, invoke make test:

$ make test

You can also specify a TESTARGS variable to filter tests like so:

$ make test TESTARGS='--run=TestConfig'

vault-plugin-secrets-kv's Issues

indirect dependency github.com/mitchellh/osext has been deleted

I cannot upgrade to the latest due to indirect dependency being deleted https://github.com/mitchellh/osext/. Details here https://gist.github.com/mitchellh/90029601268e59a29e64e55bab1c5bdc?permalink_comment_id=4802235#gistcomment-4802235

go get -u github.com/hashicorp/vault-plugin-secrets-kv
produces error:

go: github.com/hashicorp/vault-plugin-secrets-kv@upgrade (v0.16.2) indirectly requires github.com/mitchellh/[email protected]: github.com/mitchellh/[email protected]: invalid version: git ls-remote -q origin in /Users/hharbin/go/pkg/mod/cache/vcs/...: exit status 128:
        remote: Repository not found.
        fatal: repository 'https://github.com/mitchellh/osext/' not found

Upgrade Plugin using same storage backend

How can we upgrade an existing custom vault plugin such that storage backend data remains intact and can be used with the new upgraded plugin?

Improve documentation

Can you please improve the documentation of this plugin, like how to configure (config path) it or how to use it.

Incompatible type Backend vs Factory

Using the master branch:

make dev

==> Checking that code complies with gofmt requirements...
go generate
...
./main.go:21:21: cannot use kv.Factory (type func(context.Context, *"github.com/hashicorp/vault-plugin-secrets-kv/vendor/github.com/hashicorp/vault/logical".BackendConfig) ("github.com/hashicorp/vault-plugin-secrets-kv/vendor/github.com/hashicorp/vault/logical".Backend, error)) as type "github.com/hashicorp/vault/logical".Factory in field value

Upgrading from non-versioned to versioned data. This backend will be unavailable for a brief period and will resume service shortly.

Hi,
I get this error for kv v2 secrets engine with v2 syntax with -mount option, vault kv put -mount=secret my-secret foo=a bar=b.
Vault is deployed in kubernetes in HA mode with raft storage and UI enabled.

[Feature request]Add salt hash kv path

Overview

Currently, it is difficult to enforce secret lifecycle best practices on KV secrets in a secure manner. Three best practices are (1) secrets are not stale (2) the same secret isn’t reused in multiple paths (3) secrets are not in source code.
A component to enforce these best practices (external to Vault), would encounter some difficulties with the current KV mount today.

(1) Secrets are not stale
While KV metadata provides the age of a particular version of a secret, it does not provide information whether the current version is the same as a previous version of the secret. To ensure the case that a secret value has been updated, the component needs access to KV data. To check how long a secret remains unchanged, a component needs to compare values between different versions.

(2) Same secret isn’t reused in multiple paths
This again needs access to KV data. If secrets are not to be stored outside of the vault, it is resource intensive since each secret needs to be compared with every other secret.

(3) Secrets are not in source code
Source code scanners use heuristics – regexes and entropy. An improvement would be to scan source code directly and check directly whether the KV store contains those secrets.
To enforce all 3 parts, the current KV API requires the “component” to have read access to all secrets in the KV mount, which makes this component nearly as sensitive as Vault itself.

Here is our proposal, instead of relying on reading all secret values from KV, the component could use the salted hash of all the values from path, with a new API (//salt-hash/:path). The salt is similar to the salt in /sys/audit-hash
By providing such an API, we could 1) avoid security issues that a secret lifecycle management component has broad access to actual KV secrets 2) improve the policy management.

If a user wants to compare two secret versions, they now only need to compare the hashes of the two versions. It is also easy to compare the hashes of different secret paths to find secrets reused in multiple paths (and the enforce component can store hashes to save computation time). Finally, a source code scanner can generate hashes of candidate strings and use the hash to see if there is a match with an existing secret without accessing the secret.

Design of Change

A GetSaltHashValue handler was added to the salt path (//salt-hash/:path). A GetSaltHashValue request must be performed on an existing resource. If a key metadata entry exists without data, a 404 is returned. If no key metadata or data entry exists for the path requested, the GetSaltHashValue short circuits and will ultimately bubble up to a 404 within Vault's response handling logic. For an existing secret path with both data and metadata in place, when GetSaltHashValue is invoked, it will return the salt hash version of each attribute in the data object in json format. Meanwhile, GetSaltHashValue also takes one optional parameters as input, version number. If the parameters are not set, it will use the default version. For instance, if the version is not specified, it will by default use the latest version.

/kv/salt-hash:path 
Request  { "version" : 1 }
Response { "foo" : <salt hash of foo>, "value" : <salt hash of value> }
Policy in hclPath "kv/salt-hash/*" {capabilitiy = ["read"]}