virtuslab/crypt

Universal cryptographic tool with AWS KMS, GCP KMS, GnuPG and Azure Key Vault support

crypt's Issues

GPG - support for keyring

The GnuPG keyring format has changed, unfortunately, opengpg sdk doesn't support this atm.

https://www.gnupg.org/faq/whats-new-in-2.1.html#nosecring
golang/go#29082
helm/helm#2843

Dependency constraint issue

You have a the following constraint in your Gopkg.toml

[[constraint]]
  name = "google.golang.org/api"
  branch = "master"

Please could you update this to a tagged version as this caused unresolvable conflicts when working with other projects that import this package.

Crypt does not support encrypting long secrets (more than ~500 bytes) with Azure KeyVault

Problem

Crypt does not support encrypting long secrets (more than ~500 bytes) with Azure KeyVault

Details

We tried to encrypt whole directory with one file that contains a long secret (ssh private key) and this happened:

encrypt-all-key-vault-secrets.sh 39005e2e-c0e8-40c4-aa7e-17619494c2b8 euw-prod-138-location-gl
DEBU[2020-01-28T16:00:08+01:00] Debug logging enabled
INFO[2020-01-28T16:00:09+01:00] Directory mode selected: '/Users/michalfudala/aloa/scripts/secret-management/../../key-vault/39005e2e-c0e8-40c4-aa7e-17619494c2b8/euw-prod-138-location-gl' -> '/Users/michalfudala/aloa/scripts/secret-management/../../key-vault/39005e2e-c0e8-40c4-aa7e-17619494c2b8/euw-prod-138-location-gl'
DEBU[2020-01-28T16:00:09+01:00] Skipping '/Users/michalfudala/aloa/key-vault/39005e2e-c0e8-40c4-aa7e-17619494c2b8/euw-prod-138-location-gl/git-secrets.backup'
DEBU[2020-01-28T16:00:09+01:00] Skipping '/Users/michalfudala/aloa/key-vault/39005e2e-c0e8-40c4-aa7e-17619494c2b8/euw-prod-138-location-gl/key-name'
DEBU[2020-01-28T16:00:09+01:00] Skipping '/Users/michalfudala/aloa/key-vault/39005e2e-c0e8-40c4-aa7e-17619494c2b8/euw-prod-138-location-gl/key-version'
DEBU[2020-01-28T16:00:09+01:00] Processing '/Users/michalfudala/aloa/key-vault/39005e2e-c0e8-40c4-aa7e-17619494c2b8/euw-prod-138-location-gl/preprod-docker-credentials.secret'
INFO[2020-01-28T16:00:09+01:00] Target directory was created: '/Users/michalfudala/aloa/key-vault/39005e2e-c0e8-40c4-aa7e-17619494c2b8/euw-prod-138-location-gl'
INFO[2020-01-28T16:00:10+01:00] Encryption succeeded                          key=git-secrets keyVaultURL="https://euw-prod-138-location-gl.vault.azure.net/" keyVersion=b7384be6d6b24efa86f264e43ae84052
DEBU[2020-01-28T16:00:10+01:00] Skipping '/Users/michalfudala/aloa/key-vault/39005e2e-c0e8-40c4-aa7e-17619494c2b8/euw-prod-138-location-gl/preprod-docker-credentials.secret.crypt'
DEBU[2020-01-28T16:00:10+01:00] Processing '/Users/michalfudala/aloa/key-vault/39005e2e-c0e8-40c4-aa7e-17619494c2b8/euw-prod-138-location-gl/preprod-git-private-key.secret'
INFO[2020-01-28T16:00:10+01:00] Target directory was created: '/Users/michalfudala/aloa/key-vault/39005e2e-c0e8-40c4-aa7e-17619494c2b8/euw-prod-138-location-gl'
ERROR: keyvault.BaseClient#Encrypt: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: Service returned an error. Status=400 Code="BadParameter" Message="The parameter is incorrect.\r\n"
crypt/vendor/github.com/VirtusLab/crypt/azure.(*KeyVault).encrypt
	/Users/michalfudala/.gopath/src/crypt/vendor/github.com/VirtusLab/crypt/azure/azure.go:91
crypt/vendor/github.com/VirtusLab/crypt/azure.(*KeyVault).Encrypt
	/Users/michalfudala/.gopath/src/crypt/vendor/github.com/VirtusLab/crypt/azure/azure.go:77
crypt/vendor/github.com/VirtusLab/crypt/crypto.(*crypt).Encrypt
	/Users/michalfudala/.gopath/src/crypt/vendor/github.com/VirtusLab/crypt/crypto/crypt.go:181
crypt/vendor/github.com/VirtusLab/crypt/crypto.(*crypt).EncryptFile
	/Users/michalfudala/.gopath/src/crypt/vendor/github.com/VirtusLab/crypt/crypto/crypt.go:144
crypt/vendor/github.com/VirtusLab/crypt/crypto.transformFiles
	/Users/michalfudala/.gopath/src/crypt/vendor/github.com/VirtusLab/crypt/crypto/crypt.go:84
crypt/vendor/github.com/VirtusLab/crypt/crypto.(*crypt).EncryptFiles
	/Users/michalfudala/.gopath/src/crypt/vendor/github.com/VirtusLab/crypt/crypto/crypt.go:115
main.encryptDirectory
	/Users/michalfudala/.gopath/src/crypt/main.go:369
main.action
	/Users/michalfudala/.gopath/src/crypt/main.go:338
main.encryptAction
	/Users/michalfudala/.gopath/src/crypt/main.go:361
main.encrypt.func1
	/Users/michalfudala/.gopath/src/crypt/main.go:207
crypt/vendor/github.com/urfave/cli.HandleAction
	/Users/michalfudala/.gopath/src/crypt/vendor/github.com/urfave/cli/app.go:514
crypt/vendor/github.com/urfave/cli.Command.Run
	/Users/michalfudala/.gopath/src/crypt/vendor/github.com/urfave/cli/command.go:171
crypt/vendor/github.com/urfave/cli.(*App).RunAsSubcommand
	/Users/michalfudala/.gopath/src/crypt/vendor/github.com/urfave/cli/app.go:395
crypt/vendor/github.com/urfave/cli.Command.startApp
	/Users/michalfudala/.gopath/src/crypt/vendor/github.com/urfave/cli/command.go:383
crypt/vendor/github.com/urfave/cli.Command.Run
	/Users/michalfudala/.gopath/src/crypt/vendor/github.com/urfave/cli/command.go:103
crypt/vendor/github.com/urfave/cli.(*App).Run
	/Users/michalfudala/.gopath/src/crypt/vendor/github.com/urfave/cli/app.go:265
main.main
	/Users/michalfudala/.gopath/src/crypt/main.go:101
runtime.main
	/usr/local/Cellar/go/1.13.5/libexec/src/runtime/proc.go:203
runtime.goexit
	/usr/local/Cellar/go/1.13.5/libexec/src/runtime/asm_amd64.s:1357
encrypting failed, file '/Users/michalfudala/aloa/key-vault/39005e2e-c0e8-40c4-aa7e-17619494c2b8/euw-prod-138-location-gl/preprod-git-private-key.secret'
crypt/vendor/github.com/VirtusLab/crypt/crypto.(*crypt).EncryptFile
	/Users/michalfudala/.gopath/src/crypt/vendor/github.com/VirtusLab/crypt/crypto/crypt.go:146
crypt/vendor/github.com/VirtusLab/crypt/crypto.transformFiles
	/Users/michalfudala/.gopath/src/crypt/vendor/github.com/VirtusLab/crypt/crypto/crypt.go:84
crypt/vendor/github.com/VirtusLab/crypt/crypto.(*crypt).EncryptFiles
	/Users/michalfudala/.gopath/src/crypt/vendor/github.com/VirtusLab/crypt/crypto/crypt.go:115
main.encryptDirectory
	/Users/michalfudala/.gopath/src/crypt/main.go:369
main.action
	/Users/michalfudala/.gopath/src/crypt/main.go:338
main.encryptAction
	/Users/michalfudala/.gopath/src/crypt/main.go:361
main.encrypt.func1
	/Users/michalfudala/.gopath/src/crypt/main.go:207
crypt/vendor/github.com/urfave/cli.HandleAction
	/Users/michalfudala/.gopath/src/crypt/vendor/github.com/urfave/cli/app.go:514
crypt/vendor/github.com/urfave/cli.Command.Run
	/Users/michalfudala/.gopath/src/crypt/vendor/github.com/urfave/cli/command.go:171
crypt/vendor/github.com/urfave/cli.(*App).RunAsSubcommand
	/Users/michalfudala/.gopath/src/crypt/vendor/github.com/urfave/cli/app.go:395
crypt/vendor/github.com/urfave/cli.Command.startApp
	/Users/michalfudala/.gopath/src/crypt/vendor/github.com/urfave/cli/command.go:383
crypt/vendor/github.com/urfave/cli.Command.Run
	/Users/michalfudala/.gopath/src/crypt/vendor/github.com/urfave/cli/command.go:103
crypt/vendor/github.com/urfave/cli.(*App).Run
	/Users/michalfudala/.gopath/src/crypt/vendor/github.com/urfave/cli/app.go:265
main.main
	/Users/michalfudala/.gopath/src/crypt/main.go:101
runtime.main
	/usr/local/Cellar/go/1.13.5/libexec/src/runtime/proc.go:203
runtime.goexit
	/usr/local/Cellar/go/1.13.5/libexec/src/runtime/asm_amd64.s:1357
can't encrypt/decrypt a file
crypt/vendor/github.com/VirtusLab/crypt/crypto.transformFiles
	/Users/michalfudala/.gopath/src/crypt/vendor/github.com/VirtusLab/crypt/crypto/crypt.go:86
crypt/vendor/github.com/VirtusLab/crypt/crypto.(*crypt).EncryptFiles
	/Users/michalfudala/.gopath/src/crypt/vendor/github.com/VirtusLab/crypt/crypto/crypt.go:115
main.encryptDirectory
	/Users/michalfudala/.gopath/src/crypt/main.go:369
main.action
	/Users/michalfudala/.gopath/src/crypt/main.go:338
main.encryptAction
	/Users/michalfudala/.gopath/src/crypt/main.go:361
main.encrypt.func1
	/Users/michalfudala/.gopath/src/crypt/main.go:207
crypt/vendor/github.com/urfave/cli.HandleAction
	/Users/michalfudala/.gopath/src/crypt/vendor/github.com/urfave/cli/app.go:514
crypt/vendor/github.com/urfave/cli.Command.Run
	/Users/michalfudala/.gopath/src/crypt/vendor/github.com/urfave/cli/command.go:171
crypt/vendor/github.com/urfave/cli.(*App).RunAsSubcommand
	/Users/michalfudala/.gopath/src/crypt/vendor/github.com/urfave/cli/app.go:395
crypt/vendor/github.com/urfave/cli.Command.startApp
	/Users/michalfudala/.gopath/src/crypt/vendor/github.com/urfave/cli/command.go:383
crypt/vendor/github.com/urfave/cli.Command.Run
	/Users/michalfudala/.gopath/src/crypt/vendor/github.com/urfave/cli/command.go:103
crypt/vendor/github.com/urfave/cli.(*App).Run
	/Users/michalfudala/.gopath/src/crypt/vendor/github.com/urfave/cli/app.go:265
main.main
	/Users/michalfudala/.gopath/src/crypt/main.go:101
runtime.main
	/usr/local/Cellar/go/1.13.5/libexec/src/runtime/proc.go:203
runtime.goexit
	/usr/local/Cellar/go/1.13.5/libexec/src/runtime/asm_amd64.s:1357
DEBU[2020-01-28T16:00:10+01:00] exiting with 1

Root cause

After investigation, it turns out that azure-sdk-for-go that crypt uses (and underlying key vault encrypt REST API only supports encrypting a single block of data, the size of which is dependent on the target key and the encryption algorithm.

Root cause reproducer:

1. Create some KeyVault
2. Execute

token=$(az account get-access-token --resource 'https://vault.azure.net' | jq .accessToken -r)
secret=$(python -c 'print("5" * 600)')

curl -X POST -H "Authorization: Bearer ${token}" -H "Content-Type: application/json" https://{azure-key-vault-url}/keys/{key-name}/{key-version}/encrypt\?api-version\=7.0 -d "{\"alg\": \"RSA-OAEP-256\", \"value\": \"${secret}\"}"

Avoid setting the "default" profile if not passed

The profile should not be enforced as default if not passed through the --profile flag. This could break the session creation on environments where the authentication is not managed with a profile, like on EKS where it can be configured through service accounts.

Add GPG support

Similar to what https://github.com/shyiko/kubesec is doing.

HashiCorp Vault support

The transit secrets engine handles cryptographic functions on data in-transit. Vault doesn't store the data sent to the secrets engine. It can also be viewed as "cryptography as a service" or "encryption as a service". The transit secrets engine can also sign and verify data; generate hashes and HMACs of data; and act as a source of random bytes.

https://www.vaultproject.io/docs/secrets/transit/index.html

Azure Encrypt error 404

echo "This_is_A_secret" | ./crypt encrypt azure --vaultURL https://XXXX.vault.azure.net --name testsecret --version 8dc4f7c4-046d-401b-88c6-7467d4680764

ERROR: keyvault.BaseClient#Encrypt: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: error response cannot be parsed: "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\" \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\">\r\n<html xmlns=\"http://www.w3.org/1999/xhtml\">\r\n<head>\r\n<meta http-equiv=\"Content-Type\" content=\"text/html; charset=iso-8859-1\"/>\r\n<title>404 - File or directory not found.</title>\r\n<style type=\"text/css\">\r\n<!--\r\nbody{margin:0;font-size:.7em;font-family:Verdana, Arial, Helvetica, sans-serif;background:#EEEEEE;}\r\nfieldset{padding:0 15px 10px 15px;} \r\nh1{font-size:2.4em;margin:0;color:#FFF;}\r\nh2{font-si" error: invalid character '<' looking for beginning of value

Also figured I'd ask while I'm making an Issue, if I am decrypting a key, what is the file I need to provide if I did not encrypt it using crypt (or is this not possible)?

./crypt decrypt azure --vaultURL https://xXxxxXx.vault.azure.net --name name-of-secret

ERROR: expected a pipe stdin

Add CONTRIBUTING.md

We can consider adding Contributing guide (aligned with VL)

some templates:
https://github.com/nayafia/contributing-template
https://gist.github.com/PurpleBooth/b24679402957c63ec426

what do you think?
cc @pdolega

Document crypt recipes

crypt can be useful especially in combination with https://github.com/VirtusLab/render or any other tool.

A few examples:

• secrets managements (Kubernetes, PKI, etc.)
• versioning PKI in VCS
• infrastructure as code (putting sensitive files in CloudFormation stack templates via write_files:)

Switch to go modules from dep

As Go modules became standard for dependency management we should consider migration.

• https://github.com/golang/go/wiki/Modules
• https://blog.golang.org/migrating-to-go-modules

crypt as a Helm plugin

Client-side Helm plugin for encryption/decryption
https://github.com/helm/helm/blob/master/docs/plugins.md

Support multiple Azure authentication methods

Currently, only the authentication through configuration from Azure Cli 2.0 is supported (configured from Azure CLI 2.0). It would be useful to leverage the Chained Credential strategy, where multiple type credentials are linked together, detected from the environment, and all of them are tried until one authentication method succeeds.
The current implementation does not support the authentication through the Azure Managed Identity, typically used on AKS.

Required flags

It would be nice to enforce required flags and validate them (if nil or empty) before executing any encrypt/decrypt action.

Few examples of required flags:
gcp: --location, --project, key, keyring
aws: --region, key-id
etc.

Unfortunately, library what we use doesn't support Required Flags.
urfave/cli#85 - issue is still open

distribution packaging [mac - linux]

As an ops engineer, I frequently lookup crypt and installation page over and over again. It would be great to have brew install vlcrypt or apt-get install vlcrypt from command line. (or whatever unique name you come up with)

Not able to connect to AWS using crypt

crypt enc aws --key-id <> --region us-east-1 --out /tmp/dd1 --in /tmp/dd
ERROR: encrypting failed, file '/tmp/dd': NoCredentialProviders: no valid providers in chain
caused by: EnvAccessKeyNotFound: failed to find credentials in the environment.
SharedCredsLoad: failed to load profile, default.
EC2RoleRequestError: no EC2 instance role found
caused by: RequestError: send request failed
caused by: Get "http:///latest/meta-data/iam/security-credentials/": dial tcp :80: i/o timeout (Client.Timeout exceeded while awaiting headers)

I am able to connect to aws using AWS CLI but not able to connect using crypt

what could be the issue? Any clue? Thanks.

Version flag doesn't return correct value

The latest release returns the following:

$ crypt --version
Crypt version 0.0.1

The version install via go get returns the following:

$ ~/go/bin/crypt --debug --version
crypt version -

Encrypt adds a new line

illustration of the problem with, logging added for highlight

crypt encrypt aws --profile dev --region eu-west-1 --kms alias/eks1-eu-west-1-dev --in <(echo test) | crypt decrypt aws --profile dev --region eu-west-1
INFO 'test
'                                      
INFO 'test
'                                      
INFO 'test
'                                      
test