Giter Site home page Giter Site logo

icy / genvsub Goto Github PK

View Code? Open in Web Editor NEW
12.0 4.0 5.0 77 KB

Another way to substitute environment variables in shell format strings ${FOO}, designed for k8s stuff

License: MIT License

Go 42.46% Makefile 2.64% Shell 54.89%
shell variable substitution template-engine k8s helm kustomization terraform

genvsub's Introduction

icy

Alternative to Gettext's envsubst

Substitutes environment variables in shell format strings, and can raise error if any environment is not set. There are two kinds of format string $FOO and ${FOO}. This tool only works with the later form and can serve a simple template engine for your shell scripts ;) The program can also limit its action to a small set of variables whose names match a predefined prefix/regexp.

TOC

Usage

Supported options

  • -v: Scan and output all occurrences of variables in the input.
  • -u: Raise error when environment variable is not set. When being used with -v, the program scans through the whole input; otherwise, the program stops immediately when there is any undefined (environment) variable.
  • -p regexp: Limit substitution to variables that match this prefix. You can use some regular expression as prefix. Default to [^}]+. Can be used as an alternative to SHELL-FORMAT option in the original GNU envsubst

It's highly recommended to use -u option. It's the original idea why this tool was written.

Installation. Examples

Starting from v1.2.2, you can download binary files generated automatically by Github-Action action (via goreleaser tool). You find the files from the release listing page: https://github.com/icy/genvsub/releases

To install on your laptop by local compiling process, please try the popular way

$ go get -v github.com/icy/genvsub
$ export PATH=$PATH:$(go env GOPATH)/bin

If you don't want to install, you can use it like this:

$ echo '${HOME}' |  go run github.com/icy/genvsub@latest -u

The program works with STDIN and write their output to STDOUT

$ echo 'My home is $HOME'   | ./genvsub # This will output "My home is $HOME"
$ echo 'My home is ${HOME}' | ./genvsub # This will output "My home is /home/your-home-path"
$ echo 'Raise error with unset variable ${XXHOME}' | ./genvsub -u

To limit substitution to variables that match some prefix, use -p option:

$  echo 'var=${TEST_VAR}' | ./genvsub -u -p SAFE_
:: genvsub is reading from STDIN and looking for variables with regexp '\${(SAFE_)}'
var=${TEST_VAR}

$ echo '${TEST_VAR}' | ./genvsub -u -p 'TEST_.*'
:: genvsub is reading from STDIN and looking for variables with regexp '\${(TEST_.*)}'
<TEST_VAR::error::variable_unset>
:: Environment variable 'TEST_VAR' is not set.

The second command raises an error because the variable TEST_VAR matches the expected prefix TEST_ and its value is not set.

You can also specify exactly a few variables to be substituted (which is exactly an alternative to the shell-format option in the original GNU tool envsubst):

$ echo '${TEST_VAR}' | ./genvsub -u -p 'VAR_NAME_3|VAR_NAME_3|VAR_NAME_3'

Note on variable prefix

When using -p string to specify the variable prefix, you can also use some simple regular expression. However, please note that for the given input argument -p PREFIX, the program will build the final regexp \${(PREFIX)}.

  1. Hence you can't use for example -p '^FOO'.
  2. You can also easily trick the program with some fun PREFIX ;) However, as seen in

    genvsub/sub.go

    Line 42 in 33e6804

    if in_st[0:2] != "${" || in_st[len(in_st)-1:len(in_st)] != "}" {
    the program enforces input data to follow the form ${VARIABLE_NAME}. I'm still thinking if we can allow more tricks here.

Problems

Clean up on Dec 18th 2020.

Development

Smoke tests

We don't likely have tests with Golang. We have some smoke tests instead.

$ make build tests

Tests are written in shell script. Please have a look at tests/test.sh.

Ruby version

Removed on Dec 18th 2020.

References

  • Original tool: https://www.gnu.org/software/gettext/manual/html_node/envsubst-Invocation.html (which can't raise error if some variable is not set.)
  • https://github.com/a8m/envsubst : A Go tool; It tries to behave like Bash with some advanced feature dealing with empty/non-set/default values https://github.com/a8m/envsubst#docs; it can also raise error if some variable is empty and/or not-set. This tool has a few features that I don't need, and at the same time it doesn't have some features I need (reg-exp variable name filter, strict variable naming format, bla bla)
  • https://github.com/gdvalle/envsub : A Rust tool; It introduces new syntax %VAR%, which can be refined with ENVSUB_PREFIX=% and ENVSUB_SUFFIX=%. When hitting unset variables it will exit rather than expanding as empty strings. It also fully buffers input before writing, so in-place replacement is possible.
  • https://github.com/s12v/exec-with-secrets: Fetch configuration/secrets/variables at run time, and provides them to your program. Now you have to rebuild your docker images with the tool atop your original executable command.

License

This work is writtedn by Ky-Anh Huynh and it's released under a MIT license.

genvsub's People

Contributors

guettli avatar icy avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

genvsub's Issues

Support the "popular" form '$ENV_NAME'

The expansion form $ENV_NAME is widely use and the tool now ignores them. It's great if we can have some simple way to warn users to change to ${ENV_NAME} and/or to support that form.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.