cellpointmobile/vk

vk installs and updates single-binary programs from Github and Hashicorp.

vk installs and updates single-binary programs from Github and Hashicorp.

vk is an abbreviation of værktøjskasse [ˈvæɐ̯gtʌjs-], which is a danish word meaning toolbox.

Table of contents

• Table of contents
• Getting started
• Supported tools
• Usage
• Configuring
• Github API rate limiting
• Problem abstract
• The solution
• Limitations
• Tool definitions

Getting started

vk is made to install tools for a single user, not system-wide. It defaults to use the directory $HOME/.local/bin for tools, but this is configurable. Before you start, you have to ensure this directory is present and that you modify your PATH to include it. When this has happened, go download the latest release of vk, rename the binary to just vk, move it into the directory and give it execute permission with chmod +x vk. Because vk is created just like the tools it supports, it can keep itself updated (and even uninstall itself!).

Quick install:

curl -Lo ~/.local/bin/vk https://github.com/cellpointmobile/vk/releases/download/v0.2.11/vk_0.2.11_Linux_x86_64 && chmod +x ~/.local/bin/vk

Supported tools

Here are some tools that are supported. The list is not exhaustive and can will probably be expanded in the future.

• Ark (recently renamed to Velero)
• Dive
• Docker-compose
• Fly
• Helm
• Helmfile
• Img
• K8sec
• Kops
• Kustomize
• Minikube
• Stern
• Terraform
• terraform-docs
• tflint

Usage

To list tools that vk can install use the subcommand "available":

vk available

Optionally add the flag --all to also include already installed tools in the list.

To install a tool use the subcommand "install":

vk install minikube

To update all installed tools use the subcommand "update":

vk update

To install a specific tool only, also specify the tool in the "update" subcommand:

vk update minikube

To list installed tools use the subcommand "installed":

vk installed

To uninstall a tool use the subcommand "uninstall":

vk uninstall minikube

If you want to run update in a cronjob, there is a quiet flag you can use:

vk update --quiet

Update and install can also be forced to download and overwrite the local version with the latest version, even if the are the same:

vk [install|update] minikube --force

Bash/Zsh completion is available from the "completion" subcommand:

source <(vk completion [bash|zsh])

It is possible to change which directory to use for bin files, by using the global flag --bindir. You can also set the bindir config var, to avoid passing the flag on every run. The flag overrides the config var.

It is also possible to change which URL to download the definitions from by using the global flag --definitions. This is also available as a config variable.

Configuring

Certain things can be configured by creating a file called ~/.vk/config.yaml. The following variables can be set:

• bindir - the path to where bin files are located. Defaults to $HOME/.local/bin The directory must exist and should be in your $PATH.
• github-api-token - A Github personal access token with the scope public_repo. This token is used for API calls to Github to allow for a larger rate limit.
• definitions - The URL where the definitions file is available. Can be a local path.

Github API rate limiting

vk normally talks to the Github API unauthenticated, which means it is subject to quite strict rate limiting - 60 requests per hour. If you run into this problem (vk will tell you), you can either wait an hour or add a Github personal access token to your vk config. You can create a new token here: https://github.com/settings/tokens/new. The token only needs the public_repo scope and nothing else. When you have generated a token you have to insert the following in ~/.vk/config.yaml (create it, if it does not exist):

github-api-token: YOUR-TOKEN-HERE

Problem abstract

Infrastructure engineers and architects (the sysadmins of yesteryears) who are working with cloud, containers, 12-factor etc need a bunch of modern tools to do their jobs effectively. A lot of these tools have adopted a new paradigm where they are released as single-binaries (often written in Go) on Github. This makes for a fairly easy workflow when releasing, as there are nice tools like goreleaser available that will build a binary, create a changelog and do a release on Github for you. For a single tool, this isn't too bad for a user - just go to the requisite Github repo, download the latest release, optionally extract the binary from a tarball or zipfile, copy the binary to a directory in your PATH and you are ready to work. You then have to follow along on Github to be notified of new releases and have to go through the same steps to update your tool. This becomes quite tedious when you have more than three of these tools installed. For the most part these tools aren't packaged for systems like apt, yum or snap. And for the few tools that have unofficial packages, these are rarely kept up-to-date. Quite a bit of these tools do have brew formulas available, which does somewhat mitigate the problem. However not all tools have formulas available, and brew for Linux requires a lot to run, while also mostly ending up with your machine compiling the tools itself, if they even work.

The solution

vk is a simple tool that can download, extract, install, update and uninstall these types of tools. It does so by talking to the Github (and Hashicorps Checkpoint) API for information about the latest release, which it can check against the version that is installed locally. For a user it is easy, convenient and fast to install and keep up-to-date these tools with vk.

Limitations

vk is not a full blown package manager. It can not install specific versions of tools, only the latest version. vk is also only for Linux AMD64 platforms (for now).

Tool definitions

All tools that vk knows about are defined in a JSON file in the vk-definitions Github repo (https://github.com/cellpointmobile/vk-definitions).

vk's Issues

Add flag to clear cache

It would be nice to have a flag that will clear out the cache (for both definitions and the Github/Hashicorp API).

Add graceful handling when assets are missing

When a definition does not lead to finding an expected asset in a Github release, vk currently hard exits. Instead we should give an error message and possibly continue what we are doing (fx during an update, it is better to continue updating everything else, rather than just die in the middle).

Add --offline to available subcommand

It would be nice to be able to list all available tools without looking up version information online. Add --offline to just list the known tools.

Move tool definitions into JSON that is cached and loaded on runtime.

To be able to define tools without doing a full new release of vk, the structs must be loaded from JSON which can be put online and updated independently from vk.

Add verbose flag to update subcommand

We should add --verbose to update to have it describe what it is doing in details.

Give better output when failing

Today if I run vk update or vk installed and one of the installed tools have changes which makes the definition no longer compatible, the output vk gives is less than helpful. At the very least, we should make sure the name of the tool that fails, is shown, which is not the case right now.
We should catch which ie. GetLocalVersion fails and output a helpful error message, rather than the go panic that happens.

Add --installed flag to vk debug

It would be helpful to be able to run the debug command against all installed tools, rather than all known tools or just a specified tool.

Make bindir configurable

Add flag and config var to set the path for the bin directory.

Add logging

Instead of just fmt.Printf() a logger should be used so the output level can be controlled.

Add completion support for tools

We should make it possible to dynamically tab-complete the names of tools for install, update and debug commands.

Add --no-versions to vk available

It takes API requests and time to run vk available. It would be helpful to have a --no-versions flag to have it just output the list of known tools.

Make URL for vk-definitions configurable

We should add a flag and config var to set the URL for vk-definitions.json.

Use google/renameio to avoid broken installs upon interruptions

If the network connection dies or vk is aborted at the wrong time, we can end up with a broken install of a tool (workaround: vk uninstall followed by vk install). We try to do smart stuff to avoid it already, but Google has made a nice Go package that handles it after best practices. We should refactor to use it: https://github.com/google/renameio

Add a default regexp for version extraction

Various tools often decide to change the version output, thus breaking the definitions. We should instead create a default regexp matching a triple-number sequence (eg. 0.23.0), which will match correctly for most definitions. The regexp field in the individual definitions can then override this default regexp for tools where the output is ambigous.
This will make version extraction more resilient against changes to version output from tools.

Add caching to http transport

For at least Github client it makes sense to do local caching to avoid excessive API calls.
https://github.com/gregjones/httpcache can be used and is recommended by https://github.com/google/go-github