Giter Site home page Giter Site logo

rhysd / go-github-selfupdate Goto Github PK

View Code? Open in Web Editor NEW
588.0 10.0 75.0 118 KB

Binary self-update mechanism for Go commands using GitHub

Home Page: https://godoc.org/github.com/rhysd/go-github-selfupdate/selfupdate

License: MIT License

Go 98.45% Shell 0.76% Ruby 0.79%
golang go github update selfupdate cli

go-github-selfupdate's Introduction

Self-Update Mechanism for Go Commands Using GitHub

GoDoc Badge TravisCI Status AppVeyor Status Codecov Status

go-github-selfupdate is a Go library to provide a self-update mechanism to command line tools.

Go does not provide a way to install/update the stable version of tools. By default, Go command line tools are updated:

  1. using go get -u, but it is not stable because HEAD of the repository is built
  2. using system's package manager, but it is harder to release because of depending on the platform
  3. downloading executables from GitHub release page, but it requires users to download and put it in an executable path manually

go-github-selfupdate resolves the problem of 3 by detecting the latest release, downloading it and putting it in $GOPATH/bin automatically.

go-github-selfupdate detects the information of the latest release via GitHub Releases API and checks the current version. If a newer version than itself is detected, it downloads the released binary from GitHub and replaces itself.

  • Automatically detect the latest version of released binary on GitHub
  • Retrieve the proper binary for the OS and arch where the binary is running
  • Update the binary with rollback support on failure
  • Tested on Linux, macOS and Windows (using Travis CI and AppVeyor)
  • Many archive and compression formats are supported (zip, tar, gzip, xzip)
  • Support private repositories
  • Support GitHub Enterprise
  • Support hash, signature validation (thanks to @tobiaskohlbau)

And small wrapper CLIs are provided:

Slide at GoCon 2018 Spring (Japanese)

Try Out Example

Example to understand what this library does is prepared as CLI.

Install it at first.

$ go get -u github.com/rhysd/go-github-selfupdate/cmd/selfupdate-example

And check the version by -version. -help flag is also available to know all flags.

$ selfupdate-example -version

It should show v1.2.3.

Then run -selfupdate

$ selfupdate-example -selfupdate

It should replace itself and finally show a message containing release notes.

Please check the binary version is updated to v1.2.4 with -version. The binary is up-to-date. So running -selfupdate again only shows 'Current binary is the latest version'.

Real World Examples

Following tools are using this library.

Usage

Code Usage

It provides selfupdate package.

  • selfupdate.UpdateSelf(): Detect the latest version of itself and run self update.
  • selfupdate.UpdateCommand(): Detect the latest version of given repository and update given command.
  • selfupdate.DetectLatest(): Detect the latest version of given repository.
  • selfupdate.DetectVersion(): Detect the user defined version of given repository.
  • selfupdate.UpdateTo(): Update given command to the binary hosted on given URL.
  • selfupdate.Updater: Context manager of self-update process. If you want to customize some behavior of self-update (e.g. specify API token, use GitHub Enterprise, ...), please make an instance of Updater and use its methods.

Following is the easiest way to use this package.

import (
    "log"
    "github.com/blang/semver"
    "github.com/rhysd/go-github-selfupdate/selfupdate"
)

const version = "1.2.3"

func doSelfUpdate() {
    v := semver.MustParse(version)
    latest, err := selfupdate.UpdateSelf(v, "myname/myrepo")
    if err != nil {
        log.Println("Binary update failed:", err)
        return
    }
    if latest.Version.Equals(v) {
        // latest version is the same as current version. It means current binary is up to date.
        log.Println("Current binary is the latest version", version)
    } else {
        log.Println("Successfully updated to version", latest.Version)
        log.Println("Release note:\n", latest.ReleaseNotes)
    }
}

Following asks user to update or not.

import (
    "bufio"
    "github.com/blang/semver"
    "github.com/rhysd/go-github-selfupdate/selfupdate"
    "log"
    "os"
)

const version = "1.2.3"

func confirmAndSelfUpdate() {
    latest, found, err := selfupdate.DetectLatest("owner/repo")
    if err != nil {
        log.Println("Error occurred while detecting version:", err)
        return
    }

    v := semver.MustParse(version)
    if !found || latest.Version.LTE(v) {
        log.Println("Current version is the latest")
        return
    }

    fmt.Print("Do you want to update to", latest.Version, "? (y/n): ")
    input, err := bufio.NewReader(os.Stdin).ReadString('\n')
    if err != nil || (input != "y\n" && input != "n\n") {
        log.Println("Invalid input")
        return
    }
    if input == "n\n" {
        return
    }

    exe, err := os.Executable()
    if err != nil {
        log.Println("Could not locate executable path")
        return
    }
    if err := selfupdate.UpdateTo(latest.AssetURL, exe); err != nil {
        log.Println("Error occurred while updating binary:", err)
        return
    }
    log.Println("Successfully updated to version", latest.Version)
}

If GitHub API token is set to [token] section in gitconfig or $GITHUB_TOKEN environment variable, this library will use it to call GitHub REST API. It's useful when reaching rate limits or when using this library with private repositories.

Note that os.Args[0] is not available since it does not provide a full path to executable. Instead, please use os.Executable().

Please see the documentation page for more detail.

This library should work with GitHub Enterprise. To configure API base URL, please setup Updater instance and use its methods instead (actually all functions above are just a shortcuts of methods of an Updater instance).

Following is an example of usage with GitHub Enterprise.

import (
    "log"
    "github.com/blang/semver"
    "github.com/rhysd/go-github-selfupdate/selfupdate"
)

const version = "1.2.3"

func doSelfUpdate(token string) {
    v := semver.MustParse(version)
    up, err := selfupdate.NewUpdater(selfupdate.Config{
        APIToken: token,
        EnterpriseBaseURL: "https://github.your.company.com/api/v3",
    })
    latest, err := up.UpdateSelf(v, "myname/myrepo")
    if err != nil {
        log.Println("Binary update failed:", err)
        return
    }
    if latest.Version.Equals(v) {
        // latest version is the same as current version. It means current binary is up to date.
        log.Println("Current binary is the latest version", version)
    } else {
        log.Println("Successfully updated to version", latest.Version)
        log.Println("Release note:\n", latest.ReleaseNotes)
    }
}

If APIToken field is not given, it tries to retrieve API token from [token] section of .gitconfig or $GITHUB_TOKEN environment variable. If no token is found, it raises an error because GitHub Enterprise API does not work without authentication.

If your GitHub Enterprise instance's upload URL is different from the base URL, please also set the EnterpriseUploadURL field.

Naming Rules of Released Binaries

go-github-selfupdate assumes that released binaries are put for each combination of platforms and archs. Binaries for each platform can be easily built using tools like gox

You need to put the binaries with the following format.

{cmd}_{goos}_{goarch}{.ext}

{cmd} is a name of command. {goos} and {goarch} are the platform and the arch type of the binary. {.ext} is a file extension. go-github-selfupdate supports .zip, .gzip, .tar.gz and .tar.xz. You can also use blank and it means binary is not compressed.

If you compress binary, uncompressed directory or file must contain the executable named {cmd}.

And you can also use - for separator instead of _ if you like.

For example, if your command name is foo-bar, one of followings is expected to be put in release page on GitHub as binary for platform linux and arch amd64.

  • foo-bar_linux_amd64 (executable)
  • foo-bar_linux_amd64.zip (zip file)
  • foo-bar_linux_amd64.tar.gz (tar file)
  • foo-bar_linux_amd64.xz (xzip file)
  • foo-bar-linux-amd64.tar.gz (- is also ok for separator)

If you compress and/or archive your release asset, it must contain an executable named one of followings:

  • foo-bar (only command name)
  • foo-bar_linux_amd64 (full name)
  • foo-bar-linux-amd64 (- is also ok for separator)

To archive the executable directly on Windows, .exe can be added before file extension like foo-bar_windows_amd64.exe.zip.

Naming Rules of Versions (=Git Tags)

go-github-selfupdate searches binaries' versions via Git tag names (not a release title). When your tool's version is 1.2.3, you should use the version number for tag of the Git repository (i.e. 1.2.3 or v1.2.3).

This library assumes you adopt semantic versioning. It is necessary for comparing versions systematically.

Prefix before version number \d+\.\d+\.\d+ is automatically omitted. For example, ver1.2.3 or release-1.2.3 are also ok.

Tags which don't contain a version number are ignored (i.e. nightly). And releases marked as pre-release are also ignored.

Structure of Releases

In summary, structure of releases on GitHub looks like:

  • v1.2.0
    • foo-bar-linux-amd64.tar.gz
    • foo-bar-linux-386.tar.gz
    • foo-bar-darwin-amd64.tar.gz
    • foo-bar-windows-amd64.zip
    • ... (Other binaries for v1.2.0)
  • v1.1.3
    • foo-bar-linux-amd64.tar.gz
    • foo-bar-linux-386.tar.gz
    • foo-bar-darwin-amd64.tar.gz
    • foo-bar-windows-amd64.zip
    • ... (Other binaries for v1.1.3)
  • ... (older versions)

Hash or Signature Validation

go-github-selfupdate supports hash or signature validatiom of the downloaded files. It comes with support for sha256 hashes or ECDSA signatures. In addition to internal functions the user can implement the Validator interface for own validation mechanisms.

// Validator represents an interface which enables additional validation of releases.
type Validator interface {
	// Validate validates release bytes against an additional asset bytes.
	// See SHA2Validator or ECDSAValidator for more information.
	Validate(release, asset []byte) error
	// Suffix describes the additional file ending which is used for finding the
	// additional asset.
	Suffix() string
}

SHA256

To verify the integrity by SHA256 generate a hash sum and save it within a file which has the same naming as original file with the suffix .sha256. For e.g. use sha256sum, the file selfupdate/testdata/foo.zip.sha256 is generated with:

sha256sum foo.zip > foo.zip.sha256

ECDSA

To verify the signature by ECDSA generate a signature and save it within a file which has the same naming as original file with the suffix .sig. For e.g. use openssl, the file selfupdate/testdata/foo.zip.sig is generated with:

openssl dgst -sha256 -sign Test.pem -out foo.zip.sig foo.zip

go-github-selfupdate makes use of go internal crypto package. Therefore the used private key has to be compatbile with FIPS 186-3.

Development

Running tests

All library sources are put in /selfupdate directory. So you can run tests as following at the top of the repository:

$ go test -v ./selfupdate

Some tests are not run without setting a GitHub API token because they call GitHub API too many times. To run them, please generate an API token and set it to an environment variable.

$ export GITHUB_TOKEN="{token generated by you}"
$ go test -v ./selfupdate

The above command runs almost all tests and it's enough to check the behavior before creating a pull request. Some tests are still not tested because they depend on my personal API access token, though; for repositories on GitHub Enterprise or private repositories on GitHub.

Debugging

This library can output logs for debugging. By default, logger is disabled. You can enable the logger by the following and can know the details of the self update.

selfupdate.EnableLog()

CI

Tests run on CIs (Travis CI, Appveyor) are run with the token I generated. However, because of security reasons, it is not used for the tests for pull requests. In the tests, a GitHub API token is not set and API rate limit is often exceeding. So please ignore the test failures on creating a pull request.

Dependencies

This library utilizes

  • go-github to retrieve the information of releases
  • go-update to replace current binary
  • semver to compare versions
  • xz to support XZ compress format

Copyright (c) 2013 The go-github AUTHORS. All rights reserved.

Copyright 2015 Alan Shreve

Copyright (c) 2014 Benedikt Lang

Copyright (c) 2014-2016 Ulrich Kunitz

What is different from tj/go-update?

This library's goal is the same as tj/go-update, but it's different in following points.

tj/go-update:

  • does not support Windows
  • only allows v for version prefix
  • does not ignore pre-release
  • has only a few tests
  • supports Apex store for putting releases

License

Distributed under the MIT License

go-github-selfupdate's People

Contributors

bertuss avatar bhamail avatar databus23 avatar flowonyx avatar fredbi avatar jbfarez avatar michaelbirdflyt avatar pieterclaerhout avatar rhysd avatar songmu avatar tobiaskohlbau avatar wayneashleyberry avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

go-github-selfupdate's Issues

[Feature request] Optional pre-release support

I'd like to allow specific users to optionally update to a pre-release in order to give them bleeding-edge updates. My use case is something like:

  • cmd update: updates to latest release
  • cmd update --pre-release: updates to latest release OR pre-release

Setting the archive binary name

Please excuse me if I have overlooked something, but is there a way to manually specify the binary name in the downloaded archive?

Currently it appears to be hardcoded to the running binary name, however if myapp was renamed locally to myapp-dev, then the update fails because "myapp-dev" doesn't exist in the archive.

Ideally there should be a way to set the binary name the updater should look for in the archive, so that myapp-dev would get updated with the myapp binary in the archive.

[Feature Request] Support Conditional Requests to reduce github api usage.

go-github supports github API's conditional requests, but it requires you to use your own Transport that is set up for it.

In one of my own projects that uses go-github, I would create an http.Client and set it's Transport to one created using https://github.com/gregjones/httpcache

I would then attach that Client to a context using it's WithValue method.

In order to support the various caching methods, it would be nice if we could pass in a Context object when setting up go-github-selfupdate, that way we could implement Conditional Requests on our end when using it.

Private repository support

Currently go-github-selfupdate downloads an asset via download URL directly. So downloading from private repos is not possible.

However, using GitHub asset API with Accept: application/octet-stream header: https://developer.github.com/v3/repos/releases/

go-github already supported it: https://godoc.org/github.com/google/go-github/github#RepositoriesService.DownloadReleaseAsset

I need to change the internal design to use the go-github client in both detection and update

Dependency package not found

Hi,
Thank you for this amazing project.
I was trying to get the package using go get github.com/rhysd/go-github-selfupdate
And I encountered this error message:

package github.com/google/go-github/v30/github: cannot find package "github.com/google/go-github/v30/github" in any of:
        /.../src/github.com/google/go-github/v30/github (from $GOROOT)
        /.../src/github.com/google/go-github/v30/github (from $GOPATH)

Could you please help me with this issue?
Thank you very much.

Use /tmp instead of current dir

Hi there,

I'm currently writing a system deamon in golang and it's running as /usr/bin/rcsm, the problem is that the auto update tries to create /usr/bin/.rcsm.new /usr/bin/.rcsm.old but /usr/bin is not writable by my deamon, but its own binary is. It would be nice if we could avoid doing modifications in the same directory and use /tmp.

Thanks!

Release name compatible with AppImage

Thanks for making such a great package!

I am trying to use it to update AppImage, which is the release format I have for the repo. Since AppImage has its own naming convention e.g. reponame-version-x86_64.AppImage , I wonder whether the selfupdate can be compatible with that. If you don't have time and give me a direction, I am glad to try to fix it with a PR.

UpdateSelf not returning error in case the GithubToken doesn't have required access

UpdateSelf method: latest, err := up.UpdateSelf(v, "myname/myrepo") not returning error in case the GithubToken doesn't have required access like in case repo access is not provided to the token. It shows the current version as latest and error is returned as nil. Instead it should return error that tool can't fetch latest update or something.

tls: bad record MAC

Hello, I encountered some problems in the process of using it. This is the result of my run. I don't know how to solve it. I just run an example program.

Do you want to update to1.2.4? (y/n): y
2021/04/19 22:15:54 Error occurred while updating binary: Failed to create buffer for zip file: local error: tls: bad record MAC

Selfupdate not working.

Hi, your notes-cli looked like an interesting tool, but I found that it depends on your selfupdate tool my assumption is that the GO function changed and your tool hasn't perhaps it is something you can fix.

My go version is go1.14.1 below is the error message I get.

go/src/github.com/rhysd/go-github-selfupdate/selfupdate/update.go:59:67: not enough arguments in call to up.api.Repositories.DownloadReleaseAsset
        have (context.Context, string, string, int64)
        want (context.Context, string, string, int64, *http.Client)
go/src/github.com/rhysd/go-github-selfupdate/selfupdate/update.go:81:87: not enough arguments in call to up.api.Repositories.DownloadReleaseAsset
        have (context.Context, string, string, int64)
        want (context.Context, string, string, int64, *http.Client)

Redirect is not supported

go-github-selfupdate only checks if the status is 200 or not. So 302 is not handled. Actually redirect may occur when repository is moved after the release was made public.

Cannot isolate asset from releases with multiple artifacts

When building a release with multiple artifacts (e.g. with goreleaser),
I found out that the self update cannot figure out which is the correct one.
As a matter of facts, this line tells that the first asset found with matching release, arch etc is
selected:

if strings.HasSuffix(name, s) {

I am proposing a fix here along the following lines:

  • add a type Option func(*settings) to capture some specific settings
  • add a type settings{filters: []*regexp.Regexp} to capture specifically a filter on assets (might be extended later on)
  • add an option functor like func AssetFilter(filter string) Option` to add some filtering regexp (note: invalid regexp panics...)
  • supplement the line above by a check on optional filters and select the first asset which matches any of the provided filters. If no filter is provided, keep the existing behavior
  • most public API calls (not all of them, though) are amended with a opt ...Option optional parameter, so we may pass the filter down to findAssetFromRelease

Would that approach look acceptable?

GitHub Enterprise support

Enable to change base URL of go-github client.

I need to clarify that it's possible to download asset from repositories on GHE at first.

Add support for .tgz files

My archives I’ve uploaded to GitHub have the extension .tgz rather than .tar.gz.

From what I can see, the selfupdate package doesn’t detect them as possible downloads.

Issue with linux ARM processors

Hi, I typically have an issue where the update does not work on raspberry pi.

When you build for GOARCH="arm", you have 3 more options:

  • GOARM="5"
  • GOARM="6"
  • GOARM="7"

Sadly the variable is not available to the runtime (like a runtime.GOARM)

Automated build like goreleaser will build for each architecture and tag them as arm_v5, arm_v6, arm_v7.

Now we could say let's pick one and stick to it but it's not that easy:
Raspberry Pi 1 and Zero are armv6 while the versions 2, 3 and 4 are armv7

You can easily get the version of arm processor you're currently running with the shell command

$ uname -m
armv7l

(on a raspberry pi 3)

If this is something you would consider fixing using a shell command, I could eventually propose a PR?

Allow file name to include version information

Even if the archive file name contains version information (e.g. cmdname_v0.0.1_linux_amd64.tar.gz), it also works now, but it is not documented and I want it to become a spec.

Many of my tools are such, and if we crossbuild using goxc or goxz with the -pv option, we actually get the archive files in which name included version information.

In other words, it is useful to include the version information in the filename when comparing version at locally, and many package system (cpan, gem, rpm etc.) file name conventions usually include version information as well.

blang/semver v4 is released, but go-github-selfupdate depends v3

Since v4, blang/semver follows the major version strategies of Go modules, and the import path has a v4 suffix. Due to this change, the selfupdate.UpdateSelf method is incompatible with v4, because UpdateSelf expects a v3 semver.Version struct as an argument.

func UpdateSelf(current semver.Version, slug string) (*Release, error) {

The structure of Version struct has not changed between v3 and v4, so the following workaround is available.

import (
	semverv3 "github.com/blang/semver"
	"github.com/blang/semver/v4"
	"github.com/rhysd/go-github-selfupdate/selfupdate"
)

v := semver.MustParse("1.2.3")

var v3PRVersions []semverv3.PRVersion
for _, version := range v.Pre {
	v3PRVersions = append(v3PRVersions, semverv3.PRVersion(version))
}

v3v := semverv3.Version{
	Major: v.Major,
	Minor: v.Minor,
	Patch: v.Patch,
	Pre:   v3PRVersions,
	Build: v.Build,
}
latest, err := selfupdate.UpdateSelf(v3v, slug)

However, go-github-selfupdate needs changes to support v4.
I think there are two options: just accept v4 Version as an argument, or define an own Version struct so that it does not depend on the version of blang/semver. Either way, it will be breaking changes.

go.mod question

Hi @rhysd
I left it as an issue because I had a question.
First of all, thank you so much for making a good module!

I'm using your module on my tools, but there's a problem. After adding go-github-selfupdate to go.mod, the following error occurred when installing my tool as go get.

my go.mod: https://github.com/hahwul/dalfox/blob/master/go.mod
error

$ go get -u github.com/hahwul/dalfox
package github.com/google/go-github/v30/github: cannot find package "github.com/google/go-github/v30/github" in any of:
	/usr/local/Cellar/go/1.12.4/libexec/src/github.com/google/go-github/v30/github (from $GOROOT)
	/Users/kakao/go/src/github.com/google/go-github/v30/github (from **$GOPATH)**

There's no problem except go get. (both go install and binary build are possible)
Do you know anything? I need help!

go1 tag support

go get is dynamically switching the build target following its Go version by branch name or tag name. For now, go1 tag or branch is referred if exists.

https://golang.org/cmd/go/#hdr-Download_and_install_packages_and_dependencies

This library should also follow the feature. But the feature is not popular (I don't know any tools using the feature except for golang/go).

So if anyone is actually using the feature and want this library to refer it, I'd implement the support for the feature.

Root privileges required

This package works pretty well, thanks for developing it !

But whenthe binary is in a folder that requires admin/root rights to write in like /usr/bin/ or in windows' programs folder, the update fails. Here is the error I get on linux : Error occurred while updating binary: open /usr/bin/.hatt.new: permission denied (the name of my program is hatt)

I was wondering how this issue should be addressed (ask the user for admin rights ? if so, how ? something else ?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.