paketo-buildpacks/go-dist

A Cloud Native Buildpack for Go

Go Distribution Cloud Native Buildpack

The Go Distribution CNB provides the Go binary distribution that can be used to execute Go tooling. The buildpack installs the Go binary distribution onto the $PATH which makes it available for subsequent buildpacks. These buildpacks can then use that distribution to run Go tooling including building Go application binaries. Examples of buildpacks that perform this binary building process include the Go Mod CNB and the Dep CNB.

Integration

The Go Distribution CNB provides Go as a dependency. Downstream buildpacks, like Go Mod or Dep, can require the go dependency by generating a Build Plan TOML file that looks like the following:

[[requires]]

  # The name of the  dependency is "go". This value is considered
  # part of the public API for the buildpack and will not change without a plan
  # for deprecation.
  name = "go"

  # The version of the Go dependency is not required. In the case it
  # is not specified, the buildpack will provide the default version, which can
  # be seen in the buildpack.toml file.
  # If you wish to request a specific version, the buildpack supports
  # specifying a semver constraint in the form of "1.*", "1.13.*", or even
  # "1.13.9".
  version = "1.13.9"

  # The Go buildpack supports some non-required metadata options.
  [requires.metadata]

    # Setting the build flag to true will ensure that the Go
    # depdendency is available on the $PATH for subsequent buildpacks during
    # their build phase. If you are writing a buildpack that needs to run Go
    # during its build process, this flag should be set to true.
    build = true

    # Setting the launch flag to true will ensure that the Go
    # dependency is available on the $PATH for the running application. If you are
    # writing an application that needs to run Go at runtime, this flag should
    # be set to true.
    launch = true

Usage

To package this buildpack for consumption:

$ ./scripts/package.sh

This builds the buildpack's Go source using GOOS=linux by default. You can supply another value as the first argument to package.sh.

Go Build Configuration

To configure the Go version, please use the BP_GO_VERSION environment variable at build time either directly (ex. pack build my-app --env BP_GO_VERSION=~1.14.1) or through a project.toml file.

go-dist's Issues

IMPORTANT NOTICE: End of Support for golang versions 1.14.x after 2021-04-23

Go Buildpack users
The first release of the Go buildpack after April 23, 2021 will no longer include Go versions 1.14.x. These Go versions will no longer be supported upstream.[1] Please migrate your Go apps to supported versions of Go before that time.

Note: Unless you are manually specifying a version of Go for the buildpack to use, or you have customized your Go buildpack, no action is required.

As always, the buildpacks team is happy to answer questions you may have about this deprecation in the #buildpacks Slack channel.

Buildpack maintainers
The first release of the Go buildpack after April 23, 2021 will no longer include Go versions 1.14.x. These Go versions will no longer be supported upstream.[1] Please cut a release of the buildpack that removes support for Go versions 1.14.x.

[1] - https://golang.org/doc/devel/release.html#policy

Can we cache downloaded assets earlier?

I got an error during a subsequent buildpack (deb-cnb) but when I re-ran pack build I see that it downloads the large go1.13.linux-amd64-cflinuxfs3-8b3c3fb8.tar.gz again. I'm on very slow internet and this is very noticable. Can we cache these deps somehow regardless if the entire build fails, please?

Why are deps being downloaded multiple times? Is there a pack build flag I need so that they are cached to my local machine? Downloading go1.13 or a JDK multiple times for multiple apps seems quirky?

===> BUILDING
[builder]
[builder] Go Compiler Buildpack v0.0.27
[builder]   Go 1.13: Contributing to layer
[builder]     Downloading from https://buildpacks.cloudfoundry.org/dependencies/go/go1.13.linux-amd64-cflinuxfs3-8b3c3fb8.tar.gz
[builder]     Verifying checksum
[builder]     Expanding to /layers/org.cloudfoundry.go-compiler/go
[builder]
[builder] Dep Buildpack v0.0.26
[builder]     failure running build: no import-path found in buildpack.yml
[builder] Error: failed to build: exit status 102

/cc @sclevine

Remove Go 1.18

Go 1.18 should be removed and the default version of the buildpack should be bumped to 1.19

Update import statements

Make sure all import statements are updated to point at the paketo-buildpacks org!

Configuring GitBot is recommended

Pivotal provides the GitBot service to synchronize pull requests and/or issues made against public GitHub repos with Pivotal Tracker projects. This service does not track individual commits.

If you are a Pivotal employee, you can configure Gitbot to sync your GitHub repo to your Pivotal Tracker project with a pull request. An ask+cf@ ticket is the fastest way to get write access if you get a 404 to the config repo.

If you do not want have pull requests and/or issues copied from GitHub to Pivotal Tracker, you do not need to take any action.

If there are any questions, please reach out to [email protected].

Generate SBOM for go distribution

To implement Paketo RFC0038, this buildpack will need to move from storing SBOM information in layer metadata to storing it in files that the CNB lifecycle can manipulate during the build. The RFC outlines what these files are and what they should contain.

Failure: Go get update workflow

Go get update workflow failed.
Please take a look to ensure CVE patches can be released. (cc @pivotal-cf/commercial-buildpacks).

Failure: Create Draft Release workflow

Create Draft Release workflow failed.

Failure: Create Draft Release workflow

Create Draft Release workflow failed.

Failure: Update GitHub config workflow

Update GitHub config workflow failed.

Failure: Update Dependencies workflow

Update Dependencies From Metadata workflow failed.

Implement Dependency RFC 0004 for go

Implement Dependency Management RFC Phase 1 for go. Check out the RFC for more details and background.

1. Determine dependency source strategy

Background

When possible, dependencies should be used directly from upstream, rather than undergoing any additional compilation or modifications performed by Paketo-maintained code. For each dependency, the corresponding buildpack maintainer group will decide if the dependency can be used directly from upstream, and must identify the location from which the dependency will be pulled from. Some of the Paketo Java buildpacks perform directory stripping during the buildpack build process itself. This could be a viable alternative to performing directory modifications during the dependency management process for maintainers to consider.

• Determine whether the dependency can be used directly from its upstream, rather than undergoing additional compilation/modification.

• If using directly from upstream, note where dependencies will come from
• If compiled or modified, note the decision for this

• Document decision in a language-family level RFC. This decision can be combined in the RFC with the decision made for other buildpacks in the language family.

2. Version retrieval and metadata generation code

Please refer to the retrieval RFC section on version retrieval for details around the API and background.

• Create retrieval code for the dependency

• Will live in the buildpack under: dependency/retrieval/
• Per the RFC, this will be a combination of (1) discovering new dependency versions based on the buildpack.toml, and (2) generating metadata for each new version.
• Note : Per the RFC caveat - if the dependency is to be compiled, the SHA256 and URI field from the metadata should be omitted in this step.

3. Compilation code (if needed)

If the dependency will be compiled/modified, then refer to the compilation RFC section for API details and background.

• Create compilation action

• Will live in the buildpack under: dependency/actions/compile/

4. Dependency testing (optional)

It's up to maintainer discretion if the dependency will be tested. It's recommended to test the dependency if it's been compiled. Check out the testing RFC section for details.

• Add tests for dependency

• Will live in the buildpack under: dependency/test.

5. Makefile Setup

When using the generalized workflows for dependency management down the line, version retrieval and dependency testing will be executed via a Makefile in order to provide the workflow a standardized way to run the code, regardless of what language it was written in. Check out the RFC section for what this should look like.

• Add Makefile

• Will live in the buildpack at: dependency/Makefile

6. Leveraging the new code

This issue serves to set up all the main logic for the dependency management. The work to actually leverage this code and migrate off of the dep-server will be completed in a separate issue once workflows and infrastructure is set up.

IMPORTANT NOTICE: End of Support for golang versions 1.13.x after 2020-10-02

The first release of the Go buildpack after October 2, 2020 will no longer include Go versions 1.13.x. These Go versions will no longer be supported upstream.[1] Please migrate your Go apps to supported versions of Go before that time.

Note: Unless you are manually specifying a version of Go for the buildpack to use, or you have customized your Go buildpack, no action is required.

[1] - https://golang.org/doc/devel/release.html#policy

Consume the Go dependency from the Paketo dep server

Per paketo-buildpacks/dep-server#45, we should start consuming the Go runtime dependency from the dep-server (See https://api.deps.paketo.io/v1/dependency?name=go for example dependency URIs) instead of the https://buildpacks.cloudfoundry.org/dependencies/go/... location we currently get the dependencies from.

This will make the dependency publishing/consumption process more transparent than the process we use for the dependencies available via the dependency-builds pipeline.

We have already done this switch-over in the Node Engine and Yarn Buildpacks. The outline of what this work will entail can be found in the dep-server issue linked at the top.

Week of 10/3/22: Release this buildpack!

New Golang dependency versions are in the buildpack. Also, transitive dependencies and packit have been updated. Let's cut a release this week.

Rewrite with packit

This codebase should be rewritten to take advantage of the new packit library.

Confirm to RFC RFC0043: Reproducible builds

To confirm to RFC0043 this buildpack should ensure that builds are reproducible. Specifically it should not include a built_at metadata field. In the tests that leverage this field to assert layer reuse, we should instead compare layer SHA values across rebuilds.

See also the tracking issue: paketo-buildpacks/rfcs#165.

Failure: Approve bot PR workflow

Approve bot PR workflow failed.