paketo-buildpacks/go

A Cloud Native Buildpack for Go

Go Paketo Buildpack

gcr.io/paketo-buildpacks/go

The Go Paketo Buildpack provides a set of collaborating buildpacks that enable the building of a Go-based application. These buildpacks include:

• Go Distribution CNB
• Go Mod Vendor CNB
• Go Build CNB

The buildpack supports building applications that use either the built-in Go modules feature for managing their dependencies. Usage examples can be found in the samples repository under the go directory.

The Go buildpack is compatible with the following builder(s):

• Paketo Jammy Full Builder
• Paketo Jammy Base Builder
• Paketo Jammy Tiny Builder
• Paketo Jammy Static Buildpackless Builder†
• Paketo Full Builder
• Paketo Base Builder
• Paketo Tiny Builder

This buildpack also includes the following utility buildpacks:

• Git CNB
• Procfile CNB
• Environment Variables CNB
• Image Labels CNB
• CA Certificates CNB

Check out the Go Paketo Buildpack docs for more information.

† To build with the static buildpackless builder, use the following command:

pack build \
  --builder paketobuildpacks/builder-jammy-buildpackless-static \
  --buildpack paketo-buildpacks/go \
  --env "CGO_ENABLED=0" \
  --env "BP_GO_BUILD_FLAGS=-buildmode=default"
  <app-name>

go's Issues

Week of 10/3/22: Release this buildpack!

New versions of the Go dependency are available. Let's cut a release.

Add Procfile to group order

We should add the Procfile Paketo Buildpack to the Go Language family buildpack so that users can define start commands via a Procfile (see paketo-buildpacks/rfcs#7)

Release Paketo Go buildpack

Release in the week of May 17th if relevant changes or dependency updates are merged.

What steps did you take to close this issue? What resources did you use? How long did you spend on this task this week? Answer in a comment.

Week of Feb 08: Cut buildpack releases

Upgrade to packit v2

Please upgrade to the latest packit v2 release to enable new features and extended support.

New structure for the Go language family

Proposal
In the new structure there will be 5 buildpacks as opposed to 3 they will be as follows.

• go-compiler this buildpack will function the same as the current go-compiler.
• go-mod-install this buildpack is a division of the current go-mod buildpack and will only be responsible for installing the apps go modules.
• dep this will be a division of the current dep buildpack and will be responsible for only downloading the dep dependency.
• dep-install this will be be another division of the current dep buildpack and will be responsible for installing the apps go modules using dep.
• go-build this buildpack will be responsible for building the go binary and writing a start command for the app.

The order groupings would look as follows.

[[order]]
  [[order.group]]
    id = "go-compiler"
  [[order.group]]
    id = "go-mod-install"
  [[order.group]]
    id = "go-build"

[[order]]
  [[order.group]]
    id = "go-compiler"
  [[order.group]]
    id = "dep"
  [[order.group]]
    id = "dep-install"
  [[order.group]]
    id = "go-build"

Motivation
The above plan has several benefits over the existing structure.

• The structure is much more modular allowing for much more customization and insertion of custom buildpacks.
• It separates the concerns of the buildpacks allowing them to be simpler and easier to maintain, while also helping separate different points of failure.
• It condenses common binary building logic into one buildpack in the form of go-build.

Week of Feb 22: Cut buildpack releases

Implement Go RFC0002: Decide Which Go Dependencies Will Be Paketo-hosted

RFC

Summary

Remove the Paketo-hosted dependency.

The buildpack can use the dependency that is provided by Go which can be parsed from a JSON payload of thier download page with little to no modification. The precedence for this payload page existing can be found in the Go Docs for the website. The payload includes a SHA256 from Go meaning the artifact can be verified from the upstream. Here is a paketo-buildpacks/go-dist#442 showing the buildpack using the dependencies directly from the Go upstream. Because this dependency can be consumed from a trusted source, we should stop hosting it ourselves.

Failure: Create Draft Release workflow

Create Draft Release workflow failed.

Go Buildpack should support apps using glide

We should look into adding support for apps using glide.

Add Environment Variables Buildpack to Group Order

We should consider adding the Paketo Environment Variables Buildpack to the Go Language family buildpack so that users can define environment variables during build time to get set on the application image.

Create a go-build Buildpack

Per the Go language family re-architecture RFC we should implement a packit written go-build Buildpack that builds the go binary and writes a start command for the app.

After this change is made, ordering in the Language Family Go Buildpack should look something like

..........................................
[[order]]
  [[order.group]]
    id = "go-dist"
  [[order.group]]
    id = "go-build"

Add CA Certificates Buildpack to Group Ordering

We should add the CA Certificates Buildpack to each group in the buildpack. This buildpack will allow application developers to provide their own CA certificates and have them included in the root truststore at build and runtime.

Week of Feb 15: Cut buildpack releases

Implement Paketo RFC 0038: CycloneDX + Syft SBOM

To implement Paketo RFC0038, this buildpack (and the implementation buildpacks inside) will need to move from storing SBOM information in layer metadata to storing it in files that the CNB lifecycle can manipulate during the build. The RFC outlines what these files are and what they should contain.

This issue serves as a meta-issue for work required to complete this work for the Go language family. This will require (link Github issues as they are created):

• Updating SBOM format for go-dist buildpack paketo-buildpacks/go-dist#341
• Updating SBOM format for dep buildpack paketo-buildpacks/dep#256
• Adding an SBOM for Go modules (paketo-buildpacks/go-mod-vendor/issues/13)
• Potentially, changes to go-build buildpacks to generate an SBOMs for the built app binary paketo-buildpacks/go-build#256

Support Jammy Jellyfish

Describe the Enhancement

Ultimately, we want to be able to build Golang apps on top of the Paketo Jammy stacks. Currently, builds fail because buildpacks like go-dist don't have the Jammy stack listed in their supported stacks.

Possible Solution

• Update buildpack.toml and dependencies to add io.buildpacks.stacks.jammy and io.buildpacks.stacks.jammy.tiny as compatible stacks
• (If necessary) Recompile installed dependencies to work on Jammy
• Add some integration tests that test the buildpack against Jammy stacks

Motivation

Part of paketo-buildpacks/builder-jammy-base#1

Subtasks

• - paketo-buildpacks/dep-server#161
• - paketo-buildpacks/dep-server#185
• - paketo-buildpacks/go-dist#402
• paketo-buildpacks/go-dist#415
• - paketo-buildpacks/dep#310
• - paketo-buildpacks/dep#307
• - paketo-buildpacks/dep-ensure#250

Go prerelease versions

I'm trying to build some projects that require Go 1.18, but it's not a supported version. Is it possible to build against official Go tagged versions, or build from a branch of Go? It's not clear to me how to do that.

Go Paketo Language Family Buildpacks should receive implementation buildpack updates

Context
We should set up our pipelines to start updating the language family CNB level buildpack.toml anytime there is a new version of one of its Implementation Paketo Buildpacks.

Proposal
Attempt to use Github Actions to start auto-PRing implementation CNB versions in the Paketo Go Language family buildpacks. If pipelines pass, the update PR should be automatically merged.

Acceptance Criteria
GIVEN any implementation CNB in a language family CNB(Go) is updated
THEN I see that there is a PR to the language family CNB with the implementation CNB updates
AND when the pipelines pass
THEN I see that the PR has been auto-merged

Notes
Here is a concourse task that is used bump cnb dependencies:
https://github.com/cloudfoundry/buildpacks-ci/tree/ce29842908d77018051422560db38df9fdbfa33c/tasks/cnb/update-cnb-dependency

Add git buildpack into the all order groups

What happened?

We want to add the git buildpack into all order groups. This adds two use-cases:

• add git revision metadata to the built image
• clone private repositories

See git buildpack for more detail.

Resolves paketo-buildpacks/go-mod-vendor#140.

Acceptance Criteria

• Automated acceptance test that verifies that the git buildpack is participating in at least one build
• git buildpack has been added in the buildpack.toml (prior to go-mod-vendor) and package.toml for all order groups
  • should be marked optional=true since it is not required for any existing workflows / order groups to succeed. Git is not a requirement for golang!

Notes

• jam will automatically keep the versions of the git buildpack updated in buildpack.toml.

Go Paketo Buildpack should support layer caching

This might already be supported, but it would be great if the Go Paketo Buildpack supports layer caching.

Upgrade this buildpack to Buildpack API Version 0.5

What happened?

• What were you attempting to do?

I wanted to see that Paketo buildpacks are up to date with the latest CNB Spec for buildpacks.

• What did you expect to happen?
  I expected to see that in the buildpack.toml for the buildpack, its API version was 0.5 since that's the latest release of the spec.

• What was the actual behavior? Please provide log output, if possible.
  The actual API version is 0.2 on the main branch of this repo (and in the latest release).

Build Configuration

• What platform (pack, kpack, tekton buildpacks plugin, etc.) are you
  using? Please include a version.

• What buildpacks are you using? Please include versions.

• What builder are you using? If custom, can you provide the output from pack inspect-builder <builder>?

• Can you provide a sample app or relevant configuration (buildpack.yml,
  nginx.conf, etc.)?

Checklist

• I have included log output.
• The log output includes an error message.
• I have included steps for reproduction.

Investigate adding support for apps using godep

Godep preceded dep and is essentially unsupported at this point, but filing this issue to open discussion on whether we should include support for it in our Paketo Go Buildpack.

Go Language family CNB releases should contain a buildpackage artifact

Context
As buildpackages start to become the standard of shipping our CNBs, we should build a buildpackage of our Paketo language family Go CNB. This will make it way easier for users to consume specific language family buildpacks.

Proposal
start publishing a Paketo Go buildpackage whenever we ship a new release artifact.

Notes/Helpful Links

• The pack command that we will have to run to do this is pack package-buildpack <NAME-OF-ARTIFACT> --package-config <Path/To/Package.toml>.

Acceptance/Deliverables

• Go Language Family CNB github releases should contain an online buildpackage artifact in the release assets.
• Go Buildpackage can be downloaded from Github and used to pack build apps

Week of January 11: Cut buildpack releases

Implement Go RFC 0001: Remove `dep` and `dep-ensure`

RFC

Summary

The dep and dep-ensure Buildpacks should be removed.

Subtasks

• Add deprecation warnings
  • paketo-buildpacks/dep#313
  • paketo-buildpacks/dep-ensure#252
• Release language family with deprecation warnings included

[ After 30 days ]

• Remove unsupported buildpacks from language family order groups
• Archive unsupported buildpack repos

Week of January 25: Cut buildpack releases

Failure: Push Buildpackage workflow

Push Buildpackage workflow failed.

Implement rfcs/0001-restructure.go

We've recently merged https://github.com/paketo-buildpacks/go/blob/master/rfcs/0001-restructure-go.md. Below is a set of steps we can take to implement this RFC.

• Add a language-family test suite: #51
• Rename go-compiler to go-dist: paketo-buildpacks/go-dist#65 and #92
• Create go-build: #93
• Create new dep-ensure that runs dep ensure #97 and paketo-buildpacks/dep-ensure#4
• Rename go-mod to go-mod-vendor paketo-buildpacks/go-mod-vendor#64
• Pull go build code out of go-mod-vendor and replace with go-build in the order group paketo-buildpacks/go-mod-vendor#31 and paketo-buildpacks/go-mod-vendor#71
• Pull go build code out of dep and replace with go-build in the order group
• Pull dep ensure code out of dep and replace with dep-ensure in the order group

Migrate docs into a Hugo module

In order to locate Paketo documentation closer to the code it documents, we can migrate the docs for this language family from where they currently live in the paketo website repo into a Hugo module that sits inside this repository.

For an example of this, see the Hugo module that sits inside the Paketo Dotnet Core language family repo.

To do this successfully, we need

• A workflow that automatically tags a release of the docs module when a release is cut on this language family repo. This has been handled in a PR to the centralized Paketo github-config repository
• A Hugo module inside this language family repo (This module needs to have the name github.com/paketo-buildpacks/<name of this repo>/docs in order for the existing automation to work properly.) It should have the same structure as the Hugo module inside the dotnet-core repo.
• An additional entry in the go.mod of the paketo-website repo for this repo's docs module
• An additional [[module.imports.mounts]]in the paketo-website repo's hugo config.toml that mounts the content in this repo's docs module into the proper path in the site's directory structure. That path should match the path to the existing docs file in that repo (Hint: The docs file is probably in here).

Failure: Create Draft Release workflow

Create Draft Release workflow failed.

Document using project.toml for env vars

What happened?

Now that the buildpacks in the language family have moved from buildpack.yml to env vars for configuration, the READMEs of the buildpacks should include some reference to project.toml documentation because it's not obvious to buildpack users that they can still store their env var settings in a file. I think example code blocks like

[[build.env]]
name = "BP_GO_VERSION"
value = "1.16"

would be valuable, so people don't have to be CNB spec experts just to know this option is available to them.

Checklist

• I have included log output.
• The log output includes an error message.
• I have included steps for reproduction.

Week of January 18: Cut buildpack releases

Add Image Labels Buildpack to Group Order

We should add the Image Labels Paketo Buildpack to the group order so that the Go Paketo Buildpack conforms to the image labels RFC

Week of 10/31/22: Release the buildpack

Release updated component buildpacks, then release the language family composite buildpack!

Run go generate before building

What happened?

When building our go app images with dockerfiles, we normally run go generate ./... before installing/building our application in order to run a tool that embeds files.

Would it be possible to create a go-generate buildpack that could be included in custom builders that would be run before the go-build buildpack in order to do things like embed files. In Go 1.16, there will be native file embedding support so this will likely not be necessary after then.

Build Configuration

Please provide some details about your build configuration.

Using pack v0.13.1 and the go-buildpack v0.2.4

Checklist

Please confirm the following:

• I have included log output.
• The log output includes an error message.
• I have included steps for reproduction.

Week of Mar 08: Cut buildpack releases

Add top level [[buildpack.licenses]] field

What happened?

We should add the top level [[buildpack.licenses]] field to buildpack.toml (and all implementation buildpacks) so users can see the buildpack's license through an app image's bill of materials.

Go Paketo Implementation CNBs should receive dependency updates

Context
We should set up our pipelines to start updating the implementation CNB buildpack.toml anytime there is a dependency bump in an existing version line.

Proposal
OSS Concourse Pipelines should be added to start auto-PRing dependency bumps for existing version lines in implementation Go Paketo Buildpacks. If the pipelines pass, the PR should be automatically merged.

Acceptance Criteria
GIVEN there is a new version of any dependency in any of the Go Paketo Implementation CNBs
THEN I see that there is a PR to the implementation CNB with the dependency bump
AND when the pipelines pass
THEN I see that the PR has been auto-merged

Notes
Here is a concourse task that used to bump implementation CNB dependencies:
https://github.com/cloudfoundry/buildpacks-ci/tree/ce29842908d77018051422560db38df9fdbfa33c/tasks/cnb/update-cnb-dependency

Support nodejs asset compilation, similar to `rails-assets`

Describe the Enhancement

Support asset compilation with node (ex: with webpacker) for Go apps, without the need to specify one of the nodejs buildpacks

Possible Solution

A new go-assets buildpack that wraps some of the existing nodejs BPs (probably everything except for *-start ones), users can then specify the build CMD with BP_NODE_RUN_SCRIPTS

Motivation

It's possible to achieve this by simply including the full nodejs buildpack in a group alongside the go BP, but it'd be nice to be able to specify a single group for our go apps. One could also argue that there's a potential need for doing this for other languages like java and maybe others

Implement RFC0052: Graceful Stack Upgrades

RFC

Summary

All Paketo Buildpacks should support graceful stack upgrades between the current Bionic and Jammy stacks. For most buildpacks, this should not require any changes, but for some buildpacks, layers created in previous builds that would normally be reused should be recreated if the stack ID changes.

Action Item

Consider what types of buildpacks would be impacted by a stack upgrade and make any relevant changes that would help to mitigate issues caused by that upgrade. If there is no work to be done, please comment on this issue explaining the context that helped to arrive at that conclusion.

Week of Dec 14: Cut buildpack releases

Self-nomination as Go contributor

In keeping with the Paketo Buildpacks Governance document, I am self-nominating as a contributor to the Go language family.

I have recently contributed by opening issues addressing a layer reuse issue related to go build targets and another related to the buildpack respecting GOFLAGS. I have also submitted a PR simplifying go build flags.

Document buildpack's compatibility with utility buildpacks

The readme of the Paketo Nodejs buildpack lists the utility buildpacks included in the group orders AND links out to docs on the Paketo site that explain how to use these utility buildpacks. This should be added for the Go buildpack now that the utility buildpacks are in the group orders.

Assert that builds are reproducible

Describe the Enhancement

Builds with this buildpack should be reproducible, meaning given identical inputs, the SHAs of resulting buildpack-built images are the same. This means, for a given app, if I run:

pack build my-app -b paketo-buildpacks/go

and then run

pack build my-app-copy -b paketo-buildpacks/go

with the same source code and configurations, the resulting image SHAs should be the same.

Currently, builds are not reproducible because of SBOMs included in the final app image. See paketo-buildpacks/packit#367 and paketo-buildpacks/packit#368. But once those issues are resolved and a new version of packit has been released, we should expect that the buildpack builds are reproducible.

Possible Solution

Add assertions to existing integration tests that show that two builds with the same inputs produce identical outputs.

Motivation

Build reproducibility is a selling point of CNBs that we want to provide to Paketo buildpack users. We want to know if future implementation decisions compromise build reproducibility.

Week of Mar 01: Cut buildpack releases

Nominating @fg-j as a Go subteam maintainer

@fg-j has made many contributions to Go buildpacks. Their most significant contribution was lobbying for the introduction of the go-generate buildpack:

• Issue #337
• PR #367

They also wrote the implementation to expand the functionality of the go-generate buildpack.

@ryanmoran looking for your review and approval.

Week of (Feb 01): Cut buildpack releases

Create a dep-ensure Buildpack

Per the Go language family re-architecture RFC we should implement a packit written dep-ensure Buildpack that runs dep ensure

Week of 2022/10/10 - Go buildpack releases

Several of the Go implementation buildpacks have updated their transitive dependencies since the last release. We should release the buildpacks this week.

Week of January 4: Cut buildpack releases