Giter Site home page Giter Site logo

slsa-framework / slsa-github-generator Goto Github PK

View Code? Open in Web Editor NEW
379.0 16.0 114.0 28.08 MB

Language-agnostic SLSA provenance generation for Github Actions

License: Apache License 2.0

Go 86.13% Makefile 3.65% TypeScript 2.01% Shell 6.70% Java 1.51%
security security-hardening security-tools slsa slsaprovenance

slsa-github-generator's Introduction

SLSA GitHub Generator

OpenSSF Scorecard CII Best Practices Go Report Card Slack SLSA 3

Overview

This repository contains free tools to generate and verify SLSA Build Level 3 provenance for native GitHub projects using GitHub Actions. Developers can build their software using a secure process that protects against many supply chain attacks and tampering. Users of their software can verify a tamper-proof statement of the process to know how the software was created.

What is SLSA?

Supply-chain Levels for Software Artifacts, or SLSA (salsa), is a security framework, a checklist of standards and controls to prevent tampering, improve integrity, and secure packages and infrastructure in your projects, businesses or enterprises.

SLSA defines an incrementally adoptable set of levels which are defined in terms of increasing compliance and assurance. SLSA levels are like a common language to talk about how secure software, supply chains and their component parts really are.

What is provenance?

Provenance is information, or metadata, about how a software artifact was created. This could include information about what source code, build system, and build steps were used, as well as who and why the build was initiated. Provenance can be used to determine the authenticity and trustworthiness of software artifacts that you use.

As part of the framework, SLSA defines a provenance format which can be used to hold this metadata.

What is slsa-github-generator?

slsa-github-generator is a set of tools for generation of SLSA3+ provenance for native GitHub projects. It allows projects to generate SLSA provenance safely and accurately using GitHub Actions.

Specifically, this repository contains:

While slsa-github-generator can help you achieve SLSA Build level 3, use of the provided GitHub Actions reusable workflows alone is not sufficient to meet all of the requirements at SLSA Build level 3. Specifically, these workflows do not address provenance distribution or verification. You can use the slsa-verifier to verify the provenance.

Hall of Fame

Generation of Provenance

Below is a non-exhaustive list of projects that use the builders in this repository to generate provenance:

flask stars flatbuffers stars grpc-gateway stars argo-cd stars click stars SOPS stars jib stars jinja stars docker-bench-security stars sentencepiece stars werkzeug stars ko stars micronaut-core stars kubeedge stars osv-scanner stars flux2 stars kyverno stars flask-sqlalchemy stars scorecard stars urllib3 stars pdns stars powertools-lambda-python stars hishtory stars PrivateBin stars NoPorts stars openfga stars

Edit this file to add your repository!

Builder Creation

Several builders have been built using the "Build Your Own Builder" (BYOB) framework:

  1. nodejs builder, by @ianlewis
  2. JReleaser builder, by @aalmiray
  3. Maven builder, by @AdamKorcz
  4. Gradle builder, by @AdamKorcz
  5. Coming soon! Bazel builder, by @enteraga6

Generate provenance

Below we describe the various builders and generators in this repository. They build and / or generate non-forgeable provenance using a trusted / isolated re-usable workflow. You can read up on the design in our technical design document.

To select the right option to generate provenance for your use case, take into account the programming language and build toolchain you already use, e.g. go, mvn, bazel, etc. Select a builder for your ecosystem. For example, if you use Go, use the Go builder. If you use Java and build Maven packages, use the Maven builder, and so on. If your release scripts are more complex than what the builder supports; or if there is no builder for your ecosystem, use a provenance generator instead.

Referencing SLSA builders and generators

At present, the GitHub Actions provided in this repository as builders and generators MUST be referenced by tag in order for the slsa-verifier to be able to verify the ref of the trusted builder/generator's reusable workflow. It also needs to be referred as @vX.Y.Z, because the build will fail if you reference it via a shorter tag like @vX.Y or @vX.

This is contrary to the GitHub best practice for third-party actions which recommends referencing by digest, but intentional due to limits in GitHub Actions. The desire to be able to verify reusable workflows pinned by hash, and the reasons for the current status, are tracked as Issue #12 in the slsa-verifier project.

For guidance on how to configure renovate see RENOVATE.md.

Builders

Builders build and generate provenance. They let you meet the provenance generation and isolation strength requirements for SLSA Build level 3 and above.

This repository hosts the following builders:

Ecosystem Builder Description Status
Go projects Go Builder Builds and generates provenance for Go projects available since v1.0.0
Node.js projects Node.js Builder Builds and generates provenance for npm packages Beta since v1.6.0. Expected GA release Sept 2023
Maven projects Maven builder Build Maven packages and generates provenance. Can be uploaded to Maven central Beta since v1.9.0
Gradle projects Gradle builder Build Gradle projects and generates provenance. Can be uploaded to Maven central Beta since v1.9.0
Bazel projects Bazel builder Builds Bazel projects and generates provenance WIP
docker images Container Builder Builds docker containers and generates provenance. The generated provenance is compatible with cosign's attestation format WIP
Any Container-based Builder Builds projects whose build pipeline is defined with a Dockerfile Beta since v1.7.0

There are other available builders using this repository's BYOB framework and not hosted in this repository:

Ecosystem Builder Description Status
JReleaser projects JReleaser builder Builds and generates provenance using JReleaser since v1.0.0-java

If none of these options fit your needs, use a generator as described below:

Generators

Generators only generate provenance for you. They let you meet the provenance generation and isolation strength requirements for SLSA Build level 3 and above.

Generators create an attestation to a software artifact coming from your repository.

This repository hosts the following generators:

Artifact type Generator Description Status
file (binary, package tarball etc.) Generic generator Generates provenance for arbitrary file-based artifacts, for any ecosystem and programming language available since v1.2.0
container Container generator Generate provenance for container images. The generated provenance is compatible with cosign's attestation format. available since v1.4.0

Verify provenance

To verify provenance created by any of the builders in this repository, use the github.com/slsa-framework/slsa-verifier project.

Installation

To install the verifier, see slsa-framework/slsa-verifier#installation.

Inputs

The inputs of the verifier are described in slsa-framework/slsa-verifier#available-options.

Command line examples

A command line example is provided in slsa-framework/slsa-verifier#example.

Known Issues

error updating to TUF remote mirror: invalid

This will occur when generating provenance with all builders and generators.

Affected versions: all versions up and including v1.9.0

error updating to TUF remote mirror: invalid

This issue is tracked by issue #3350. You must update to v1.10.0 to fix this issue.

Build Your Own Builder

Use the BYOB framework to create your own SLSA builder on GitHub. If you have an existing GitHub Action, you can use the BYOB framework to wrap it into a SLSA builder. This will harden the build process by running the Action in an isolated environment. Generated artifacts will meet Build Level 3 expectations and produce Build Level 3 provenance. To verify the provenance, your users can use the slsa-verifier.

Project Roadmap

The project roadmap is tracked via milestones. You can track progress and open issues via the milestones page. Each milestone includes a description of what is being worked on and a rough timeline for completion.

Technical design

The initial technical design was described in the blog post "Improving software supply chain security with tamper-proof builds".

Specifications

For a more in-depth technical dive, read the SPECIFICATIONS.md.

Provenance format

The format of the provenance is available in PROVENANCE_FORMAT.md.

Contributing

Please see the Contributor Guide for more info.

slsa-github-generator's People

Contributors

34fathombelow avatar adamkorcz avatar andresgalante avatar asraa avatar danalbert avatar dependabot[bot] avatar developer-guy avatar diogoteles08 avatar dongheelee92 avatar enteraga6 avatar haydentherapper avatar ianlewis avatar jkreileder avatar joshuagl avatar justinabrahms avatar kpk47 avatar lakshya8066 avatar laurentsimon avatar loosebazooka avatar lumjjb avatar manice18 avatar mihaimaruseac avatar naveensrinivasan avatar pnacht avatar ramonpetgrave64 avatar rarkins avatar rbehjati avatar renovate-bot avatar saisatishkarra avatar suzuki-shunsuke avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

slsa-github-generator's Issues

Feature: automated update of builder's API

Our builders need to use the API of https://github.com/slsa-framework/slsa-github-generator at HEAD.
This means every time we make an update to the generic API, we need to update the builder's go.sum/mod

I don't think we can check for it in pre-submits, because the hash to update to is unknown until the commit is pushed to main.

I think we have the following options:

  1. Figure out how to configure dependabot to send us PR
  2. Write our own schedule-trigger workflows once every X days to create an PR.

Note that this may quickly become an endless cycle if we're not careful: each new commit will create a new HEAD, which would trigger a new PR, etc. We may want to send a PR only if the code of the generic API has changed.

Feature: add e2e tests

We need e2e tests. We will need to create test repos that generate provenance and verify via schedule events; and report an issue automatically if they fail

Discussion: Use a Docker image for building binaries/files prior to generating SLSA provenances

Follow-up discussion about the idea of using a Docker image as the builder/releaser, as we have in project Oak.

In project Oak, and as part of our transparent-release work, we use a builder image for building binaries. The builder image is a Docker image, which has all the tools required for building the binary installed, and the required environment variables set. It might be interesting to use a similar idea here for building the binaries and generating the provenances. This can be used as an alternative to tools like go-releaser.

Currently in our tooling for transparent-release, the build command is a docker run command that runs a given command in the builder image. When generating SLSA provenances, we include this information as the BuildConfig. See also our custom buildType. In addition, we include the builder image in the list of materials. The build tool fetches the specified docker image and ensures that the command for building the binary is executed using the fetched builder image. The builder image is identified by a URI containing the digest of the image. If the versions of the toolchains are fixed in the Dockerfile (example from Oak) and the checksums are verified, then this can get very close to the idea of a trusted builder.

Here is an example of such a SLSA provenance file, with BuildConfig and materials as described above.

This is our GitHub action that generates provenances. We generate provenances for each commit that is merged into the main branch. It currently doesnโ€™t use the build tool from transparent-release (because the build does not yet generate a provenance file), but we plan to use this build toll with a simple TOML file similar to this example. The idea is to have the TOML file checked into the repo as a static file (containing only the command, the output_path and a few other fields), and let the GitHub actions job fill out the commit_hash and the builder_image URI that are different for each commit and invocation of the build tool.

A similar minimal TOML can be used here for building the binary using a builder image provided by the maintainers of the repo.

Note that in our approach, we donโ€™t fill out the invocation part in the SLSA predicate, as we think all the information is provided in the buildConfig, and materials.

cc @laurentsimon, @tiziano88

Merge all ecosystem builders into this repo

I was discussing with @MarkLodato about the cost of maintenance of the (possibly) many builder repos we will eventually have.

The main reason we opted to have separate ecosystem repos was to avoid asking users to update their workflow hash/version when another ecosystem's builder was updated. However, we're starting to wonder if this is the right thing to do or not.

An alternative would be to have all builders in this repository, which could reduce maintenance for us. I imagine it would look like this:

  • Ecosystem builders: .github/workflows/slsa3_go.yml, .github/workflows/slsa3_ko.yml, etc
  • Each builder's specific code could live under ./go, ./ko, etc. Maybe overtime we would find better ways to share code between them, e.g. thru a standard codebase with "interface/template" for each ecosystem: dry-run command, download dependency command, compilation command, etc. which would allow us to abstract away the underlying compiler

Thoughts?

cc @asraa @ianlewis @joshuagl

e2e testing for provenance-only workflow

Related #76

e2e tests:

General Tasks:

  • Auto-close issues when e2e tests pass
  • Update documentation as necessary

Drop `.git` in URIs

The generated URIs append .git to the end, but this is wrong.

        "uri": "git+https://github.com/ianlewis/actions-test@refs/heads/main.git",

I suggest removing this altogether. (Another option would be to move it before the @, but that doesn't seem to have any value.)

Docker build image workflow

Based on #23 it would be possible to allow the use of a generic Docker image to perform the build step prior to provenance generation.

Considerations:

  • We could allow users to specify a publicly or privately available Docker image by name and run that.
  • We could allow users to specify a Dockerfile which we build for them before running the image.

Feature: less contrained variable names

Dynamic arguments can be used in the config file, e.g/, {{ .OS }}. We currently only allow an exact match, and may need to be more flexible and allow {{.OS}}, for example. Go releaser seems to allow that.

Re-organize docs

We need to organize docs by workflow.

  • README.md - base readme for the project. Links to docs for individual workflows
    • builders/go/README.md - Doc for Go builder.
    • builders/slsa/README.md - Doc for provenance-only workflow
    • etc.

Verify requested OIDC tokens

@asraa said:

we should probably verify that the client ID is set to the requested audience and that the issuer was github This library would be useful! https://github.com/coreos/go-oidc/blob/v2.2.1/verify.go#L73

obviously we are generating this internally and not expecting this to be passed in thru user args, but it is probably good to verify

also see where this is used for usage https://github.com/sigstore/fulcio/blob/b2186c01da1ddf807bde3ea8c450226d8e001d88/pkg/config/config.go#L195-L199

Generic workflow/library

In addition to the generic API we have, there is a need for a generic workflow that calls a CLI/API with:

  1. A lock file -> the reusable workflow can generate an SBOM from it. Example builder generate-sbom -lock-file <lock-file>
  2. A list of commands to vendor the dependencies and tarball them to be shared with the building VM (useful for hermetic builds). In Npm, the download part would be npm ci. Example: builder vendor -cmds "npm ci"
  3. A list of commands to build. In Npm, this would be npm build or npm pack. Example: builder build -cmds "untar-deps, npm pack"
  4. Provenance generation: we already have this part built. Example: builder generate-provenance <TODO>

/cc @lumjjb @bcoe @MarkLodato @ianlewis @asraa @joshuagl

Feature: add builder info to provenance

We should investigate capturing runner OS's information if available in /etc
Maybe also provide an SBOM of our builder, etc.

GitHub may also expose info. In the logs, for example, VM setup shows info like:

Current runner version: '2.288.1'
Environment: ubuntu-20.04
  Version: 20220227.1
  Included Software: https://github.com/actions/virtual-environments/blob/ubuntu20/20220227.1/images/linux/Ubuntu2004-Readme.md
  Image Release: https://github.com/actions/virtual-environments/releases/tag/ubuntu20%2F20220227.1
Virtual Environment Provisioner
  1.0.0.0-main-20220307-1

Support buildStartedOn, buildFinishedOn in Go builder.

Add buildStartedOn and buildFinishedOn to metadata in the Go builder.

    "metadata": {
      "buildInvocationId": "<STRING>",
      "buildStartedOn": "<TIMESTAMP>",
      "buildFinishedOn": "<TIMESTAMP>",
      "completeness": {
        "parameters": true/false,
        "environment": true/false,
        "materials": true/false
      },
      "reproducible": true/false
    },

Doc on provenance field formats

The README doc could have some notes on the data and format of fields determined by the buildType. These include invocation, buildConfig, materials, and environment.

Feature: Should provenance include materials?

SLSA materials are:

materials array of objects, optional

The collection of artifacts that influenced the build including sources, dependencies, build tools, base images, and so on.

This is considered to be incomplete unless metadata.completeness.materials is true.

It seems like we currently include the source repo and digest, but should we also be including:

  • the go compiler?
  • linked libraries?

I don't see any examples of this, and since it's optional I understand it's not necessary but I would like to have a plan in mind. Go seems easy enough for dependencies because they are listed in the source anyway with hashes.

Improve WorkflowRun customization

Consider adding utility functions on the WorkflowRun to customize fields. This will allow generators that use the API to set their own build type and fields easier.

Some earlier suggestions from @laurentsimon

Just a last-minute thought. We could also have WithXXX() functions, which may be simple too:

r := slsa.NewWorkflowRun(subjects, githubContext)
       .WithBuildType("some_string")
       .WithBuildConfig(some_struct)
func (self * WorkflowRun) WithBuildConfig(arg interface{}) *WorkflowRun{
  self.BuildConfig = arg
  return self
}

Can do that in a follow-up PR if we want

Maybe AddMaterial() and AddEnvironment() would work. Let's just file an issue for later discussion.

Feature: official release

Feature: output public key certificate

Public key certficiate should be saved with the intoto artifact. This gives us sufficient information to verify against rekor.

rekor-cli verify --artifact <artifact> --public-key <certificate> --type intoto

Support `name` containing whitespace

The current implementation disallows whitespace in the subject name because it uses strings.Fields. The spec does not disallow whitespace, and spaces in filenames are not uncommon, so it would be best if the implementation allowed that.

To fix, using strings.Cut (or strings.SplitN or a regexp) would avoid the issue.

Feature: Customize the signer

Allow options for

  1. Default Fulcio signer
  2. A private key with an encrypted password per cosign's COSIGN_PASSWORD
  3. A KMS key

Using the second will also allow us to avoid using default fulcio signers in e2e tests -- that can't happen right now because pull_request workflows don't have id-token write access

Entrypoint is the workflow name

At it stands the provenance invocation.entryPoint is set to the workflow field from the Github Actions workflow. This is the name of the calling workflow or the path if the name is not set.

This seems wrong to me. I think the path to the calling workflow yaml file would be more useful but I'm not sure. We could get from event.workflow in the github context.

Set arch key in Environment

Get set the architecture for the build.

Since the job that built the artifact could be using a different CPU architecture than the job running the provenance generation, this probably needs to be added as an optional input.

Dockerfile workflow

We can provide a builder which builds a Docker image based on a Dockerfile as the build artifact and generate SLSA provenance for it.

Support more metadata fields

Add support for more metadata fields:

    "metadata": {
      "buildInvocationId": "<STRING>",
      "buildStartedOn": "<TIMESTAMP>",
      "buildFinishedOn": "<TIMESTAMP>",
      "completeness": {
        "parameters": true/false,
        "environment": true/false,
        "materials": true/false
      },
      "reproducible": true/false
    },

Feature: harden against material leaks

Leaks of:

  1. GITHUB_TOKEN which allows requesting certs from Fulcio. Not sure yet what we can do here, besides hardening our implementation and verifying that the keys don't leak in coredump, in logs, etc. Somehow we'd like to add scope to the GITHUB_TOKEN to be one-time-use only, but that would require support from GitHub. Maybe something we can propose to them. Let me know if you have other ideas.
  2. The signing key: shall we enforce, during verification, that there exists a single rekor entry with the certificate?

Feature: support for hermeticity

The go project does support hermeticity at the moment by building in 2 steps:

  1. Vendoring go mod vendor
  2. Build with go build --mod=vendor

However we currently:

  1. Do not add this information to the provenance
  2. Do not set up networking rules to verify that nothing is fetched during build - this is enforced by the option --mod=vendor though.

In the case where we filter certain arguments (currently the case), I think this gives us hermeticity. However, if we relax and decide to not filter the user-provided argument (e.g., a user could provide --mod=XXX that overwrites the --mod=vendor), we may need some OS-level guarantees to enforce hermeticity. We can do that:

  1. Thru setting IP tables ourselves, and maybe dropping privileges/locking down sudo
  2. Use dockerfiles to run the build - which achieves the same as previous point

Support outputting unsigned provenance

In case a user wants to create GitHub workflow provenance and then attach that to a container image using cosign CLI. By doing this, the attached container image would include rekor bundle and certificate in the cosign simple signing schema.

e.g.

$ ./slsa-github-generator attest --unsigned --output predicate.json

$ ./cosign attest --predicate predicate.json --type intoto gcr.io/asra-ali/hello-ko

The generic provenance builder should only output the predicate

cc @laurentsimon

Refactor directory structure

I was thinking about where was best to put builders long term. A couple of options were

  • internal/builders/* : builder code is generally not meant to be imported so this might make the most sense?
  • cmd/* : Where provenance-only is currently. This seems like a common pattern but seems arbitrary. Not sure I like it.
  • builders/* : Where go is currently. This is ok, but code is importable.

I kind of like the idea of putting main packages under a .../cmd/ structure so maybe this is also an option but maybe I'm overthinking it.

  • internal/cmd/{go-builder,python-builder,slsa-builder,...}: commands
  • internal/builders/{go,python,slsa,...}: supporting build code

I think the most important thing is just to be consistent with our structure.

Support private PKI

It would be useful to support private PKI.

  • Private sigstore instances (#3607)
  • Private keys obtained from vault or something else? (#326)

GitHub action for common tasks

Across reusable workflows, we need:

  • Checkout the repo at the right ref (#49)
  • Build the builder
  • Download-verify the builder
  • Declaration of common env variables (VERIFIER_BINARY, etc at the top of each workflow)
  • Push assets to a release
  • e2e and e2e pre-submit scripts (wrap their installation thru an Action)

We need common GitHub actions to perform these task, to avoid copy/paste of the same code.

The actions can be hosted in this repo and called by reusable workflows:

uses: slsa-framework/slsa-github-generator/actions/checkout-builder // Note: may just be hidden inside build-builder, I think
...
uses: slsa-framework/slsa-github-generator/actions/build-builder
  with:
    ref: needs.detect-env.outputs.ref
    repo: needs.detect-env.outputs.repo

...

uses: slsa-framework/slsa-github-generator/actions/download-builder
  with:
    name: ...
    sha256: ${{ needs.builder.outputs.sha256 }}

Support for multiple Go builds

Our current config file is inspired by goreleaser's config file but was simplified to show feasibility of the approach. We need to enhance the config file to support multiple builds.

One question to answer is how we will isolate each builds from one another. Note that the strategy.matrix field is not supported by the user workflow when calling a reusable workflow.

We can then include multiple Subject Digests inside provenance. See from https://github.com/gossts/slsa-go/issues/40

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.