philips-labs / slsa-provenance-action Goto Github PK
View Code? Open in Web Editor NEWGithub Action implementation of SLSA Provenance Generation
License: MIT License
Github Action implementation of SLSA Provenance Generation
License: MIT License
On MacOS there is a different version of sed
.
By installing gsed
and adding an alias in your .bashrc
/.zshrc
you can have compatibility.
Make however is not picking aliases and still uses the version of sed
that ships with MacOS.
We should find another way in the Makefile to use gsed
when running on MacOS. e.g. Detect the OS from the makefile and then conditionally use gsed
as opposed to sed
.
Implement logic to populate the environment using Github available data.
https://slsa.dev/provenance/v0.2#github-actions
This requires #78 to be finalised first.
Bump the provenance format to use the 0.2 spec
https://github.com/slsa-framework/slsa/blob/main/docs/provenance/v0.2-draft.md
Manually create a provenance file for curl, based on this example: https://slsa.dev/example
By using cosigns experimental feature it will start using fulcio and rekor.
This adds additional security by adding a transparancy log as well PKCS11 based certificates.
As the feature is still experimental we might want to hold back for a bit to have this feature mature a bit more.
We need to add the certificate lines back
as well enable the experimental feature see https://github.com/caarlos0/goreleaserfiles/blob/main/cosign_docker.yml#L6
Doing so enables us to do keyless code signing of our releases.
When creating multiple subjects in multiple steps in a workflow, all predicates are the same.
Basically the Provenance states: We ran workflow w and it produced subject x with digest y on timestamp z. There is no information on which step and what arguments are used.
{
"_type": "https://in-toto.io/Statement/v0.1",
"predicateType": "https://slsa.dev/provenance/v0.2",
"subject": [
{
"name": "index.docker.io/philipssoftware/blackduck",
"digest": {
"sha256": "e4b7db02469ea627818cfd8bcd0cff1c377e64189ef670184e56d0685096374f"
}
}
],
"predicate": {
"builder": {
"id": "https://github.com/philips-software/docker-blackduck/Attestations/GitHubHostedActions@v1"
},
"buildType": "https://github.com/Attestations/GitHubActionsWorkflow@v1",
"invocation": {
"configSource": {
"uri": "git+https://github.com/philips-software/docker-blackduck",
"digest": {
"sha1": "64a263c8bb206b256e0507f1998d81e7f647f8fc"
},
"entryPoint": "build"
}
},
"metadata": {
"buildInvocationID": "https://github.com/philips-software/docker-blackduck/actions/runs/1752676715",
"buildFinishedOn": "2022-01-26T19:39:00Z",
"completeness": {
"parameters": true,
"environment": false,
"materials": false
},
"reproducible": false
},
"materials": [
{
"uri": "git+https://github.com/philips-software/docker-blackduck",
"digest": {
"sha1": "64a263c8bb206b256e0507f1998d81e7f647f8fc"
}
}
]
}
}
The invocation
node shows which repository
with given sha1
and the entryPoint
. The entrypoint relates back to the name of the workflow, not the step.
In the metadata
we see the buildInvocationID
and the buildFinishedOn
fields. This is not enough to know what was used to build it. We also need to known what github-action-name
is invoked and what arguments
were passed.
In the SLSA-provenance v0.2 spec we see an example for "GitHub Actions Workflow".
They are using the entrypoint
like we are doing. (they added filename in front of the workflow name). The parameters
node is used for workflow_dispatch
parameters.
The environment
part is to provide context how to reproduce it.. I think we need to add the step
/ github-action-name
and arguments
there.
See also: philips-software/docker-ci-scripts#97
Add generating the SBOM to our CI pipeline and attach it to the Github Release.
https://golangrepo.com/repo/spdx-spdx-sbom-generator-go-miscellaneous
Multiple layers with file downloads from previous layers.
Example:
Job 1 creates two artifacts, 1 jar, and 1 provenance file.
Job 2 is depended on job 1 and downloads the artifacts. We should generate a provenance file based on the existing provenance file from job 1
Right now, CI runs in my own fork (and PRs) fail because some secrets are not available. This is the GITHUB_TOKEN
required to query the GitHub APIs. It would be good to have the workflows not run these obviously unrunnable tests :)
When releasing a new version of our action we push a new tag.
This results in a release of the docker image with that specific tag as well a github release.
Now to ensure people are using this tag the exact same commit referenced by the tag should also point to the to-be released version of our image within action.yml
https://github.com/philips-labs/slsa-provenance-action/blob/github-releases/action.yaml#L29
Basically we have the chicken and egg issue that needs to be resolved to be able to automate this.
Currently we will have to make a new commit on the main branch increasing the version, then tag this commit and do
git push --follow-tags
When using the current GH Action, you always have to use the action. In some use cases, you want to only install it and use it later on.
See GoReleaser as an example.
How do we create a beta release?
What if we want to overwrite previous beta releases of the same version?
Hey folks, thanks for creating this useful action.
I've found an issue when generating the provenance that if a commit contains an apostrophe it fails to encode the github context correctly:
/home/runner/work/_temp/23f196b1-6f36-4322-a6e0-0596e1da9544.sh: line 229: unexpected EOF while looking for matching `''
This appears to happen in this part of the action:
https://github.com/philips-labs/slsa-provenance-action/blob/main/action.yaml#L33
Signing of provenance files should allow for compatibility with:
Both Cosign and notation allow for attaching additional artefacts to the image. Eventually this signed provenance should also be attachable to the container image (#88). That would allow us to fetch the provenance for a given docker image and use the tooling (notation/cosign) to verify the signature.
What does it do?
Example:
"predicate": {
"builder": {
"id": "https://github.com/philips-labs/SLSA-Provenance-Action/Attestations/GitHubHostedActions@v1"
},
What does the URL do and what precisely does the Attesations mean?
Currently we are using the cosign cli to attach the provenance file to a Docker image.
Investigate if we can use https://github.com/sigstore/cosign/tree/bad18e5cb25f2cb86301d248e7e4ed39d49df143/pkg/cosign as a library to natively integrate this step in slsa-provenance.
Makefile that automates releasing
By integrating the logic into our binary we should reduce the attack surface of using third party libraries and actions.
Remove old example and create a new workflow which tests the action.
Add the recommendations to JSON files in the vscode folder to create a uniform development environment. See https://github.com/philips-labs/dct-notary-admin/tree/develop/.vscode as an example.
The complete provenance.json
is attached to the container.
cosign attest --predicate provenance.json --key cosign.key ${{ matrix.repo }}@${{ needs.release.outputs.container_digest }}
This is not recognised by cosign as a slsaprovenance predicate-type.
This results in a custom attestation, which can be seen if you look at the payload of the attestation.
The payload contains a predicate type: cosign.sigstore.dev/attestation/v1
with our escaped provenance.json file.
https://github.com/philips-labs/slsa-provenance-action/runs/4627740174?check_suite_focus=true
Payload:
eyJfdHlwZSI6Imh0dHBzOi8vaW4tdG90by5pby9TdGF0ZW1lbnQvdjAuMSIsInByZWRpY2F0ZVR5cGUiOiJjb3NpZ24uc2lnc3RvcmUuZGV2L2F0dGVzdGF0aW9uL3YxIiwic3ViamVjdCI6W3sibmFtZSI6ImluZGV4LmRvY2tlci5pby9waGlsaXBzc29mdHdhcmUvc2xzYS1wcm92ZW5hbmNlIiwiZGlnZXN0Ijp7InNoYTI1NiI6IjhlYjE0YmRkZDE0ZTZmZjc1YzY4YTllZTYxZDcxM2Y2ODA1ZjhiMzIzYjNlMTgzYjlmNDFlNjdlODZkNzY5MTQifX1dLCJwcmVkaWNhdGUiOnsiRGF0YSI6IntcbiAgXCJfdHlwZVwiOiBcImh0dHBzOi8vaW4tdG90by5pby9TdGF0ZW1lbnQvdjAuMVwiLFxuICBcInN1YmplY3RcIjogW1xuICAgIHtcbiAgICAgIFwibmFtZVwiOiBcInBoaWxpcHNzb2Z0d2FyZS9zbHNhLXByb3ZlbmFuY2U6YmEyOWU5OWZhZDA5YTY3NmE1NWRjYzQ3Y2VlMjBlZmU3Zjk4MDA4N1wiLFxuICAgICAgXCJkaWdlc3RcIjoge1xuICAgICAgICBcInNoYTI1NlwiOiBcIjhlYjE0YmRkZDE0ZTZmZjc1YzY4YTllZTYxZDcxM2Y2ODA1ZjhiMzIzYjNlMTgzYjlmNDFlNjdlODZkNzY5MTRcIlxuICAgICAgfVxuICAgIH0sXG4gICAge1xuICAgICAgXCJuYW1lXCI6IFwicGhpbGlwc3NvZnR3YXJlL3Nsc2EtcHJvdmVuYW5jZTp2MC41LjBcIixcbiAgICAgIFwiZGlnZXN0XCI6IHtcbiAgICAgICAgXCJzaGEyNTZcIjogXCI4ZWIxNGJkZGQxNGU2ZmY3NWM2OGE5ZWU2MWQ3MTNmNjgwNWY4YjMyM2IzZTE4M2I5ZjQxZTY3ZTg2ZDc2OTE0XCJcbiAgICAgIH1cbiAgICB9XG4gIF0sXG4gIFwicHJlZGljYXRlVHlwZVwiOiBcImh0dHBzOi8vc2xzYS5kZXYvcHJvdmVuYW5jZS92MC4yXCIsXG4gIFwicHJlZGljYXRlXCI6IHtcbiAgICBcImJ1aWxkZXJcIjoge1xuICAgICAgXCJpZFwiOiBcImh0dHBzOi8vZ2l0aHViLmNvbS9waGlsaXBzLWxhYnMvc2xzYS1wcm92ZW5hbmNlLWFjdGlvbi9BdHRlc3RhdGlvbnMvR2l0SHViSG9zdGVkQWN0aW9uc0B2MVwiXG4gICAgfSxcbiAgICBcImJ1aWxkVHlwZVwiOiBcImh0dHBzOi8vZ2l0aHViLmNvbS9BdHRlc3RhdGlvbnMvR2l0SHViQWN0aW9uc1dvcmtmbG93QHYxXCIsXG4gICAgXCJpbnZvY2F0aW9uXCI6IHtcbiAgICAgIFwiY29uZmlnU291cmNlXCI6IHtcbiAgICAgICAgXCJlbnRyeVBvaW50XCI6IFwiQ29udGludW91cyBpbnRlZ3JhdGlvblwiLFxuICAgICAgICBcInVyaVwiOiBcImdpdCtodHRwczovL2dpdGh1Yi5jb20vcGhpbGlwcy1sYWJzL3Nsc2EtcHJvdmVuYW5jZS1hY3Rpb25cIixcbiAgICAgICAgXCJkaWdlc3RcIjoge1xuICAgICAgICAgIFwic2hhMVwiOiBcImJhMjllOTlmYWQwOWE2NzZhNTVkY2M0N2NlZTIwZWZlN2Y5ODAwODdcIlxuICAgICAgICB9XG4gICAgICB9LFxuICAgICAgXCJwYXJhbWV0ZXJzXCI6IG51bGwsXG4gICAgICBcImVudmlyb25tZW50XCI6IG51bGxcbiAgICB9LFxuICAgIFwibWV0YWRhdGFcIjoge1xuICAgICAgXCJidWlsZEludm9jYXRpb25JZFwiOiBcImh0dHBzOi8vZ2l0aHViLmNvbS9waGlsaXBzLWxhYnMvc2xzYS1wcm92ZW5hbmNlLWFjdGlvbi9hY3Rpb25zL3J1bnMvMTYxOTU5NTcyM1wiLFxuICAgICAgXCJidWlsZEZpbmlzaGVkT25cIjogXCIyMDIxLTEyLTI0VDE1OjQxOjA1WlwiLFxuICAgICAgXCJjb21wbGV0ZW5lc3NcIjoge1xuICAgICAgICBcInBhcmFtZXRlcnNcIjogdHJ1ZSxcbiAgICAgICAgXCJlbnZpcm9ubWVudFwiOiBmYWxzZSxcbiAgICAgICAgXCJtYXRlcmlhbHNcIjogZmFsc2VcbiAgICAgIH0sXG4gICAgICBcInJlcHJvZHVjaWJsZVwiOiBmYWxzZVxuICAgIH0sXG4gICAgXCJtYXRlcmlhbHNcIjogW1xuICAgICAge1xuICAgICAgICBcInVyaVwiOiBcImdpdCtodHRwczovL2dpdGh1Yi5jb20vcGhpbGlwcy1sYWJzL3Nsc2EtcHJvdmVuYW5jZS1hY3Rpb25cIixcbiAgICAgICAgXCJkaWdlc3RcIjoge1xuICAgICAgICAgIFwic2hhMVwiOiBcImJhMjllOTlmYWQwOWE2NzZhNTVkY2M0N2NlZTIwZWZlN2Y5ODAwODdcIlxuICAgICAgICB9XG4gICAgICB9XG4gICAgXVxuICB9XG59IiwiVGltZXN0YW1wIjoiMjAyMS0xMi0yNFQxNTo0MTowNloifX0=
Payload decoded:
{
"_type": "https://in-toto.io/Statement/v0.1",
"predicateType": "cosign.sigstore.dev/attestation/v1",
"subject": [
{
"name": "index.docker.io/philipssoftware/slsa-provenance",
"digest": {
"sha256": "8eb14bddd14e6ff75c68a9ee61d713f6805f8b323b3e183b9f41e67e86d76914"
}
}
],
"predicate": {
"Data": "{\n \"_type\": \"https://in-toto.io/Statement/v0.1\",\n \"subject\": [\n {\n \"name\": \"philipssoftware/slsa-provenance:ba29e99fad09a676a55dcc47cee20efe7f980087\",\n \"digest\": {\n \"sha256\": \"8eb14bddd14e6ff75c68a9ee61d713f6805f8b323b3e183b9f41e67e86d76914\"\n }\n },\n {\n \"name\": \"philipssoftware/slsa-provenance:v0.5.0\",\n \"digest\": {\n \"sha256\": \"8eb14bddd14e6ff75c68a9ee61d713f6805f8b323b3e183b9f41e67e86d76914\"\n }\n }\n ],\n \"predicateType\": \"https://slsa.dev/provenance/v0.2\",\n \"predicate\": {\n \"builder\": {\n \"id\": \"https://github.com/philips-labs/slsa-provenance-action/Attestations/GitHubHostedActions@v1\"\n },\n \"buildType\": \"https://github.com/Attestations/GitHubActionsWorkflow@v1\",\n \"invocation\": {\n \"configSource\": {\n \"entryPoint\": \"Continuous integration\",\n \"uri\": \"git+https://github.com/philips-labs/slsa-provenance-action\",\n \"digest\": {\n \"sha1\": \"ba29e99fad09a676a55dcc47cee20efe7f980087\"\n }\n },\n \"parameters\": null,\n \"environment\": null\n },\n \"metadata\": {\n \"buildInvocationId\": \"https://github.com/philips-labs/slsa-provenance-action/actions/runs/1619595723\",\n \"buildFinishedOn\": \"2021-12-24T15:41:05Z\",\n \"completeness\": {\n \"parameters\": true,\n \"environment\": false,\n \"materials\": false\n },\n \"reproducible\": false\n },\n \"materials\": [\n {\n \"uri\": \"git+https://github.com/philips-labs/slsa-provenance-action\",\n \"digest\": {\n \"sha1\": \"ba29e99fad09a676a55dcc47cee20efe7f980087\"\n }\n }\n ]\n }\n}",
"Timestamp": "2021-12-24T15:41:06Z"
}
}
We can only send the predicate part of our provenance and add the type: slsaprovenance
Currently, Files are included.
Other provenance types include:
In the provenance file there is a section on recipe
.
This contains a type. In our case: https://github.com/Attestations/GitHubActionsWorkflow@v1
This is not a valid url. We think this is still under discussion with GitHub.
Or, it's perfectly fine, see the definition: https://github.com/in-toto/attestation/blob/main/spec/field_types.md#TypeURI
...
"recipe": {
"type": "https://github.com/Attestations/GitHubActionsWorkflow@v1",
"definedInMaterial": 0,
"entryPoint": "Create a provenance",
"arguments": {
"text_input": "Garlic"
},
"environment": null
},
...
Similar to #6
Due to adding more and more features the UX of the cli becomes more complicated as well the logic within the CLI to validate the commandline options is becoming more complicated.
e.g. when creating provenance for containers it requires different arguments then generating provenance for file assets.
Draft Proposal:
bin/slsa-provenance containers generate -repo ghcr.io/philips-labs/slsa-provenance -tags v0.4.0,abc12345fe7878d9 -digest 9817abcd1238gfa -output_path provenance.json
bin/slsa-provenance files generate -artifact_path ./build_output -output_path provenance.json
bin/slsa-provenance gh-release generate -release v0.4.0 -output_path provenance.json
bin/slsa-provenance sign -key signing.key -provenance_file provenance.json -output_path provenance.signed
This should allow us to bring some consistency on options used in the different commands as well bring more clarity/transparency for the end-user on what will be the effect of providing an option.
Current command in the container-provenance branch looks as following and is becoming quite cluttered. Both from an UX perspective as well from a implementation perspective with all the combinations possible/not possible.
USAGE
slsa-provenance generate
FLAGS
-artifact_path ... The file or dir path of the artifacts for which provenance should be generated.
-container_digest ... The digest for the oci artifact
-container_repo ... The repository for the oci artifact
-container_tags ... the given tags for this oci release
-extra_materials ... paths to files containing SLSA v0.1 formatted materials (JSON array) in to include in the provenance
-github_context ... The '${github}' context value.
-output_path build.provenance The path to which the generated provenance should be written.
-runner_context ... The '${runner}' context value.
-tag_name ... The github release to generate provenance on.
GoReleaser does not publish to the marketplace when a release is published.
Is this a shortcoming in the GH API?
Generate a simple use case and provide the provenance of a single artifact.
Generate a simple use case and provide the provenance of multiple artifact.
Add some unit tests that cover the basic functionality that is currently part of this action.
Add a workflow that executes the Unit tests and integration tests every time someone opens a PR to default branch.
Generated docs can't handle multiline documentation. It results in a new table row messing up the docs.
See:
https://github.com/philips-labs/slsa-provenance-action#inputs
See tabia, spiffe-vault for some examples on releasing the docker image and the binaries.
We only need to verify if something specific has to be done for the GH action itself.
Generate a simple use case and provide the provenance of a single artifact with manual input.
This should result in the following changes in the provenance file:
predicate.metadata.completeness
should have the content of the manual inputpredicate.recipe.arguments
should have the content of the manual inputDependabot build fails on generating the provenance file.
2021/10/18 08:01:01 failed to unmarshal github context json: invalid character 'h' in string escape code
https://github.com/philips-labs/slsa-provenance-action/pull/55/checks?check_run_id=3923877684
Generate a software bill of material and provide it with releases. This can be used as a reference in blog posts.
Research manifest file and see how it fits into the provenance file.
Should manifest be generated beforehand or during the generation of provenance?
https://carlosbecker.com/posts/multi-platform-docker-images-goreleaser-gh-actions/
Using docker_manifests as described in this build might also give us the opportunity to conditionally do latest releases. Didn't go into the details yet.
Once we have provenance generated, it would also be handy to be able to verify the provenance.
In practice this means we need to be able to execute the attestations from the in-toto statement.
Verifying the provenance file would in general be something that is executed in a admission-controller before something is installed in production.
See here a reference of the in-toto statement
https://github.com/in-toto/attestation
See here another binary implementing in-toto including a verify command.
The action is not able to pull private images at this point. This is due to the library that is used internally to retrieve the images is not passing along the proper tokens.
Dependabot do continuous integration tests are failing.
Dependabot does not have access to the CODECOV secrets.
Two solutions are possible:
The @philips-labs/secure-software-supply-chain team isn't assigned to PRs automatically, despite the fact this team is configured as code owner.
The current tests use the production GitHub API for testing generating provenance for release assets and attaching it to GitHub.
Instead of using the API, we should mock it to avoid things breaking out of the blue.
When releasing docker images to various registries using various tags it would be good to be able to generate provenance using this action independent from the tooling/scripts that create the images and tags.
e.g.:
For each registry we would like to be able to generate provenance for the tags that are correlated to each other. We could leverage the docker image digest as a subject.
This provenance file could then be attached using cosign (sigstore) or notation (Notary Project).
ghcr.io/philips-labs/slsa-provenance:latest
ghcr.io/philips-labs/slsa-provenance:v0.2.0
ghcr.io/philips-labs/slsa-provenance:v0.2
ghcr.io/philips-labs/slsa-provenance:v0
philipssoftware/slsa-provenance:latest
philipssoftware/slsa-provenance:v0.2.0
philipssoftware/slsa-provenance:v0.2
philipssoftware/slsa-provenance:v0
In the current spec that would however be some duplication to express the same image using the various tags.
To be discussed:
Implement logic to populate the buildConfig.
This requires #78 to be finalised first.
Version is not set in build artifact.
$ docker run ghcr.io/philips-labs/slsa-provenance:v0.5.0 version
GitVersion:
GitCommit:
GitTreeState: clean
BuildDate: '2021-12-24T15:39:07Z'
GoVersion: go1.17.5
Compiler: gc
Platform: linux/amd64
This happens in version v0.2.0
, v0.3.0
, v0.4.0
and v0.5.0
Align the provenance extension with the extension used by cosign.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.