Giter Site home page Giter Site logo

ulisesgascon / openssf-scorecard-monitor Goto Github PK

View Code? Open in Web Editor NEW
27.0 5.0 6.0 2.16 MB

Simplify OpenSSF Scorecard tracking in your organization with automated markdown and JSON reports, plus optional GitHub issue alerts

Home Page: https://github.com/marketplace/actions/openssf-scorecard-monitor

License: Apache License 2.0

Shell 1.05% JavaScript 87.96% EJS 10.99%
github-actions open-source-management openssf-scorecard security security-audit security-tools

openssf-scorecard-monitor's Introduction

OpenSSF Scorecard Monitor

Simplify OpenSSF Scorecard tracking in your organization with automated markdown and JSON reports, plus optional GitHub issue alerts.


๐Ÿ“ข IMPORTANT

This project is now part of the OpenSSF organization. You can find more information about it in this announcement.


๐Ÿ”ฎ About

If you're feeling overwhelmed by an avalanche of repository scorecards in your organization, you can breathe easy: Automation is here to make your life easier! It will streamline the process of keeping track of them all by providing a comprehensive report in Markdown and a local database in JSON with all the scores. Furthermore, to stay on top of any changes in the scores, you can choose to get notifications through Github Issues.

โœ… Requirements

Please ensure that any repository you wish to track with Scorecard Monitor has already been analyzed by OpenSSF Scorecard at least once. This can be accomplished using the official GitHub Action or the Scorecard CLI.

It's also possible that some repositories in your organization are already being automatically tracked by OpenSSF in this CSV file via weekly cronjob. One caveat: Automatically tracked projects do not include certain checks in their analysis (CI-Tests,Contributors,Dependency-Update-Tool,Webhooks).

If you're not sure whether a specific project is already using Scorecard, you can always spot-check with the following URL pattern: https://securityscorecards.dev/viewer/?uri=github.com/<ORG_NAME>/<REPO_NAME> (substitute <ORG_NAME> and <REPO_NAME> as appropriate). The Scorecard API is also able to fetch scores for a given repository.

๐Ÿ“บ Tutorial

This section is coming soon. If you would like to contribute to the documentation, please feel free to open a pull request for review.

โค๏ธ Awesome Features

  • Easy to use with great customization
  • Easy to patch the scoring as the reports includes a direct link to StepSecurity
  • Easy way to visualize the scorecard results with The Scorecard Visualizer or deps.dev
  • Cutting-edge feature that effortlessly compares OpenSSF scorecards between previous and current commits with The Scorecard Visualizer Comparator
  • Discovery mode: list all the repos in one or many organizations that are already being tracked with OpenSSF Scorecard
  • Reporting in Markdown with essential information (hash, date, score) and comparative against the prior score
  • Self-hosted: The reporting data is stored in JSON format (including previous records) in the repo itself
  • Generate an issue (assignation, labels..) with the last changes in the scores, including links to the full report
  • Automatically create a pull request for repositories that have branch protection enabled
  • Easy to exclude/include new repositories in the scope from any GitHub organization
  • Extend the markdown template with you own content by using tags
  • Easy to modify the files and ensure the integrity with JSON Schemas
  • The report data is exported as an output and can be used in the pipeline
  • Great test coverage (in progress)

๐ŸŽ‰ Demo

Sample Report

sample report

Sample report

Sample Issue

sample issue preview

Sample issue

:shipit: Used By

๐Ÿ“ก Usage

Standalone with auto-discovery version

With the following workflow, you will get the most out of this action:

  • Trigger manually or by Cron job every Sunday
  • It will scan the org(s) in scope looking for repositories that are available in the OpenSSF Scorecard
  • It will store the database and the scope files in the repo
  • It will generate an issue if there are changes in the score
name: "OpenSSF Scoring"
on: 
  # Scheduled trigger
  schedule:
    # Run every Sunday at 00:00
    - cron: "0 0 * * 0"
  # Manual trigger
  workflow_dispatch:

permissions:
  # Write access in order to update the local files with the reports
  contents: write
  # Write access only required if creating PRs (see Advanced Tips below)
  pull-requests: none 
  # Write access in order to create issues
  issues: write
  packages: none

jobs:
  security-scoring:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: OpenSSF Scorecard Monitor
        uses: ossf/[email protected]
        with:
          scope: reporting/scope.json
          database: reporting/database.json
          report: reporting/openssf-scorecard-report.md
          auto-commit: true
          auto-push: true
          generate-issue: true
          # The token is needed to create issues, discovery mode and pushing changes in files
          github-token: ${{ secrets.GITHUB_TOKEN }}
          discovery-enabled: true
          # As an example nodejs Org and Myself
          discovery-orgs: 'UlisesGascon,nodejs'

Options

  • scope: Defines the path to the file where the scope is defined
  • database: Defines the path to the JSON file usage to store the scores and compare
  • report: Defines the path where the markdown report will be added/updated
  • auto-commit: Commits the changes in the database and report files
  • auto-push: Pushes the code changes to the branch
  • generate-issue: Creates an issue with the scores that had been updated
  • issue-title: Defines the issue title
  • issue-assignees: List of assignees for the issue
  • issue-labels: List of labels for the issue
  • github-token: The token usage to create the issue and push the code
  • max-request-in-parallel: Defines the total HTTP Request that can be done in parallel
  • discovery-enabled: Defined if the discovery is enabled
  • discovery-orgs: List of organizations to be includes in the discovery, example: discovery-orgs: owasp,nodejs. The OpenSSF Scorecard API is case sensitive, please use the same organization name as in the GitHub url, like: https://github.com/NodeSecure is NodeSecure and not nodesecure. See example
  • report-tags-enabled: Defines if the markdown report must be created/updated around tags by default is disabled. This is useful if the report is going to be include in a file that has other content on it, like docusaurus docs site or similar
  • report-start-tag Defines the start tag, default <!-- OPENSSF-SCORECARD-MONITOR:START -->
  • report-end-tag: Defines the closing tag, default <!-- OPENSSF-SCORECARD-MONITOR:END -->
  • render-badge: Defines if the OpenSSF badge must be rendered in the reportor to only show the score
  • report-tool: Defines the reporting review tool in place: scorecard-visualizer Example or deps.dev Example, by default scorecard-visualizer

Outputs

  • scores: Score data in JSON format
name: "OpenSSF Scoring"
on: 
  # ...

permissions:
  # ...

jobs:
  security-scoring:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: OpenSSF Scorecard Monitor
        uses: ossf/[email protected]
        id: openssf-scorecard-monitor
        with:
          # ....
      - name: Print the scores
        run: |
          echo '${{ steps.openssf-scorecard-monitor.outputs.scores }}'  

๐Ÿš€ Advanced Tips

Avoid committing directly to the branch and instead generate a PR

If you have implemented the recommended branch protection rules from the OpenSSF Scorecard, committing and pushing directly to the main branch will be impossible. An easy alternative is to extend the pipeline to automatically generate a PR for you:

name: "OpenSSF Scoring"
on: 
  # ...

permissions:
  contents: write
  pull-requests: write
  issues: write
  packages: none

jobs:
  security-scoring:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: OpenSSF Scorecard Monitor
        uses: ossf/[email protected]
        id: openssf-scorecard-monitor
        with:
          auto-commit: false
          auto-push: false
          generate-issue: true
          # ....
      - name: Print the scores
        run: |
          echo '${{ steps.openssf-scorecard-monitor.outputs.scores }}'
      - name: Create Pull Request
        uses: peter-evans/create-pull-request@38e0b6e68b4c852a5500a94740f0e535e0d7ba54 # v4.2.4
        with:
            token: ${{ secrets.GITHUB_TOKEN }}
            commit-message: OpenSSF Scorecard Report Updated
            title: OpenSSF Scorecard Report Updated
            body: OpenSSF Scorecard Report Updated
            base: main
            assignees: ${{ github.actor }}
            branch: openssf-scorecard-report-updated
            delete-branch: true

Embed Report version

If you want to mix the report in markdown format with other content, then you can use report-tags-enabled=true then report file will use the tags to add/update the report summary without affecting what is before or after the tagged section.

This is very useful for static websites, here is an example using docusaurus.

Custom tags

By default we use <!-- OPENSSF-SCORECARD-MONITOR:START --> and <!-- OPENSSF-SCORECARD-MONITOR:END -->, but this can be customize by adding your custom tags as report-start-tag and report-end-tag

Increase HTTP request in parallel

You can control the amount of parallel requests performed against the OpenSSF Scorecard Api by defining any numerical value in max-request-in-parallel, like max-request-in-parallel=15.

By default the value is 10, higher values might not be a good use of the API and you can hit some limits, please check with OpenSSF if you want to rise the limits safely.

Exclude repos

In some scenarios we want to enable the auto-discovery mode but we want to ignore certain repos, the best way to achieve that is by editing the scope.json file and add any report that you want to ignore in the excluded section for that specific organization.

๐Ÿฟ Other

Scoping Structure

Just for reference, the scope will be stored this way:

File: reporting/scope.json

{
    "github.com": {
      "included": {
        "UlisesGascon":[
          "tor-detect-middleware", 
          "check-my-headers", 
          "express-simple-pagination"
        ]
      },
      "excluded": {
        "UlisesGascon": [
          "demo-stuff"
        ]
      }
    }

}

Database structure

Just for reference, the database will store the current value and previous values with the date:

{
  "github.com": {
    "UlisesGascon": {
      "check-my-headers": {
        "previous": [ {
          "score": 6.7,
          "date": "2022-08-21"
        }],
        "current": {
          "score": 4.4,
          "date": "2022-11-28"
        }
      }
    }
  }
}

๐Ÿ’ช Contributing

Please read CONTRIBUTING.md for details on our code of conduct and the process for submitting pull requests to us. You need to accept DCO 1.1 in order to make contributions.

openssf-scorecard-monitor's People

Contributors

dependabot[bot] avatar justaugustus avatar kooltheba avatar lelia avatar rajbos avatar ulisesgascon avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

openssf-scorecard-monitor's Issues

Clarify Scorecard requirements in docs

We've recently been exploring the use of OpenSSF Scorecard Monitor for our various open source organizations, but hit a roadbump after finding that Scorecard Monitor was only discovering a handful of repositories in our orgs (regardless of which projects were listed in scope.json).

After some investigation, I realized that OpenSSF was automatically tracking a select group of our projects upstream, which is why some projects already scores and others didn't. I'd like to update the README docs to clarify these requirements, and provide some hints in case others experience similar issues.

sorting options

Main Objective

Let's offer a way to sort the scores:

  • by higher scores sort-by="ASC"
  • by lower scores sort-by="DESC"

Context

Suggested in OSFF Slack

Technical requirements

It is important to notice that one user can scan several organizations so the sorting without a grouping support can be very confusing. So let's ship this feature with a grouping option like: group-by="ORG"

It will be great to support CAPITAL an LOWER values like ASC or asc

Also this feature must be documented

BREAKING NEWS: The scorecard Monitor is part of the OSSF :partying_face:

TL;DR:

I am very glad to announce that this repository is now part of the OSSF Organization, so the Scorecard Visualizer is now an official tool in the OSSF Scorecard ecosystem. ๐ŸŽŠ ๐ŸŽŠ

Important Details

As part of the migration process, the repository has been transferred to the OSSF (from UlisesGascon/openssf-scorecard-monitor to ossf/scorecard-monitor). The redirection should be working, so no additional steps are required from your part. Starting from version v2.0.0-beta8, we will use the new URLs (soon to be released).

Let's celebrate this moment together ๐Ÿค—

This journey started a long time ago, even before the first commit on Feb 2023 when we started to adopt the OSSF Scorecard in the Node.js Organization (nodejs/security-wg#851) in Dec'22, thanks to the GOSST Team (@gabibguti, @joycebrum, @pnacht, @diogoteles08 and others) that helped us understand in detail what this project is about and how it can help our organization be more secure (full video).

As soon as I understood how important this was for the Open Source Community, I tried to spread this idea into the ecosystem, so I started to blog about it and discuss it with the community on social media.

The real challenge came when we needed to adopt it at the scale of Node.js, in the Node.js' security WG (nodejs/security-wg#851 (comment)). We realized that we needed a tool to help us monitor the scoring over time. In the following weeks, we started to iterate over this idea until we had the most basic features of the Monitor, especially thanks to Security WG (@mhdawson, @RafaelGSS, @marco-ippolito, @fraxken and others) for all the patience, feedback, ideas, and contributions to consolidate this tool and make it extensible to the community.

Once we had a clear idea on how to track the scores in our repositories, we realized that it was very hard for us to spot the evolution in terms of scoring differences. So, @KoolTheba joined the efforts by creating the Scorecard Visualizer that allowed us to showcase the scorecard details per project using commit hashes and to compare between two different commits. This was a game-changer for us as it allowed us to quickly spot the differences and act on them on a bi-weekly basis, especially when the diff details were added.

Our next big problem was how to reduce the Time To Remediation (TTR). One day, the Step Security team did an eye-opening demo for the Node.js Security WG (#37). Thanks, @varunsh-coder and @boahc077, for showing us the right way. Since then, there is a fix it link in the report to quickly apply many scorecard recommendations in any GitHub project.

I want to especially thank all the collaborators (@KoolTheba, @justaugustus, @lelia, @rajbos and others...) who helped us on this amazing journey, as well as all the users (@inigomarquinez, carpasse and others) and orgs that were early adopters and provided invaluable feedback and perspectives to the project!

Finally, thanks to the OpenJS Security Collab (@ruddermann, @ctcpip, @ljharb, @mrutkows, @shusak, @joesepi, @rginn, @bensternthal and others) for the endless discussions and invaluable knowledge shared in every session. Also, to the OSSF team for building these amazing tools and sharing them with me in advance (@laurentsimon, @naveensrinivasan and others...), and to the OSSF for helping us in all the donation journey and making all the changes required for us to join (@justaugustus, @afmarcum, @bbpursell1 and others).


๐Ÿ“ข You can follow this discussion on Twitter, Linkedin and Mastodoon

Show the scorecard badges in the table

I'd like to show the information in the table with a badge similar to what I have done in my org readme here:

image

Setup is like this:
[![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/devops-actions/issue-comment-tag/badge)](https://api.securityscorecards.dev/projects/github.com/devops-actions/issue-comment-tag)

No issue created on first or second run

I was expecting an issue to be created with my configuration (first test, love the functionality!) here.

If it only does that after the first delta (2 different dates?), then indicate it in the docs.

Allow Categories to split the output table into several smaller tables

As discussed in nodejs/security-wg#949, it will great to allow the repos to we categorized like primary, secondary.. so the output table can be split into several minor tables. That way is easier to focus only in a specific repos when the discovery mode is enabled. That way the user can control easy all the repos in a specific organization and segment the results for the analysis.

@KoolTheba This is very related to #35 but is a different feature

Full feedback in this video (from 16:12 to 17:05)

Update docs for v2.0.0

Pending:

  • Add auto-scoping example as the main example
  • Explain tags
  • Include policies samples about include vs exclude...
  • Move current example to manual scope
  • List orgs using the action
  • Add reference to NodeSecure (NodeSecure/Governance#23)
  • Add reference to All the users/orgs that use the tool
  • Update the images
  • Add alternative pipeline for storing the database and scope externally
  • Include video tutorial

Hide titles if the tags mode is enabled

If the user uses report-tags-enabled=true then titles should not be included in the markdown report file:

For reference this are the titles:

# OpenSSF Scorecard Report

## Summary

Side note: This won't affect issues template

Add auto-scoping

Overview

As a user, I want to be able to configure auto-scoping feature as true and include several github organizations as auto-scope-orgs (e.g. ['nodejs', 'OWASP']) in the Github action, so that the Github Action will list all the public repos available for the org, update the allowed scope with repos that were not previously included or in the exclusion list, save the changes in the file, commit the changes, and continue with the regular process.

Technical Requirements

  • New input auto-scope to enable/disable auto-scoping feature
  • New input auto-scope-orgs in the Github action that allow to include several github organizations, like ['nodejs', 'OWASP']
  • Add new properties to the scope file to divide the content in allowed or excluded repositories {allowed: {org: [repo]}, excluded: {org: [repo]}}
  • Collect the Org repos and compare against the included / excluded list
  • Check if the api has indexed the repo or ignore it

Main logic

  • Check that the Github token has been added as input or throw an error
  • Generate a list from all the repos (public only) available for the org using the github api and the token
  • Update the allowed scope with the repos that were not previously included or present in the exclusion list
  • Save the changes in the scope file
  • Commit the changes
  • Continue with the regular process (ensure that the regular process is using the latest version of the scope ).

Context

Add commit hash

Overview

As a user, I want to quickly understand how fresh the reports are. Therefore, I need to be able to see the HASH from the last commit, as well as a link to the repository, so I can easily understand the state of the reports on that particular moment.

Technical Requirements

  • Extract additional data from the HTTP Request
  • Generate a crafted URL with the hash
  • Include this Hash information to the reports (markdown and issue)

Main logic

  • Check that the Hash is included in the response
  • Pass and store the hash as part of the database information just as the date or score
  • Add an additional Column with the title last commit in the reports and include the url as value [{hash}]({hash_link}).

Context

Improve output

Context

If there are no changes, the process will stop without notifying the changes or the output.

Todo

  • Ensure that the current scores are available as output even if there are no changes
  • Return and document an additional prop in the output hasChanged: Bool

Add proper output

As a user, I want to consume the output of this action in a different way

Technical Requirements

  • Export the Score Analysis (the same used to render the report) as a JSON string (in a single line)

Alternative URLs

As proposed by @rajbos in #44 (comment), it will be nice to include custom urls with an argument like custom-api-url=https://myalternativeapi.com.

Do we need to map the org and repo to the new url? I mean.. currently we use https://api.securityscorecards.dev/projects/github.com/nodejs/node but this can be replaced by https://myalternativeapi.com/projects/github.com/nodejs/node, but we might need something more flexible than replace the domain like https://myapi.com/v1/security/{org}/reporting/{repo}/scorecard

Add Missing tests

Testing Strategy:

  • Move not critical parts as external libraries with a proper testing (minimal for v2-beta1)
  • Let's increase the test coverage from the action itself (minimal for v2)
  • Let's cover scenarios in e2e

Support tags

It will great to allow the usage of tags, to allow the output markdown to be integrated with existing files:

This text should remind the same...

<!-- OPENSSF-SCORECARD-MONITOR:START -->
<!-- OPENSSF-SCORECARD-MONITOR:END -->

This text will be also unaffected

Example of this implementation in gautamkrishnar/blog-post-workflow

Don't require `GITHUB_TOKEN` when it is not needed

From what I can see, the GITHUB_TOKEN is not needed for discovery (it would only have access to the current repo anyway), so we can remove it from this check. I ran into it during debugging (output tested in a ACTIONS_STEP_SUMMARY) and since it is not needed....

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.