actions-rs/grcov

📈 GitHub Action for code coverage reporting with grcov

grcov Action

This GitHub Action collects and aggregates code coverage data with the grcov tool.

Example workflow

on: [push]

name: Code Coverage

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v1
      - uses: actions-rs/toolchain@v1
        with:
          toolchain: nightly
          override: true
      - uses: actions-rs/cargo@v1
        with:
          command: test
          args: --all-features --no-fail-fast
        env:
          CARGO_INCREMENTAL: '0'
          RUSTFLAGS: '-Zprofile -Ccodegen-units=1 -Cinline-threshold=0 -Clink-dead-code -Coverflow-checks=off -Cpanic=abort -Zpanic_abort_tests'
          RUSTDOCFLAGS: '-Zprofile -Ccodegen-units=1 -Cinline-threshold=0 -Clink-dead-code -Coverflow-checks=off -Cpanic=abort -Zpanic_abort_tests'
      - uses: actions-rs/[email protected]

Usage

1. As grcov works with coverage data generated by the unstable -Z profile feature, you need to install nightly toolchain, for example, with the actions-rs/toolchain Action:

   - uses: actions-rs/toolchain@v1
     with:
       toolchain: nightly
       override: true

2. It is strongly recommended to call cargo clean command if any of cargo check, cargo build, cargo test or other similar commands were executed already in this workspace.

   - uses: actions-rs/cargo@v1
     with:
       command: clean

3. Create a configuration file for grcov, see config section for details.

4. Execute the cargo test command. It is required to add CARGO_INCREMENTAL, RUSTFLAGS and RUSTDOCFLAGS environment variables for this command, see grcov page for details.

   - uses: actions-rs/cargo@v1
     with:
       command: test
       args: --all-features --no-fail-fast  # Customize args for your own needs
     env:
       CARGO_INCREMENTAL: '0'
       RUSTFLAGS: '-Zprofile -Ccodegen-units=1 -Cinline-threshold=0 -Clink-dead-code -Coverflow-checks=off -Cpanic=abort -Zpanic_abort_tests'
       RUSTDOCFLAGS: '-Zprofile -Ccodegen-units=1 -Cinline-threshold=0 -Clink-dead-code -Coverflow-checks=off -Cpanic=abort -Zpanic_abort_tests'

   Note that -Clink-dead-code flag might be broken for macOS environments, see #63047

5. Add the actions-rs@grcov Action to the workflow.

   - id: coverage  
     uses: actions-rs/[email protected]

6. After the successful execution, actions-rs@grcov will set an Action output called report with an absolute path to the coverage report file.

   This file can be uploaded to any code coverage service, ex. with codecov or coveralls Actions help:

   - name: Coveralls upload
     uses: coverallsapp/github-action@master
     with:
       github-token: ${{ secrets.GITHUB_TOKEN }}
       path-to-lcov: ${{ steps.coverage.outputs.report }}

Inputs

• config: Configuration file path (relative to repository root, default to .github/actions-rs/grcov.yml)

Outputs

• report: Absolute path to the report file

Config

grcov execution can be tuned with the configuration file.
By default this Action tries to load it from the .github/actions-rs/grcov.yml path (relative to the repository root directory); you can change file location with the config input, ex.

- uses: actions-rs/[email protected]
  with:
    config: configs/grcov.yml

If configuration file is missing, default values will be used by the grcov.

All possible keys in the config are optional and directly mapped to the grcov flags and options.
Note that not all parameters can be changed via this file, see example below for config file format and available options.

Example

branch: true
ignore-not-existing: true
llvm: true
filter: covered
output-type: lcov
output-path: ./lcov.info
prefix-dir: /home/user/build/
ignore:
  - "/*"
  - "C:/*"
  - "../*"
path-mapping:
  - "/path1"
  - "/path2"

Notes

1. Coveralls Action is expecting LCOV format, do not use the coveralls or coveralls+ output type for the grcov.
   Instead, set the output-type config value to lcov.

2. Generated report file is stored in the temporary directory by default, which might not be accessible by the Docker-based Actions, such as codecov.
   Consider either mount it as a Docker volume or use the output-path option in the config to store report in the path accessible by container.

grcov's Issues

Support Rust stable

Do the checklist before filing an issue:

• Is this related to the actions-rs Actions?
  If you think it's a problem related to Github Actions in general, use GitHub Community forum instead: https://github.community
• You've read the Contributing section about bugs reporting: https://github.com/actions-rs/.github/blob/master/CONTRIBUTING.md#reporting-bugs
• Is this something you can debug and fix? Send a pull request! Bug fixes and documentation fixes are welcome.

Description

rust-lang/rust#90132

Grcov is not working with codecov

Hi!
Maybe I'm doing something wrong, but I spend a few hours trying to make it working, unfortunately without any luck. I don't have any errors - everything passes successfully. Could I ask you to confirm if this is a bug?

My repo's GitHub action config is here: https://github.com/luna/basegl/blob/master/.github/workflows/build.yml
And the build results are here:
https://github.com/luna/basegl/actions

As you can see, the coverage file gets generated and gets successfully passed to actions-rs/grcov. The codecov token is correct as well, however, codecov does not display any results and no errors are reported anywhere :( I'd be thankful for any hint here!

Configuration using `with: ...` in workflow yaml?

Motivation

It would be nice to configure grcov inline with my other workflow commands, similar to how the other actions-rs actions work.

Workflow example

- uses: actions-rs/[email protected]
  with:
    branch: true
    ignore-not-existing: true
    llvm: true
    filter: covered
    output-type: lcov
    output-path: ./lcov.info
    prefix-dir: /home/user/build/
    ignore:
      - "/*"
      - "C:/*"
      - "../*"
    path-mapping:
      - "/path1"
      - "/path2"

Additional context

I think this method of configuration should probably be mutually exclusive with the separate configuration file.

`excl-*` config values are missing when running the action

Do the checklist before filing an issue:

• Is this related to the actions-rs Actions?
  If you think it's a problem related to Github Actions in general, use GitHub Community forum instead: https://github.community
• You've read the Contributing section about bugs reporting: https://github.com/actions-rs/.github/blob/master/CONTRIBUTING.md#reporting-bugs
• Is this something you can debug and fix? Send a pull request! Bug fixes and documentation fixes are welcome.

Description

My .github/actions-rs/grcov.yml contains:

branch: true
ignore-not-existing: true
llvm: true
filter: covered
output-type: lcov
output-path: ./lcov.info
excl-line: "#\\[derive\\("
excl-br-line: "#\\[derive\\("
excl-start: "mod tests \\{"
excl-br-start: "mod tests \\{"

The debug statement when the action is run prints:

##[debug]User configuration: {"branch":true,"ignoreNotExisting":true,"llvm":true,"filter":"covered","outputType":"lcov","outputPath":"./lcov.info"}

It looks like the excl* config values are missing.

Workflow code

https://github.com/apollographql/rust/blob/8a2f4ca73c9529274661fcc710573c49b7647409/.github/workflows/coverage.yml

Action output

  env:
    CARGO_TERM_COLOR: always
    CARGO_INCREMENTAL: 0
    RUSTFLAGS: -Zprofile -Ccodegen-units=1 -Cinline-threshold=0 -Clink-dead-code -Coverflow-checks=off -Cpanic=abort -Zpanic_abort_tests
    RUSTDOCFLAGS: -Zprofile -Ccodegen-units=1 -Cinline-threshold=0 -Clink-dead-code -Coverflow-checks=off -Cpanic=abort -Zpanic_abort_tests
##[debug]User configuration: {"branch":true,"ignoreNotExisting":true,"llvm":true,"filter":"covered","outputType":"lcov","outputPath":"./lcov.info"}
::group::Searching for coverage files
Searching for coverage files
  Found project crates: apollo-query-planner,graphql-parser
  ##[debug]Creating an archive with coverage files at /tmp/coverage-wmwe4iwq1z.zip
  Archiving coverage file: /home/runner/work/rust/rust/target/debug/deps/apollo_query_planner-48cd324783e6f8e1.gcno
  Archiving coverage file: /home/runner/work/rust/rust/target/debug/deps/apollo_query_planner-9a2d2ac00302e9ac.gcda
  Archiving coverage file: /home/runner/work/rust/rust/target/debug/deps/apollo_query_planner-9a2d2ac00302e9ac.gcno
  Archiving coverage file: /home/runner/work/rust/rust/target/debug/deps/graphql_parser-13f552932c4a8288.gcda
  Archiving coverage file: /home/runner/work/rust/rust/target/debug/deps/graphql_parser-13f552932c4a8288.gcno
  Archiving coverage file: /home/runner/work/rust/rust/target/debug/deps/graphql_parser-989eb3afebc0a136.gcda
  Archiving coverage file: /home/runner/work/rust/rust/target/debug/deps/graphql_parser-989eb3afebc0a136.gcno
  Coverage files archive was created at the /tmp/coverage-wmwe4iwq1z.zip
  ::endgroup::
::group::Execute grcov
Execute grcov
  /usr/share/rust/.cargo/bin/grcov /tmp/coverage-wmwe4iwq1z.zip --branch --ignore-not-existing --llvm --commit-sha 8a2f4ca73c9529274661fcc710573c49b7647409 --filter covered --output-path /home/runner/work/rust/rust/lcov.info --output-type lcov --service-name Code Coverage --source-dir /home/runner/work/rust/rust
  ::endgroup::
Generated coverage report at /home/runner/work/rust/rust/lcov.info
::set-output name=report::/home/runner/work/rust/rust/lcov.info
##[debug]steps.coverage.outputs.report='/home/runner/work/rust/rust/lcov.info'
##[debug]Node Action run completed with exit code 0
##[debug]Finishing: Run actions-rs/[email protected]

Expected behavior

I believe the result of those options not showing up is that some lines are detected as not covered, e.g. all of the lines in the FetchGroup struct here: https://codecov.io/gh/apollographql/rust/src/8a2f4ca73c9529274661fcc710573c49b7647409/query-planner/src/groups.rs

Ability to run grcov action for a sub-folder containing Rust sources.

Hi! We've been succesfully using grcov for some time now and we have recently refactored our codebase. Our Cargo.toml is no longer located at the top-level. Instead it is inside src/rust folder. When running the CI we got error now: no such file or directory, open '/home/runner/work/ide/ide/Cargo.lock', which is true, as it should point to no such file or directory, open '/home/runner/work/ide/ide/src/rust/Cargo.lock' instead. The problem is that GitHub Actions do not provide a way to set up CWD for third party actions and I haven't found any configuration here how to point grcov to some specific root folder. I'd be very thankful for any help! In case this would be important, this is our repo: https://github.com/luna/ide (branch wip/wd/dev, soon the changes will be on master)

Request for adding exclusion settings to the config file.

Do the checklist before filing an issue:

• Is this related to the actions-rs Actions?
  If you think it's a problem related to Github Actions in general, use GitHub Community forum instead: https://github.community
• You've read the Contributing section about feature requests: https://github.com/actions-rs/.github/blob/master/CONTRIBUTING.md#feature-requests
• Is this something you can debug and fix? Send a pull request! Bug fixes and documentation fixes are welcome.

Motivation

I want the --excl-* options added in grcov v0.5.14 (mozilla/grcov#416) to be mapped to grcov.yml so that one can exclude #[derive(...)], mod tests { ... }, etc. from coverage analysis.

grcov.yml

excl-line: "#\\[derive\\("
excl-br-line: "#\\[derive\\("
excl-start: "mod tests \\{"
excl-br-start: "mod tests \\{"

Cannot find any coverage files for crates like 'foo-bar-baz'

coverage.getCoverageFiles does not seem to work correctly for packages like:

[package]
name = "foo-bar-baz"

grcov/src/coverage.ts

Line 45 in 3ce5aed

const replacement = crate.replace('-', '_');

$ node
> 'foo-bar-baz'.replace('-', '_')
'foo_bar-baz'

Example

https://github.com/qryxip/gh-releases-from-actions/runs/493381276

Optionally use binstall to speed up CI

Do the checklist before filing an issue:

• Is this related to the actions-rs Actions?
  If you think it's a problem related to Github Actions in general, use GitHub Community forum instead: https://github.community
• You've read the Contributing section about feature requests: https://github.com/actions-rs/.github/blob/master/CONTRIBUTING.md#feature-requests
• Is this something you can debug and fix? Send a pull request! Bug fixes and documentation fixes are welcome.

Motivation

From-source builds in CI with grcov take time and resources, so utilizing binstall would mean milliseconds instead of minutes for installation.
This requires binstall to be present on the action runner, which can be solved with its action.

Workflow example

- uses: codecov/codecov-action@v3
  with:
    binstall: true # default = false

Generater report failed

Do the checklist before filing an issue:

• Is this related to the actions-rs Actions?
  If you think it's a problem related to Github Actions in general, use GitHub Community forum instead: https://github.community
• You've read the Contributing section about bugs reporting: https://github.com/actions-rs/.github/blob/master/CONTRIBUTING.md#reporting-bugs
• Is this something you can debug and fix? Send a pull request! Bug fixes and documentation fixes are welcome.

Description

In today, I am runing my github action with actions-rs/[email protected], then run coverallsapp/github-action , it is return Error :

Error: ENOENT: no such file or directory, open '/tmp/grcov-report-n2jbbrhhsc'

first time, I thought it was their ( coverallsapp/github-action) problem. then I tried to change it settings multiple times :

• one

- name: Coveralls upload
        uses: coverallsapp/github-action@v1
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          path-to-lcov: ${{ steps.coverage.outputs.report }}

• two

 - name: Coveralls upload
        uses: coverallsapp/github-action@master
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          path-to-lcov: ${{ steps.coverage.outputs.report }}

• three

 - name: Coveralls upload
        uses: coverallsapp/github-action@v2
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          file: ${{ steps.coverage.outputs.report }}

• four

 - name: Coveralls upload
        uses: coverallsapp/github-action@v2
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}

They all failed

Workflow code

Listed up

Action output

This is some time github action log : https://github.com/tu6ge/oss-rs/actions

Expected behavior

success output report

Additional context

no more info

Wrong coverage using the example Github Workflows in the documentaiton

Do the checklist before filing an issue:

• Is this related to the actions-rs Actions?
  If you think it's a problem related to Github Actions in general, use GitHub Community forum instead: https://github.community
• You've read the Contributing section about bugs reporting: https://github.com/actions-rs/.github/blob/master/CONTRIBUTING.md#reporting-bugs
• Is this something you can debug and fix? Send a pull request! Bug fixes and documentation fixes are welcome.

Description

Using a Github Workflow like the one suggested in the documentation, macro invocation is considered as not covered by tests, although it is.

Workflow code

  test:
    name: Test
    runs-on: ubuntu-latest
  
    steps:
      - uses: actions/checkout@v3
        with:
          lfs: true

      - uses: actions-rs/toolchain@v1
        with:
          toolchain: nightly
          override: true
   
      - uses: actions-rs/cargo@v1
        with:
          command: test        
        args: --all-features
        env:
          CARGO_INCREMENTAL: '0'
          RUSTFLAGS: '-Zprofile -Ccodegen-units=1 -Cinline-threshold=0 -Clink-dead-code -Coverflow-checks=off -Cpanic=abort -Zpanic_abort_tests -Cinstrument-coverage'
          RUSTDOCFLAGS: '-Zprofile -Ccodegen-units=1 -Cinline-threshold=0 -Clink-dead-code -Coverflow-checks=off -Cpanic=abort -Zpanic_abort_tests -Cinstrument-coverage'
      
      - uses: actions-rs/[email protected]
        id: coverage
        name: Generate code coverage

      - name: Upload to codecov.io
        uses: codecov/codecov-action@v3
        with:
          files: ${{ steps.coverage.outputs.report }}
          token: ${{ secrets.CODECOV_TOKEN }}

Expected behavior

This code of the thiserror macro be covered, instead it is not:

#[derive(Error, Debug, PartialEq, Eq)]
pub enum EncodingFormatParsingError {
    #[error("Unexpected encoding {0}")] // line not covered
    UnexpectedEncoding(String),
}

cargo-grcov is not working with hypens

Do the checklist before filing an issue:

• Is this related to the actions-rs Actions? If not, use GitHub Community forum to ask questions about Actions in a whole: https://github.community
• You've read the Contributing section about bugs reporting: https://github.com/actions-rs/.github/blob/master/CONTRIBUTING.md#reporting-bugs
• Is this something you can debug and fix? Send a pull request! Bug fixes and documentation fixes are welcome.

Description

When I run the cargo-grcov action, I need to replace hypens with underscores in order to make it work. Unfortunately, if another subcrate is added, changing hypens to underscores break its compilation.

Used environment

• os: ubuntu-latest

Workflow code

https://github.com/lu-zero/tree-sitter-core/blob/da91815b916a8a17fe3ef4ba63495df02e92d698/.github/workflows/tree-sitter-core.yml#L116

Action output

Searching for coverage files
  Found project crates: tree-sitter-core,tree-sitter-tests
##[error]Unable to find any coverage files, was `cargo test` executed correctly?

Expected behavior

cargo-grcov should run regardless of the hypens present in a crate name.

Additional context

Add any other context about the problem here.

grcov v0.5.14 broke the --output-file option

After v0.5.14 of grcov was released, actions-rs/grcov invocations stopped working because the --output-file command-line option is no longer recognized by the grcov binary.

Looking at the compare, (upstream does not keep a proper changelog) it appears that mozilla/grcov@26acaa0 is the culprit, and does not include a backwards compatibility mechanism. It appears that a simple change from output-file to output-path is all that is necessary to resolve this, and I'm preparing a patch for that. This issue is mostly just for tracking.

codecov not work for pr from fork repository

A pr from fork repository branch always failed:

Please provide the repository token to upload reports via `-t :repository-token`

workflow like this

jobs:
  grcov:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v1

      - name: Install toolchain
        uses: actions-rs/toolchain@v1
        with:
          toolchain: nightly
          override: true

      - name: Clean
        uses: actions-rs/cargo@v1
        with:
          command: clean
      - name: Execute tests
        uses: actions-rs/cargo@v1
        with:
          command: test
          args: --all
        env:
          CARGO_INCREMENTAL: 0
          RUSTFLAGS: "-Zprofile -Ccodegen-units=1 -Cinline-threshold=0 -Clink-dead-code -Coverflow-checks=off -Zno-landing-pads"

      - name: Gather coverage data
        id: coverage
        uses: actions-rs/[email protected]
        with:
          config: configs/grcov.config.yml
          coveralls-token: ${{ secrets.COVERALLS_TOKEN }}
      - name: Upload coverage to Codecov  
        uses: codecov/codecov-action@v1
        with:
          token: ${{ secrets.CODECOV_TOKEN }}
          file: ${{ steps.coverage.outputs.report }}
          fail_ci_if_error: true

pr from same repository branch is ok.

this is the full report: https://github.com/rxRust/rxRust/pull/59/checks

Install grcov from binary?

Install grcov from cargo install command cost a lot of time.
Is there any possible to install grcov from binary assets of mozilla/grcov releases.

Example:

• https://github.com/mozilla/grcov/releases/download/v0.5.5/grcov-linux-x86_64.tar.bz2
• https://github.com/mozilla/grcov/releases/download/v0.5.5/grcov-osx-x86_64.tar.bz2
• https://github.com/mozilla/grcov/releases/download/v0.5.5/grcov-win-x86_64.tar.bz2

crate with same name as repository ignored

Description

I have a repository with a workspace containing 4 crates.

• Repository named fantoch: https://github.com/vitorenesduarte/fantoch
• Crates:
  1. fantoch
  2. fantoch_ps
  3. fantoch_timing
  4. fantoch_bote

Expected behavior

The 4 crates should appear in the coverage report but 1 is ignored. The crate that is ignored is the first one, fantoch, which has the same name as the repository.

To make sure that's not a coincidence, I've created a repository named test with the same content as fantoch and now all the 4 crates appear in the coverage report.

• coveralls for fantoch: https://coveralls.io/github/vitorenesduarte/fantoch
• coveralls for test: https://coveralls.io/github/vitorenesduarte/test

grcov.yml

ignore-not-existing: true
output-type: lcov
output-path: ./lcov.info
ignore:
  - "/*"
  - "fantoch_ps/src/bin/*"
  - "fantoch_bote/src/main.rs"

Further considerations

I have also tried 0.2-proto in this branch (it was suggested in #51 for a different issue), but the problem still occurs.

While typing I started wondering if the issue is in coverallsapp/github-action@master and not in this action.

Enable new Rust code coverage instrumentation

grcov merged mozilla/grcov#509 (comment) - it would be great to have access to it from the action.

Clarify handling proc-macro panicking

The RUSTFLAGS provided in the README does not work with proc-macro crates because compiling procedural macros requires the panic_unwind runtime. We would get this error instead:

error: the linked panic runtime `panic_unwind` is not compiled with this crate's panic strategy `abort`

It would be useful to document in the README what is suggested to be done in the case of workspaces with procedural macros. For example, are users advised to override the panic strategy through a certain cfg filter in Cargo.toml? Or are there other suggested tricks?