octocov is a toolkit for collecting code metrics (code coverage, code to test ratio, test execution time and your own custom metrics).
Key features of octocov are:
- Useful both as a CI tool and as a CLI tool
- Support multiple coverage report formats.
- Support multiple code metrics.
- Support for even generating coverage report badges.
- Have a mechanism to aggregate reports from multiple repositories.
GitHub Actions for octocov is here !!
First, run test with coverage report output.
For example, in case of Go language, add -coverprofile=coverage.out option as follows
$ go test ./... -coverprofile=coverage.outAnd generete .octocov.yml to your repository.
$ octocov init
.octocov.yml is generatedAnd set up a workflow file as follows and run octocov on GitHub Actions.
# .github/workflows/ci.yml
name: Test
on:
pull_request:
jobs:
test:
runs-on: ubuntu-latest
steps:
-
uses: actions/checkout@v3
-
uses: actions/setup-go@v4
with:
go-version-file: go.mod
-
name: Run tests with coverage report output
run: go test ./... -coverprofile=coverage.out
-
uses: k1LoW/octocov-action@v0Then, octocov comment the report of the code metrics to the pull request.
It is also possible to add reports to GitHub Actions Job Summaries by editing .octocov.yml.
It can also be inserted into the body of a pull request.
Note that only pull requests from the same repository can be commented on (Reporting to GitHub Actions Job Summaries is permitted). This is because the workflow token of a forked pull request does not have write permission.
octocov acts as a code metrics viewer on the terminal.
For example, in case of Go language, add -coverprofile=coverage.out option as follows
$ go test ./... -coverprofile=coverage.outAnd run octocov ls-files , octocov view [FILE...] and octocov diff [REPORT_A] [REPORT_B]
By setting comment:, comment the reports to pull request.
# .octocov.yml
comment:
hideFooterLink: false # hide octocov linkoctocov checks for "Code Coverage" by default. If it is running on GitHub Actions, it will also measure "Test Execution Time".
If you want to measure "Code to Test Ratio", set codeToTestRatio:.
comment:
codeToTestRatio:
code:
- '**/*.go'
- '!**/*_test.go'
test:
- '**/*_test.go'By setting report: ( report.path: or report.datastores ) and diff: ( diff.path: or diff.datastores ) additionally, it is possible to show differences from previous reports as well.
comment:
report:
datastores:
- artifact://${GITHUB_REPOSITORY}
diff:
datastores:
- artifact://${GITHUB_REPOSITORY}
By setting coverage.acceptable:, the condition of acceptable coverage is specified.
If this condition is not met, the command will exit with exit status 1.
# .octocov.yml
coverage:
acceptable: 60%$ octocov
Error: code coverage is 54.9%. the condition in the `coverage.acceptable:` section is not met (`60%`)By setting codeToTestRatio.acceptable:, the condition of acceptable "Code to Test Ratio" is specified.
If this condition is not met, the command will exit with exit status 1.
# .octocov.yml
codeToTestRatio:
acceptable: 1:1.2
code:
- '**/*.go'
- '!**/*_test.go'
test:
- '**/*_test.go'$ octocov
Error: code to test ratio is 1:1.1, the condition in the `codeToTestRatio.acceptable:` section is not met (`1:1.2`)By setting testExecutionTime.acceptable:, the condition of acceptable "Test Execution Time" is specified (on GitHub Actions only) .
If this condition is not met, the command will exit with exit status 1.
# .octocov.yml
testExecutionTime:
acceptable: 1 min$ octocov
Error: test execution time is 1m15s, the condition in the `testExecutionTime.acceptable:` section is not met (`1 min`)By setting *.badge.path:, generate badges self.
# .octocov.yml
coverage:
badge:
path: docs/coverage.svg# .octocov.yml
codeToTestRatio:
badge:
path: docs/ratio.svg# .octocov.yml
testExecutionTime:
badge:
path: docs/time.svgYou can display the coverage badge without external communication by setting a link to this badge image in README.md, etc.
# mytool
  
By setting push:, git push report badges self.
# .octocov.yml
coverage:
badge:
path: docs/coverage.svg
push:By setting report:, store the reports to datastores and local path.
# .octocov.yml
report:
datastores:
- github://owner/coverages/reports
- s3://bucket/reports# .octocov.yml
report:
path: path/to/report.json- GitHub repository
- GitHub Actions Artifacts
- Amazon S3
- Google Cloud Storage (GCS)
- BigQuery
- Local
By enabling central:, octocov acts as a central repository for collecting reports ( example ).
# .octocov.yml for central mode
central:
root: . # root directory or index file path of collected coverage reports pages. default: .
reports:
datastores:
- bq://my-project/my-dataset/reports # datastore paths (URLs) where reports are stored. default: local://reports
badges:
datastores:
- local://badges # directory where badges are generated.
push: # enable self git push- GitHub repository
- GitHub Actions Artifacts
- Amazon S3
- Google Cloud Storage (GCS)
- BigQuery
- Local
octocov ls-files command can be used to list files logged in code coverage report.
octocov view (alias: octocov cat) command can be used to view the file coverage report.
The name of the repository.
It should be in the format owner/repo.
By default, the value of the environment variable GITHUB_REPOSITORY is set.
In case of monorepo, code metrics can be reported to datastore separately by specifying owner/repo/project-a or owner/repo@project-a.
repository: k1LoW/octocovTimeout for octocov execution. (default: 30sec)
timeout: 5minConfiguration for code coverage.
coverage.path: has been deprecated. Please use coverage.paths: instead.
The path to the coverage report file.
If no path is specified, the default path for each coverage format will be scanned.
coverage:
paths:
- tests/coverage.xmlExclude files from the coverage report.
coverage:
exclude:
- 'cmd/*.ts'
- 'proto/**/*.pb.ts'acceptable coverage condition.
coverage:
acceptable: 60%coverage:
acceptable: current >= 60% && diff >= 0.5%The variables that can be used are as follows.
| value | description |
|---|---|
current |
Current code metrics value |
prev |
Previous value. This value is taken from diff.datastores:. |
diff |
The result of current - prev |
It is also possible to omit the expression as follows
| Omitted expression | Expanded expression |
|---|---|
60% |
current >= 60% |
> 60% |
current > 60% |
Set this if want to generate the badge self.
The path to the badge.
coverage:
badge:
path: docs/coverage.svgConditions for measuring code coverage.
coverage:
if: is_default_branchConfiguration for code to test ratio.
Files to count.
codeToTestRatio:
code: # files to count as "Code"
- '**/*.go'
- '!**/*_test.go'
test: # files to count as "Test"
- '**/*_test.go'acceptable ratio condition.
codeToTestRatio:
acceptable: 1:1.2codeToTestRatio:
acceptable: current >= 1.2 && diff >= 0.0The variables that can be used are as follows.
| value | description |
|---|---|
current |
Current code metrics value |
prev |
Previous value. This value is taken from diff.datastores:. |
diff |
The result of current - prev |
It is also possible to omit the expression as follows
| Omitted expression | Expanded expression |
|---|---|
1:1.2 |
current >= 1.2 |
> 1:1.2 |
current > 1.2 |
Set this if want to generate the badge self.
The path to the badge.
codeToTestRatio:
badge:
path: docs/ratio.svgConditions for measuring code to test ratio.
codeToTestRatio:
if: is_default_branchConfiguration for test execution time.
acceptable time condition.
testExecutionTime:
acceptable: 1mintestExecutionTime:
acceptable: current <= 1min && diff <= 1secThe variables that can be used are as follows.
| value | description |
|---|---|
current |
Current code metrics value |
prev |
Previous value. This value is taken from diff.datastores:. |
diff |
The result of current - prev |
It is also possible to omit the expression as follows
| Omitted expression | Expanded expression |
|---|---|
1min |
current <= 1min |
< 1min |
current < 1min |
The name of the step to measure the execution time.
testExecutionTime:
steps:
- Run test
- Run slow testIf not specified, the step where the coverage report file is generated is used as the measurement target.
Set this if want to generate the badge self.
The path to the badge.
testExecutionTime:
badge:
path: docs/time.svgConditions for measuring test execution time.
testExecutionTime:
if: is_pull_requestConfiguration for git push files self.
Conditions for pushing files.
# .octocov.yml
push:
if: is_default_branchThe variables available in the if section are here.
message for commit.
# .octocov.yml
push:
message: Update by octocov [skip ci]Set this if want to comment report to pull request
Hide footer octocov link.
comment:
hideFooterLink: trueDelete previous code metrics report comments instead of hiding them
comment:
deletePrevious: trueConditions for commenting report.
# .octocov.yml
comment:
if: is_pull_requestThe variables available in the if section are here.
Set this if want to add report to job summary page.
Hide footer octocov link.
summary:
hideFooterLink: trueConditions for adding report to job summary page.
# .octocov.yml
summary:
if: trueThe variables available in the if section are here.
Set this if want to insert report to body of pull request.
Hide footer octocov link.
body:
hideFooterLink: trueConditions for inserting report body of pull request.
# .octocov.yml
body:
if: is_pull_requestThe variables available in the if section are here.
Configuration for comparing reports.
Path of the report to compare.
diff:
path: path/to/coverage.ymldiff:
path: path/to/report.jsonDatastores where the report to be compared is stored.
diff:
datastores:
- local://.octocov # Use .octocov/owner/repo/report.json
- s3://my-bucket/reports # Use s3://my-bucket/reports/owner/repo/report.jsonConditions for comparing reports
# .octocov.yml
report:
if: is_pull_request
path: path/to/report.jsonThe variables available in the if section are here.
Configuration for reporting to datastores.
Path to save the report.
report:
path: path/to/report.jsonDatastores where the reports are stored.
report:
datastores:
- github://owner/coverages/reports
- s3://bucket/reportsUse github:// scheme.
github://[owner]/[repo]@[branch]/[prefix]
Required environment variables:
GITHUB_TOKENorOCTOCOV_GITHUB_TOKENGITHUB_REPOSITORYorOCTOCOV_GITHUB_REPOSITORYGITHUB_API_URLorOCTOCOV_GITHUB_API_URL(optional)
Use artifact:// or artifacts:// scheme.
artifact://[owner]/[repo]/[artifactName]
artifact://[owner]/[repo]/[artifactName]artifact://[owner]/[repo]( default artifactName:octocov-report)
Note that reporting to the artifact can only be sent from the GitHub Actions of the same repository.
Required environment variables:
GITHUB_TOKENorOCTOCOV_GITHUB_TOKENGITHUB_REPOSITORYorOCTOCOV_GITHUB_REPOSITORYGITHUB_API_URLorOCTOCOV_GITHUB_API_URL(optional)
Use s3:// scheme.
s3://[bucket]/[prefix]
Required permission:
s3:PutObject
Required environment variables:
AWS_ACCESS_KEY_IDorOCTOCOV_AWS_ACCESS_KEY_IDAWS_SECRET_ACCESS_KEYorOCTOCOV_AWS_SECRET_ACCESS_KEYAWS_SESSION_TOKENorOCTOCOV_AWS_SESSION_TOKEN(optional)
Use gs:// scheme.
gs://[bucket]/[prefix]
Required permission:
storage.objects.createstorage.objects.delete
Required environment variables:
GOOGLE_APPLICATION_CREDENTIALSorGOOGLE_APPLICATION_CREDENTIALS_JSONorOCTOCOV_GOOGLE_APPLICATION_CREDENTIALSorOCTOCOV_GOOGLE_APPLICATION_CREDENTIALS_JSON
Use bq:// scheme.
bq://[project ID]/[dataset ID]/[table]
Required permission:
bigquery.datasets.getbigquery.tables.getbigquery.tables.updateData
Required environment variables:
GOOGLE_APPLICATION_CREDENTIALSorGOOGLE_APPLICATION_CREDENTIALS_JSONorOCTOCOV_GOOGLE_APPLICATION_CREDENTIALSorOCTOCOV_GOOGLE_APPLICATION_CREDENTIALS_JSON
Datastore schema:
If you want to create a table, execute the following command ( require bigquery.datasets.create ).
$ octocov migrate-bq-tableNote: Only works with
report.datastoresorcentral.reReport.datastores
Use mackerel:// or mkr:// scheme.
mackerel://[Service Name]
Required permission:
readwrite
Required environment variables:
MACKEREL_API_KEYorOCTOCOV_MACKEREL_API_KEY
Use local:// or file:// scheme.
local://[path]
Example:
If the absolute path of .octocov.yml is /path/to/.octocov.yml
local://reports.../path/to/reportsdirectorylocal://./reports.../path/to/reportsdirectorylocal://../reports.../path/reportsdirectorylocal:///reports.../reportsdirectory.
Conditions for storing a report.
# .octocov.yml
report:
if: env.GITHUB_REF == 'refs/heads/main'
datastores:
- github://owner/coverages/reportsThe variables available in the if section are here.
Note: It supports expr-lang/expr expressions.
The variables available in the if section are as follows
| Variable name | Type | Description |
|---|---|---|
year |
int |
Year of current time (UTC) |
month |
int |
Month of current time (UTC) |
day |
int |
Day of current time (UTC) |
hour |
int |
Hour of current time (UTC) |
weekday |
int |
Weekday of current time (UTC) (Sunday = 0, ...) |
github.event_name |
string |
Event name of GitHub Actions ( ex. issues, pull_request ) |
github.event |
object |
Detailed data for each event of GitHub Actions (ex. github.event.action, github.event.label.name ) |
env.<env_name> |
string |
The value of a specific environment variable |
is_pull_request |
boolean |
Whether the job is related to an pull request (ex. a job fired by on.push will be true if it is related to a pull request) |
is_draft |
boolean |
Whether the job is related to a draft pull request |
labels |
array |
Labels that are set for the pull request |
is_default_branch |
boolean |
Whether the job is related to default branch of repository |
Note: When central mode is enabled, other functions are automatically turned off.
The root directory or index file ( index file example ) path of collected coverage reports pages. default: .
central:
root: path/toDatastore paths (URLs) where reports are stored. default: local://reports
central:
reports:
datastores:
- local://reports
- gs://my-gcs-bucket/reportsWhen using GitHub Actions Artifacts as a datastore, perform badge generation via on.schedule.
# .octocov.yml
report:
datastores:
- artifact://${GITHUB_REPOSITORY}# .octocov.yml for central repo
central:
reports:
datastores:
- artifact://owner/repo
- artifact://owner/other-repo
- artifact://owner/another-repo
[...]
push:Code metrics and badges of my open source projects using octocov central mode is here.
When using the central repository as a datastore, perform badge generation via on.push.
# .octocov.yml
report:
datastores:
- github://owner/central-repo/reports# .octocov.yml for central repo
central:
reports:
datastores:
- github://owner/central-repo/reports
push:or
# .octocov.yml for central repo
central:
reports:
datastores:
- local://reports
push:When using the S3 bucket as a datastore, perform badge generation via on.schedule.
# .octocov.yml
report:
datastores:
- s3://my-s3-bucket/reports# .octocov.yml for central repo
central:
reports:
datastores:
- s3://my-s3-bucket/reports
push:Required permission (Central Repo):
s3:GetObjects3:ListObject
Required environment variables (Central Repo):
AWS_ACCESS_KEY_IDorOCTOCOV_AWS_ACCESS_KEY_IDAWS_SECRET_ACCESS_KEYorOCTOCOV_AWS_SECRET_ACCESS_KEYAWS_SESSION_TOKENorOCTOCOV_AWS_SESSION_TOKEN(optional)
When using the GCS bucket as a datastore, perform badge generation via on.schedule.
# .octocov.yml
report:
datastores:
- gs://my-gcs-bucket/reports# .octocov.yml for central repo
central:
reports:
datastores:
- gs://my-gcs-bucket/reports
push:Required permission (Central Repo):
storage.objects.getstorage.objects.liststorage.buckets.get
Required environment variables (Central Repo):
GOOGLE_APPLICATION_CREDENTIALSorGOOGLE_APPLICATION_CREDENTIALS_JSONorOCTOCOV_GOOGLE_APPLICATION_CREDENTIALSorOCTOCOV_GOOGLE_APPLICATION_CREDENTIALS_JSON
When using the BigQuery table as a datastore, perform badge generation via on.schedule.
# .octocov.yml
report:
datastores:
- bq://my-project/my-dataset/reports# .octocov.yml for central repo
central:
reports:
datastores:
- bq://my-project/my-dataset/reports
push:Required permission (Central Repo):
bigquery.jobs.createbigquery.tables.getData
Required environment variables (Central Repo):
GOOGLE_APPLICATION_CREDENTIALSorGOOGLE_APPLICATION_CREDENTIALS_JSONorOCTOCOV_GOOGLE_APPLICATION_CREDENTIALSorOCTOCOV_GOOGLE_APPLICATION_CREDENTIALS_JSON
Datastore paths (URLs) where badges are generated. default: local://badges
central:
badges:
datastores:
- local://badges
- s3://my-s3-buckets/badgesConfiguration for git push index file and badges self.
Conditions for central mode.
# .octocov.yml
central:
if: env.GITHUB_REF == 'refs/heads/main'
reports:
datastores:
- s3://my-s3-bucket/reportsThe variables available in the if section are here.
Store collected reports in yet another datastores.
Conditions for re storing reports.
Datastores where the reports are re-stored.
octocov supports multiple coverage report formats.
And octocov searches for the default path for each format.
If you want to specify the path of the report file, set coverage.path
coverage:
paths:
- /path/to/coverage.txtDefault path: coverage.out
Default path: coverage/lcov.info
Support SF DA only
Default path: coverage/.resultset.json
Default path: coverage.xml
Default path: coverage.xml
Default path: build/reports/jacoco/test/jacocoTestReport.xml
- Code Coverage
- Code to Test Ratio
- Test Execution Time (on GitHub Actions only)
octocov accepts custom metrics in addition to the three supporting metrics.
Specify the path to the custom metrics JSON file in an environment variable prefixed with OCTOCOV_CUSTOM_METRICS_ to collect the code metrics at the same time.
The JSON schema for custom metrics can be found here.
If there are multiple custom metrics JSON files, specify each file path in a separate environment variable (example here) or combine the JSONs that satisfy the JSON schema into an array.
octocov detect pull request number following order.
- Get pull request number from
GITHUB_PULL_REQUEST_NUMBERorOCTOCOV_GITHUB_PULL_REQUEST_NUMBER. - Get pull request number from
GITHUB_REF( e.g.refs/pull/1/merge). - Get branch name from
GITHUB_REF( e.g.refs/heads/branch/branch/name) and detect pull request number using GitHub API.
If an environment variable with prefix OCTOCOV_ is set, it is used as an unprefixed environment variable in octocov.
For example, if OCTOCOV_GITHUB_REF is set, it is handled as GITHUB_REF in octocov.
This feature allows environment variables that cannot normally be overridden to be changed on octocov.
deb:
$ export OCTOCOV_VERSION=X.X.X
$ curl -o octocov.deb -L https://github.com/k1LoW/octocov/releases/download/v$OCTOCOV_VERSION/octocov_$OCTOCOV_VERSION-1_amd64.deb
$ dpkg -i octocov.debRPM:
$ export OCTOCOV_VERSION=X.X.X
$ yum install https://github.com/k1LoW/octocov/releases/download/v$OCTOCOV_VERSION/octocov_$OCTOCOV_VERSION-1_amd64.rpmapk:
$ export OCTOCOV_VERSION=X.X.X
$ curl -o octocov.apk -L https://github.com/k1LoW/octocov/releases/download/v$OCTOCOV_VERSION/octocov_$OCTOCOV_VERSION-1_amd64.apk
$ apk add octocov.apkhomebrew tap:
$ brew install k1LoW/tap/octocovaqua:
$ aqua g -i k1LoW/octocovmanually:
Download binary from releases page
go install:
$ go install github.com/k1LoW/octocov@latestdocker:
$ docker pull ghcr.io/k1low/octocov:latest![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "github-actions[bot]")
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent