markusn/coveralls-erl

Erlang module to convert and send cover data to coveralls.io (or similar). Available as a hex package on https://hex.pm/packages/coveralls.

coveralls-erl

Erlang module to convert and send cover data to coveralls. Available as a hex package on https://hex.pm/packages/coveralls.

Example usage: rebar3 and Travis CI

In order to use coveralls-erl + Travis CI in your project you will need to add the following lines to your rebar.config.script:

case os:getenv("TRAVIS") of
  "true" ->
    JobId   = os:getenv("TRAVIS_JOB_ID"),
    lists:keystore(coveralls_service_job_id, 1, CONFIG, {coveralls_service_job_id, JobId});
  _ ->
    CONFIG
end.

This will ensure that the rebar coveralls plugin will have access to the needed JobId and that the plugin is only run from Travis CI.

You will also need to add the following lines to your rebar.config:

{plugins                , [coveralls]}. % use hex package
{cover_enabled          , true}.
{cover_export_enabled   , true}.
{coveralls_coverdata    , "_build/test/cover/eunit.coverdata"}. % or a string with wildcards or a list of files
{coveralls_service_name , "travis-ci"}. % use "travis-pro" when using with travis-ci.com

When using with travis-ci.com coveralls repo token also has to be added as {coveralls_repo_token, "token_goes_here"}

These changes will add coveralls-erl as a dependency, tell rebar3 where to find the plugin, make sure that the coverage data is produced and exported and configure coveralls-erl to use this data and the service travis-ci.

And you send the coverdata to coveralls by issuing: rebar3 as test coveralls send

Note: If you have dependencies specific to the test profile, or if you only add the coveralls dependency or any of its' configuration variables to the test profile you need to run coveralls using: rebar3 as test coveralls send

Example: rebar3 and CircleCI

Example rebar.config.script:

case {os:getenv("CIRCLECI"), os:getenv("COVERALLS_REPO_TOKEN")} of
    {"true", Token} when is_list(Token) ->
        JobId   = os:getenv("CIRCLE_BUILD_NUM"),
        CONFIG1 = lists:keystore(coveralls_service_job_id, 1, CONFIG, {coveralls_service_job_id, JobId}),
        lists:keystore(coveralls_repo_token, 1, CONFIG1, {coveralls_repo_token, Token});
    _ ->
        CONFIG
end.

Example rebar.config:

{plugins                , [coveralls]}. % use hex package
{cover_enabled          , true}.
{cover_export_enabled   , true}.
{coveralls_coverdata    , "_build/test/cover/ct.coverdata"}.
{coveralls_service_name , "circle-ci"}.

Note that you'll need to set COVERALLS_REPO_TOKEN in your CircleCI environment variables!

Example usage: rebar3 and GitHub Actions

In order to use coveralls-erl + GitHub Actions in your project, you will need to add the following lines to your rebar.config.script:

case {os:getenv("GITHUB_ACTIONS"), os:getenv("GITHUB_TOKEN")} of
  {"true", Token} when is_list(Token) ->
    CONFIG1 = [{coveralls_repo_token, Token},
               {coveralls_service_job_id, os:getenv("GITHUB_RUN_ID")},
               {coveralls_commit_sha, os:getenv("GITHUB_SHA")},
               {coveralls_service_number, os:getenv("GITHUB_RUN_NUMBER")} | CONFIG],
    case os:getenv("GITHUB_EVENT_NAME") =:= "pull_request"
        andalso string:tokens(os:getenv("GITHUB_REF"), "/") of
        [_, "pull", PRNO, _] ->
            [{coveralls_service_pull_request, PRNO} | CONFIG1];
        _ ->
            CONFIG1
    end;
  _ ->
    CONFIG
end.

This will ensure that the rebar coveralls plugin will have access to the needed JobId and that the plugin is only run from GitHub Actions.

You will also need to add the following lines to your rebar.config:

{plugins                , [coveralls]}. % use hex package
{cover_enabled          , true}.
{cover_export_enabled   , true}.
{coveralls_coverdata    , "_build/test/cover/eunit.coverdata"}. % or a string with wildcards or a list of files
{coveralls_service_name , "github"}.

These changes will add coveralls-erl as a dependency, tell rebar3 where to find the plugin, make sure that the coverage data is produced and exported and configure coveralls-erl to use this data and the service github.

And you send the coverdata to coveralls by adding a step like:

- name: Coveralls
  env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
  run: rebar3 as test coveralls send

Other available GitHub Actions Environment Variables are available here

Optional settings

The plugin also support the coveralls_service_pull_request and coveralls_parallel settings. See the Coveralls documentation for the meaning of those.

Author

Markus Ekholm (markus at botten dot org).

License

3-clause BSD. For details see COPYING.

coveralls-erl's Issues

source_digest not part of the source file object

As per coveralls api source_digest is required

{
  "service_job_id": "1234567890",
  "service_name": "travis-ci",
  "source_files": [
    {
      "name": "example.rb",
      "source_digest": "asdfasdf1234asfasdf2345",
      "coverage": [null, 1, null]
    },
    {
      "name": "lib/two.rb",
      "source_digest": "asdf1234asdfsdfggsfgd9423",
      "coverage": [null, 1, 0, null]
    }
  ]
}

but it seems like source_digest is not send with the json_file

coveralls-erl/src/coveralls.erl

Lines 198 to 203 in 749eb63

	Str =
	"{~n"
	"\"name\": \"~s\",~n"
	"\"source\": \"~s\",~n"
	"\"coverage\": ~p~n"
	"}",

rebar3 reports

I have a project and with rebar 2, the reports are shown on coveralls.io, while with rebar3 they don't. Is there something I am missing? (I followed the described usage examples to the point).

Specify .coverdata files with a wildcard string

In my project (esl/MongooseIM) I have 4 different coverdata files. It would be easier for me if I were able to specify all of them by a wildcard (see filelib:wildcard/1) . If this is sth you would be interested in, I can submit a PR with this functionality.

Fails on rebar3 "need to specify coveralls_* and cover_export_enabled in rebar.config" but already set i rebar.config

rebar.config:
{require_min_otp_vsn, "17"}.
{plugins , [{coveralls, {git, "https://github.com/markusn/coveralls-erl", "master"}}]}.
{cover_enabled, true}.
{cover_print_enabled, true}.
{cover_export_enabled, true}.
{cover_opts, [verbose]}.
{coveralls_coverdata, "log/all.coverdata"}.
{coveralls_service_name , "travis-ci"}.
{ct_opts, [{logdir, "log/ct"},
{dir, "./test/"},
{include, ["include"]},
{suites, "test", all},
{cover,"test/replication.coverspec"},
{suite, [
replication_SUITE
]}
]}.

replication.coverspec:
{import, ".eunit/cover.coverdata"}.
{export, "logs/all.coverdata"}.
{level, details}.

env DEBUG=1 rebar3 coverall send
===> Evaluating config script "rebar.config.script"
===> Evaluating config script "/opt/test/ReplicationEngine/_build/default/plugins/coveralls/rebar.config.script"
===> Evaluating config script "/opt/test/ReplicationEngine/build/default/plugins/coveralls/rebar.config.script"
===> Expanded command sequence to be run: []
===> Expanded command sequence to be run: [{default,app_discovery},
{coveralls,send}]
===> Evaluating config script "/opt/test/ReplicationEngine/rebar.config.script"
===> Running coveralls...
===> need to specify coveralls* and cover_export_enabled in rebar.config
make: *** [coverage-report] Error 1

Can coverallsget the locations of coverdata from coverspec file?

Hi,

The coverspec file already indicates the location of exported coverdata
We can read from the spec file if present. There is a sister project but no a rebar plugin (see link end of message)
Also each run of ct creates its own directory for logs and coverdata, can you enhance the tool to support that?
Ref:

• http://www.erlang.org/doc/apps/common_test/cover_chapter.html
• https://github.com/nifoc/ecoveralls

Bad appdir imports with non-test rebar3 profile

When coveralls is used in a non-test rebar3 profile, cover_paths is trying to add incorrect application directories. You can reproduce it by making coveralls to run in a rebar3 profile. Alternatively, you can use my project (requires python-dev, virtualenv in $PATH, and Erlang 17.0+):

1. Clone https://github.com/Motiejus/tchannel-erlang/tree/74ed3a12bdcc69cf08016dad71007dc2e5658080
2. make setup proper test cover coveralls
3. Observe the stack trace. With the diff to coveralls below, it is trying to add the following directory to the code path: AppDir: "/Users/motiejus/code/tchannel-erlang/_build/coveralls/lib/tchannel", which doesn't exist.

Failed travis-ci run (look at the bottom for make coveralls): https://travis-ci.org/Motiejus/tchannel-erlang/jobs/120550013

Patch to coveralls to reveal the added directory:

motiejus ~/code/tchannel-erlang/_checkouts/coveralls $ git diff
diff --git a/src/rebar3_coveralls.erl b/src/rebar3_coveralls.erl
index fecffb0..c813ebf 100644
--- a/src/rebar3_coveralls.erl
+++ b/src/rebar3_coveralls.erl
@@ -86,6 +86,7 @@ format_error(Reason) ->
 cover_paths(State) ->
   lists:foreach(fun(App) ->
                     AppDir = rebar_app_info:out_dir(App),
+                    io:format(user, "AppDir: ~p~n", [AppDir]),
                     true   = code:add_path(filename:join([AppDir, "ebin"])),
                     true   = code:add_path(filename:join([AppDir, "test"]))
                 end,

Stack trace (you can also see it in travis-ci):

===> Uncaught error in rebar_core. Run with DEBUG=1 to see stacktrace
===> Uncaught error: {badmatch,{error,bad_directory}}
===> Stack trace to the error location: [{rebar3_coveralls,
                                                 '-cover_paths/1-fun-0-',1,
                                                 [{file,
                                                   "/Users/motiejus/code/tchannel-erlang/_checkouts/coveralls/src/rebar3_coveralls.erl"},
                                                  {line,90}]},
                                                {lists,foreach,2,
                                                 [{file,"lists.erl"},
                                                  {line,1337}]},
                                                {rebar3_coveralls,
                                                 cover_paths,1,
                                                 [{file,
                                                   "/Users/motiejus/code/tchannel-erlang/_checkouts/coveralls/src/rebar3_coveralls.erl"},
                                                  {line,87}]},
                                                {rebar3_coveralls,do,1,
                                                 [{file,
                                                   "/Users/motiejus/code/tchannel-erlang/_checkouts/coveralls/src/rebar3_coveralls.erl"},
                                                  {line,68}]},
                                                {rebar_core,do,2,
                                                 [{file,
                                                   "/home/travis/build/rebar/rebar3/_build/default/lib/rebar/src/rebar_core.erl"},
                                                  {line,125}]},
                                                {rebar_prv_do,do_tasks,2,
                                                 [{file,
                                                   "/home/travis/build/rebar/rebar3/_build/default/lib/rebar/src/rebar_prv_do.erl"},
                                                  {line,68}]},
                                                {rebar_core,do,2,
                                                 [{file,
                                                   "/home/travis/build/rebar/rebar3/_build/default/lib/rebar/src/rebar_core.erl"},
                                                  {line,125}]},
                                                {rebar_prv_do,do_tasks,2,
                                                 [{file,
                                                   "/home/travis/build/rebar/rebar3/_build/default/lib/rebar/src/rebar_prv_do.erl"},
                                                  {line,68}]}]

Thank you for looking into this!

Is it compatible with Github Action?

Hi, @markusn Thanks for providing so convenient for coveralls. But I've only seen examples related to travis-ci. Does it work in GitHub Action?

any idea how to make this work with circle ci?

any idea how to make this work with circle ci?

Coveralls only updates PRs when used with GitHub actions

Hi,

I noticed that, using the latest version of the plugin (2.2.0) and the suggested configuration, Coveralls correctly receives all updates from GitHub, but that the "master" branch is not identified as such:

The question marks there correspond to pushes (merges) to master, which is set as default branch. Anyone else experiencing the same behaviour?

Rebar3 support

This plugin works only with rebar2, I think supporting rebar3 is good idea

rebar3 as test coveralls send

Hi, I'm trying this plugin, and I've found that rebar3 coveralls send fails in my project. I had to run rebar3 as test coveralls send for it to pick up the coverdata file. This does look like the most frequent case, I can send a PR updating the README if it makes sense.

Task for sending data

Plugin sends covedata after eunit or ct tasks but sometimes I want to merge coverdatas before send

It would look like rebar skip_deps=true eunit ct send-coveralls

What do you think about?

Using meck

Now coveralls-erl is supporting meck.
Maybe we could use this instead of self-made mocking tool?