gcc 8 creates gcno files that are unreadable by lcov

[linux-test-project]

gcc 8 creates gcno files that are unreadable by lcov

[Zitrax]

@oberpar How does it look regarding a release? 1.13 is over 2 years old now and newer distros are defaulting to gcc 8 (although I think for example ubuntu has manually included this patch on top of 1.13).

[marxin]

As being developer of GCOV in GCC, I would recommend to use intermediate format (-i) of gcov tool instead of reading of *.gc{da,no} files. Thanks for understanding.

[oberpar]

@marxin: Excellent, thank you! This should provide a path for LCOV to no longer require parsing of binary .gcno files in the future. Intermediate file support is now at the top of my list for lcov enhancements to work on, though at this time I cannot yet provide a time frame for when it will be done.

[oberpar]

It makes sense to release a new version of LCOV including this and other fixes. I'll have to see when I get the time to create it.

[marxin]

The GCC 8.1 has been released. A new version of LCOV would be great.

[marxin]

@marxin: GCOV's intermediate format looks promising. Can it be used to generate zero coverage data for .gcno files where no .gcda file is present? This is relevant for example for projects which consist of multiple executables, to get a total coverage view independent of the number of executables actually run during a test.

@oberpar This works now:

$ gcc --version
gcc (SUSE Linux) 7.3.1 20180323 [gcc-7-branch revision 258812]
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
$ gcc empty.c --coverage && ./a.out ; rm empty.gcda && gcov empty.c && cat empty.c.gcov
empty.gcda:cannot open data file, assuming not executed
File 'empty.c'
Lines executed:0.00% of 10
Creating 'empty.c.gcov'

        -:    0:Source:empty.c
        -:    0:Graph:empty.gcno
        -:    0:Data:-
        -:    0:Runs:0
        -:    0:Programs:0
    #####:    1:int foo(int argc)
        -:    2:{
    #####:    3:  if (argc % 2)
    #####:    4:    return 1;
        -:    5:  else
    #####:    6:    return 2;
        -:    7:}
        -:    8:  
        -:    9:int
    #####:   10:main(int argc, char **argv)
        -:   11:{
    #####:   12:  if (argc == 123)
    #####:   13:    __builtin_abort ();
    #####:   14:  else if(argc != 1111)
    #####:   15:    return foo (argc);
        -:   16:
    #####:   17:  return 0;
        -:   18:}
$ gcc empty.c --coverage && ./a.out ; rm empty.gcda && gcov -i empty.c && cat empty.c.gcov
empty.gcda:cannot open data file, assuming not executed
File 'empty.c'
Lines executed:0.00% of 10
Creating 'empty.c.gcov'

file:empty.c
function:1,0,foo
function:10,0,main
lcount:1,0
lcount:3,0
lcount:4,0
lcount:6,0
lcount:10,0
lcount:12,0
lcount:13,0
lcount:14,0
lcount:15,0
lcount:17,0

[oberpar]

Thanks for reporting. The issue actually appears to occur for any kind of source code compiled with GCC 8. I've added commit a5dd952 to address the changes introduced with GCC 8. Could you verify if this fixes the problem you are seeing?

[berrange]

I confirm that works for the simple demo program I was given by the original bug reporter. So that's good enough for me until someone complains about something else :-)

[oberpar]

@marxin: GCOV's intermediate format looks promising. Can it be used to generate zero coverage data for .gcno files where no .gcda file is present? This is relevant for example for projects which consist of multiple executables, to get a total coverage view independent of the number of executables actually run during a test.

Some additional thoughts/observations:

• Adding the base directory path (current directory at the time of compilation) as a separate field to the output would be of massive value as it would make it easier to recreate absolute source code paths from relative paths.
• An option to have output on standard output instead of file would be nice (no more file operation/conflict in case of parallel processing)
• If the intermediate file format is intended for consumption by programs, additions to the format should be done in a backwards compatible way (counter example: addition of end_line_number to function data for example would have broken parsing relying on 3 CSV fields per line)
• Not sure if this occurs on upstream GCC, but I just noticed that running gcov -iab causes a core dump on GCC 7.2 (7.2.1 20170915 (Red Hat 7.2.1-2)). Using gcov -i alone on GCC 8 throws an assertion (8.0.1 20180222 (Red Hat 8.0.1-0.16)).

[marxin]

Thanks for the ideas, I've just created: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84846 which I will implement in next GCC release (9.1).

Not sure if this occurs on upstream GCC, but I just noticed that running gcov -iab causes a core dump on GCC 7.2 (7.2.1 20170915 (Red Hat 7.2.1-2)). Using gcov -i alone on GCC 8 throws an assertion (8.0.1 20180222 (Red Hat 8.0.1-0.16)).

Can you please provide an example of a source file that causes that?

[oberpar]

Excellent! Looking forward to those gcov improvements.
Regarding the gcc 7 segmentation fault: it appears to occur with any kind of source file:

$ gcc --version
gcc (GCC) 7.2.1 20170915 (Red Hat 7.2.1-2)
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ echo "int main() { return 0; }" > t.c
$ gcc t.c -o t --coverage
$ ./t
$ gcov -iab t.c
File 't.c'
Lines executed:100.00% of 1
No branches
No calls
Creating 't.c.gcov'
Segmentation fault (core dumped)

Edit: this is on s390x (big-endian) if that makes a difference.

[marxin]

Thanks again, useful feedback. I've just created:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84879

[PetterS]

This is great. Thanks! Will there be a new version released with this fix?

[JKorbelRA]

@oberpar, you've saved my day, thanks!

[marxin]

Adding the base directory path (current directory at the time of compilation) as a separate field to the output would be of massive value as it would make it easier to recreate absolute source code paths from relative paths.

Implemented in current trunk:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84846

An option to have output on standard output instead of file would be nice (no more file operation/conflict in case of parallel processing)

Likewise.

If the intermediate file format is intended for consumption by programs, additions to the format should be done in a backwards compatible way (counter example: addition of end_line_number to function data for example would have broken parsing relying on 3 CSV fields per line)

This should be done at consumer's side. Now we provide version which tools should follow and adjust parsing in a corresponding way.

Thus all feature requests mentioned in this issue are done at GCC's side.

[marxin]

Don't hurry, you have the whole year before GCC 9.1 will be released ;) If you see any feature requests that will be handy, don't hesitate and create a PR:
https://gcc.gnu.org/bugzilla/

[iirina]

@oberpar Is this the right issue to follow for progress on lcov supporting gcov intermediate file format?

[oberpar]

@iirina No, I don't think so. This issue is closed and it was about a different problem that only touched on the subject of intermediate files as a side note. That said, there has not been any progress on intermediate file support in lcov so far on my side. Code contributions in that direction are of course always welcome.

[Farwaykorse]

With @Zitrax.
Lcov works only with GCC-8 using the Ubuntu version: 1.13-4, but only available on cosmic (18.10) and disco (devel). (packages.ubuntu.com)

[oberpar]

@oberpar How does it look regarding a release? 1.13 is over 2 years old now and newer distros are defaulting to gcc 8 (although I think for example ubuntu has manually included this patch on top of 1.13).

I'm planning for a release in February (2019). Ideally this will include the intermediate file format support that I've started working on.

[marxin]

I would like to mention that as written in #43, GCOV intermediate format will change in GCC 9.1 into a JSON-based format. I would consider implementing the new one and use current output for older versions of GCC.

[oberpar]

I would like to mention that as written in #43, GCOV intermediate format will change in GCC 9.1 into a JSON-based format. I would consider implementing the new one and use current output for older versions of GCC.

Yes, I am aware of that. I checked all intermediate file format implementations in gcov starting from GCC 5 to 9 and found that even pre-9, the fact that gcov can be run in any directory when using the intermediate format can be leveraged to implement parallel lcov processing even with older versions of GCC. I'm currently planning to start with support for the old format, and add the GCC 9 format after that.

[marxin]

Yes, I am aware of that. I checked all intermediate file format implementations in gcov starting from GCC 5 to 9 and found that even pre-9, the fact that gcov can be run in any directory when using the intermediate format can be leveraged to implement parallel lcov processing even with older versions of GCC. I'm currently planning to start with support for the old format, and add the GCC 9 format after that.

Great, maybe you would be interested in newly created wrapper that can help you:
https://github.com/RPGillespie6/fastcov