tomtom-international/cpp-dependencies

Tool to check C++ #include dependencies (dependency graphs created in .dot format)

cpp-dependencies's Issues

Ubuntu 22.10 permission denied error with cpp-dependencies installed via snap

$pwd
/work/fg/next/flightgear
$ cpp-dependencies --stats
terminate called after throwing an instance of 'boost::filesystem::filesystem_error'
what(): boost::filesystem::directory_iterator::construct: Permission denied: "."
Aborted (core dumped)

PrintF not working correctly on mingw64

While compiling went smoothly, g++ threw lots of warnings saying that it had "unknown conversion type character"s in a lot of the printf statements. Here are the errors:

$ make
/c/msys64/mingw64/bin/g++.exe -c -Wall -Wextra -Wpedantic -o obj/main.o src/main.cpp -std=c++11 -O3
src/main.cpp: In lambda function:
src/main.cpp:91:75: warning: unknown conversion type character 'z' in format [-Wformat=]
                     components.size(), totalPublicLinks, totalPrivateLinks);
                                                                           ^
src/main.cpp:91:75: warning: unknown conversion type character 'z' in format [-Wformat=]
src/main.cpp:91:75: warning: unknown conversion type character 'z' in format [-Wformat=]
src/main.cpp:91:75: warning: too many arguments for format [-Wformat-extra-args]
src/main.cpp:91:75: warning: unknown conversion type character 'z' in format [-Wformat=]
src/main.cpp:91:75: warning: unknown conversion type character 'z' in format [-Wformat=]
src/main.cpp:91:75: warning: unknown conversion type character 'z' in format [-Wformat=]
src/main.cpp:91:75: warning: too many arguments for format [-Wformat-extra-args]
src/main.cpp:93:80: warning: unknown conversion type character 'z' in format [-Wformat=]
             fprintf(stderr, "Detected %zu nodes in cycles\n", NodesWithCycles());
                                                                                ^
src/main.cpp:93:80: warning: too many arguments for format [-Wformat-extra-args]
src/main.cpp:93:80: warning: unknown conversion type character 'z' in format [-Wformat=]
src/main.cpp:93:80: warning: too many arguments for format [-Wformat-extra-args]
src/main.cpp: In function 'int main(int, char**)':
src/main.cpp:275:72: warning: unknown conversion type character 'z' in format [-Wformat=]
             printf("Found %zu ambiguous includes\n\n", ambiguous.size());
                                                                        ^
src/main.cpp:275:72: warning: too many arguments for format [-Wformat-extra-args]
src/main.cpp:275:72: warning: unknown conversion type character 'z' in format [-Wformat=]
src/main.cpp:275:72: warning: too many arguments for format [-Wformat-extra-args]
/c/msys64/mingw64/bin/g++.exe -c -Wall -Wextra -Wpedantic -o obj/Component.o src/Component.cpp -std=c++11 -O3
/c/msys64/mingw64/bin/g++.exe -c -Wall -Wextra -Wpedantic -o obj/Configuration.o src/Configuration.cpp -std=c++11 -O3
/c/msys64/mingw64/bin/g++.exe -c -Wall -Wextra -Wpedantic -o obj/generated.o src/generated.cpp -std=c++11 -O3
/c/msys64/mingw64/bin/g++.exe -c -Wall -Wextra -Wpedantic -o obj/Input.o src/Input.cpp -std=c++11 -O3
/c/msys64/mingw64/bin/g++.exe -c -Wall -Wextra -Wpedantic -o obj/Output.o src/Output.cpp -std=c++11 -O3
src/Output.cpp: In function 'void PrintLinksForTarget(Component*)':
src/Output.cpp:178:57: warning: unknown conversion type character 'z' in format [-Wformat=]
     printf("Public linked (%zu):", sortedPubLinks.size());
                                                         ^
src/Output.cpp:178:57: warning: too many arguments for format [-Wformat-extra-args]
src/Output.cpp:178:57: warning: unknown conversion type character 'z' in format [-Wformat=]
src/Output.cpp:178:57: warning: too many arguments for format [-Wformat-extra-args]
src/Output.cpp:184:61: warning: unknown conversion type character 'z' in format [-Wformat=]
     printf("\nPrivate linked (%zu):", sortedPrivLinks.size());
                                                             ^
src/Output.cpp:184:61: warning: too many arguments for format [-Wformat-extra-args]
src/Output.cpp:184:61: warning: unknown conversion type character 'z' in format [-Wformat=]
src/Output.cpp:184:61: warning: too many arguments for format [-Wformat-extra-args]
src/Output.cpp: In function 'void PrintInfoOnTarget(Component*)':
src/Output.cpp:199:44: warning: unknown conversion type character 'z' in format [-Wformat=]
     printf("Lines of Code: %zu\n", c->loc());
                                            ^
src/Output.cpp:199:44: warning: too many arguments for format [-Wformat-extra-args]
src/Output.cpp:199:44: warning: unknown conversion type character 'z' in format [-Wformat=]
src/Output.cpp:199:44: warning: too many arguments for format [-Wformat-extra-args]
src/Output.cpp:204:63: warning: unknown conversion type character 'z' in format [-Wformat=]
     printf("Public dependencies (%zu): ", sortedPubDeps.size());
                                                               ^
src/Output.cpp:204:63: warning: too many arguments for format [-Wformat-extra-args]
src/Output.cpp:204:63: warning: unknown conversion type character 'z' in format [-Wformat=]
src/Output.cpp:204:63: warning: too many arguments for format [-Wformat-extra-args]
src/Output.cpp:209:66: warning: unknown conversion type character 'z' in format [-Wformat=]
     printf("\nPrivate dependencies (%zu):", sortedPrivDeps.size());
                                                                  ^
src/Output.cpp:209:66: warning: too many arguments for format [-Wformat-extra-args]
src/Output.cpp:209:66: warning: unknown conversion type character 'z' in format [-Wformat=]
src/Output.cpp:209:66: warning: too many arguments for format [-Wformat-extra-args]
src/Output.cpp:217:45: warning: unknown conversion type character 'z' in format [-Wformat=]
     printf("\nFiles (%zu):", c->files.size());
                                             ^
src/Output.cpp:217:45: warning: too many arguments for format [-Wformat-extra-args]
src/Output.cpp:217:45: warning: unknown conversion type character 'z' in format [-Wformat=]
src/Output.cpp:217:45: warning: too many arguments for format [-Wformat-extra-args]
/c/msys64/mingw64/bin/g++.exe -c -Wall -Wextra -Wpedantic -o obj/CmakeRegen.o src/CmakeRegen.cpp -std=c++11 -O3
/c/msys64/mingw64/bin/g++.exe -c -Wall -Wextra -Wpedantic -o obj/Analysis.o src/Analysis.cpp -std=c++11 -O3
/c/msys64/mingw64/bin/g++.exe -o cpp-dependencies obj/main.o obj/Component.o obj/Configuration.o obj/generated.o obj/Input.o obj/Output.o obj/CmakeRegen.o obj/Analysis.o -lboost_filesystem-mt -lboost_system-mt -O3

After compilation, the output of cpp-dependencies.exe" --stats is:

zu components with zu public dependencies, zu private dependencies Detected zu nodes in cycles

I compiled it using the following compiler:

$ /c/msys64/mingw64/bin/g++.exe --version
g++.exe (Rev1, Built by MSYS2 project) 6.1.0
Copyright (C) 2016 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Support Mac OS X

In Input.cpp, the Linux function memrchr() is used. This is a non-standard function, and should be east to replace by something more portable.

Compatibility with Makefiles.

Is it possible to make it work for makefiles, instead of CMakeList , since they have almost same information.

Consider adding travis shields to README

You may wish to consider adding shields to your README to indicate successful builds and/or test coverage. (For example, check the README.md in the SpeedTools repo on how to use such shields.)

Allow output to stdout using "-"

Instead of writing to a file literally called "-". Requested by @mattgodbolt

Includes from the root project to others are not taken into account

This causes cases where files or components are claimed unused, but are not entirely.

Support Windows

The tool so far has not been tried or tested on Windows.

show snap installation instructions on readme?

i packaged cpp-dependencies as a snap app which should run on almost all linux distros:

https://snapcraft.io/cpp-dependencies
sudo apt install cpp-dependencies on most linux distros

Might want to compare with the features in my dependency analyser

Just watched the Meeting C++ 2016 video.

The analyser is a Visual Studio extension / NuGet command line DeepEnds that, for Visual C++, will scan for #include statements or slowly parse with libclang. Output options include an interactive graph file and a set of statistics.

C++17 std::filesystem instead of boost

Can we use std::filesystem (in c++17) instead of Boost::filesystem?

Add unit tests

Many parts of the tool are logical operations and can be unit-tested. So far they just haven't been.

Does it support bazel build system ?

I am using Bazel build system for my c++ project and i was trying to generate the dependency graphs for the project. After running the tool, it showed me this error:

Warning: Analyzing your project resulted in a very low amount of components. This either points to a small project, or
to cpp-dependencies not recognizing the components.

It tries to recognize components by the existence of project build files - CMakeLists.txt, Makefiles, MyProject.vcxproj
or similar files. If it does not recognize any such files, it will assume everything belongs to the project it is
contained in. You can invert this behaviour to assume that any code file will belong to a component local to it - in
effect, making every folder of code a single component - by using the --infer option.

Another reason for this warning may be running the tool in a folder that doesn't have any code. You can either change
to the desired directory, or use the --dir <myProject> option to make it analyze another directory.

Cannot scan libwebp due to parenthesis level check

Hi,

this is follow up of #57 . The fix provided doesn't help for all projects I use.
In particular running cpp-dependecies over libwebp causes the issue:

$ cpp-dependencies --stats
final level of parentheses=7
cpp-dependencies: /build/cpp-dependencies/parts/cpp-depenencies/src/src/Input.cpp:310: void ReadCmakelist(const Configuration&, std::unordered_map<std::__cxx11::basic_string<char>, Component*>&, const boost::filesystem::path&): Assertion `parenLevel == 0 || (printf("final level of parentheses=%d\n", parenLevel), 0)' failed.
Aborted (core dumped)

Project code:

https://github.com/webmproject/libwebp / tag v1.2.4 (current HEAD also fails)

It worth noting that before the #61 final level of parentheses was 1 (not 7).
And it was possible to overcome the issue by modifying following line in CMakeLists.txt of libwebp:

405: "AC_INIT\\([^\n]*\\[[0-9\\.]+\\]"

to

405: "AC_INIT\\([^\n]*\\[[0-9\\.]+\\]\\)"

With recent changes this no longer works.

dose this also work for C only?

Or is this only for C++

assertion failed in ReadCmakelist while running cpp-dependencies on itself

Running cpp-dependencies on itself results in a failure:

../build/src/cpp-dependencies --stats .
final level of parentheses=1
cpp-dependencies: /work/cpp-dep_ws/cpp-dependencies/src/Input.cpp:310: void ReadCmakelist(const Configuration&, std::unordered_map<std::__cxx11::basic_string, Component*>&, const boost::filesystem::path&): Assertion `parenLevel == 0 || (printf("final level of parentheses=%d\n", parenLevel), 0)' failed.
Aborted (core dumped)

#include finding is not robust and has false positives and false negatives

This is a known issue.

It assumes that there will be no white space between the hash character and the "include" keyword.

It also does not understand the /* ... */ comments and will consider include statements between them as actual includes.

To fix this would require a fairly hefty rewrite of the C++ source file reading.

CMakeLists.txt files in subdirectories are unconditionally included in analysis

CMakeLists.txt files in subdirectories, like those from submodules, are included in analysis regardless of whether they are in the transitive-include graph of the top-level CMakeLists.txt file.

src/Input.cpp L 267 will pull in any file named CMakeLists.txt.

This results in misdiagnosis of ambiguous includes in projects like https://github.com/jbcoe/polymorphic_value which include the popular Catch testing framework.

The online help text for cpp_dependencies should include a version number

The online help text for cpp_dependencies should include a version number.
It's hard to see which version the tool is now.

std::bad_alloc (core dumped) on Ubuntu 16.04

Build went fine on Ubuntu 16.04
But execution failed
/usr/local/bin/cpp-dependencies --stats . terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc Aborted (core dumped)

Add 'cpp-dependencies' to .gitignore

Add cpp-dependencies to .gitignore - on Linux/Mac, this is the executable which should not be (inadvertently) stored in the repo.

CMake Error about "Could NOT find Boost (missing: Boost_INCLUDE_DIR filesystem system)"

I clone the source code from github and run the command cmake cpp-dependencies. Then, the cmake print error below

CMake Error at /usr/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find Boost (missing: Boost_INCLUDE_DIR filesystem system)
Call Stack (most recent call first):
  /usr/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
  /usr/share/cmake-3.22/Modules/FindBoost.cmake:2360 (find_package_handle_standard_args)
  CMakeLists.txt:76 (find_package)

Make very explicit what the tool does with CMakeLists

I started using the tool on my codebase recently and for some components I could make some improvements already.

But with bigger contexts the tool still struggles. I tried to find out why. As far as I can see, the tool is not able to handle the INCLUDE() Cmake directive correctly, so some crucial targets are missing and my depedency graph is wonky.

Is this support indeed missing?
Where would on start adding it?

Warn if the last command is a further-command-affecting option

This typically happens when you forget that commands are run in order. The last command won't do anything in this case and it can be detected up front.

Command-line options -regen and --dryregen are unclear and undocumented

Please document the -regen and -dryregen options.

• You might only need 1, not 2.
• If it is to generate CMakeLists.txt consider renaming the option to --generate-makelists

No such file or directory error

I am getting this error

cpp-dependencies-1.1/example/Engine/./Engine.h:1:33: fatal error: Framework/framework.h: No such file or directory
#include "Framework/framework.h"

when calling "make" after "cmake example"

What am I doing wrong can you help me?

Specifying directories to exclude from the analysis

Hi,

I've just started using the tool an a fair sized legacy codebase and am immediately getting some interesting results (GREAT tool thanks). Unfortunately a near full version of boost is contained inside the root directory of the codebase which can't be helping the performance of the parser and is also adding noise to the results.

It would be nice to specify a (list of) directories that should be excluded from the the analysis (including all subdirectories).

Error when compiling with VS on Windows - memrchr

memrchr() function is a GNU extension. Input.cpp in ReadCodeFrom()

Be nice to clearly state that you need to be using CMake

The README didn't make it clear to me that it needs CMakeLists.txt to do anything useful. I managed to get it building on WIndows but our code uses MSVC projects so all I got was:

1 components with 0 public dependencies, 0 private dependencies
Detected 0 nodes in cycles

Release pre-built binaries for Linux/MacOS/Windows

Hi,

I think this is a great tool and it would be even better if pre-built binaries were packaged up as releases (can be done automatically by the CI when a new release is made).

I personally feel it is unreasonable to expect every user to compile the tool from source in order to use it.

Thanks.

Assertion failed in src/Input.cpp:304

final level of parentheses=-1
cpp-dependencies: ../src/Input.cpp:304: void ReadCmakelist(const Configuration&, std::unordered_map<std::__cxx11::basic_string<char>, Component*>&, const boost::filesystem::path&): Assertion `parenLevel == 0 || (printf("final level of parentheses=%d\n", parenLevel), 0)' failed.
Aborted (core dumped)

Create clang-format file & run it in CI system somewhere

So that questions about formatting can be simply answered with "run clang-format".

Can I use cpp-dependencies to analyse C project?

Hello, thanks for making this tool.
I need to perform dependency analysis on the C project, but I can't find a good tool. Can cpp-dependencies analyze the C file?

Support case-insensitive header finding as an option (for OSX / Windows)

One of its functions is to identify header names which conflict with other components. This comparison should be done case-insensitively on Windows and OSX, as the underlying OS will also read the same files case-insensitively.

Warn if no sensible project(s) could be found

For new users it's confusing if it says there's one component when there are others, but it only won't find it because there are no cmakelists or other supported project files. Instead, tell the user explicitly that it didn't find anything because of this.

Make includesize output more useful by outputting the highest scores only

Right now it outputs the scores for all headers, leading to a 10k+ line output and user action to find out what's the worst contributor. It would be better if it would output the top N (<= 30?) scoring headers.

boost dependency necessary for Linux?

Great tool! The project's README says C++ Boost's Filesystem is necessary, but the file "src/FilesystemInclude.h" gives an alternative（"experimental") by a conditional directive.
On Linux, there is indeed an "experimental" directory inside GCC's include directory.
So is this necessary for Linux? Installing boost seems not a cakewalk (at least for me).

Add support for d3.js visualisations

@dascandy I'll make a start on this soon, been a tad busy and been unable to work on personal stuff lately.

Clarify purpose and status of this repository

IMPORTANT PLEASE READ: NO RESPONSE WILL RESULT IN REPO ARCHIVAL AND FINALLY DELETION

Dear maintainer,

One important topic in our TomTom open source engineering community is to make our public organization on GitHub, as well as the repositories hosted on it, more welcoming to contributors. That process starts which making it easy for other to understand the purpose of a codebase is and how people can contribute.

In order to do so we want to learn which repo's are still actively maintained and important to keep available to the open source community and which are not. We have therefore decided to start a clean-up proces which will be done as follows:

1. Please respond to this issue within 30 days and provide us with the necessary information requested below.
2. If there is no reply to this issue after these 30 days, the repository will be archived
3. After your repo has been archived you have an additional 30 days to respond and provide information and discuss with us whether you want to revert, or keep archived status.
4. IMPORTANT After the above mentioned additional 30 days no response is given the repository will be removed from the TomTom-International organization or will be DELETED

Could you please be so kind to provide us with the following information

Contact details

Repo owner/primary contact: [please provide email address]

Repository details

Describe the purpose/objective of this codebase in 1-5 sentences:
[enter description here]

Please select how this repository contributes to the following (thick all that apply):

• Internal use: The code is used inside TomTom and there is a vested interest in long term maintenance
• Employer branding: Project gives an insight into the scale and problem-solving of TomTom tech.
• Hiring: contributors and users of the project could be attractive job candidates
• Learning: TomTom employees can collaborate with and learn from external experts on the topic
• Enhancement: the project could gain new functionality and refinement by external use and contribution
• Product: The project is needed to make our tech or products integrate with other products or tech stacks
• License Compliance
• Fork: Fork needed for for upstream contributions, long-term upstream collaboration or product / CI/CD pipeline dependency or other need. If this is the case you can move your fork to a new GitHub organization https://github.com/tomtom-forks. If you need access to this org reach out at the #ospo channel on slack with your TomTom Github user account.

Additional context information about the purpose of this codebase:
[enter addition info here]

Questions?

Please respond to this issue or drop a line to [email protected] mentioning this issue nr.
Or for TomTom'ers reach out on the #ospo slack channel.

Thanks so much for helping keep our public repositories up to date and welcoming to all contributors.

- Add lists of OS headers for verifying that you don't have headers that (can) conflict with OS headers / std headers

Many projects use header names that conflict with system headers. It would be useful to identify these before they are in widespread use, so that collisions can be prevented rather than later on cured.