molpro / profiler Goto Github PK

View Code? Open in Web Editor NEW

0.0 0.0 1.0 1.5 MB

Simple class to collect and display code execution time statistics

License: MIT License

CMake 3.49% Dockerfile 0.08% C++ 82.11% Fortran 14.32%

profiler's People

Contributors

Watchers

Forkers

pjknowles

profiler's Issues

Embed C++ language standard in library - [merged]

In GitLab by @KnowlesPJ on Mar 21, 2020, 08:08

Merges embed-c++-language-standard-in-library -> master

Repair fortran i8 detection - [closed]

In GitLab by @KnowlesPJ on Jan 17, 2021, 12:12

Merges repair-fortran-i8-detection -> master

Improvement to documentation and website - [merged]

In GitLab by @mxs301 on Nov 1, 2020, 04:19

Merges improve_documentation -> master

Cleaned up the README file and changed the format of hosted website. I've described structure of the new profiler, since it is much simpler, but the implementation still needs to be finished.

Complete cmake configuration - [merged]

In GitLab by @KnowlesPJ on Jul 1, 2019, 16:43

Merges cmake -> master

Separated Profiler into serial and MPI - [merged]

In GitLab by @mxs301 on Mar 19, 2020, 13:14

Merges separate_mpi_serial -> master

Split profiler into two targets, Profiler::serial and Profiler::mpi. Each one has it's own Profiler.h exposing Profiler struct that inherits from either serial or parallel profiler.

When using Profiler as a dependency, Profiler::serial is always built. If Fortran compiler is found, it will also built Fortran interface. If mpi target is found, Profiler::mpi is also built. User has to decide which library to link in their own CMakeLists.

CMake was also cleaned up. configure_library() is not used, and install() was implemented for each target explicitly.

Fix spacing in output - [merged]

In GitLab by @KnowlesPJ on Jan 28, 2021, 08:28

Merges fix-spaces-in-output -> master

Just trivial formatting

Parallel output - [closed]

In GitLab by @DavidKreplin on Dec 10, 2019, 14:55

Merges parallel -> master

Hello Peter,
I have been using your profiler for quite a while, and I like it very much.
However, I found there are some features missing for parallel profiling.
What I'm looking for is something like this:

Step Proc Min Max Aver
Init 0.07 0.07 0.07 0.07
MuFock 1465.74 1324.72 1468.98 1401.78
Cholesky 5.21 5.20 5.25 5.22

Proc: Time spend on the processor of output.out_xxx
Min: Minimum over all processors
Max: Maximum over all processors
Aver: Average over all processors

I did some changes in profiler.cpp to get those numbers (only tested within Molpro).
But this is not a proper merge, since the changes are most likely not at the right place.
Maybe we can discuss here whether this is a reasonable feature and where I could do some changes.

Best wishes
David

Library manager - [merged]

In GitLab by @mxs301 on Jul 30, 2020, 17:24

Merges LibraryManager -> master

Fixes issue #2

Doxygen warnings

In GitLab by @mxs301 on Jun 4, 2020, 18:33

Fix doxygen warnings

Searching for member function documentation...
/home/marat/Programs/profiler/src/molpro/Profiler/ProfilerSerial.cpp:19: warning: documented symbol `molpro::profiler::ProfilerSerial::ProfilerSerial' was not declared or defined.
/home/marat/Programs/profiler/src/molpro/Profiler/ProfilerSerial.cpp:25: warning: documented symbol `void molpro::profiler::ProfilerSerial::reset' was not declared or defined.
/home/marat/Programs/profiler/src/molpro/Profiler/ProfilerSerial.cpp:34: warning: no matching class member found for 
  void molpro::profiler::ProfilerSerial::active(const int level, const int stopPrint)

/home/marat/Programs/profiler/src/molpro/Profiler/ProfilerSerial.cpp:41: warning: no matching class member found for 
  void molpro::profiler::ProfilerSerial::start(const std::string &name)

/home/marat/Programs/profiler/src/molpro/Profiler/ProfilerSerial.cpp:78: warning: documented symbol `void molpro::profiler::ProfilerSerial::totalise' was not declared or defined.
/home/marat/Programs/profiler/src/molpro/Profiler/ProfilerSerial.cpp:96: warning: no matching class member found for 
  void molpro::profiler::ProfilerSerial::stop(const std::string &name, long operations)

/home/marat/Programs/profiler/src/molpro/Profiler/ProfilerSerial.cpp:137: warning: documented symbol `void molpro::profiler::ProfilerSerial::stopall' was not declared or defined.
/home/marat/Programs/profiler/src/molpro/Profiler/ProfilerSerial.cpp:142: warning: no matching class member found for 
  ProfilerSerial::resultMap molpro::profiler::ProfilerSerial::totals() const

/home/marat/Programs/profiler/src/molpro/Profiler/ProfilerSerial.cpp:154: warning: documented symbol `std::string molpro::profiler::ProfilerSerial::resources::str' was not declared or defined.
/home/marat/Programs/profiler/src/molpro/Profiler/ProfilerSerial.cpp:215: warning: documented symbol `std::string molpro::profiler::ProfilerSerial::str' was not declared or defined.
/home/marat/Programs/profiler/src/molpro/Profiler/ProfilerSerial.cpp:257: warning: documented symbol `void molpro::profiler::ProfilerSerial::accumulate' was not declared or defined.
/home/marat/Programs/profiler/src/molpro/Profiler/ProfilerSerial.cpp:279: warning: documented symbol `struct ProfilerSerial::resources molpro::profiler::ProfilerSerial::getResources' was not declared or defined.
/home/marat/Programs/profiler/src/molpro/Profiler/ProfilerSerial.cpp:309: warning: documented symbol `struct ProfilerSerial::resources & molpro::profiler::ProfilerSerial::resources::operator+=' was not declared or defined.
Creating members for template instances...

Tree design - [merged]

In GitLab by @KnowlesPJ on Sep 3, 2020, 16:43

Merges TreeDesign -> master

Update fortran -i8 detection - [merged]

In GitLab by @KnowlesPJ on Jan 17, 2021, 12:19

Merges fortran-i8 -> master

Use official Docker image for CI - [merged]

In GitLab by @KnowlesPJ on Mar 20, 2020, 09:58

Merges use-official-docker-image -> master

Use MPI_COMM_WORLD as default - [merged]

In GitLab by @mxs301 on Aug 27, 2020, 18:34

Merges default_to_MPI_COMM_WORLD -> master

Changes ProfilerMPI default to MPI_COMM_WORLD without consideration for PPIDD or GA.

less strict on cmake version - [merged]

In GitLab by @KnowlesPJ on Jul 4, 2019, 18:33

Merges master -> master

Complete rewrite of Profiler - [merged]

In GitLab by @mxs301 on Nov 3, 2020, 10:19

Merges profiler_tree -> master

The profiler has two main tasks. The first is to profile the code by tracing the call tree and timing each node. This can be naturally and efficiently represented using a tree with timer or counter on each node.
The second job is to report the profile and visually this should be a list of paths to each node (call stack) and corresponding statistics. The tree structure is no longer very convenient and for reporting it is better to have a flat list with call stack and statistics from the counter.

The original profiler was written in the mindset of a reporter and as such stored the call stack and statistics in a flat list. However, this implementation is not the most efficient and gets worse as depth of tree increases.

The current re-implementation is an attempt to use more efficient data structures for the two tasks of building the profile and reporting it. The Profiler is now constructed as a tree structure using Node<Counter> class with Counter collecting statistics. This leads to about x10 speed up compared to the old Profiler. The reporting is done in report.h where the tree is converted to a flat list of TreePath container that store the call stack and corrected statistics (cumulative effects) in an ordered fashion (see SortBy).

The weak singleton pattern was also made more concrete and light-weight, although it's documentation is a bit featherweight.

I've pushed for logical separation of between serial and MPI sections of Profiler. In parallel, the communicator is only needed during reporting, which does not over-ride the serial pathway. In practice, even parallel applications should write a report for each process, probably to a logger, to allow developers access to useful info i.e. !9. The parallel report shows a picture that the user is most likely to be interested in. We previously had a lot of headache thinking about what to do with communicators, so we should reflect on it now.

I've attempted to make the interface as close to original as possible, especially in the wrappers where the interface is unchanged. However, not everything might make sense now, especially since things like communicator and sort ordered are no longer necessary on construction. The wrappers should probably be redesigned.

There is minor polishing on my mind which can be done later but I will note it for the record.

write_timing in report.h is too simplistic. The old profiler had a very good implementation which should be copied back.
(Keep it) Profiler::reset seems redundant, one can simply use move assignment
(Deal with it later) option to sort by name was removed, but also it was never exposed in the wrappers. Should it be brought back? The tree is sorted by name by construction, so this would be trivial. This is not a bad idea, but also should have an option to write the call tree in the same order it was constructed.
WeakSingleton should have erase() method
profiler-singelton-example is broken
add report report() that takes a node and reports on the subtree
in parallel add report_master() which only writes on master
introduce stop-watch functionality in Timer. If Timer is not stoppped and cumulative_time() is called it will return timing at the moment it was called. Otherwise profiler has to be fully stopped before reporting to avoid zeros.
improve documentation in Profiler.h

If everybody is satisfied with the design, than we can make it 0.2.0 and polish things later.

Cmake - [merged]

In GitLab by @KnowlesPJ on Jul 4, 2019, 14:20

Merges cmake -> master

made operator<< a friend function of ProfilerSerial - [merged]

In GitLab by @mxs301 on Aug 21, 2020, 20:44

Merges hotfix_ostream -> master

There was a problem finding it with ADL in LinearAlgebra. I think it's because Profiler is now a dummy that inherits from ProfilerSerial or ProfilerMPI. Everything worked with ProfilerMPI because operator<< is a friend, so I did the same with ProfilerSerial to fix it.

Fortran interface for dotgraph() should be provided

In GitLab by @KnowlesPJ on Aug 12, 2021, 06:46

Some work on the F- and C-wrappers - [merged]

In GitLab by @yapolyak on Jan 15, 2021, 12:00

Merges fortran_wrapper -> master

Fixed a bug and switched to calling a synchronised reporting procedure (if using MPI) in the C-wrapper. Done some work on passing and initiating the global MPI communicator, but the corresponding functions in the F-wrapper are currently uncommented due to the fact we don't use molpro::utilities as a dependency.

Fix fortran -i8 detection - [merged]

In GitLab by @KnowlesPJ on Jan 17, 2021, 14:12

Merges fortran-i8 -> master

Fix gitlab ci for pages - [merged]

In GitLab by @mxs301 on Nov 5, 2020, 08:51

Merges fix-gitlab-ci-pages -> master

Pages were not building correctly

$ if [ $(git describe --tags --abbrev=0 --always HEAD) = $(git describe --tags --abbrev=0 --always origin/master) ]; then cd $PUBLIC; rm -f latest; ln -s ${PROJECT_VERSION_MAJOR}.${PROJECT_VERSION_MINOR} latest ; rm -f ./index.html; cd ..; fi
fatal: Not a valid object name origin/master
/usr/bin/bash: line 119: [: 0.2.0: unary operator expected

The if statement is now always executed.

WeakSingleton does not have to be initialized. - [merged]

In GitLab by @KnowlesPJ on Apr 27, 2021, 07:25

Merges default-weak-singleton -> master

WeakSingleton does not have to be initialized. If it's asked for and doesn't yet exist, a new instance is created, and given zero depth.

library cmake configuration - [merged]

In GitLab by @KnowlesPJ on Jul 7, 2019, 15:24

Merges master -> master

ProfilerSingle should use weak_ptr

In GitLab by @mxs301 on Jun 28, 2020, 10:21

In the current model the singleton creates shared_ptr objects and has full ownership. As a result, the user has to call ProfilerSingle.destroy() before the program terminates, otherwise MPI version raises an exception on exit.

There is no need for ProfilerSingle to have ownership. It should use weak_ptr instead and return shared_ptr instance. The user can than destroy underlying Profiler by scoping or explicitly resetting.

documentation fix - [merged]

In GitLab by @KnowlesPJ on Sep 2, 2021, 09:34

Merges fortran-dotgraph -> master

Fortran wrapper - destructor

In GitLab by @yapolyak on Jan 15, 2021, 11:51

I don't know if this has been discussed before, but I figured out that the FINAL subroutine (i.e. destructor in our case) is not invoked before END statement of the main program (see https://community.intel.com/t5/Intel-Fortran-Compiler/FINAL-subroutine-not-called/td-p/754732) - hence it is not called in the profiler-fortran-example (and likewise in the Fortran examples in LinearAlgebra) unless we wrap them in a subroutine called from the main function. We then have to call profiler%destroy(), otherwise a memory error is thrown upon destruction of the Profiler object.

Just wondering if we should do anything about it, or it's just fine and we should keep having both the implicit and explicit "destructors".

cmake - [merged]

In GitLab by @KnowlesPJ on Jul 5, 2019, 06:26

Merges cmake -> master

get rid of compiler warnings - [merged]

In GitLab by @KnowlesPJ on Aug 11, 2021, 17:43

Merges eliminate-warnings -> master

Support for graphviz dot output - [merged]

In GitLab by @KnowlesPJ on Jun 25, 2021, 14:34

Merges write_dotfile -> master

Resolve #4 with a comment in the code; the language standard prevents a more substantive fix - [merged]

In GitLab by @KnowlesPJ on Aug 12, 2021, 06:28

Merges fix-4 -> master

correct detection of fortran i8 option - [merged]

In GitLab by @KnowlesPJ on Jan 17, 2021, 12:54

Merges fortran-i8 -> master

Fortran wrapper - MPI communicator for print()

In GitLab by @yapolyak on Jan 12, 2021, 12:55

I noticed that we are not passing a communicator through the Fortran interface when asking Profiler to report timing. Does it mean that in parallel run timing will not be reported correctly (each process will report it's own wall time)?

cmake libconfig - [merged]

In GitLab by @KnowlesPJ on Jul 9, 2019, 11:17

Merges tuesday -> master

Add a singleton Profiler wrapper - [merged]

In GitLab by @mxs301 on Feb 29, 2020, 22:19

Merges singleton -> master

There is generally a single profiler instance per program and current model requires use of global variables. A singleton class is quite natural for this purpose. It hides the global variable inside the class and the user only needs to work with a wrapper to get its instance.

I've implemented a singleton class wrapper for Profiler, called ProfilerSingle. Access to underlying Profiler is with ProfilerSingle::Instance(), which returns a shared pointer. In serial there is only one Profiler object, and in parallel there is one for each MPI communicator.

Profiler itself could be made into a singleton, but that wouldn't be backwards compatible.

I'm not sure whether MPI_Comm is hashable in all implementations and whether we should allow for multiple global Profiler objects even without mpi.