Include C/C++ code in documentation

[alazzaro]

Include C/C++ code in documentation

[dev-zero]

It seems that there is no Doxygen to FORD converter atm.

The simple documentation entries (single \brief) can be easily converted:

# remove empty doxygen params and tags with info already recorded in Git
find src -type f -iname '*.f*' -print0 | xargs -0 sed -i \
    -e '/^\!>.*\.\.\.$/d' \
    -e '/^\!> \\author.*$/d' \
    -e '/^\!> \\date.*$/d' \
    -e '/^\!> \\version.*$/d' \
    -e '/^\!>[[:space:]]*$/d'

# remove history tags (use `git blame`)
find src -type f -iname '*.f*' -print0 | xargs -0 perl -i -0pe 's/^!>\ \\par\ History\n(!>\ {5}.+\n)+//mg'
find src -type f -iname '*.f*' -print0 | xargs -0 perl -i -0pe 's/^!>\ <b>Modification history:<\/b>\n(!>\ \-\ .+\n)+//mg'

# remove now empty doxygen headers
find src -type f -iname '*.f*' -print0 | xargs -0 perl -i -0pe 's/^!\ \*{98}\n!\ \*{98}\n//smg'

# convert the single \brief elements to FORD-style documentation
find src -type f -iname '*.f*' -print0 | xargs -0 perl -i -0pe 's/^!\ \*{98}\n!>\ \\brief\s+(.+)\n!\ \*{98}\n(\s*)(.+)\n/\2\3\n\2   !! \1\n/mg'

the reminder (~700 functions) seems to be a bit harder.

What I tried so far is to use a XML output Doxygen configuration:

INPUT                  = src
FILE_PATTERNS          = *.f90 *.F
RECURSIVE              = YES
GENERATE_HTML          = NO
GENERATE_LATEX         = NO
GENERATE_XML           = YES
XML_OUTPUT             = xml
XML_PROGRAMLISTING     = NO

to get a list of XML files.

After that, one can iterate through those and get for each function the complete documentation plus the location of the header and body in the source file in a well-defined format.
What is missing is the original type of the parameter, requiring that we have to parse it out of the Fortran code.

The other way to approach this is to add a pre-commit hook such that every changed file must be manually converted by the person changing that file.

[dev-zero]

the PoC can be found here: https://github.com/dev-zero/dbcsr/tree/feature/convert-to-ford

Run with:

ford project-file.md

fypp and cpp must be in your $PATH.

[dev-zero]

Doxygen comment blocks are interpreted as descriptions for the entire function and the \brief, \param tags are ignored as expected. Generating dependency graphs works too, but takes 5-10 minutes.

[dev-zero]

As per discussion:

• switching to FORD means removing doxify and the part of prettify which does dummy argument reordering and grouping since it interferes with FORD's documentation of arguments
• in my branch I removed empty and redundant Doxygen tags (redundant in the sense that \history and \author information is also contained in the VCS), I furthermore converted lone \brief tags to FORD style documentation !! <content of \brief> (after the function declaration)
• FORD can easily be installed by the user via pip install --user ford, inclusion in DBCSR is therefore not necessary nor desired. Integration in the build system should be done, though.
• Current Doxygen comments are included as documentation, but the Doxygen tags will be ignored:
  
• ~700 Doxygen headers would need to be rewritten to get proper documentation:

diff --git a/src/acc/dbcsr_acc_devmem.F b/src/acc/dbcsr_acc_devmem.F
index 4a596e94..8ede2bff 100644
--- a/src/acc/dbcsr_acc_devmem.F
+++ b/src/acc/dbcsr_acc_devmem.F
@@ -175,20 +175,18 @@ MODULE dbcsr_acc_devmem
 
 CONTAINS
 
-! **************************************************************************************************
-!> \brief Ensures that given devmem has at least the requested size.
-!> \param[in,out] this device memory
-!> \param[in] stream on which zeroing and memcopying is performed
-!> \param[in] requested_size_in_bytes requested size in bytes
-!> \param[in] nocopy (optional) if after growin old content should NOT be copied over. Default: false.
-!> \param[in] zero_pad (optional) if after growing the new memory should be zeroed. Default: false.
-! **************************************************************************************************
    SUBROUTINE acc_devmem_ensure_size_bytes(this, stream, requested_size_in_bytes, nocopy, zero_pad)
-      TYPE(acc_devmem_type), &
-         INTENT(INOUT)                          :: this
-      TYPE(acc_stream_type), INTENT(IN) :: stream
-      INTEGER, INTENT(IN)                      :: requested_size_in_bytes
-      LOGICAL, INTENT(IN), OPTIONAL            :: nocopy, zero_pad
+      !! Ensures that given devmem has at least the requested size.
+      TYPE(acc_devmem_type), INTENT(INOUT) :: this
+         !! this device memory
+      TYPE(acc_stream_type), INTENT(IN)    :: stream
+         !! stream on which zeroing and memcopying is performed
+      INTEGER, INTENT(IN)                  :: requested_size_in_bytes
+         !! requested size in bytes
+      LOGICAL, INTENT(IN), OPTIONAL        :: nocopy
+         !! if after growin old content should NOT be copied over. Default: false.
+      LOGICAL, INTENT(IN), OPTIONAL        :: zero_pad
+         !! if after growing the new memory should be zeroed. Default: false.
 
 #if ! defined (__DBCSR_ACC)
       MARK_USED(this)

• Empty subroutines/function will be listed, but without any comment of course:
  
• Integration of extra documentation is supported, but I haven't tested it
• Grouped parameters will get the same documentation:

   SUBROUTINE acc_devmem_ensure_size_bytes(this, stream, requested_size_in_bytes, nocopy, zero_pad)
      !! Ensures that given devmem has at least the requested size.
      TYPE(acc_devmem_type), INTENT(INOUT) :: this
         !! this device memory
      TYPE(acc_stream_type), INTENT(IN)    :: stream
         !! stream on which zeroing and memcopying is performed
      INTEGER, INTENT(IN)                  :: requested_size_in_bytes
         !! requested size in bytes
      LOGICAL, INTENT(IN), OPTIONAL        :: nocopy, zero_pad
         !! if after growing the new memory should be zeroed. Default: false.

[alazzaro]

Well, let's actually keep this issue open for FORD documentation improvements...

[alazzaro]

• Add DBCSR version to the report
• Add link in the README and wiki
• Include C/C++ files
• Include .f90 files

Note: currently the documentation runs on .F files only

[dev-zero]

* [ ]  Add DBCSR version to the report

* [x]  Add link in the README and wiki

* [ ]  Include C/C++ files

* [] Include .f90 files

Note: currently the documentation runs on .F files only

the documentation runs on fypp-processed source files, actually. Hence the f90 files are implicitly included in that.

[dev-zero]

and the DBCSR version is now included in the documentation

[dev-zero]

I took the liberty to rename the issue to reflect the remaining issue