goddard-fortran-ecosystem/pfunit

Parallel Fortran Unit Testing Framework

pfunit's Issues

missleading README.md information

After following the README guide, I have experienced the following error:

./testSuites.inc(1): error #5082: Syntax error, found END-OF-STATEMENT when expecting one of: => = . [ % (
ADD_TESTS_SUITE(helloWorld_suite)
---------------------------------^
./testSuites.inc(1): error #5082: Syntax error, found END-OF-STATEMENT when expecting one of: => = . [ % (
ADD_TESTS_SUITE(helloWorld_suite)
---------------------------------^
./testSuites.inc(1): error #5082: Syntax error, found END-OF-STATEMENT when expecting one of: => = . [ % (
ADD_TESTS_SUITE(helloWorld_suite)
---------------------------------^
compilation aborted for /linkhome/rech/lmd/rlmd998/own_libs/pffunit-serial/include/driver.F90 (code 1)
make: *** [tests.x] Error 1

It turns out the README.md tells you to add testSuites.inc with the following content:

! Add a test suite to the build.
ADD_TESTS_SUITE(helloWorld_suite)

But there is a problem, because there is an extra S, from TESTS to TEST. After applying this simple modification it successfully compiles:

! Add a test suite to the build.
ADD_TEST_SUITE(helloWorld_suite)

It took me a few days to realize about it.

README-INSTALL inconsistent with build: no Makefile

It looks like support for building with GNU make has been removed, but the documentation in the READMEs (especially the main README.md documentation) still refers to the make build using GNUMakefile, which no longer exists.

Issues with oclint and Travis

Travis recently started removing the troublesome oclint from its osx builds. Thus, our explicit uninstall causes an error now. Remove from .travis.yml

regression tests broken in travis?

@mathomp4 I'm a bit confused why the latest regression tests for merge requests onto develop are breaking. My own test for gfortran+OSX with a fresh clone works just fine.

My best bet is that I've introduced a new dependency with some of my C interoperability updates. Could use your help in fixing.

add support for testing ESMF components

Extensions exist in other projects. Should bring those into framework as optional.

complete TapListener

This work was only partially completed before.

Create instructions for using from a project using "make" instead of "cmake".

CMake/CTest Integration?

Does pFUnit have integration with CMake that allows the tests to run as part of the build? Something like this for GTest:

https://cmake.org/cmake/help/v3.10/module/GoogleTest.html

Idea for improving PFUNIT_EXTRA_INITIALIZE

Framework should have an (ordered) table of pre and post actions to take. MPI driver can then simply be a matter of registering mpi_init() and mpi_finalize().

Move ChangeLog to ChangeLog.md

In an intrinsic assignment statement, variable shall not be polymorphic

The following error shows up when trying to compile fArgparse from pFUnit.

Environment variables:

FC=ifort
F90_VENDOR=INTEL
F90=ifort

Cmake command output:

cmake -DSKIP_OPENMP=YES -DSKIP_MPI=YES -DCMAKE_INSTALL_PREFIX=$PFUNIT ..

/ccc/cont003/home/drf/p529jorn/pFUnit/build
[p529jorn@irene171 build]$ cmake -DSKIP_OPENMP=YES -DSKIP_MPI=YES -DCMAKE_INSTALL_PREFIX=$PFUNIT ..
-- The Fortran compiler identification is Intel 17.0.4.20170411
-- The C compiler identification is GNU 4.8.5
-- Check for working Fortran compiler: /ccc/products/ifort-17.0.4.196/system/default/17.0.4.196/bin/intel64/ifort
-- Check for working Fortran compiler: /ccc/products/ifort-17.0.4.196/system/default/17.0.4.196/bin/intel64/ifort  -- works
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Checking whether /ccc/products/ifort-17.0.4.196/system/default/17.0.4.196/bin/intel64/ifort supports Fortran 90
-- Checking whether /ccc/products/ifort-17.0.4.196/system/default/17.0.4.196/bin/intel64/ifort supports Fortran 90 -- yes
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done

Error when compiling:

[p529jorn@irene171 build]$ make
[ 17%] Built target generate-type-incs
[ 22%] Built target generate-template-incs
[ 22%] Built target force-generation-of-includes
[ 43%] Built target gftl-shared
[ 43%] Built target demo.x
[ 44%] Building Fortran object extern/fArgParse/src/CMakeFiles/fargparse.dir/BaseAction.F90.o
/ccc/cont003/home/drf/p529jorn/pFUnit/extern/fArgParse/src/BaseAction.F90(110): error #8304: In an intrinsic assignment statement, variable shall not be polymorphic.   [N_ARGUMENTS]
         this%n_arguments = n_arguments
--------------^
/ccc/cont003/home/drf/p529jorn/pFUnit/extern/fArgParse/src/BaseAction.F90(112): error #8304: In an intrinsic assignment statement, variable shall not be polymorphic.   [N_ARGUMENTS]
         this%n_arguments = None
--------------^
/ccc/cont003/home/drf/p529jorn/pFUnit/extern/fArgParse/src/BaseAction.F90(123): error #8304: In an intrinsic assignment statement, variable shall not be polymorphic.   [DEFAULT]
            this%default = String(default)
-----------------^
/ccc/cont003/home/drf/p529jorn/pFUnit/extern/fArgParse/src/BaseAction.F90(125): error #8304: In an intrinsic assignment statement, variable shall not be polymorphic.   [DEFAULT]
            this%default = default
-----------------^
/ccc/cont003/home/drf/p529jorn/pFUnit/extern/fArgParse/src/BaseAction.F90(128): error #8304: In an intrinsic assignment statement, variable shall not be polymorphic.   [DEFAULT]
         this%default = NONE
--------------^
/ccc/cont003/home/drf/p529jorn/pFUnit/extern/fArgParse/src/BaseAction.F90(132): error #8304: In an intrinsic assignment statement, variable shall not be polymorphic.   [CONST]
         this%const = const
--------------^
/ccc/cont003/home/drf/p529jorn/pFUnit/extern/fArgParse/src/BaseAction.F90(134): error #8304: In an intrinsic assignment statement, variable shall not be polymorphic.   [CONST]
         this%const = NONE
--------------^
/ccc/cont003/home/drf/p529jorn/pFUnit/extern/fArgParse/src/BaseAction.F90(199): warning #6178: The return value of this FUNCTION has not been defined.   [MATCHES]
   logical function matches(this, argument)
--------------------^
/ccc/cont003/home/drf/p529jorn/pFUnit/extern/fArgParse/src/BaseAction.F90(309): error #8304: In an intrinsic assignment statement, variable shall not be polymorphic.   [N_ARGUMENTS]
      n_arguments = this%n_arguments
------^
compilation aborted for /ccc/cont003/home/drf/p529jorn/pFUnit/extern/fArgParse/src/BaseAction.F90 (code 1)
make[2]: *** [extern/fArgParse/src/CMakeFiles/fargparse.dir/BaseAction.F90.o] Error 1
make[1]: *** [extern/fArgParse/src/CMakeFiles/fargparse.dir/all] Error 2
make: *** [all] Error 2

It is a Linux machine with Intel Compiler 17.0.4.196

Fix installation location and layout

The documentation in the README says that if -DCMAKE_INSTALL_PREFIX is not set then it will default to /usr/local/. However, in reality it defaults to build/installed. Furthermore, the layout of the installation directory is that of a Windows system rather than a Linux one. While cmake can cope with this easily enough, it would be nice if a unix-like installation could be done when running on Linux.

How to run just one test?

Is it possible to run just a single tests (or one of a few tests) out of a full suite? By running the executable and searching for command line options, I don't see any way:

 Command line arguments:
  
  Options: 
    '-h', '--help'    : Prints this message
    '-v', '--verbose' : Logs start/stop of each test
    '-d', '--debug'   : Logs start/stop of each test (same as -v)
    '-o <file>'       : Diverts output to specified file
    '-robust'         : (experimental) runs tests in a separate shell
                        Attempts to detect/handle hangs and crashes
    '-max-timeout-duration <duration>' : Limit detection time for robust
    '-max-launch-duration  <duration>' : Limit detection time for robust
    '-skip n'         : used by remote start with 'robust' internally
                        This flag should NOT be used directly by users.
    '-xml <file>'     : output JUnit XML to specified file
                        XML can be used with e.g. Jenkins.
    '-name <name>'    : give tests an identifying name in XML output

CMake error in the logic in include/add_pfunit_sources.cmake or callers

Hi @tclune et al.,

There seems to be a logic error somewhere in either include/add_pfunit_sources.cmake or one of the functions calling this:

CMake Error at include/add_pfunit_sources.cmake:33 (file):
  file problem creating directory:
  /tmp.F90unit-20190507-36706-1qjb902/pFUnit-4.0.0/build/tests/funit-core
Call Stack (most recent call first):
  include/add_pfunit_ctest.cmake:47 (add_pfunit_sources)
  tests/funit-core/CMakeLists.txt:98 (add_pfunit_ctest)

Your code is taking the current working directory and substituting .F90 in the directory path. e.g.,

/tmp/pfunit-20190507-36706-1qjb902/pFUnit-4.0.0/build ➡️ /tmp/pfunit-20190507-36706-1qjb902/pFUnit-4.0.0/build

I think I know the fix for this and can open a PR. This is a blocker for adding a homebrew formula, so the sooner a minor release can be minted after fixing, the better.

Pollution of CPP #define namespace

I got the error below:

[ 98%] Building Fortran object model/lipluggable/tests/CMakeFiles/tests.dir/home2/rpfische/spack6/opt/spack/linux-centos7-x86_64/gcc-4.9.3/pfunit-3.2.9-lefpp6xzvqfxyixvfjo3gcdrwugtgyel/include/driver.F90.o
/home2/rpfische/spack6/opt/spack/linux-centos7-x86_64/gcc-4.9.3/pfunit-3.2.9-lefpp6xzvqfxyixvfjo3gcdrwugtgyel/include/driver.F90:253.34:

         allocate(context, source=newMpiContext())
                                  1
Error: Function 'newmpicontext' at (1) has no IMPLICIT type

This is caused by a comedy of errors. driver.F90 has an "#ifdef USE_MPI" in it. I don't know where USE_MPI was supposed to be defined. This installed version of pFUnit was built with -DMPI=NO, so it was NOT supposed to be defined:

==> 'cmake' '/home2/rpfische/spack6/var/spack/stage/pfunit-3.2.9-lefpp6xzvqfxyixvfjo3gcdrwugtgyel/pFUnit-3.2.9' '-G' 'Unix Makefiles' '-DCMAKE_INSTALL_PREFIX:PATH=/home2/rpfische/spack6/opt/spack/linux-centos7-x86_64/gcc-4.9.3/pfunit-3.2.9-lefpp6xzvqfxyixvfjo3gcdrwugtgyel' '-DCMAKE_BUILD_TYPE:STRING=RelWithDebInfo' '-DCMAKE_VERBOSE_MAKEFILE:BOOL=ON' '-DCMAKE_INSTALL_RPATH_USE_LINK_PATH:BOOL=FALSE' '-DCMAKE_INSTALL_RPATH:STRING=/home2/rpfische/spack6/opt/spack/linux-centos7-x86_64/gcc-4.9.3/pfunit-3.2.9-lefpp6xzvqfxyixvfjo3gcdrwugtgyel/lib;/home2/rpfische/spack6/opt/spack/linux-centos7-x86_64/gcc-4.9.3/pfunit-3.2.9-lefpp6xzvqfxyixvfjo3gcdrwugtgyel/lib64' '-DCMAKE_PREFIX_PATH:STRING=/home2/rpfische/spack6/opt/spack/linux-centos7-x86_64/gcc-4.9.3/python-3.5.2-o4sbclg3sf5uuroc53qpu4axrvr62ar5;/home2/rpfische/spack6/opt/spack/linux-centos7-x86_64/gcc-4.9.3/cmake-3.11.1-joeqauye6ejnz5j3lahtwbpdgc6fpgwa' '-DPYTHON_EXECUTABLE=/home2/rpfische/spack6/opt/spack/linux-centos7-x86_64/gcc-4.9.3/python-3.5.2-o4sbclg3sf5uuroc53qpu4axrvr62ar5/bin/python3.5' '-DBUILD_SHARED=YES' '-DCMAKE_Fortran_MODULE_DIRECTORY=/home2/rpfische/spack6/opt/spack/linux-centos7-x86_64/gcc-4.9.3/pfunit-3.2.9-lefpp6xzvqfxyixvfjo3gcdrwugtgyel/include' '-DBUILD_DOCS=NO' '-DOPENMP=NO' '-DMPI=NO'

However... my project defines its own USE_MPI. And it becomes a problem because my project compiles the driver.F90 source code, which then mis-interprets my project's USE_MPI definition as its own.

Given that pFUnit source code is compiled by the user, the solution would be to use longer/more obscure preprocessor symbols. Eg: USE_MPI -> _PFUNIT_USE_MPI.

*.pf synthax highlighting on GitHub

You could add the file .gitattributes to the main directory that contains the following line:

*.pf linguist-language=fortran

That way, the *.pf files have fortran synthax highlighting on github.
This is already done for your pFUnit_demos repo: https://github.com/Goddard-Fortran-Ecosystem/pFUnit_demos/blob/master/.gitattributes

Thanks for the great unit testing environment!

Installation issues with sub command strip and git submodule update

Some issues came up during the installation. Check below the error:

 cmake -DSKIP_OPENMP=YES -DSKIP_MPI=YES ..
-- The Fortran compiler identification is GNU 4.8.5
-- The C compiler identification is GNU 4.8.5
-- Check for working Fortran compiler: /usr/bin/f95
-- Check for working Fortran compiler: /usr/bin/f95  -- works
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Checking whether /usr/bin/f95 supports Fortran 90
-- Checking whether /usr/bin/f95 supports Fortran 90 -- yes
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- *** Setting default install prefix to /home/users/ajornet/pFUnit/build/installed.
-- *** Override with -DCMAKE_INSTALL_PREFIX=<path>.
-- Performing Test _LOGICAL_DEFAULT_KIND: SUCCESS (value=4)
-- Performing Test _INT_DEFAULT_KIND: SUCCESS (value=4)
-- Performing Test _ISO_INT8: SUCCESS (value=1)
-- Performing Test _ISO_INT16: SUCCESS (value=2)
-- Performing Test _ISO_INT32: SUCCESS (value=4)
-- Performing Test _ISO_INT64: SUCCESS (value=8)
-- Performing Test _REAL_DEFAULT_KIND: SUCCESS (value=4)
-- Performing Test _DOUBLE_DEFAULT_KIND: SUCCESS (value=8)
-- Performing Test _ISO_REAL32: SUCCESS (value=4)
CMake Error at cmake/CheckFortranSource.cmake:13 (string):
  string sub-command STRIP requires two arguments.
Call Stack (most recent call first):
  cmake/CheckCompilerCapabilities.cmake:57 (CHECK_FORTRAN_SOURCE_RUN)
  CMakeLists.txt:55 (include)

 -- Performing Test _REAL32_IEEE_SUPPORT: SUCCESS (value=)
-- Performing Test _ISO_REAL64: SUCCESS (value=8)
CMake Error at cmake/CheckFortranSource.cmake:13 (string):
  string sub-command STRIP requires two arguments.
Call Stack (most recent call first):
  cmake/CheckCompilerCapabilities.cmake:57 (CHECK_FORTRAN_SOURCE_RUN)
  CMakeLists.txt:55 (include)


 -- Performing Test _REAL64_IEEE_SUPPORT: SUCCESS (value=)
-- Performing Test _ISO_REAL128: SUCCESS (value=10)
CMake Error at cmake/CheckFortranSource.cmake:13 (string):
  string sub-command STRIP requires two arguments.
Call Stack (most recent call first):
  cmake/CheckCompilerCapabilities.cmake:57 (CHECK_FORTRAN_SOURCE_RUN)
  CMakeLists.txt:55 (include)


-- Performing Test _REAL128_IEEE_SUPPORT: SUCCESS (value=)
You need to run this command from the toplevel of the working tree.
CMake Error at cmake/build_submodule.cmake:26 (message):
  git submodule update --init failed with 1, please checkout submodules
Call Stack (most recent call first):
  extern/CMakeLists.txt:2 (build_submodule)


-- Configuring incomplete, errors occurred!
See also "..../pFUnit/build/CMakeFiles/CMakeOutput.log".

If works after applying the following fixes:

git diff origin/master
diff --git a/cmake/CheckFortranSource.cmake b/cmake/CheckFortranSource.cmake
index 0c0883e..7259c29 100644
--- a/cmake/CheckFortranSource.cmake
+++ b/cmake/CheckFortranSource.cmake
@@ -10,7 +10,7 @@ macro (CHECK_FORTRAN_SOURCE_RUN file var)
 
   # Successful runs return "0", which is opposite of CMake sense of "if":
   if (NOT run)
-    string(STRIP ${${var}} ${var})
+    string(STRIP "${${var}}" ${var})
     if (NOT CMAKE_REQUIRED_QUIET)
       message(STATUS "Performing Test ${var}: SUCCESS (value=${${var}})")
     endif ()
diff --git a/cmake/build_submodule.cmake b/cmake/build_submodule.cmake
index 4b55ce9..4f70542 100644
--- a/cmake/build_submodule.cmake
+++ b/cmake/build_submodule.cmake
@@ -19,7 +19,7 @@ macro (build_submodule submodule)
        if (GIT_SUBMODULE)
          execute_process (
            COMMAND ${GIT_EXECUTABLE} submodule update --init --recursive
-           WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
+        WORKING_DIRECTORY ${PROJECT_SOURCE_DIR}
            RESULT_VARIABLE GIT_SUBMODULE_RESULT
            )
          if (NOT GIT_SUBMODULE_RESULT EQUAL "0")

default flags for pFUnit should be -O0

Dropping Python 2 support.

Isn't time to rise minimum supported Python to 3.5?
We are three months away from Python2.7 end of support.

Optional CMake flag for FTL installation

Currently the develop branch automatically downloads and installs FTL during the build process. This is convenient for users that do not want to know about this simple dependency, but bad for higher level tools such as spack. It is also bad for users that need to build in an environment that cannot access the internet.

What is needed is an optional CMake command line flag that specifies a path to an installed FTL. If present, then that path is used. If not present, then CMake will obtain FTL from GitHub.

GitHub release tarballs do not include submodules

Apologies if you're already aware of this, but the GitHub-created tarballs on releases don't include any git submodules, so they fail to build. And because they're also not git repos, you can't just run git submodule update --init --recursive to check them out.

Our solution was to add git-archive-all as a submodule, and a make target (dist, for tradition) that calls it to produce a tarball with all the submodules, and upload that manually to the GitHub release. Our wrapper script is here if that's helpful for you.

Problems building pFUnit_demos

I'm currently having a couple of issues with building the pFUnit demos:

1. The FARGPARSE_TOP_DIR variable in PFUNIT.mk is empty, so building the demos don't work:

   /home/Tools/pFUnit_demos/Trivial: ./build_with_make_and_run.x 
   /home/Tools/pFUnit-4.0/build/install/PFUNIT-4.0/include/PFUNIT.mk:10: 
   /include/FARGPARSE.mk: No such file or directory
   make: *** No rule to make target '/include/FARGPARSE.mk'.  Stop.
   ./build_with_make_and_run.x: line 9: ./my_tests: No such file or directory
   

   Fargparse does get installed though, in /path/to/install/FARGPARSE-0.9

   Transitively going through PFUNIT.mk, FARGPARSE.mk, and GFTL_SHARED.mk to set their dependency paths correctly seems to fix things. But from the looks of things, these should be set at configure time?

2. Perhaps just my opinion, but I think you're installing pFUnit (and the dependencies) in the wrong place: if I run cmake .. -DCMAKE_INSTALL_PREFIX=/path/to/install, I would expect everything to be under /path/to/install, but it's actually under /path/to/install/PFUNIT-4.0. This is a bit surprising! I have to set PFUNIT_DIR to /path/to/install/PFUNIT-4.0, rather than the path I told CMake. Maybe the modules should go under /path/to/install/includes/pfunit given the number of them, but I think "principle of least surprise" would be good here -- just install stuff in the conventional paths.

python module path for funitproc?

@MatthewHambley

I was trying to create a small demo project that uses all the advanced bells-and-whistles from pfunit 4.0 (develop branch), and am encountering several issues with the install step. I've fixed a few (not yet committed), but am hoping you can help with the python paths.

Currently PFUNITConfig.cmake has the following snippet

set (PFUNIT_PARSER "${PYTHON_EXECUTABLE}" "${CMAKE_CURRENT_LIST_DIR}/bin/funitproc")
set (PFUNIT_DRIVER "${CMAKE_CURRENT_LIST_DIR}/include/driver.F90")
set (PFUNIT_TESTUTILS "${CMAKE_CURRENT_LIST_DIR}/include/TestUtil.F90")

This is no longer accurate, given the new locations of the config files (trying to be consistent across multiple projects of mine). So I've modified them to be:

set (PFUNIT_PARSER "${PYTHON_EXECUTABLE}" "${CMAKE_CURRENT_LIST_DIR}/../bin/funitproc")
set (PFUNIT_DRIVER "${CMAKE_CURRENT_LIST_DIR}/../include/driver.F90")
set (PFUNIT_TESTUTILS "${CMAKE_CURRENT_LIST_DIR}/../include/TestUtil.F90")

That seems fine, but funitproc can now not find the parser:

Traceback (most recent call last):
  File "/Users/tclune/swdev/GitHub/pFUnit/nag/installed/PFUNIT-4.0/cmake/../bin/funitproc", line 12, in <module>
    import funit.pFUnitParser

My problem is that I don't really understand how the previous version worked. I know there are a few options for extending Python's search path, but don't want to resort to that if there is a natural fix.

Fix high level cmake options

Current documentation says that there are options:

-SKIP_OPENMP and -SKIP_HAMCREST

but these do not yet exist. Further the legacy "USE_OPENMP" conflicts.

cmake add_pfunit_test expects relative paths

The add_pfunit_test macro in cmake only works if it is passed relative paths to the .pf files. Line 101 of PFUNITConfig.cmake prepends the absolute path to the source directory of CMakeLists.txt to the file name of the pf-source files. This means that if the provided path is already absolute then cmake will look for a nonexistent file. For example, say CMakeLists.txt and test.pf are both in directory /a/b/c and the absolute path to test.pf is provided to add_pfunit_test. Then cmake will be looking for the test source file at location /a/b/c/a/b/c/test.pf where it does not exist.

The solution to this would be to either check whether an absolute path is provided or to compute the relative path using the cmake file(RELATIVE_PATH) command.

find_package() is in wrong place in config.

The find_package() logic should be moved to before the include of PFUNIT.cmake.

For some reason this works on OS X, but the problem shows up on Linux.

eliminate ./src/core directory

This is detritus from earlier refactoring - just did not notice.

incorporate fArgParse

This is to get command line arguments.

Complete implementation of command line argument parsing

Some work has already been committed to replicate some of the python command line argument parsing capabilities. Eventually hit a snag with compiler bugs, so it is not yet used in pFUnit. The work should be completed and bug reports filed with vendors if issues still exist.

incorporate gFTL-shared

This avoids the need to duplicate containers and simplifies use of containers across subcomponents.

Add ability to match a pattern for an exception in addition to an exact string match

In our code we have some preprocessing that adds the line number to some aborts that happen to make it easier to find. This is great except for our PfUnit unit testing, as if the line number happens to change -- the unit test will also fail even if everything else is the same. So what I'd like to be able to do is to do a pattern match in the exception handling rather than have to include the line number in the match. I'd just like to see this as an option, not the standard way to do things.

Problems installing on OSX 10.14.5

Hi,

I'm having issues trying to install pFUnit on OSX Mojave 10.14.5. Below are the steps that I have taken, hopefully you can see where I'm going wrong.

1. clone the repository
2. mkdir build
3. cd build
4. cmake .. -DSKIP_MPI -DSKIP_OPENMP (terminal output in attached output_1.log)
5. make tests (terminal output in attached output_2.log)
6. make install (terminal output in attached output_3.log plus errors in screenshot below about "libfunit.a" and "no symbols")

In the folder installed/PFUNIT-4.0/include I have the files listed in the attached screenshot below but no base.mk.

[output_1.txt](https://github.com/Goddard-Fortran-Ecosystem/pFUnit/files/3330734/output_1.txt) [output_2.txt](https://github.com/Goddard-Fortran-Ecosystem/pFUnit/files/3330735/output_2.txt) [output_3.txt](https://github.com/Goddard-Fortran-Ecosystem/pFUnit/files/3330736/output_3.txt)

Update location of gFTL subrepo

The permanent home was moved to a new org under GitHub. Need to update that in CMake and git submodule.

Provide snake_case procedure names for all public methods.

While there are many fans of CamelCase, a strong argument can be made that it is inappropriate for case-insensitive languages. The style I have adopted is for ClassNames to be CamelCase, and methods/variables to be snake_case. For backward compatibility, CamelCase will be be retained until pFUnit 5.0

Finish ctest work

Build with MPI=YES does not run the mpi-related tests in a canonical manner. Will add a top-level "tests" target and enable ctest to run the full suite of tests.

Implement "disable" annotation

This is partially implemented as "ignore" on develop branch. Latest JUnit has switched to "disable" so may as well keep up with them while this is not released

gfortran 5.4.0 - "Error: Assignment to an allocatable polymorphic variable at (1) is not yet supported"

This is my first time trying the software. I'm trying to build with gfortran v5.4.0 per README instructions.

$ git clone https://github.com/Goddard-Fortran-Ecosystem/pFUnit.git
$ export FC=gfortran
$ mkdir build; cd build
$ cmake ..
.
.
.
-- Configuring done
-- Generating done
-- Build files have been written to: /home/morton/tmp/pFUnit/build

Then, when I try to build the tests I get numerous "allocatable polymorphic variables..." errors that prevent me from going further...

$ make tests
.
.
.
/home/morton/tmp/pFUnit/extern/fArgParse/src/BaseAction.F90:309:6:

       n_arguments = this%n_arguments
      1
Error: Assignment to an allocatable polymorphic variable at (1) is not yet supported
.
.
.

Have looked far and wide for what this even means and am wondering if there's a simple fix or not.

Add a cmake function to build a group of tests. Existing function is outdated.

change add_pfunit_test() cmake macro

@MatthewHambley

I think I would like to modify the add_pfunit_test() macro to use cmake_parse_arguments().

The interface would then look like:

add_pfunit_test (<executable-name>  
   TEST_SOURCES src1 [, src2[, ...]] 
   OTHER_SOURCES src1 [,src2[, ...]])
   REGISTRY registry

TEST_SOURCES will be processed with funitproc. OTHER_SOURCES are treated as regular sources. And REGISTRY is to specify the 'testSuites.inc' so you can have multiple cases in a single directory. (Not particularly wed to the name 'REGISTRY' though.)

In addition to appreciating your feedback, I would like to know whether we need OTHER_SOURCES_C? It seems like we ought to be able to mix the Fortran and C sources into one add_executable() statement, no?

Roadmap for 4.0

We're looking at adopting pFUnit for unit testing in a fairly large project but before working on this integration just wanted to check if there is an expected timescale for moving from 3 to 4 and what level of backwards compatibility can we anticipate? It would be good to see if there are places we might be able to contribute.

Is there a public roadmap for the next major version of pFUnit?

Create capability to order tests within a suite

There are 2 separate use cases here.

First use case is where the order of the tests matter. Currently pfunit happens to enforce ordering to be the same as the order the tests are registered. But later we hope to have an option run in reverse and random order. Some mechanism to enforce strict ordering even when run in that mode is useful.

A second case, requested by @MatthewHambley arises when it simply does not make sense to run remaining tests once one test in the suite has failed. A question arises as to whether to mark the skipped tests as failed or skipped in the tally.

"use mpi_f08" instead of "include mpif.h"

First of all thanks for the great unit test environment!

Reading Halo_mod I noticed, that "include mpif.h" is used.
(Other modules like PF_MpiTestCase already have "use mpi")

Using mpi with "include mpif.h" is outdated and should not be done anymore. Instead think about putting "use mpi" or even better (for all functionality) "use mpi_f08".

From the mpi forum:

Instead of using mpif.h, the use of the mpi_f08 or mpi module is strongly encouraged for the following reasons:

• Most mpif.h implementations do not include compile-time argument checking.
• Therefore, many bugs in MPI applications remain undetected at compile-time, such as:
  • Missing ierror as last argument in most Fortran bindings.
  • Declaration of a status as an INTEGER variable instead of an INTEGER array with size MPI_STATUS_SIZE.
  • Incorrect argument positions; e.g., interchanging the count and datatype arguments.
  • Passing incorrect MPI handles; e.g., passing a datatype instead of a communicator.
• The migration from mpif.h to the mpi module should be relatively straightforward (i.e., substituting include 'mpif.h' after an implicit statement by use mpi before that implicit statement) as long as the application syntax is correct.
• Migrating portable and correctly written applications to the mpi module is not expected to be difficult. No compile or runtime problems should occur because an mpif.h include file was always allowed to provide explicit Fortran interfaces.

Update cmake install step on develop branch

Logic needs to be refreshed for the refactored structure.

This not been an issue during development but must be resolved before other projects can port to the 4.0 version.

Improve robust configuration

A side discussion with @MatthewHambley suggests that it would be useful to have the RobustRunner somehow launch tearDown() methods for tests that crashed or hung. At the very least, this would be useful for the common case where a crashing test leaves temp files laying around that break the opens in the subsequent setUp() invocations.

Need @assertAlmostEqual()

...like in Python

https://docs.python.org/2/library/unittest.html#unittest.TestCase.assertAlmostEqual

Add assert annotation for assert_that

Going with snake case for now.

Create annotations for disabledOn(compiler) and disabledOn(os)

Travis tests should only test mpi build

On the develop branch, there is now little reason to have travis separately test serial and parallel builds. A parallel build is explicitly just an extension of the serial build and exercises all serial tests.

Minor step to reduce burden on Travis resources.

bin/__pycache__/ files not ignored

Dear pFUnit-team,

we use pFUnit in our internal project (and its great!) but have an issue, that the python installation creates
bin/__pycache__/foo.pyc files that are not ignored with the .gitignore in bin/. As we have pFUnit as git-submodule it is a small annoyance that it is dirty all the time :/

Changing the bin/.gitignore from /*.pyc to *.pyc (as in source/) would fix that issue.

Thank you very much!

CMake command containing spaces collapses to a single string

On NREL's Peregrine compute system, which runs CentOS 7 and cmake 3.8, this cmake command found at source/CMakeLists.txt line 11 collapses to a single string:

COMMAND ${PYTHON_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/GenerateAssertsOnArrays.py --maxRank ${MAX_RANK}

The fix is to wrap the argument in quotes:

COMMAND "${PYTHON_EXECUTABLE}" "${CMAKE_CURRENT_SOURCE_DIR}/GenerateAssertsOnArrays.py" "--maxRank" "${MAX_RANK}"

I see that the CMake configuration has changed in the develop branch. I'll test that branch on our system and issue a pull request with our cmake fixes, if necessary.