slicer/slicerexecutionmodel

An open-source CMake-based project that provides macros and associated tools for the easy building of 3D Slicer command line interface (CLI) modules.

SlicerExecutionModel

Overview

The SlicerExecutionModel is a CMake-based project providing macros and associated tools allowing to easily build Slicer CLI (Command line module).

It is designed to improve the acceptance and productivity of Slicer application developers. The Execution Model provides a simple mechanism for incorporating command line programs as Slicer modules. These command line modules are self-describing, emitting an XML description of its command line arguments. Slicer uses this XML description to construct a GUI for the module.

SlicerExecutionModel is documented here:

https://www.slicer.org/wiki/Documentation/Nightly/Developers/SlicerExecutionModel.

Contributing

There are many ways to contribute to SlicerExecutionModel, with varying levels of effort. Do try to look through the documentation first if something is unclear, and let us know how we can do better.

• Ask a question on the Slicer forum
• Submit a feature request or bug, or add to the discussion on the issue tracker
• Submit a Pull Request to improve the SlicerExecutionModel or its documentation

We encourage a range of contributions, from patches that include passing tests and documentation, all the way down to half-baked ideas that launch discussions.

How to submit a PR

If you are new to SlicerExecutionModel development and you don't have push access to the SlicerExecutionModel repository, here are the steps:

1. Fork and clone the repository.
2. Create a branch.
3. Push the branch to your GitHub fork.
4. Create a Pull Request.

This corresponds to the Fork & Pull Model mentioned in the GitHub flow guides.

If you have push access to SlicerExecutionModel repository, you could simply push your branch into the main repository and create a Pull Request. This corresponds to the Shared Repository Model and will facilitate other developers to checkout your topic without having to configure a remote. It will also simplify the workflow when you are co-developing a branch.

When submitting a PR, make sure to add a Cc: @slicer/slicerexecutionmodel comment to notify SlicerExecutionModel developers of your awesome contributions. Based on the comments posted by the reviewers, you may have to revisit your patches.

Automatic testing of pull requests

When you submit a PR to the SlicerExecutionModel repo, CircleCI will run the build and test suite on the head of the branch. If you add new commits onto the branch, those will also automatically be run through the CI process. The status of the CI process (passing, failing, or in progress) will be displayed directly in the PR page in GitHub.

The CircleCI build will run according to the .circleci/config.yml file, which is useful as an example for how to set up your own environment for testing.

Your test results will be posted on SlicerExecutionModel's dashboard. These results will list any failed tests. You can reach your build by clicking the build status link on your GitHub PR.

Testing environments

With each PR, SlicerExecutionModel is built and tested against multiple version of ITK. Each environment is made available in docker images:

• Each environment is described in a Dockerfile found in a Docker-ITK-vX.Y.Z subdirectory of test.
• Testing using a given environment is enabled updating the .circleci/config.yml file.
• Before enabling an environment, associated docker image has to be (1) built locally and (2) pushed to dockerhub.

Available environments

• 

Deprecated environments

Deprecated environments are not maintained, corresponding Dockerfiles have been deleted and associated SlicerExecutionModel continuous integration settings have been removed.

• 

• 

• 

Adding or updating a testing environment

To add a testing environment:

• create a new directory in test following the existing convention (test/Docker-ITK-vX.Y.Z[_qualifier])
• add a Dockerfile
• update list of DIRECTORIES in test/Docker/Makefile
• add a new job to .circleci/config.yml
• update list of environments in README.md

To update a testing environment:

• update test/Docker-ITK-vX.Y.Z[_qualifier]/Dockerfile

Then, after adding or updating an environment, build and publish images:

cd test/Docker
make push

And finally, create a PR with your changes.

How to integrate a PR

Getting your contributions integrated is relatively straightforward, here is the checklist:

• All tests pass

• Consensus is reached. This requires that a reviewer adds an "approved" review via GitHub with no changes requested, and a reasonable amount of time passed without anyone objecting.

Next, there are two scenarios:

• You do NOT have push access: A SlicerExecutionModel core developer will integrate your PR.
• You have push access: Simply click on the "Merge pull request" button.

Then, click on the "Delete branch" button that appears afterward.

License

See http://www.slicer.org/copyright/copyright.txt for details.

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

slicerexecutionmodel's Issues

find_package() perform in SEMMacroBuildCLI and ITK_LIBRARIES issues

When using the SEMMacroBuildCLI, the macro perfoms a find_package() step:
https://github.com/Slicer/SlicerExecutionModel/blob/master/CMake/SEMMacroBuildCLI.cmake#L103
This step leads to another find_package() step in UseGenerateCLP.cmake:
https://github.com/Slicer/SlicerExecutionModel/blob/master/GenerateCLP/UseGenerateCLP.cmake.in#L20
In this last step, the CMake variable ITK_LIBRARIES is reset.
This reset could lead to link errors, mostly related to the IO factory mechanism if the ITK_LIBRARIES is used again after calling the SEMMacroBuildCLI, i.e. if one calls the macro multiple times to compile multiple CLIs in the same CMakeLists.txt file.
When trying to compile the example available here (branch ITKLinkError):
https://github.com/fbudin69500/3D4DImageConversion/blob/ITKLinkError/CMakeLists.txt
It results in linking error messages:
Linking CXX executable bin/4DTo3DImages
lib/lib4DTo3DImagesLib.a(4DTo3DImages.cxx.o):(.data+0x3a0): undefined reference to itk::NiftiImageIOFactoryRegister__Private()' lib/lib4DTo3DImagesLib.a(4DTo3DImages.cxx.o):(.data+0x3a8): undefined reference toitk::NrrdImageIOFactoryRegister__Private()'
lib/lib4DTo3DImagesLib.a(4DTo3DImages.cxx.o):(.data+0x3b0): undefined reference to `itk::MetaImageIOFactoryRegister__Private()'

One easy solution for the user is to save the value of the ITK_LIBRARIES in a different variable. However, this should be done directly in SEMMacroBuild.

This issue is partly related to:
#8

Add COMPILE_FLAGS argument to SEMMacroBuildCLI

Sometimes it might be useful to disable warnings when compiling a CLI.
The naive manual solution to call set_target_properties(${MODULE_NAME}Lib PROPERTIES COMPILE_FLAGS "-Wno-overloaded-virtual") fails because "-Dmain-=ModuleEntryPoint" is no longer defined.

Add CMake version variables and tag repository

SEM add_compiler_export_flags should be removed

loading initial cache file /Shared/johnsonhj/Binaries/Darwin/BT-11/SlicerExecutionModel-prefix/tmp/SlicerExecutionModel-cache-Release.cmake
CMake Deprecation Warning at /usr/local/Cellar/cmake/3.10.1/share/cmake/Modules/GenerateExportHeader.cmake:382 (message):
  The add_compiler_export_flags function is obsolete.  Use the
  CXX_VISIBILITY_PRESET and VISIBILITY_INLINES_HIDDEN target properties
  instead.
Call Stack (most recent call first):
  ModuleDescriptionParser/CMakeLists.txt:154 (add_compiler_export_flags)

Only installs one shared library

Only one file is installed: lib/ModuleDescriptionParser/libModuleDescriptionParser.so
Is this what's intended?

The description says: project that provides macros and associated tools.

Provide a hook so that cuda_add_executable can be used

Duplicated flags in the XML should generate errors during build

Currently, no errors / warnings are generated if flags are duplicated in the XML schema. You will then get indeterminate runtime errors.

Update tclap version to 1.22 & add UpdateFromUpstream.sh script

See also https://github.com/Slicer/tclap/

Error while compiling with ITKv4.9.0

Compilation error when building SlicerExecutionModel in a x86_64 GNU/Linux system.

[  7%] Building CXX object ModuleDescriptionParser/CMakeFiles/ModuleDescriptionParser.dir/ModuleDescriptionParser.cxx.o
In file included from include/ITK-4.9/expat.h:11:0,
                 from SlicerExecutionModel/ModuleDescriptionParser/ModuleDescriptionParser.cxx:24:
include/ITK-4.9/expatDllConfig.h:5:30: fatal error: itk_expat_mangle.h: No such file or directory
compilation terminated.

Longflag should support dash ('-') chars

command line applications typically have dashes in the names of their long flags.
It should be possible to have parameters such as:
--no-something
--use-something
...

Spec: Add support for unit

See http://www.na-mic.org/Bug/view.php?id=4225

Cc: @vovythevov

Add CLI Support for the new vtkMRMLTableNode.

We are creating a new vtkMRMLTableNode. See Slicer issue #3912 at http://na-mic.org/Mantis/view.php?id=3912
This ticket will track the development to add CLI support for this new data type in SEM.

paramType in schema improperly described

Looking at the schema allowing to validated the XML description, I noticed that the paramType is specified as a xsd:choice of name, flag, longflag, ... which is probably incorrect.

For example, I believe name is mandatory, at least flag or longflag should be specified, ...

See https://github.com/Slicer/SlicerExecutionModel/blob/master/ModuleDescriptionParser/ModuleDescription.xsd#L52

For reference, what motivated the creation of this issue if what's discussed here: http://www.na-mic.org/Bug/view.php?id=1828

Add windows testing

Define C++ pre-processor code via SEMMacroBuildCLI

Sometimes it is useful to pass CMake variables into C++ code. The way to do it is to use the CMake configure_file() macro. It takes a *.h.in as input and generate a .h file.
SEMMacroBuildCLI already generate a *CLP.h file. It would be great if we could define pre-processor variables by passing a simple SEMMacroBuildCLI parameter. For example:
in cmake:

SEMMacroBuildCLI( MyCLI PREPROCESSOR MY_CMAKE_VARIABLE ...)

If MY_CMAKE_VARIABLE is True, that would generate in the CLP.h file:
#define MY_CMAKE_VARIABLE
which could then be used in the MyCLI.cxx file:

int main(int argc, char* argv[])
{
  PARSE_ARGS
#ifdefined MY_CMAKE_VARIABLE
  // do something here
#endif
  ...
}

Relax ITK version requirement

Is ITK v4.5 really required? I was able build against ITK v4.3.1 (using all default build options) and things seem to still work fine (100% tests passed)

OUTPUT_DIRECTORY variables cannot be changed

SlicerExecutionModel_DEFAULT_CLI___OUTPUT_DIRECTORY and SlicerExecutionModel_DEFAULT_CLI_INSTALL___DESTINATION variables' default value is used when using SEMMacroBuildCLI.cmake because of find_package(SlicerExecutionModel REQUIRED GenerateCLP) (line 100). One solution is to set those varialbes as CACHE variables when setting their default value in SlicerExecutionModelConfig.cmake.

GenerateCLP should be independent of ITKv4

ITK is needed only because itksys is used. Instead, kwsys should either be integrated (copying the source :( or using a git submodule :) )

find_package(ITK 4.5 COMPONENT) corrupted by SEM

The CMakeLists.txt file below is a simple example that demonstrates that including SEM before ITK causes find_package(ITK COMPONENTS ) to be corrupted.

SlicerExecutionModel hash 865e2a7 does not seem to exhibit this behavior.

cmake_minimum_required(VERSION 3.0)
project(test)

set(ITK_IO_MODULES_USED "")
set(ITK_MODULES_USED
ITKAnisotropicSmoothing
ITKBinaryMathematicalMorphology
ITKCommon
ITKConnectedComponents
ITKCurvatureFlow
ITKDeprecated
ITKDiffusionTensorImage
ITKDisplacementField
ITKDistanceMap
ITKFFT
ITKFastMarching
ITKHDF5
ITKIOBMP
ITKIOBioRad
ITKIODCMTK
ITKIOGDCM
ITKIOGE
ITKIOGIPL
ITKIOImageBase
ITKIOJPEG
ITKIOLSM
ITKIOMeta
ITKIONIFTI
ITKIONRRD
ITKIOPNG
ITKIORAW
MGHIO
ITKIOSpatialObjects
ITKIOStimulate
ITKIOTIFF
ITKIOTransformBase
ITKIOXML
ITKImageAdaptors
ITKImageCompare
ITKImageCompose
ITKImageFeature
ITKImageFilterBase
ITKImageFunction
ITKImageFusion
ITKImageGradient
ITKImageGrid
ITKImageIntensity
ITKImageSources
ITKImageStatistics
ITKLabelMap
ITKLabelVoting
ITKLevelSets
ITKMathematicalMorphology
ITKMesh
ITKMetricsv4
ITKOptimizers
ITKOptimizersv4
ITKPDEDeformableRegistration
ITKQuadEdgeMesh
ITKQuadEdgeMeshFiltering
ITKRegionGrowing
ITKRegistrationCommon
ITKRegistrationMethodsv4
ITKReview
ITKSmoothing
ITKSpatialObjects
ITKStatistics
ITKTestKernel
ITKThresholding
ITKTransform
ITKV3Compatibility
${ITK_VTK_COMPONENTS}
${ITK_IO_MODULES_USED}
)

option(SEMFIRST "Include SEM First" ON)

if(SEMFIRST)

THIS FAILS

find_package(SlicerExecutionModel REQUIRED GenerateCLP)
include(${GenerateCLP_USE_FILE})
include(${SlicerExecutionModel_USE_FILE})
include(${SlicerExecutionModel_CMAKE_DIR}/SEMMacroBuildCLI.cmake)
message(STATUS "YYYY ${ITK_INCLUDE_DIRS}")

find_package(ITK 4.5 COMPONENTS
${ITK_MODULES_USED}
REQUIRED
)
include(${ITK_USE_FILE})
message(STATUS "XXXX ${ITK_INCLUDE_DIRS}")
else()

THIS WORKS

find_package(ITK 4.5 COMPONENTS
${ITK_MODULES_USED}
REQUIRED
)
include(${ITK_USE_FILE})
find_package(SlicerExecutionModel REQUIRED GenerateCLP)
include(${GenerateCLP_USE_FILE})
include(${SlicerExecutionModel_USE_FILE})
include(${SlicerExecutionModel_CMAKE_DIR}/SEMMacroBuildCLI.cmake)
message(STATUS "YYYY ${ITK_INCLUDE_DIRS}")

message(STATUS "XXXX ${ITK_INCLUDE_DIRS}")

endif()

Fix and simplify continuous integration

As of August 13th 2021, all jobs are failing:

Source: https://app.circleci.com/pipelines/github/Slicer/SlicerExecutionModel/19/workflows/ed1fad58-f7e9-4df7-8887-ddf82dd7e454

Suggested plan:

• Remove these obsolete jobs:

  SlicerExecutionModel/.circleci/config.yml

  Lines 56 to 58 in f19d6e8

  	- itk-v4.8.0_use_system_libraries-off
  	- itk-v4.10.1_use_system_libraries-off
  	- itk-v4.13.0_use_system_libraries-off

• Update itk-master_use_system_libraries-off docker image

• Add itk-v5.1.2_use_system_libraries-off and itk-v5.2.1_use_system_libraries-off

For more details:

• https://github.com/Slicer/SlicerExecutionModel#adding-or-updating-a-testing-environment
• https://github.com/Slicer/SlicerExecutionModel/tree/master/test

Document usage of SEMMacroBuildCLI

Comment associated with these issues could be integrated in the documentation:

• #7 (comment)
• #26 (comment)

Add support for CMake function target_compile_features

When compiling a target that has specific C++ requirements, one can use the new CMake function target_compile_features() [1].

There is currently no way of passing these requirements to SEMMacroBuildCLI, and the targets created in the macro are not exposed to the user.
It would be great to either be able:

1. to pass the requirements to the CMake macro or
2. to expose the list of targets created by the macro.

Note: Both solutions are not mutually exclusive and even if 1) is implemented, 2) might still be useful in certain corner cases.

[1] https://cmake.org/cmake/help/v3.6/command/target_compile_features.html

Add ITK/VTK plugin watcher utility classes

In order for CLI to report progress, the following output is the standard:

<filter-start>
 <filter-name>...</filter-name>
 ...
</filter-start>
<filter-progress>
...

There is in Slicer (Base/CLI) some utility classes that observe ITK and VTK filters to automatically generate such output.
Such classes should be moved in SlicerExecutionModel as conveniency (and be enabled only if ITK_DIR or VTK_DIR are passed).
A generic progress watcher (whose itkPluginFilterWatcher and vtkPluginFilterWatcher should derive from) should also be added.

class SEMProgressWatcher
{
public:
  SEMProgressWatcher(const char* name, const char* comment = 0, ModuleProcessInformation* inf = 0, double fraction = 1.0, double start = 0.0);
  virtual void Start();
  virtual void SetProgress(double progress);
  virtual void End();
  virtual double GetProgress();
protected:
  void ReportStart();
  void ReportEnd();
  void ReportProgress();
};
class itkPluginFilterWatcher: public itkSimpleFilterWatcher, SEMProgressWatcher
{
...
};
class vtkPluginFilterWatcher: public SEMProgressWatcher
{
...
};

KWSys / itksys no longer has a SharedForward.h.in file

ITK has updated its version of KWSys (the source of itksys).

InsightSoftwareConsortium/ITK@2c45eec

That update introduced a version of KWSys that does not contain the SharedForward.h.in file, as required by GenerateCLPLauncher.

SlicerExecutionModel/GenerateCLP/GenerateCLPLauncher.c.in

Line 19 in bd2efa9

#include <itksys/SharedForward.h>

SlicerExecutionModel (specifically the GenerateCLPLauncher) will not compile with ITK master.

Fix Parser1Test3 on windows

For reference:

• #124 (comment)

Error when building GenerateCLPLauncher on latest master

[100%] Building C object GenerateCLP/CMakeFiles/GenerateCLPLauncher.dir/GenerateCLPLauncher.c.o
/Users/zach/dev/covalic-score-build/SlicerExecutionModel-Build/GenerateCLP/GenerateCLPLauncher.c:15:4: error: expected identifier or '('
   "/Users/zach/dev/covalic-score-build/SlicerExecutionModel-Build/ModuleDescriptionParser/bin" CONFIG_DIR_POST \
   ^
In file included from /Users/zach/dev/covalic-score-build/SlicerExecutionModel-Build/GenerateCLP/GenerateCLPLauncher.c:29:
/Users/zach/dev/covalic-score-build/Insight-Build/Modules/ThirdParty/KWSys/src/itksys/SharedForward.h:720:76: error: expected expression
  static const char* search_path_build[] = {KWSYS_SHARED_FORWARD_PATH_BUILD, 0};
                                                                           ^
/Users/zach/dev/covalic-score-build/Insight-Build/Modules/ThirdParty/KWSys/src/itksys/SharedForward.h:721:80: error: expected expression
  static const char* search_path_install[] = {KWSYS_SHARED_FORWARD_PATH_INSTALL, 0};

ping @vovythevov @jcfr

Add support for Temporary directory

Discussion: http://slicer-devel.65872.n3.nabble.com/Temporary-directory-for-CLI-modules-td4035464.html

Reason to reject initial patch from @ prastawa : Patch Slicer/Slicer#400 will be problematic when the CLI is executed directly without Slicer.

Proposal:

• CLI should always expect the environment variable SEM_TEMPORARY_DIR to be set
• SEM_TEMPORARY_DIR can be set in three different ways:
  1. Set by reading the value of an other environment variable. This other environment variable would be set configuring SlicerExecutionModel using for example -DSEM_TEMPORARY_DIR_ENVVAR_NAME:STRING=SLICER_TEMPORARY_DIR
  2. Set by reading the value of special parameter named --sem-temporary-dir. If specified, other environment variable would be ignored
  3. If neither (1) or (2) have been set, temporary directory will be set to the operating system default
• code in charge of reading the other environment variable, or special parameter would be embedded in the CLI thanks to code generated by GenerateCLP

Remove dependency on "itksys/sstream"

Providing CLI source file having extension cpp

Respect multi-line strings while generating CLP files

See: http://na-mic.org/Mantis/view.php?id=2781

Create a Docker Organization for Pull Request #46

Pull Request #46 is adding CircleCI verification to this repository using Docker.
The actual Docker repository is a private one ( https://hub.docker.com/r/lucasgandel/slicerexecutionmodel/ ) and should be replaced by a new Organization for SlicerExecutionModel.
Once this organization is created, changes can be made to the pull request to point to that new repository.

Installation of SlicerExecutionModel

When running the install step of SlicerExecutionModel, only one file is installed:
<CMAKE_INSTALL_PREFIX>/lib/ModuleDescriptionParser/libModuleDescriptionParser.a
It is therefore not possible to install SlicerExecutionModel and use it. It would be great to have a real install step that would install all the necessary files to be able to actually install SlicerExecutionModel.