gmlc-tdc / helics Goto Github PK
View Code? Open in Web Editor NEWHierarchical Engine for Large-scale Infrastructure Co-Simulation (HELICS)
Home Page: https://docs.helics.org/en/latest/
License: BSD 3-Clause "New" or "Revised" License
Hierarchical Engine for Large-scale Infrastructure Co-Simulation (HELICS)
Home Page: https://docs.helics.org/en/latest/
License: BSD 3-Clause "New" or "Revised" License
Right there are two different JSON readers in HELICS. One can handle comments which is what is used for the federate configuration, but the one used in the players cannot handle comments. We should add the ability to strip off any comments in the player and recorder configuration file or switch JSON readers to one that handles them.
The version of Boost used by Travis is really old (1.54) -- the configuration should be changed to install the minimum version of Boost that we are targeting.
helicsGetSubscriptionType
only returns default subscription type, def
.Need some way to automatically test the Matlab interface. Given it uses Matlab and a custom version of swig this may not be workable in the automated tests, but we should be able to set something up in a nightly build test.
The echo app will echo any messages received back to the original source with a specified time delay.
The current implementation of the player and recorder have no ability to capture or send iterations on the time.
I think we should have a mac build in the travis scripts. I regularly build on Windows and Linux, but mac is a little more challenging as I don't have access to any mac systems older than Sierra. Having a build in Travis with a slightly older version using appleclang would test out a few bits of code we don't currently build against.
the json configuration for filters is not well established or tested, and a few mechanics of filters need to be worked out.
@abhyshr made the comment
Rename HelicsEnterXXXMode to HelicsXXXBegin() or HelicsXXXInitiliaze, and a corresponding HelicsXXXClose() or HelicsXXXFinalize(). I am saying this because I find the name helicsEnterXXXModelFinalize() confusing.
This is worthy of a discussion but it is different from the naming conventions so I want to separate it out.
Helics has 5 states
created (after construction and before global init) all registration and parameter modification allowed
initialized (initial value computation, value exchange before time 0)
executing (normal simulation and time advancement)
terminated (no further interfederate communication)
error
right now the function call names to switch state are somewhat confusing.
the core has
void enterInitializingState (federate_id_t federateID);
iteration_result enterExecutingState (federate_id_t federateID, iteration_request iterate = NO_ITERATION)`
virtual void finalize (federate_id_t federateID)
the application api also has asynchronous calls
such as
void enterExecutionStateAsync (iteration_request iterate = iteration_request::no_iterations);
iteration_result enterExecutionStateFinalize ();
the C interface should probably be made to mirror the application API calls but that still raises the question of whether the C++ api names and for that matter the Core names are sufficiently clear and if not how do we make them clearer
We will need a MPI core and Broker to work on the HPC systems more directly than we can do currently.
This and the TCP core will complete all currently planned cores.
The query interface needs to be further developed and tested with different queries and results for the different objects.
The argParser class uses boost::program options and creates a wrapper around that. It currently is used in the brokers and cores, but with a little more effort and a few enhancements it could be used by a number of other objects which would simplify the code and remove a number of nearly redundant functions scattered throughout HELICS.
We should have a set of test cases in the C-interface-tests that match the application api tests. Probably don't need them all but having a set of them running against the C-interface vs the C++ interface would test that, especially if we are going to be using that pretty regularly for python, java, and other bindings, and other applications.
Many of the functions in the C-shared api are currently lacking doxygen comments and descriptions.
Looks like the version check in the python CMake files doesn't work for cmake 3.10, it produces a bunch of warnings on deprecated SWIG commands.
We probably should add the python library build into Travis to make sure that doesn't break on updates
The cloning filter feature is not fully developed and is needed by the recorder for message capture
@nightlark brought up a good point in the review of the recorder update that there would probably be a need for an "echo" app that would "display" any messages received or values updates. This would be distinct from the recorder app in that it wouldn't have the ability to capture to a file or use its own endpoints, but it would use the same input formats and also a potential callback function for use in other applications.
The intent of the Echo app currently in place was to function like a "ping" in that it would reply to all messages sent to it with the same message with a specified time delay.
We can devise different names for these if necessary.
Another question is if we can think of any other simple apps that would be useful for testing?
Add CMAKE switches to separate BUILD_PYTHON from GENERATE_PYTHON
We need to add a logging callback to the C interface.
I am not entirely sure what this looks like yet, whether it should be on a per federate/core basis, or a global callback or both?
Also need to consider how callbacks will work with the python/matlab/java interfaces
Minimal Working Example below:
// test.cpp
#include "helics.h";
#include <stdio.h>
int main() {
char *result = 0;
result = (char *)helicsGetVersion();
printf("%s\n", result);
return 0;
}
clang test.cpp -I../path/to/HELICS-src/src/shared_api_library -L../path/to/helics_install/lib -lhelicsSharedLib
# (Optional) install_name_tool -change @rpath/libhelicsSharedLib.dylib ../path/to/helics_install/lib/libhelicsSharedLib.dylib
./a.out
0.3.0 (10-16-17)
Minimal Example of Mex not working below:
//test.cpp
#include "mex.h"
#include "helics.h"
void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[]) {
char *result = 0;
result = (char *)helicsGetVersion()
}
mex -I../path/to/HELICS-src/src/shared_api_library -L../path/to/helics_install/lib test.cpp helicsSharedLib
matlab
>>> test()
% Hang indefinitely
Create a simpler(Quicker) set of test cases to run for the CI integration. We want to run a subset of the test cases on a continual basis to check each commit, like we do with compilation now.
The goal would be to have this running with the tests for the 0.9 release.
The specification of remote connections to brokers and other objects needs to be a little less complicated
as an example
./messageFed --name="fed1" --target="fed2" --coreinit="--broker_address=\"tcp://10.254.21.101\"" --local_interface=\"tcp://10.254.21.86\""
ideally something like this would probably be better
./messageFed --name="fed1" --target="fed2" --broker="tcp://10.254.21.101"
with the local_interface being inferred from as either external or internal based on the broker specified
the broker had to be specified like
2 --local_interface="tcp://10.254.21.101"
This is also an interesting question of whether the broker should by default open up external ports or not.
A number of *.in files for cmake configuration are in the main directory, these should be moved to a subfolder and in the process move the cmake configuration to a better named directory.
Should have a couple examples fully documented and easy to use
I am thinking the messageFed, and valueFed and piSender and piReceiver
basically more completely documenting the code and also linking a web page in the documentation that highlights them.
and some easy scripts to run them.
I'm installing from source using HELICS develop branch and using Python 3.6
➜ python
Python 3.6.3 |Anaconda, Inc.| (default, Dec 5 2017, 17:30:25)
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import helics
Fatal Python error: PyThreadState_Get: no current thread
[1] 99554 abort python
There are no build errors, but when I import helics it causes a fatal python error.
Since we have a Matlab interface does it make sense to do the same for Octave? I noticed that Swig had a note in there releases about supporting Octave 4.2 so it seems it may be supported in the standard swig.
-- Using Boost include files : /Users/travis/build/GMLC-TDC/HELICS-src/dependencies/boost/include
CMake Error at cmake-3.4.3-Darwin-x86_64/CMake.app/Contents/share/cmake-3.4/Modules/FindPackageHandleStandardArgs.cmake:148 (message):
Could NOT find SWIG (missing: SWIG_EXECUTABLE SWIG_DIR)
Call Stack (most recent call first):
cmake-3.4.3-Darwin-x86_64/CMake.app/Contents/share/cmake-3.4/Modules/FindPackageHandleStandardArgs.cmake:388 (_FPHSA_FAILURE_MESSAGE)
cmake-3.4.3-Darwin-x86_64/CMake.app/Contents/share/cmake-3.4/Modules/FindSWIG.cmake:75 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
swig/CMakeLists.txt:4 (FIND_PACKAGE)
Looks like something is missing in the config for this build
Warning! PATH is not properly set up, '/Users/travis/.rvm/gems/ruby-2.0.0-p648/bin' is not at first place,
usually this is caused by shell initialization files - check them for 'PATH=...' entries,
it might also help to re-add RVM to your dotfiles: 'rvm get stable --auto-dotfiles',
to fix temporarily in this shell session run: 'rvm use ruby-2.0.0-p648'.
We get this warning in the build log for the mac builds. Probably should fix this at some point
How possible is it to add an Appveyor build to the integration testing?
Use the following for older versions of CMake (<3.8)
SWIG_ADD_MODULE(helics python ../helics.i)
instead of
SWIG_ADD_LIBRARY(helics TYPE MODULE LANGUAGE python SOURCES ../helics.i)
$ cmake --version
cmake version 3.10.1
$ gcc --version
Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 9.0.0 (clang-900.0.39.2)
Target: x86_64-apple-darwin16.7.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
$ # boost version
$ ls /usr/local/Cellar/boost/1.66.0/
INSTALL_RECEIPT.json include lib
make
[ 14%] Built target application_api
[ 23%] Built target helics_common
[ 24%] Building CXX object src/helics/core/CMakeFiles/helics_core.dir/udp/UdpComms.cpp.o
In file included from /Users/$USER/GitRepos/HELICS-src/src/helics/core/udp/UdpComms.cpp:12:
In file included from /Users/$USER/GitRepos/HELICS-src/src/helics/core/udp/../../common/AsioServiceManager.h:30:
/usr/local/include/boost/asio/io_service.hpp:27:20: error: typedef redefinition with different types ('boost::asio::io_context' vs
'boost::asio::io_service')
typedef io_context io_service;
^
/Users/$USER/GitRepos/HELICS-src/src/helics/core/udp/UdpComms.h:23:15: note: previous definition is here
class io_service;
^
In file included from /Users/$USER/GitRepos/HELICS-src/src/helics/core/udp/UdpComms.cpp:12:
/Users/$USER/GitRepos/HELICS-src/src/helics/core/udp/../../common/AsioServiceManager.h:39:34: error: incomplete type 'boost::asio::io_service' named in
nested name specifier
std::unique_ptr<boost::asio::io_service::work> nullwork; //!< pointer to an object used to keep a service running
~~~~~~~~~~~~~^~~~~~~~~~~~
/Users/$USER/GitRepos/HELICS-src/src/helics/core/udp/UdpComms.h:23:15: note: forward declaration of 'boost::asio::io_service'
class io_service;
^
/Users/$USER/GitRepos/HELICS-src/src/helics/core/udp/UdpComms.cpp:141:17: error: no matching constructor for initialization of 'udp::socket' (aka
'basic_datagram_socket<boost::asio::ip::udp>')
udp::socket socket (ioserv->getBaseService ());
^ ~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/include/boost/asio/basic_datagram_socket.hpp:46:7: note: candidate constructor (the implicit copy constructor) not viable: cannot convert
argument of incomplete type 'boost::asio::io_service' to 'const boost::asio::basic_datagram_socket<boost::asio::ip::udp>' for 1st argument
class basic_datagram_socket
^
/usr/local/include/boost/asio/basic_datagram_socket.hpp:73:12: note: candidate constructor not viable: cannot convert argument of incomplete type
'boost::asio::io_service' to 'boost::asio::io_context &' for 1st argument
explicit basic_datagram_socket(boost::asio::io_context& io_context)
^
/usr/local/include/boost/asio/basic_datagram_socket.hpp:151:3: note: candidate constructor not viable: cannot convert argument of incomplete type
'boost::asio::io_service' to 'boost::asio::basic_datagram_socket<boost::asio::ip::udp>' for 1st argument
basic_datagram_socket(basic_datagram_socket&& other)
^
/usr/local/include/boost/asio/basic_datagram_socket.hpp:185:3: note: candidate template ignored: could not match
'basic_datagram_socket<type-parameter-0-0>' against 'boost::asio::io_service'
basic_datagram_socket(
^
/usr/local/include/boost/asio/basic_datagram_socket.hpp:90:3: note: candidate constructor not viable: requires 2 arguments, but 1 was provided
basic_datagram_socket(boost::asio::io_context& io_context,
^
/usr/local/include/boost/asio/basic_datagram_socket.hpp:112:3: note: candidate constructor not viable: requires 2 arguments, but 1 was provided
basic_datagram_socket(boost::asio::io_context& io_context,
^
/usr/local/include/boost/asio/basic_datagram_socket.hpp:133:3: note: candidate constructor not viable: requires 3 arguments, but 1 was provided
basic_datagram_socket(boost::asio::io_context& io_context,
^
/Users/$USER/GitRepos/HELICS-src/src/helics/core/udp/UdpComms.cpp:233:19: error: no matching constructor for initialization of 'udp::resolver'
(aka 'basic_resolver<boost::asio::ip::udp>')
udp::resolver resolver (ioserv->getBaseService ());
^ ~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/include/boost/asio/ip/basic_resolver.hpp:67:7: note: candidate constructor (the implicit copy constructor) not viable: cannot convert argument
of incomplete type 'boost::asio::io_service' to 'const boost::asio::ip::basic_resolver<boost::asio::ip::udp>' for 1st argument
class basic_resolver
^
/usr/local/include/boost/asio/ip/basic_resolver.hpp:100:12: note: candidate constructor not viable: cannot convert argument of incomplete type
'boost::asio::io_service' to 'boost::asio::io_context &' for 1st argument
explicit basic_resolver(boost::asio::io_context& io_context)
^
/usr/local/include/boost/asio/ip/basic_resolver.hpp:116:3: note: candidate constructor not viable: cannot convert argument of incomplete type
'boost::asio::io_service' to 'boost::asio::ip::basic_resolver<boost::asio::ip::udp>' for 1st argument
basic_resolver(basic_resolver&& other)
^
/Users/$USER/GitRepos/HELICS-src/src/helics/core/udp/UdpComms.cpp:235:17: error: no matching constructor for initialization of 'udp::socket' (aka
'basic_datagram_socket<boost::asio::ip::udp>')
udp::socket transmitSocket (ioserv->getBaseService ());
^ ~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/include/boost/asio/basic_datagram_socket.hpp:46:7: note: candidate constructor (the implicit copy constructor) not viable: cannot convert
argument of incomplete type 'boost::asio::io_service' to 'const boost::asio::basic_datagram_socket<boost::asio::ip::udp>' for 1st argument
class basic_datagram_socket
^
/usr/local/include/boost/asio/basic_datagram_socket.hpp:73:12: note: candidate constructor not viable: cannot convert argument of incomplete type
'boost::asio::io_service' to 'boost::asio::io_context &' for 1st argument
explicit basic_datagram_socket(boost::asio::io_context& io_context)
^
/usr/local/include/boost/asio/basic_datagram_socket.hpp:151:3: note: candidate constructor not viable: cannot convert argument of incomplete type
'boost::asio::io_service' to 'boost::asio::basic_datagram_socket<boost::asio::ip::udp>' for 1st argument
basic_datagram_socket(basic_datagram_socket&& other)
^
/usr/local/include/boost/asio/basic_datagram_socket.hpp:185:3: note: candidate template ignored: could not match
'basic_datagram_socket<type-parameter-0-0>' against 'boost::asio::io_service'
basic_datagram_socket(
^
/usr/local/include/boost/asio/basic_datagram_socket.hpp:90:3: note: candidate constructor not viable: requires 2 arguments, but 1 was provided
basic_datagram_socket(boost::asio::io_context& io_context,
^
/usr/local/include/boost/asio/basic_datagram_socket.hpp:112:3: note: candidate constructor not viable: requires 2 arguments, but 1 was provided
basic_datagram_socket(boost::asio::io_context& io_context,
^
/usr/local/include/boost/asio/basic_datagram_socket.hpp:133:3: note: candidate constructor not viable: requires 3 arguments, but 1 was provided
basic_datagram_socket(boost::asio::io_context& io_context,
^
/Users/$USER/GitRepos/HELICS-src/src/helics/core/udp/UdpComms.cpp:462:31: error: no matching constructor for initialization of 'udp::resolver'
(aka 'basic_resolver<boost::asio::ip::udp>')
udp::resolver resolver (serv->getBaseService ());
^ ~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/include/boost/asio/ip/basic_resolver.hpp:67:7: note: candidate constructor (the implicit copy constructor) not viable: cannot convert argument
of incomplete type 'boost::asio::io_service' to 'const boost::asio::ip::basic_resolver<boost::asio::ip::udp>' for 1st argument
class basic_resolver
^
/usr/local/include/boost/asio/ip/basic_resolver.hpp:100:12: note: candidate constructor not viable: cannot convert argument of incomplete type
'boost::asio::io_service' to 'boost::asio::io_context &' for 1st argument
explicit basic_resolver(boost::asio::io_context& io_context)
^
/usr/local/include/boost/asio/ip/basic_resolver.hpp:116:3: note: candidate constructor not viable: cannot convert argument of incomplete type
'boost::asio::io_service' to 'boost::asio::ip::basic_resolver<boost::asio::ip::udp>' for 1st argument
basic_resolver(basic_resolver&& other)
^
/Users/$USER/GitRepos/HELICS-src/src/helics/core/udp/UdpComms.cpp:467:29: error: no matching constructor for initialization of 'udp::socket' (aka
'basic_datagram_socket<boost::asio::ip::udp>')
udp::socket transmitter (serv->getBaseService (), udp::endpoint (udp::v4 (), 0));
^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/include/boost/asio/basic_datagram_socket.hpp:112:3: note: candidate constructor not viable: cannot convert argument of incomplete type
'boost::asio::io_service' to 'boost::asio::io_context &' for 1st argument
basic_datagram_socket(boost::asio::io_context& io_context,
^
/usr/local/include/boost/asio/basic_datagram_socket.hpp:90:3: note: candidate constructor not viable: cannot convert argument of incomplete type
'boost::asio::io_service' to 'boost::asio::io_context &' for 1st argument
basic_datagram_socket(boost::asio::io_context& io_context,
^
/usr/local/include/boost/asio/basic_datagram_socket.hpp:185:3: note: candidate template ignored: could not match
'basic_datagram_socket<type-parameter-0-0>' against 'boost::asio::io_service'
basic_datagram_socket(
^
/usr/local/include/boost/asio/basic_datagram_socket.hpp:73:12: note: candidate constructor not viable: requires single argument 'io_context', but 2
arguments were provided
explicit basic_datagram_socket(boost::asio::io_context& io_context)
^
/usr/local/include/boost/asio/basic_datagram_socket.hpp:151:3: note: candidate constructor not viable: requires single argument 'other', but 2 arguments
were provided
basic_datagram_socket(basic_datagram_socket&& other)
^
/usr/local/include/boost/asio/basic_datagram_socket.hpp:46:7: note: candidate constructor (the implicit copy constructor) not viable: requires 1 argument,
but 2 were provided
class basic_datagram_socket
^
/usr/local/include/boost/asio/basic_datagram_socket.hpp:133:3: note: candidate constructor not viable: requires 3 arguments, but 2 were provided
basic_datagram_socket(boost::asio::io_context& io_context,
^
7 errors generated.
make[2]: *** [src/helics/core/CMakeFiles/helics_core.dir/udp/UdpComms.cpp.o] Error 1
make[1]: *** [src/helics/core/CMakeFiles/helics_core.dir/all] Error 2
make: *** [all] Error 2
To complete the networking protocols for the different core types we will need a core type using tcp communications.
Not that having a TCP core is particularly useful or critical at this point, but each one of the core types has distinct interthread communication methods and techniques and developing each one has led to improvements in all the others. When it comes to scalability later being able to directly compare them will be very interesting and insightful.
optional arguments that are listed as const char * in the C interface should be tolerant of nullptr or NULL as an argument.
@phlptp I am attaching the test-value-federate1.cpp
file that I am currently testing as part of the C-API testing task. As I discussed with you on Gitter, when helicsRegisterPublication
is called without the type
and units
(which are specified as ""
in the call), helicsGetPublicationKey
throws an exception.
There have been some reported issues with the Recorder not connecting properly to a federation.
Need some way to automatically test the python interface. If the python interface gets built we should have some means of testing that
this would probably be through a few test cases running in travis and ideally appveyor. as well as some more extensive tests we can do in a nightly build.
For some applications there needs to be an ability to generate multiple clones of all messages to a destination. This is currently not supported as only one destination filter is allowed.
The source app will automatically generate values for publications in a prescribed pattern
thinking
ramp source
periodic source
random source
maybe others
In [1]: import helics as h
In [2]: h.helicsCreateBroker("", "", "--help")
libc++abi.dylib: terminating with uncaught exception of type helics::HelicsException: unrecognized broker type
[1] 67607 abort ipython
object not clearing after global destruction force close
error: unable to destroy all objects giving up
libc++abi.dylib: terminating with uncaught exception of type std::__1::system_error: mutex lock failed: Invalid argument
[1] 54435 abort python bin/helics-broker.py
For some use cases it will be necessary to be able to translate between different communication mediums. For instance a broker the communicates between MPI and ZMQ when we are crossing over into an HPC boundary. This will be the role of the router. This issue is to capture development intent for the router and any questions and features on that topic.
The ZMQ broker occasionally throws errors when multiple cores connect near simultaneously. From @jeffdaily
terminate called after throwing an instance of 'zmq::error_t'
what(): Operation cannot be accomplished in current state
This occurs for example on a ZMQ_REP socket. “This socket type allows only an alternating sequence of zmq_recv(request) and subsequent zmq_send(reply) calls”. So if two sends or two recvs are issued back to back, this error occurs. I think this is happening when I start both the player and the recorder in quick succession such that their startups overlap. The broker must be attempting to respond to the connections/requests but the messages are arriving asynchronously and not ordered in a way that the current zmq sockets can handle. I’m hoping Phil you might recall the design of the zmq core and whether this is indeed possibly happening? There might be a solution where we use a ZMQ_DEALER or ZMQ_ROUTER?
I have been working on getting the install files a little better so we only install the needed public headers for the libraries. I was wondering what the opinions were on the directory organization of the headers.
I see two options:
All the headers in a single directory in the install
separate folders for the core, application api, and shared library.
the choice will alter the include_directories in cmake and in the files themselves. I kind of go back and forth on this so I was wondering what other peoples opinions were. This came up as a result of working on the helics-fmi and GridDyn stuff to import HELICS.
Need some way to automatically test the Java interface. If the java interface gets built we should have some means of testing that
this would probably be through a few test cases running in travis and ideally appveyor. as well as some more extensive tests we can do in a nightly build.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.