aphysci/gravity

A high-performance software framework for rapid prototyping and development of complex, distributed systems

Gravity

A high-performance software framework for rapid prototyping and development of complex, distributed systems.

Getting started

Please see the Wiki pages for the Gravity Build Guide that describes how to build Gravity for Linux and Windows.

Then take a look at the Gravity Examples that demonstrate and explain several key features of the Gravity framework.

gravity's Issues

Add README file that points to the wiki

People may not find the wiki on their own, but they'll normally look at the project README on where to get started.

Add ServiceDirectoryDomainUpdatePB to the python build

This will allow receiving domain updates in Python components.

Python synchronous request doesn't properly handle case where service does not exist

Application will crash as the binding code doesn't correctly handle non-GDP value returned on error.

Python binding segfaults when GravitySubscriber goes out of scope

If you subscribe using a GravitySubscriber object in a scope that later disappears, the python environment will segfault.

I suspect that the GravitySubscriber's reference count is not being appropriately incremented in GravityNode::subscribe(), but I'm not familiar enough with SWIG to fix it quickly.

This will cause a segfault:

#!/usr/bin/env python

import gravity

class MySubscriber(gravity.GravitySubscriber):
  def subscriptionFilled(self, dataProducts):
    print "Success."

def subscribe_in_disappearing_scope(node):
  subscriber = MySubscriber()
  node.subscribe("SomePublicationThatExists", subscriber)


node = gravity.GravityNode()
node.init("IAmGonnaSegfault")
subscribe_in_disappearing_scope(node)
node.waitForExit()

Problem with REQUEST_TIMEOUT init node.

I need some guidance. I am having the problem where I unable to start a node, and or unable to make a connection. I will have 6+ nodes already started, and service directory seems to be ok, but if I try starting another one I just keep getting -3 returned, which is TIMEOUT I believe. This is within a single machine, using tcp. Using netstat, I can see the connection being made between the 2 processes, but they go away, and the retry happens again. It seems to happen more often when I restart the node process. And seems worse as the more nodes I have running.

Watching netstat here is what I see.

1. I see the tcp connection to the port 5555 to ServiceDirectory, but the node local port changes every 2 seconds, then after 6 seconds or so it gives up.
2. Restarting the node doesn't help, but restarting the ServiceDirectory does work.
3. Sometimes the node does succesfully connect, but there are no tcp connections made, and data product is not received correctly.
4. When I restart ServiceDirectory I get 2x number of connected tcp sockets.

Any ideas? I am able to reproduce, so let me know if you would like some more info.

Bug in Domain sync for 64-bit architectures

There's an issue in the parsing of the DomainSyncList that can cause an infinite loop on 64-bit machines.

Python binding wrongly de-allocates handler on unsubscribe()

It seems like the C++ object is being (wrongly) deallocated on unsubscribe, which causes all sorts of issues on the Python side. Example:

#!/usr/bin/env python
import time
import gravity

class Handler(gravity.GravitySubscriber):
    def subscriptionFilled(self, dataProducts):
        pass

handler = Handler()
node = gravity.GravityNode()
node.init("SegFaultCity")
node.subscribe("SegFaultsRUs", handler)

while True:
    print handler, type(handler)
    time.sleep(0.1)
    node.unsubscribe("SegFaultsRUs", handler)

The results when running it vary:

<__main__.Handler; proxy of <Swig Object of type 'gravity::GravitySubscriber *' at 0x7f9c253cfa20> > <class '__main__.Handler'>
<__main__.Handler; proxy of <Swig Object of type 'gravity::GravitySubscriber *' at 0x7f9c253cfa20> > <class '__main__.Handler'>
<__main__.Handler;  > <class '__main__.Handler'>
Segmentation fault

or

<__main__.Handler; proxy of <Swig Object of type 'gravity::GravitySubscriber *' at 0x7f502462a9c0> > <class '__main__.Handler'>
<__main__.Handler; proxy of <Swig Object of type 'gravity::GravitySubscriber *' at 0x7f502462a9c0> > <class '__main__.Handler'>
Fatal Python error: Inconsistent interned string state.
Aborted

I also managed to have handler get interpreted as <listiterator at 0x7fbb14093f50> and (<sqlite3.Connection at 0x7fbb17831dc8>,) which are some pretty creative options.

GravityNode(std::string componentID) constructor should be deprecated

This constructor calls GravityNode::init for the user, but that should be called in their own code so that they can check the return code and retry as necessary (if desired).

We should also make sure all examples (wiki and code) show the correct usage.

Apparent memory leak in GravityPublishManager

By default, the last value of all published data products are cached so that new subscribers can receive the last published value. This can be disabled by setting CacheLastSentDataproduct to false in the Gravity.ini. There appears to be an issue in the code that's preventing memory allocated to handle the publish when this flag is set to false causing a memory leak that can quickly blow up the process if a lot of data is being published.

Service Directory Registrations are not cleared when component is removed

When a component is killed or otherwise removed from the Gravity system the Service Directory registrations are not updated to reflect this. This can introduce problems when those registered ports are later reused.

MATLABProtobufGenerator is dependent on Java 8

The newly added MATLABProtobufGenerator is dependent on Java 8 which limits its usefulness for older versions of MATLAB. This tool should be compatible with Java 6.

make distributable target is confusing and unnecessary

The default 'all' target should perform the tasks currently performed by the distributable target, including coping files to bin, lib & include.

GravityNode unregisterHeartbeatListener() within MissedHeartbeat() deadlock

Calling unregisterHeartbeatListener() within the context of MissedHeartbeat() causes an apparent thread deadlock.

Failed attempt to register data product returns SUCCESS

When no service directory is present, a failed attempt to register a data product returns as success.

test/components/cpp do not compile

None of the stuff in test/components/cpp compiles (except ServiceDirectory and ServiceDirectoryReregister).

The top level Makefile which appears generated by configure has the right libraries, but bad paths. Using -L../../../../lib instead of -L../../../lib

The directories in components/cpp all contain makefiles with seem no longer used.
They refer to libraries that don't appear to be part of gravity any more (iniparser, wsock32)

Also, the HearbeatTest attempts MyHeartbeatListener listener;, which fails because the listener class is pure virtual.

Having some shutdown problems, possible threading issue with Logger

So I am having a problem running UnitTest where I get a SIGSEGV at GravityLogger.cpp : 291. If I do not have a ServiceController running, I get this error everytime. The other info besides the SIGSEGV is I added some debug to vLog and as soon as I get the failure "Could not initialize gravity" then when it gets into vLog, the loggers list is the same size, but the Loggers are not valid.

Looking at the code the first thing that jumps out at me is there possibly multiple threads changing Log::loggers, so that when things try to shutdown, there is a race condition in vLog since after getting the iterators, the loggers list could change. Maybe should add a mutex to coordiate access to loggers?

Any ideas?

Allow for debug flag in cpp build.

So I have tried using 'export CFLAGS='-g -O0' to enable debug symbols before runing configure, and this adds this flag to the library compile, but not the objects themselves. Can you add $(AC_FLAGS) to make @$(CC) $(COUTPUT_FLAG)$@ $(COMPILE_FLAG) $(CFLAGS) $(AC_CFLAGS) $< in gravity/src/api/cpp/Makefile

Fix static analysis warnings

A few warnings that I haven't noticed before that we should fix. None of these can cause any harm currently.

GravityLogger.cpp:351:5: warning: this `while' clause does not guard... [-Wmisleading-indentation]

GravityNode.cpp:1367:5: warning: this `if' clause does not guard... [-Wmisleading-indentation]

GravityPublishManager.cpp:471:5: warning: this `if' clause does not guard... [-Wmisleading-indentation]

Examples require -lsodium on Ubuntu 16.04

Compilation of examples fails under Ubuntu 16.04 when using libraries straight from the ubuntu repository. The distributed version of zeromq requires linking explicitly against libsodium (perhaps that was previously built with zeromq?)

make[1]: Entering directory '/home/chrismurf/lib/gravity/test/examples/1-BasicDataProduct'
g++ -o BasicDataProductPublisher BasicDataProductPublisher.cpp -I../../../src/api/cpp/ -I/usr/local/include -I/usr/include -Wall -L../../../src/api/cpp/ -L../../../src/keyvalue_parser/ -L/usr/local/lib -L/usr/local/lib -g -O2 -Wl,-Bstatic -lgravity -lkeyvalue_parser -lprotobuf -lzmq -Wl,-Bdynamic -lrt -lpthread
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libzmq.a(libzmq_la-ctx.o): In function zmq::ctx_t::~ctx_t()': (.text+0xfa6): undefined reference to randombytes_close'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libzmq.a(libzmq_la-zmq_utils.o): In function zmq_curve_keypair': (.text+0x337): undefined reference to crypto_box_keypair'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libzmq.a(libzmq_la-curve_client.o): In function zmq::curve_client_t::encode(zmq::msg_t*)': (.text+0x176): undefined reference to crypto_box_afternm'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libzmq.a(libzmq_la-curve_client.o): In function zmq::curve_client_t::decode(zmq::msg_t*)': (.text+0x4ab): undefined reference to crypto_box_open_afternm'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libzmq.a(libzmq_la-curve_client.o): In function zmq::curve_client_t::curve_client_t(zmq::options_t const&)': (.text+0x87e): undefined reference to sodium_init'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libzmq.a(libzmq_la-curve_client.o): In function zmq::curve_client_t::curve_client_t(zmq::options_t const&)': (.text+0x89a): undefined reference to crypto_box_keypair'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libzmq.a(libzmq_la-curve_client.o): In function zmq::curve_client_t::produce_hello(zmq::msg_t*)': (.text+0xb9a): undefined reference to crypto_box'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libzmq.a(libzmq_la-curve_client.o): In function zmq::curve_client_t::process_welcome(unsigned char const*, unsigned long)': (.text+0xe25): undefined reference to crypto_box_open'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libzmq.a(libzmq_la-curve_client.o): In function zmq::curve_client_t::process_welcome(unsigned char const*, unsigned long)': (.text+0xf17): undefined reference to crypto_box_beforenm'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libzmq.a(libzmq_la-curve_client.o): In function zmq::curve_client_t::produce_initiate(zmq::msg_t*)': (.text+0x108a): undefined reference to randombytes'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libzmq.a(libzmq_la-curve_client.o): In function zmq::curve_client_t::produce_initiate(zmq::msg_t*)': (.text+0x10ae): undefined reference to crypto_box'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libzmq.a(libzmq_la-curve_client.o): In function zmq::curve_client_t::produce_initiate(zmq::msg_t*)': (.text+0x12b3): undefined reference to crypto_box'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libzmq.a(libzmq_la-curve_client.o): In function zmq::curve_client_t::process_ready(unsigned char const*, unsigned long)': (.text+0x15e0): undefined reference to crypto_box_open_afternm'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libzmq.a(libzmq_la-curve_server.o): In function zmq::curve_server_t::encode(zmq::msg_t*)': (.text+0x176): undefined reference to crypto_box_afternm'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libzmq.a(libzmq_la-curve_server.o): In function zmq::curve_server_t::decode(zmq::msg_t*)': (.text+0x5bb): undefined reference to crypto_box_open_afternm'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libzmq.a(libzmq_la-curve_server.o): In function zmq::curve_server_t::curve_server_t(zmq::session_base_t*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, zmq::options_t const&)': (.text+0x959): undefined reference to sodium_init'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libzmq.a(libzmq_la-curve_server.o): In function zmq::curve_server_t::curve_server_t(zmq::session_base_t*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, zmq::options_t const&)': (.text+0x975): undefined reference to crypto_box_keypair'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libzmq.a(libzmq_la-curve_server.o): In function zmq::curve_server_t::process_hello(zmq::msg_t*)': (.text+0xdc6): undefined reference to crypto_box_open'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libzmq.a(libzmq_la-curve_server.o): In function zmq::curve_server_t::produce_welcome(zmq::msg_t*)': (.text+0xeed): undefined reference to randombytes'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libzmq.a(libzmq_la-curve_server.o): In function zmq::curve_server_t::produce_welcome(zmq::msg_t*)': (.text+0xf81): undefined reference to randombytes'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libzmq.a(libzmq_la-curve_server.o): In function zmq::curve_server_t::produce_welcome(zmq::msg_t*)': (.text+0xf9e): undefined reference to crypto_secretbox'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libzmq.a(libzmq_la-curve_server.o): In function zmq::curve_server_t::produce_welcome(zmq::msg_t*)': (.text+0xfc4): undefined reference to randombytes'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libzmq.a(libzmq_la-curve_server.o): In function zmq::curve_server_t::produce_welcome(zmq::msg_t*)': (.text+0x10fb): undefined reference to crypto_box'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libzmq.a(libzmq_la-curve_server.o): In function zmq::curve_server_t::produce_ready(zmq::msg_t*)': (.text+0x13b1): undefined reference to crypto_box_afternm'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libzmq.a(libzmq_la-curve_server.o): In function zmq::curve_server_t::process_initiate(zmq::msg_t*)': (.text+0x231e): undefined reference to crypto_secretbox_open'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libzmq.a(libzmq_la-curve_server.o): In function zmq::curve_server_t::process_initiate(zmq::msg_t*)': (.text+0x2400): undefined reference to crypto_box_open'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libzmq.a(libzmq_la-curve_server.o): In function zmq::curve_server_t::process_initiate(zmq::msg_t*)': (.text+0x2513): undefined reference to crypto_box_open'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libzmq.a(libzmq_la-curve_server.o): In function zmq::curve_server_t::process_initiate(zmq::msg_t*)': (.text+0x254a): undefined reference to crypto_box_beforenm'
collect2: error: ld returned 1 exit status
Makefile:81: recipe for target 'BasicDataProductPublisher' failed
make[1]: *** [BasicDataProductPublisher] Error 1
make[1]: Leaving directory '/home/chrismurf/lib/gravity/test/examples/1-BasicDataProduct'
Makefile:50: recipe for target 'all' failed
make: *** [all] Error 2

Logger deletion code has stray semicolon

Line 352 of GravityLogger.cpp has the following piece of code, which does not do as intended...

    while(i != l_end);
    {
        delete i->first;
        i++;
    }

Note the stray semicolon. Found by g++-7.3. Will fix with a few other incoming updates.

using namespace statements in gravity header

I just found a couple cases where there's a 'using namespace' statement in a header file. The worst offender is GravitySubscriber.h since that's included almost everywhere. Upon removing it I found that many gravity .cpp files no longer compile.

The two namespaces listed were std and std::tr1 (for shared_ptr). These need to be explicitly referenced in all headers, and any code that was relying on those statements must be updated to manage that themselves.

Intermittent SIGSEGV shutting down

This may be another case that you cannot really recreate, but I get a SIGSEGV with my unit tests sometimes when test is complete. I don't see anyway to shutdown Subscription Manager, but have you seen this before?

#0  0x00007ffff7874735 in gravity::Log::vLog (level=16, format=0x7ffff78c91b0 "Interrupted, exiting (rc = %d)", args=0x7ffff3db6928) at GravityLogger.cpp:301
#1  0x00007ffff7874c53 in gravity::Log::debug (message=0x7ffff78c91b0 "Interrupted, exiting (rc = %d)") at GravityLogger.cpp:412
#2  0x00007ffff788cff3 in gravity::GravitySubscriptionManager::start (this=0x7ffff3db6d50) at GravitySubscriptionManager.cpp:101
#3  0x00007ffff7875dbe in startSubscriptionManager (context=0x7792e0) at GravityNode.cpp:76
#4  0x00007ffff6cbbe25 in start_thread () from /lib64/libpthread.so.0
#5  0x00007ffff574a34d in clone () from /lib64/libc.so.6

Add Python support for Windows

The Python binding hasn't been integrated into the VisualStudio build files, so we can't currently use it in Windows.

Ctrl-C doesn't work with waitForExit in Python binding

GravityNode.waitForExit() should block until all the Gravity spawned threads terminate. In the C++ version, this can be triggered with ctrl-c (SIGINT), but this doesn't work when used in the Python version. SIGTERM does work though.

Python Binding not working correctly with multiple inheritance

I have a case like:

class MyClass(GravitySubscriber, GravityRequestor):
    def __init__(self):
        super(MyClass, self)__init__()

then I try to subscribe like:

self.gravityNode.subscribe("MySub", self)

That works, but then when I try to make an asynchronous request like:

self.gravityNode.request("MyService", reqGDP, self)

I get an error that the args are the wrong type or number. If I switch the order of the parent classes in the MyClass declaration, then the call to subscribe fails instead with the same message.

It looks like the problem is the type checking performed in the generated C++ code - for some reason it's only seeing that self is an instance of the first parent class, not the second.

shared_ptr is in two (std/std::tr1) namespaces and cannot be resolved

Now that C++11 / C++17 are actually out, shared_ptr is in both the std:: and str::tr1:: namespaces. In many files, we have "using namespace std; using namespace std::tr1;". The compiler cannot decide which one to use.

These can be fixed by getting rid of “using namespace std::tr1” and replacing shared_ptr with tr1::shared_ptr. Even that is a bit of a kludge, since TR1 is officially deprecated now that C++11, C++14 and C++17 are out and will be removed in the future, but it meets the need for now.

I ran into this issue today using GCC7 with Ubuntu Bionic (the new LTS version).

ticket 48, issue #18

perhaps you want to delete my old branch? "prevent blocking on ack from unregistered heartbeat"

Long Python exceptions are truncated in gravity log

Would be good if the new GravityLogHandler allowed splitting multi-line log statements into separate log messages so that they won't get truncated.

Compiling examples as 'static' requires lots of of zeromq dependencies.

Under the latest Ubuntu, I again had issues linking zeromq due to the lack of a couple of libraries - this time it was libpgm and libnorm instead of libsodium. I realized that I don't have these issues with the components because they aren't compiled as static, but the examples are for some reason.

Any objections to removing '-Wl,-Bstatic' from the example Makefile.in's?

Add tutorials to github wiki

Should get the tutorials checked out and uploaded.

GravitySubscriptionManager never unsubscribes to RegisteredPublishers

When a new publisher registers with the ServiceDirectory, the SD publishes this update using the RegisteredPublishers data product id. The GravitySubscriptionManager subscribes to these message automatically when it first subscribes to a new domain/data product id pair. But if a component unsubscribes from a particular domain/data product id, it never unsubscribes from RegisteredPublishers. So if the component again subscribes to that domain/GDP id, we get duplicate RegisteredPublisher subscriptions.

Consider deleting unneeded cpp files

Minor issue.

GravityMetricsUtil.cpp only defines 5 externs. Not clear these need to be externs - and could simply be defined in the .h

GravityServiceProvider::~GravityServiceProvider() {} could be replaced with GRAVITY_API virtual ~GravityServiceProvider() = default;

GravitySubscriptionMonitor.cpp only defines 2 empty functions:

• The default destructor, which could be replaced with virtual ~GravitySubscriptionMonitor() = default;
• The subscriptionFilled function which is pure virtual and doesn't need to exist at all.

Same applies to GravitySubscriber.

GravityHeartbeatListener.h uses virtual ~GravityHeartbeatListener() { }; vice = delete;

GravityRequestor.cpp only defines 2 functions:

• Empty destructor
• virtual requestTImeout with a single Log::warning

whether a virtual func with a default implementation should be in the .h or .cpp is a matter of taste.

Only tangentially related:

• GravitySemaphore.h is implemented in Semaphore.cpp
• Most of the source files have executable permission

BroadcastEnabled should default to true

Forgetting to set this flag in Gravity.ini is a common problem when configuring domains (I've forgotten several times). If a domain is specified, the default behavior should be to broadcast - I don't believe there's a use case where someone would use domains and not want this enabled. But we need to make sure that it won't try to broadcast if no domain is specified.

It would also be good to update the domain example (example number 9) to reflect this.

Python GravityLogHandler breaks on some log messages

Unicode messages cause an exception to be thrown when calling the Gravity Logger here.

Replace pthread with std::thread

Explicit pthread calls create portability problems.
It also modernizes the code base.

GravityNode::getComponentID is declared, but never defined.

As the title suggests, GravityNode::getComponentID is declared in GravityNode.h but is never defined in GravityNode.cpp

Headers included by GravityNode are missing from build

Headers included by GravityNode do not exist in the include/protobufs directory.

Asynchronous request timeout has no effect

There's a timeout parameter to the asynchronous request method on GravityNode, but it isn't used in the code. The timeout for the synchronous call works fine.

build fails with g++ 7.3.0

This is the default compiler on Ubuntu 18.04.
The error is as follows:

gcc  -g -c -o Relay.o Relay.cpp -I../../../api/cpp -Iprotobufs -I../../../keyvalue_parser -I/usr/local/include -I/usr/local/include -Wall 
Relay.cpp: In member function ‘int Relay::run()’:
Relay.cpp:120:9: error: reference to ‘lock’ is ambiguous
         lock.Lock();
         ^~~~
Relay.cpp:28:20: note: candidates are: gravity::Semaphore lock
 gravity::Semaphore lock;
                    ^~~~
In file included from /usr/local/include/google/protobuf/stubs/mutex.h:33:0,
                 from /usr/local/include/google/protobuf/stubs/common.h:52,
                 from ../../../api/cpp/protobuf/GravityDataProductPB.pb.h:9,
                 from ../../../api/cpp/GravityDataProduct.h:34,
                 from ../../../api/cpp/GravitySubscriber.h:29,
                 from ../../../api/cpp/GravityNode.h:29,
                 from Relay.cpp:19:
/usr/include/c++/7/mutex:542:5: note:                 template<class _L1, class _L2, class ... _L3> void std::lock(_L1&, _L2&, _L3& ...)
     lock(_L1& __l1, _L2& __l2, _L3&... __l3)
     ^~~~
Relay.cpp:123:9: error: reference to ‘lock’ is ambiguous
         lock.Unlock();
         ^~~~
Relay.cpp:28:20: note: candidates are: gravity::Semaphore lock
 gravity::Semaphore lock;
                    ^~~~
In file included from /usr/local/include/google/protobuf/stubs/mutex.h:33:0,
                 from /usr/local/include/google/protobuf/stubs/common.h:52,
                 from ../../../api/cpp/protobuf/GravityDataProductPB.pb.h:9,
                 from ../../../api/cpp/GravityDataProduct.h:34,
                 from ../../../api/cpp/GravitySubscriber.h:29,
                 from ../../../api/cpp/GravityNode.h:29,
                 from Relay.cpp:19:
/usr/include/c++/7/mutex:542:5: note:                 template<class _L1, class _L2, class ... _L3> void std::lock(_L1&, _L2&, _L3& ...)
     lock(_L1& __l1, _L2& __l2, _L3&... __l3)
     ^~~~

When using ConfigServer, items are not set to lower case.

In ParseConfigFile() it calls StringCopyToLowerCase() for all the items, but when using ConfigServer, ParseConfigService() doesn't make it lower case. This makes it different for node when switching from server/local config file.

Bad delete in a few files

Code in GravityNode.cpp and FutureResponse.cpp allocate free store arrays, but deallocate using delete vice delete[].

See

gravity/src/api/cpp/GravityNode.cpp

Line 2248 in 746afaa

delete bytes;

gravity/src/api/cpp/FutureResponse.cpp

Line 54 in 746afaa

delete data;

Protobuf files should be in a 'gravity' package directory.

When protoc generates python descriptors, it sets the Python module's "module" attribute based off of the file path, as well as embedding the package information in the encoded descriptor:

serialized_pb=_b('\n\"gravity/GravityDataProductPB.proto\x12...

versus

serialized_pb=_b('\n\x1aGravityDataProductPB.proto\x12...

This problem mostly shows up as an issue when you use a protobuf type inside another protobuf type as a submessage, which I suspect is why it hasn't showed up before now.

I believe the right solution is to ensure that all protobufs have a parent folder structure which matches their package name - moving "src/api/proto/.proto" to "src/api/proto/gravity/.proto" for instance.

Possible to get updated windows build scripts for visual studio 2017?

I am having lots of trouble getting a gravity application to build/link on windows. Is it possible to get build scripts updated to work with visual studio 2017?
Thanks.

swig/python detected a memory leak of type 'int64_t *', no destructor found.

I am trying out the python interface for listening to heartbeats, and I get this error when I miss a heartbeat. Is there something wrong with my python declaration or with the code itself?

Thanks,

swig/python detected a memory leak of type 'int64_t *', no destructor found.

class MyListener(GravityHeartbeatListener):
    def __init__(self):
        super(MyListener, self).__init__()

    def MissedHeartbeat(self, componentId, microsecond_to_last_heartbeat, interval_in_microseconds):
        print("missed heartbeat")

    def ReceivedHeartbeat(self, componentID, interval_in_microseconds):
        print("received heartbeat")

Should void GravityRequestManager::sendFutureResponse() return a GravityReturnCode

Minor issue.
The local ret is set in a few places, but never used.
Was the intent that this function not be void?

gravity/src/api/cpp/GravityRequestManager.cpp

Line 249 in 746afaa

int ret = GravityReturnCodes::SUCCESS;

Add async timeout callback support to Java binding

Follow-on to #26. We made the updates to support a new async request timeout without breaking backwards compatibility for C++ and Python by providing a default implementation for the new callback method. For Java though, we use an interface to define the callback methods. Java 8 adds support for providing a default impl in an interface, but we need to maintain support for Java 6 and 7 for the MATLAB versions we use. So adding Java support here will break backwards compatibility as it will require all classes that implement GravityRequestor to implement this new method.

We will make this change when we have other pending changes that will also break backwards compatibility.

MATLABProtobufGenerator breaks (NPE) on use of Oneof field

Question: What is best way to handle many-to-one messages?

Imagine an alert data product, where many nodes can publish this data product and a single node subscribes to this alert data product. It seems that a node can only publish() data if you have called registerDataProduct(), and this makes it so only one node can publish data. And I don't want the single node to have to know all the possible nodes that can possibly connect and send alerts.

The only workaround I can see is to use the GravityRequestor interface, and send these alert type messages as request() . Do you all agree, and is this the best way to do something like this?

Any luck running with newer matlab, like 2016?

I am trying to run on a newer matlab 2016b, and followed the wiki. The only part I cant do is regarding the libstdc++.* as matlab actually uses a newer version of glib than redhat 7.4. I currently get the blow error. Any ideas?

Error using GravityNode (line 27)
java exception occurred:
java.lang.NoClassDefFoundError: Count not initialize class
com.aphysci.gravity.swig.gravityJNI at com.aphysci.gravity.swig.GravityNode.<init>(GravityNode.java:47)

ServiceDirectorySynchronizer can crash if running with more than 2 domains

When the synchronizer gets a Remove command, it removes its subscription to the remote domain's ServiceDirectory_DomainDetails. It can crash when updating its pollItems vector to remove that subscription (currently line 170) though, because the position of that pollItem in the vector may have changed if another domain entry (earlier in the vector) was already removed.