capellil/ipregel

A shared-memory vertex-centric framework.

ipregel's Issues

Broadcasting / Receiving multiple messages

Hi,

We are currently working on implementing APSP in iPregel. Our main idea is to generalize SSSP, that means: Each node holds an array of distances to each other node, and we enable nodes to send their distances from multiple other nodes to their neighbors.

For that, one node would need to be able to broadcast in a superset either:
a. multiple messages in the form of <origin-node, distance> tuples
b. one message consisting of multiple <origin-node, distance> tuples

We are currently struggling with the combine method, that as we understand it, gets always called on received messages. We obviously cannot combine our messages in our use case because we need to receive multiple tuples. We are now approaching this with a somewhat hacky solution of sending Linked Lists of tuples that can be combined easily and then evaluating just the combined Linked Lists. But that doesn't seem feasible due to memory management issues, which means that we can't really control the memory used for our Linked Lists and e.g., cannot free the memory afterward.

Our first question therefore is:

• Can the combine optimizer be disabled? / Is there an easy way of receiving every message that was sent and not a combined message? We did see that in March 2018 there used to be a no_combiner_postamble which was removed since iPregel is combinar-based.

• And that leads directly to our more general question, as we feel that we might overlook something, because implementing APSP turned out more complicated than expected: How would one implement APSP in iPregel? Just the general idea perhaps, so that we can check if we are approaching this as intended.

Thanks a lot in advance (also for the quick reply last time!),

Theresa

OS compatibility (remark in README)

Hi :)

I just spent some time trying to compile iPregel under MacOS and was able to fix some issues until I reached the following compile error:

As cpu_set_t is a Linux-only provided data structure (https://man7.org/linux/man-pages/man3/CPU_SET.3.html), I assume that OS compatibility with macOS (and probably Windows) is just not given and would only be reachable with code modifications.

If I am wrong in my assessment and there actually is a way to compile and run iPregel on macOS, I would be glad and thankful to hear it. Otherwise, only supporting Linux is, of course, totally fine, but maybe adding a remark to the README about OS compatibility could save people some time in the future.