Delta original concept was send only what is changed to the server. This would allow very minimal amount of data to be transferred between hosts. Then in GemFire 10 the team extended delta to be shared across a wide area network (WAN) connection.
While cool, we can take advantage of how delta works and have the delta be applied as an idempotent message to be applied like additions or subtraction from a counter.
For our delta counter to work we need to maintain state of counter and any changes that have been accumulating since we last stored the value in GemFire. When storing the DeltaCounter
in GemFire the hasDelta
is checked to see if we send the current value (false
) or the accumulator (true
).
If we send the current value (false
) the value that is stored is overwritten.
There are two methods of use - directly from a client or via a function. The differance in the two approaches come from needing to check the value before addition or not. Checking the value would require a get
doSomething
and finally storing. If we did this get from a client perspective there will be two network hops - first from the get
and the second from the put
of the delta.
If we did this in a function there will be only one network call - there by increasing performance substantially due to the reduction of network overhead. To ensure that we don't send the code to the wrong host we use a optimizeForWrite
function and use a partitioned region.
optimizeForWrite
signals to GemFire that this function when utilized with a filter
it will send the function invocation to the host containing the data. In the case of a Partitioned Region it will execute the function on the primary data host. Making the read and write all local.
I have attempted to make the code as simple as possible to use. There is one main caveat to using this code is that we need to initialize the DeltaCounter
BEFORE invoking the function. This has todo with a performance optimization for now the code has a race if we don't initialize before.
DeltaCounter
by the name of name
which is stored in the accumulator region.
accumulatorRegion.put("name", new DeltaCounter());
Increment the DeltaCounter
which is named name
by 1. If the counter is greater than 400000 reject the increment. The new value will be returned.
int deltaCounter = DeltaCounterFunction.increment(accumulatorRegion, "name", 1, 400000);
In some instances all we care about is sending the commands to alter state, and it is up to some other process in an event driven architecture to perform any work. Here we can just create an instance of our DeltaCounter
apply any changes and do a put
performing the same single network hop.
With the current configuration there is a problem with the DELTA across the WAN has an issue. To run the code and see how this works:
- gradlew
- startGemFireWAN.bat - this starts up to GemFire clusters site 1 and site 2. Once the two systems are up and running it then uploads the functions then it configures a WAN between the two sites with a region for the application to use.
- runWANExample.bat - this launches two applications where one client is jamming in data into one cluster while another client is jamming in data in another cluster. When the client is done the expected value is supposed to be 2k as currently committed.