infinispan/js-client

Javascript client for Infinispan, over the Hot Rod wire protocol

Hot Rod JS Client

infinispan is an asynchronous event-driven Infinispan client for Node.js. The results of the asynchronous operations are represented using Promise instances. Amongst many advantages, promises make it easy to transform/chain multiple asynchronous invocations and they improve error handling by making it easy to centralise it.

The client is under heavy development but here's a summary of its current capabilities:

• infinispan client can be constructed with a single server address or multiple servers addresses. When passing multiple addresses, it will iterate until it finds a server to which it can connect to.
• Clients can interact with a named cache whose name is passed on client construction via {cacheName: 'myCache'} option. In the absence of any cache name options, the client will interact with the default cache.
• Full CRUD operation support, e.g. put, get, remove, containsKey...etc.
• Compare-And-Swap operation support, e.g. putIfAbsent, getWithMetadata, replace, replaceWithVersion, removeWithVersion..etc.
• Expiration with absolute lifespan or relative maximum idle time is supported. This expiration parameters as passed as optional parameters to create/update methods and they support multiple time units, e.g. {lifespan: '1m', maxIdle: '1d'}.
• Update and remove operations can optionally return previous values by passing in {previous: true} option.
• Bulk store/retrieve/delete operations are supported, e.g. putAll, getAll, clear...etc.
• Cache contents can be iterated over using the iterator method.
• Cache size can be determined using the size method.
• Remote cache listeners can be plugged using the addListener method, which takes the event type (create, modify, remove or expiry) and the function callback as parameter.
• Clients can store scripts using addScript and then they can be remotely executed using the execute operation. Executing a script remotely optionally takes per-invocation parameters.
• Server-side statistics can be retrieved using the stats operation.
• Clients can connect using encryption with the server via SSL/TLS with optional TLS/SNI support.
• Clients can talk to clusters of Infinispan Server instances, using Consistent-Hash based algorithms to route key-based operations.
• Multi-key or key-less operations are routed in round-robin fashion.
• Clients only need to be configure with a single node's address and from that node the rest of the cluster topology can be discovered. As nodes are added or destroyed, clients get notified of changes in the cluster topology dynamically.
• Clients can talk to multiple clusters that are separated into different site clusters. The client is normally connected to one of the sites, but if its members fail to respond, it will automatically switch to an alternative site to which it can connect.
• Clients have methods, such as switchToCluster(clusterName) and switchToDefaultCluster that allows users to manually change to which site cluster to connect.
• Finally, clients can stop communication with the server(s) using the disconnect method.

Hot Rod JS client documentation

Find installation, configuration, and example usage in the Hot Rod JS Client Guide at infinispan.org/documentation.

You can also build the Hot Rod JS Client Guide as follows:

1. Clone the source repository.

$ git clone [email protected]:infinispan/js-client.git

2. Build the HTML from the asciidoc source.

$ asciidoctor documentation/asciidoc/titles/js_client.asciidoc

3. Open documentation/asciidoc/titles/js_client.html in any browser.

API docs

Review Hot Rod JS client API documentation.

You can also build API docs from the source repository as follows:

1. Generate JSDoc formatted API docs.

$ npm install jsdoc
$ ./node_modules/.bin/jsdoc lib/*.js

2. Open open out/index.html in any browser.

Testing

Before executing any tests, Infinispan Server instances need to be started up so that testsuite can run against those. To ease this process, a script has been created in the root directory to start all the expected server instances.

Go to the root of the repo and execute:

$ npm install

Next, start the Infinispan Servers via:

$ ./run-server.sh

To run the testsuite once execute:

$ ./run-testsuite.sh

To run tests continuously execute:

$ ./node_modules/.bin/jasmine-node spec --autotest --watch lib --captureExceptions

To run individual tests execute:

$ node node_modules/jasmine-node/lib/jasmine-node/cli.js spec/infinispan_local_spec.js --captureExceptions

To help with testing, you can quickly run the smoke tests via:

$ ./smoke-tests.sh

Both testsuite and smoke tests can be run with older protocol versions, e.g.

$ protocol=2.5 ./smoke-tests.sh

Note for Mac Users:

You might experience MPING issues running an Infinispan cluster.

13:37:15,561 ERROR (jgroups-5,server-two) [org.jgroups.protocols.MPING]

If you run into the errors above, add the following to the routes of your host

sudo route add -net 224.0.0.0/5 127.0.0.1
sudo route add -net 232.0.0.0/5 192.168.1.3

Manual stress tests

The testsuite now contains manual stress tests that take several minutes to run. To run these tests, execute:

$ ./node_modules/.bin/jasmine-node spec-manual --captureExceptions

Memory profiling

The source code comes with some programs that allow the client's memory consumption to be profiled. Those programs rely on having access to the global garbage collector. So, to run them you must pass --expose-gc command line parameter. Example:

node --expose-gc memory-profiling/infinispan_memory_many_get.js

So of programs might only report the memory usage before/after. Others might generate heap dumps which can be visualized using Google Chrome. Within Chrome, the Developer Tools UI contains a Memory tab where heap dumps can be loaded.

Debugging

To debug tests with IDE:

node --inspect-brk node_modules/jasmine-node/lib/jasmine-node/cli.js spec/codec_spec.js

Or:

node --inspect-brk node_modules/jasmine-node/lib/jasmine-node/cli.js spec/infinispan_local_spec.js

And then start a remote Node.js debugger from IDE on port 9229.

Tests, servers and ports

Here's some more detailed information on which tests interact with which servers and on which ports. On top of that, you can find information on which tests are always running as opposed to those that are started (and stopped) by the tests themselves.

Test	Server Profile	Ports (Auto/Manual)
local spec	local	11222 (A)
expiry spec	local	11222 (A)
cluster spec	clustered	11322 (A), 11332 (A), 11342 (A)
failover spec	clustered	11422 (M), 11432 (M), 11442 (M)
ssl spec	local	11232 (A), 12242 (A), 12252 (A)
xsite spec	earth, moon	11522 (earth, M), 11532 (moon, M)

Reporting an issue

This project does not use Github issues. Instead, please report them via JIRA (project HRJS).

js-client's Issues

Typescript adaptor

A Typescript adaptation layer should be provide so to ease Infinispan-based Typescript applications dev

Incorporate sasl lbrary code and remove dependency

Current code rely on the library https://www.npmjs.com/package/saslmechanisms, to perform sasl authentication.

Task
Remove library dependecy

js-client/package.json

Line 30 in bac3b43

"saslmechanisms": "^0.1.1",

and import its source code directly in the project

add a linter for code consistency

This module should probably have a linter. It's common to use eslint with a predefined style.

Note: this may (and arguably should) enforce some modernisation of the code syntax (for example, moving away from var's).

Refs: https://nodeshift.dev/nodejs-reference-architecture/development/code-consistency

Missing License file

It looks like there is no license file. Can we get a license file so that I can use it in the product (JDG)?

Implement PING v3.0 operation

Client now support 2.9 hotrod protocol.

Task
Implement PING operation v3.0 so client can work with hotrod 3.0 protocol

Add a MediaType for `application/x-protostream`

Protostream is a Protobuf based serialization library, is used by Infinispan to serialize cache entries preserving their type.

Protostream message is defined here link

Client should support protobuf encode/decode for mediatype application/x-protostream.
MediaType initialization seems to be here link

Protostream encoder/decoder should be added here link

I've found an example of js protobuf usage here link based on protobufjs, but maybe better examples are available...

wrong protobuff path

When I run import infinispan from 'infinispan'; with version 0.11.0

It has a critical error

Exception during run: Error: ENOENT: no such file or directory, open 'C:\Users\david\Documents\GitHub\node-databases\redhat\infinispan\node_modules\infinispan\lib\protostream\Protos\message-wrapping.proto'
    at Object.openSync (node:fs:581:18)
    at Object.readFileSync (node:fs:457:35)
    at fetch (C:\Users\david\Documents\GitHub\node-databases\redhat\infinispan\node_modules\protobufjs\src\root.js:173:34)
    at process (C:\Users\david\Documents\GitHub\node-databases\redhat\infinispan\node_modules\protobufjs\src\root.js:133:29)
    at fetch (C:\Users\david\Documents\GitHub\node-databases\redhat\infinispan\node_modules\protobufjs\src\root.js:179:13)
    at Root.load (C:\Users\david\Documents\GitHub\node-databases\redhat\infinispan\node_modules\protobufjs\src\root.js:207:13)
    at Root.loadSync (C:\Users\david\Documents\GitHub\node-databases\redhat\infinispan\node_modules\protobufjs\src\root.js:248:17)
    at Object.loadSync (C:\Users\david\Documents\GitHub\node-databases\redhat\infinispan\node_modules\protobufjs\src\index-light.js:69:17)
    at C:\Users\david\Documents\GitHub\node-databases\redhat\infinispan\node_modules\infinispan\lib\codec.js:489:23
    at Object.<anonymous> (C:\Users\david\Documents\GitHub\node-databases\redhat\infinispan\node_modules\infinispan\lib\codec.js:498:4)
    at Object.<anonymous> (C:\Users\david\Documents\GitHub\node-databases\redhat\infinispan\node_modules\infinispan\lib\codec.js:500:3)
    at Module._compile (node:internal/modules/cjs/loader:1376:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1435:10)
    at Module.load (node:internal/modules/cjs/loader:1207:32)
    at Module._load (node:internal/modules/cjs/loader:1023:12)
    at Module.require (node:internal/modules/cjs/loader:1235:19)
    at require (node:internal/modules/helpers:176:18)
    at Object.<anonymous> (C:\Users\david\Documents\GitHub\node-databases\redhat\infinispan\node_modules\infinispan\lib\infinispan.js:14:15)
    at Object.<anonymous> (C:\Users\david\Documents\GitHub\node-databases\redhat\infinispan\node_modules\infinispan\lib\infinispan.js:960:3)
    at Module._compile (node:internal/modules/cjs/loader:1376:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1435:10)
    at Module.load (node:internal/modules/cjs/loader:1207:32)
    at Module._load (node:internal/modules/cjs/loader:1023:12)
    at Module.require (node:internal/modules/cjs/loader:1235:19)
    at require (node:internal/modules/helpers:176:18)
    at Object.<anonymous> (C:\Users\david\Documents\GitHub\node-databases\redhat\infinispan\node_modules\infinispan\index.js:1:18)
    at Module._compile (node:internal/modules/cjs/loader:1376:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1435:10)
    at Module.load (node:internal/modules/cjs/loader:1207:32)
    at Module._load (node:internal/modules/cjs/loader:1023:12)
    at cjsLoader (node:internal/modules/esm/translators:345:17)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:294:7)
    at ModuleJob.run (node:internal/modules/esm/module_job:218:25)
    at async ModuleLoader.import (node:internal/modules/esm/loader:329:24)
    at async importModuleDynamicallyWrapper (node:internal/vm/module:431:15)
    at async formattedImport (C:\Users\david\Documents\GitHub\node-databases\redhat\infinispan\node_modules\mocha\lib\nodejs\esm-utils.js:9:14)
    at async exports.requireOrImport (C:\Users\david\Documents\GitHub\node-databases\redhat\infinispan\node_modules\mocha\lib\nodejs\esm-utils.js:42:28)
    at async exports.loadFilesAsync (C:\Users\david\Documents\GitHub\node-databases\redhat\infinispan\node_modules\mocha\lib\nodejs\esm-utils.js:100:20)
    at async singleRun (C:\Users\david\Documents\GitHub\node-databases\redhat\infinispan\node_modules\mocha\lib\cli\run-helpers.js:125:3)
    at async exports.handler (C:\Users\david\Documents\GitHub\node-databases\redhat\infinispan\node_modules\mocha\lib\cli\run.js:370:5) {
  errno: -4058,
  code: 'ENOENT',
  syscall: 'open',
  path: 'C:\\Users\\david\\Documents\\GitHub\\node-databases\\redhat\\infinispan\\node_modules\\infinispan\\lib\\protostream\\Protos\\message-wrapping.proto'
}

Add Client Intelligence

infinispan.client.hotrod.client_intelligence
should be supported in the client

https://issues.redhat.com/browse/ISPN-14337

Implement Hotrod query operation without projection

Reference in the Hotrod implementation is here
Query request and response proto definition are here

Query without projection: i.e. `from sample_bank_account.User u where u.name = "John"
How to unroll the QueryResponse object. Some c# code as reference is here

Let's leave the query with projection, i.e. select u.name, u.surname, u.age from sample_bank_account.User u where u.age is null to a different issue

Implement query with projection

Implement query with projection: i.e. select u.name, u.surname, u.age from sample_bank_account.User u where u.age is null.

Depends from #86

migrate off of request module

Request has been formally deprecated - https://www.npmjs.com/package/request

It's only used for tests in this module so is not urgent. But, it could be a potential good first issue to move off of it.

I'd probably recommend node-fetch as an alternative. But axios and got are also good options (though have more dependencies, and axios has not yet reached v1.0.0).

Update Infinispan Simple Tutorial

Add typescript and update version

https://github.com/infinispan/infinispan-simple-tutorials

testsuite hangs on xsite server startup

testsuite hangs on xsite server startup

Replace heapdump dependency with NodeJs internal heap dump functionality

It is not possible to build (run 'npm install') for js-client on RHEL9 or Fedora 35 with Node.JS version v16.14.0, because of heapdump dependency. Getting the folling error:

npm ERR! code 1
npm ERR! path /home/amanukya/IdeaProjects/js-client/node_modules/heapdump
npm ERR! command failed
npm ERR! command sh -c node-gyp rebuild
npm ERR! make: Entering directory '/home/amanukya/IdeaProjects/js-client/node_modules/heapdump/build'
npm ERR!   CXX(target) Release/obj.target/addon/src/heapdump.o
npm ERR! make: Leaving directory '/home/amanukya/IdeaProjects/js-client/node_modules/heapdump/build'
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using [email protected]
npm ERR! gyp info using [email protected] | linux | x64
npm ERR! gyp info find Python using Python version 3.10.2 found at "/usr/bin/python3"
npm ERR! gyp info spawn /usr/bin/python3
npm ERR! gyp info spawn args [
npm ERR! gyp info spawn args   '/usr/local/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
npm ERR! gyp info spawn args   'binding.gyp',
npm ERR! gyp info spawn args   '-f',
npm ERR! gyp info spawn args   'make',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/home/amanukya/IdeaProjects/js-client/node_modules/heapdump/build/config.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/usr/local/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/home/amanukya/.cache/node-gyp/16.14.0/include/node/common.gypi',
npm ERR! gyp info spawn args   '-Dlibrary=shared_library',
npm ERR! gyp info spawn args   '-Dvisibility=default',
npm ERR! gyp info spawn args   '-Dnode_root_dir=/home/amanukya/.cache/node-gyp/16.14.0',
npm ERR! gyp info spawn args   '-Dnode_gyp_dir=/usr/local/lib/node_modules/npm/node_modules/node-gyp',
npm ERR! gyp info spawn args   '-Dnode_lib_file=/home/amanukya/.cache/node-gyp/16.14.0/<(target_arch)/node.lib',
npm ERR! gyp info spawn args   '-Dmodule_root_dir=/home/amanukya/IdeaProjects/js-client/node_modules/heapdump',
npm ERR! gyp info spawn args   '-Dnode_engine=v8',
npm ERR! gyp info spawn args   '--depth=.',
npm ERR! gyp info spawn args   '--no-parallel',
npm ERR! gyp info spawn args   '--generator-output',
npm ERR! gyp info spawn args   'build',
npm ERR! gyp info spawn args   '-Goutput_dir=.'
npm ERR! gyp info spawn args ]
npm ERR! gyp info spawn make
npm ERR! gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
npm ERR! make: g++: No such file or directory
npm ERR! make: *** [addon.target.mk:111: Release/obj.target/addon/src/heapdump.o] Error 127
npm ERR! gyp ERR! build error 
npm ERR! gyp ERR! stack Error: `make` failed with exit code: 2
npm ERR! gyp ERR! stack     at ChildProcess.onExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:194:23)
npm ERR! gyp ERR! stack     at ChildProcess.emit (node:events:520:28)
npm ERR! gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:291:12)
npm ERR! gyp ERR! System Linux 5.16.12-200.fc35.x86_64
npm ERR! gyp ERR! command "/usr/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
npm ERR! gyp ERR! cwd /home/amanukya/IdeaProjects/js-client/node_modules/heapdump
npm ERR! gyp ERR! node -v v16.14.0
npm ERR! gyp ERR! node-gyp -v v8.4.1
npm ERR! gyp ERR! not ok

I want to suggest to remove the heapdump dependency (it is used for testing purposes for dumping heap and is not used regularly) and replace it with heap dump functionality that NodeJs new version has in it's Core API.

Extend protostream enc/decoder to basic types

Current protostream implementation only decodes user message, but protrostream has dedicated fields for basic types.
Each supported basic type has its own tag and accessor and needs to be managed correctly.
an example of code handling "basic types" vs "user message" in C# is here (this is improperly provided as part of the query feature)
An equivalent implementation is needed for the enc/decoderProtobuf