alexguan / node-zookeeper-client Goto Github PK
View Code? Open in Web Editor NEWA pure Javascript ZooKeeper client for Node.js
License: Other
A pure Javascript ZooKeeper client for Node.js
License: Other
ConnectionManager.prototype.findNextServer() hangs with 100% cpu on Windows when it calls setTimeout() with a fractional delay:
ConnectionManager.prototype.findNextServer = function (callback) {
...
if (self.serverAttempts === self.servers.length) {
setTimeout(function () {
...
}, Math.random() * self.spinDelay); <<<<<<<<<<<<<<<
} else {
...
}
This was observed with node v0.10.30 on a Windows Server 2008 system with version 0.2.0 of node-zookeeper-client. It is easily reproducible by specifying a single zookeeper server that is not responsive (down or is the ensemble leader and leaderServes is true). The bug is actually in the Windows version of node (I've not observed it on Linux), but is easily avoided by using
Math.floor(Math.random() * self.spinDelay)
instead. Other timeouts in ConnectionManager.js are already computed with Math.floor()
Some times I want to make some cleanup after the client is closed but due to current code, Client.close() method is only push request to queue but there is no way for Client to know when the ConnectionManager just have done the work.
So I think 'closed' event is also interested by Client and should not ignored in code.
I have my own fork t to just do simple work to just emit 'closed' when the close job is done.
Please see in villadora/node-zookeeper-client@ae37fc7
As the title. I'm using [email protected] LTS. and jslint throws an error.
After I update jslint, seems code on master does not pass the lint
Since jslint is too strict, why not ESLint ๐
Will this library support connections to Zookeeper using SASL and Kerberos/NTLM? I can't find any info on this.
If that's not currently possible, is it likely to be added soon? Or would love to hear your thoughts on how feasible it would be for us to add that kind of functionality in a PR, and what the best strategy to do so might be.
Thank you for the library, and your time!
Hey, really great module.
If a client has connected to a server, and then later on, there is a network partition and the client is unable to reconnect -- no SESSION_EXPIRED event is ever emitted by the client.
From my testing all the client emits is a DISCONNECTED event -- even if the client never reconnects to another server (which means that from the server point of view our session has expired since we couldn't reconnect to it). This is because the only time we emit SESSION_EXPIRED is when we get data from the socket: https://github.com/alexguan/node-zookeeper-client/blob/master/lib/ConnectionManager.js#L432
But if the socket was already closed and no data events will be emitted. In our case when there is a network partition, we'll never emit the SESSION_EXPIRED event. This means that if downstream consumers are only listening for SESSION_EXPIRED, they'll never get it on a network partition and hang forever. Specifically, if I've got a watch set on a znode, I won't get any more watch events or SESSION_EXPIRED events and I'll hang forever.
When a watch fires, the WatcherManager keeps the node's path in its re-watch dictionaries instead of clearing it out like the Java client does.
This example will print out the watchManager dictionary after watching and triggering for 100 different nodes:
util = require('util');
zk = require('node-zookeeper-client');
server = process.env.ZOOKEEPER_ADDRESS
testBasePath = '/test';
watchesToLeak = 100;
client = zk.createClient(server);
client.once('connected', function() {
client.mkdirp(testBasePath, function(err, path) {
asplode(watchesToLeak);
});
});
client.connect();
function asplode(i) {
client.create(testBasePath + '/i-', zk.CreateMode.EPHEMERAL_SEQUENTIAL, function(err, path) {
client.exists(path, function(ev) {
console.log(path + " " + ev.getName());
}, function(err, stat) {
client.remove(path, function(err) {
if(i > 0) asplode(i - 1);
else {
console.log(util.inspect(client.connectionManager.watcherManager));
client.close();
}
});
});
});
}
I used kaka-node to connect to kafka. kafka-node use node-zookeeper-client to get connect to zookeeper to get brokers.
For abnormal testing, there will be two zookeeper connection after stop zookeeper and start zookeeper. If repeating these processes again and again, there will be so many connections even more than 100 which leads to zookeeper stuck.
zk.once('disconnected', function () { if (!zk.closed) { zk.close(); self.connect(); self.emit('zkReconnect'); } });
The problem I found is after zk.close(), the socket still does not closed and self.connect() will create another one.
So I tried to figure out the root problem in node-zookeeper-client.
`ConnectionManager.prototype.onSocketClosed = function (hasError) {
var retry = false,
errorCode,
pendingPacket;
switch (this.state) {
case STATES.CLOSING:
errorCode = Exception.CONNECTION_LOSS;
retry = false;
break;
case STATES.SESSION_EXPIRED:
errorCode = Exception.SESSION_EXPIRED;
retry = false;
break;
case STATES.AUTHENTICATION_FAILED:
errorCode = Exception.AUTH_FAILED;
retry = false;
break;
default:
errorCode = Exception.CONNECTION_LOSS;
retry = true;
}
this.cleanupPendingQueue(errorCode);
this.setState(STATES.DISCONNECTED);
if (retry) {
this.connect();
} else {
this.setState(STATES.CLOSED);
}
};`
the socket will firstly catches error and closed event will be trigger. The 'retry' will be true. It will set state as 'disconnected' which invokes disconnected event for kaka-node to do zk.close(). Then state become 'closing' and then execute this.connect(), which will set state to 'connecting' and retry and retry.
The solution just add checking in the connect. to make sure if state == closing, just do not connect.
if (self.state === STATES.CLOSING) return;
`ConnectionManager.prototype.connect = function () {
var self = this;
if (self.state === STATES.CLOSING) return;
self.setState(STATES.CONNECTING);
self.findNextServer(function (server) {
self.socket = net.connect(server);
self.connectTimeoutHandler = setTimeout(
self.onSocketConnectTimeout.bind(self),
self.connectTimeout
);
// Disable the Nagle algorithm.
self.socket.setNoDelay();
self.socket.on('connect', self.onSocketConnected.bind(self));
self.socket.on('data', self.onSocketData.bind(self));
self.socket.on('drain', self.onSocketDrain.bind(self));
self.socket.on('close', self.onSocketClosed.bind(self));
self.socket.on('error', self.onSocketError.bind(self));
});
};`
I will send a pull request for it.
Thanks
Hello everyone,
The code given below connects to zookeeper. But the issue is that if the connection is unsuccessful then the code stucks. Is there a way to check weather connection successful or not
var zookeeper = require('node-zookeeper-client');
var client = zookeeper.createClient("localhost:2182", { sessionTimeout: 1000, retries: 0 });
let timeoutTimeperiod = 10000
client.connect();
Code stucks in the codes below:
client.setData(path, buffer, defaultVersion, function (error, stat) {
// failure
if (error) {
client.close();
response['message'] = 'Failed to set data on node : ' + path;
response['path'] = path;
response['error'] = error;
reject(response);
}
// success
client.close();
response['message'] = 'Data has been set on node : ' + path;
response['error'] = 'NIL';
response['status'] = true;
resolve(response);
});
Need help, thanks in advance
Is the distributed deployment cluster pattern available?
socket could be undefined or null
defense code needed
node-zookeeper-client/lib/ConnectionManager.js 625 line
if ( this.socket && !this.socket.write(packet.request.toBuffer())) { // Back pressure is handled here, when the socket emit // drain event, this method will be invoked again. break; }
var kafka = require('kafka-node'),
Consumer = kafka.Consumer,
client = new kafka.Client(),
consumer = new Consumer(
client,
[
{ topic: '1234', partition: 0 }
]
);
consumer.on('message', function (message) {
console.log(message);
});
Kafka consumer created using the above code crashes with NO_NODE[-101] error.
Exception: NO_NODE[-101]
at ConnectionManager.onSocketData (/home/sk/ws/kafka/node_modules/kafka-node/node_modules/node-zookeeper-client/lib/ConnectionManager.js:570:23)
at Socket.EventEmitter.emit (events.js:95:17)
at Socket.<anonymous> (_stream_readable.js:746:14)
at Socket.EventEmitter.emit (events.js:92:17)
at emitReadable_ (_stream_readable.js:408:10)
at emitReadable (_stream_readable.js:404:5)
at readableAddChunk (_stream_readable.js:165:9)
at Socket.Readable.push (_stream_readable.js:127:10)
at TCP.onread (net.js:526:21)
More information on the node path that is not found will help a lot....
When I use [email protected]
on npm, there is a require error on WatcherManager.js
. Does the code on npm is the same on github?
module.js:340
throw err;
^
Error: Cannot find module './path.js'
at Function.Module._resolveFilename (module.js:338:15)
at Function.Module._load (module.js:280:25)
at Module.require (module.js:362:17)
at require (module.js:378:17)
at Object.<anonymous> (node-hbase-client/node_modules/zookeeper-watcher/node_modules/node-zookeeper-client/lib/WatcherManager.js:10:12)
at Module._compile (module.js:449:26)
at Object.Module._extensions..js (module.js:467:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Module.require (module.js:362:17)
``
Is there a plan to add ssl support for the client?
Does this have the same support for Client cert authentication as outlined here: https://cwiki.apache.org/confluence/display/ZOOKEEPER/ZooKeeper+SSL+User+Guide
Would be nice if there was one too.
see line 227,
...An array of [ACL](#acl) objects, option...
In case server is offline, ConnectionManager
will try to reconnect infinite times without emitting errors or any other notifying. Because of that in node-kafka
which uses ConnectionManager
it's impossible to handle connection errors. It also goes 100% CPU (I guess because of retries frequency) with node 10.30 OS X. It doesn't consume all the CPU on 10.33, but connection loss is still unmanageable.
Can we have those errors emitted or fixed retries count or something else.
I want to use diffrent client in one file, and I first write as follow:
for (client in clients) {
client.on(something, fucntion(){
client.....
});
}
i find evrytime the client in client.on is the same client, and
for (client in clients) {
function test (){
client.on(something, fucntion(){
client.....
});
}
test();
}
So i can get diffrent client in clinet.on, but i don't know why it works, I have struggle for hours, so can u explain or just give me the way to bind diffrent client? thanks very much : )
I am trying to read ctime
and mtime
from the Stat object but I am unable to do that.
When I print Stat
object I see below output. But I am unable to convert ctime
buffer to meaningful value.
I tried stat.mtime.toString('utf8')
and other buffer functions, but nothing helped.
Have anyone has tried converting mtime
buffer to String, something like Tue Mar 28 22:53:39 PDT 2017
? Any pointers will be appreciated.
Record {
specification:
[ { name: 'czxid', type: 'long' },
{ name: 'mzxid', type: 'long' },
{ name: 'ctime', type: 'long' },
{ name: 'mtime', type: 'long' },
{ name: 'version', type: 'int' },
{ name: 'cversion', type: 'int' },
{ name: 'aversion', type: 'int' },
{ name: 'ephemeralOwner', type: 'long' },
{ name: 'dataLength', type: 'int' },
{ name: 'numChildren', type: 'int' },
{ name: 'pzxid', type: 'long' } ],
chrootPath: undefined,
czxid: <Buffer 00 00 00 43 02 23 af 4f>,
mzxid: <Buffer 00 00 00 43 02 23 af 4f>,
ctime: <Buffer 00 00 01 5b 18 bc d3 f8>,
mtime: <Buffer 00 00 01 5b 18 bc d3 f8>,
version: 0,
cversion: 0,
aversion: 0,
ephemeralOwner: <Buffer 00 00 00 00 00 00 00 00>,
dataLength: 748,
numChildren: 0,
pzxid: <Buffer 00 00 00 43 02 23 af 4f> }
I want to stop the connection and notice the user when I got the error event of slef.connect
....but the function ConnectionManager.prototype.onSocketClosed
invoke connect again....Is there any way for me to stop retry and I can add a listener for the client connection error in my code?
I tried to connect two zk and create two client , but everytime in the callback I can only get the last event, so is the zookeeper is singleton?
่ฑๆๆฐดๅนณๅคชๆฌก๏ผๆ
ๅฟ่ฑ่ฏญๆ่ฟฐไธๆธ
ๆฅ๏ผ็จๅฝ่ฏญๅงใ
ๆ็ไฝฟ็จๆ
ๅตๆฏ่ฟๆ ท็๏ผๆๅกๅฏๅจ็ๆถๅไผๅzookeeper้ๆทปๅ ไธไธช็ป็น๏ผๅพๆฎ้ๅพๅนณๅธธ็ไธไธช็ป็นใ็ถๅๆๆๅก้ๅๅฏๅจไบไธไธชๅฆๅคไธไธชๆๅก็็ๅฌใๅฝๆ็งๅผๅธธๆ
ๅตๆๅฆๅค้ฃไธช็ๅฌๆๅก็่ฟๆฅไผๅคฑ่ดฅ๏ผ่ฟไธชๆถๅๆ่ฆ่ชๅจ้ๅฏ็ๅฌ๏ผ่ๅๆถๅข๏ผๆไผๅๆฌกๆทปๅ ๅzookeeperไธญๆทปๅ ็ป็น๏ผๅฐฑๅจ่ฟไธชๆถๅ๏ผ็ๆๅกๆฅๅฟๆฏ่ฟไธชๆถๅ๏ผ๏ผ็ป็นๆทปๅ ไธไธ๏ผไปzkuiไธญ็ไธๅฐ็ป็น๏ผไฝๆฏไปฃ็ ไธญๅนถๆฒกๆๆฅๅๅปบ็ป็นๅคฑ่ดฅ็้่ฏฏใ
ๅ ไธบ่ฟ็งๆ
ๅตไธๅฎนๆ้็ฐ๏ผ็ฎๅไนๆฒกๆๆพๅฐๅๆณๆจกๆ๏ผไฝๆฏ่ฟไธช้ฎ้ข็กฎๅฎๅบ็ฐ่ฟๅ ๆฌกใ็ฐๅจๆๅๆไธๅบๆฅๅๅ ไบ๏ผๅธๆๅไฝ่ๅคงๅธฎๅธฎๅฟ๏ผไธๅๆๆฟใ @alexguan
So, as the title says, I'm seeing an exception when I attempt to use a client instance that has successfully connected when it includes multiple servers. These servers all contain a chroot path, but I'm not sure if that makes a difference in the issue or not. I assume that you have tried this module out with multiple servers, but maybe not multiple servers chroot'd? The chroot path is the same for all servers, so that doesn't matter.
The connection succeeds, I handle the 'connected' event, but when I attempt to do anything after that, for example I try to do a mkdirp in the 'connected' handler to initialize things, it fails with NO_NODE exception. I tried following things down through the jute code, but I couldn't find anything that would send back a NO_NODE error.
I don't know what 'node' the error is referring to and I don't know why things work with one server in the connect string, but not with multiple servers. Can anyone help me out? Thanks.
node-zookeeper-client/lib/Transaction.js
Line 123 in 290e468
If in transaction call Transaction.prototype.setData = function (path, data, version)
and version has value 0, but an output is internally evaluated as -1 but should be 0 otherwise this use case doesn't work:
I have znode in zookeeper with version e.i. 33 and I will call aforementioned function
Transaction.prototype.setData = function (path, data, 0)
and everything will be updated even though this should throw an Exception.BAD_VERSION because version 0 and 33 is different.
I don't know what does they mean on their site (https://www.npmjs.com/package/fs) but the package is creating build issues in my vue and nuxt js application. Kindly look into it as I need Zookeeper client for service discovery.
we are using "node-zookeeper-client" in our node.js server to watch backend zookeeper event change.
we are using getData like here https://github.com/alexguan/node-zookeeper-client/blob/master/examples/get.js
And recently, the backend zookeeper kept changing leader every 10 milliseconds(which is flapping, we will fix this issue on the backend for sure). however, in the meantime, our Web app is crashing due to page fault because the event change watcher continually process leadership change event on node.js.
is there a way to fix this?
Hello,
I maintain zkplus, which several large(ish) consumers of Zookeeper use in node to interact with the ZK API. I recently made a side project to port to this library as opposed to the native one, since the native one is riddled with bugs, and exacerbated by poor interactions with libuv. However, upon testing with lots of states where Zookeeper is offline, I am able to reproduce crashes that resemble this (note that the specific error code is non-deterministic, leading me to believe you are either incorrectly parsing Jute or have parsing logic where your offset gets mangled):
/Users/mcavage/work/registrar/node_modules/zkplus/node_modules/node-zookeeper-client/lib/Exception.js:51
throw new Error('Unknown code: ' + code);
^
Error: Unknown code: 935133184
at validateCode (/Users/mcavage/work/registrar/node_modules/zkplus/node_modules/node-zookeeper-client/lib/Exception.js:51:15)
at Function.create (/Users/mcavage/work/registrar/node_modules/zkplus/node_modules/node-zookeeper-client/lib/Exception.js:140:5)
at ConnectionManager.onSocketData (/Users/mcavage/work/registrar/node_modules/zkplus/node_modules/node-zookeeper-client/lib/ConnectionManager.js:571:35)
at Socket.EventEmitter.emit (events.js:95:17)
at Socket.<anonymous> (_stream_readable.js:746:14)
at Socket.EventEmitter.emit (events.js:92:17)
at emitReadable_ (_stream_readable.js:408:10)
at emitReadable (_stream_readable.js:404:5)
at readableAddChunk (_stream_readable.js:165:9)
at Socket.Readable.push (_stream_readable.js:127:10)
The stack trace is pretty much a simple application I was using to drive testing of ephemeral nodes. Note that this is not at all anything my process/application can act on, as your library is throwing an assertion on itself: https://github.com/alexguan/node-zookeeper-client/blob/master/lib/Exception.js#L51
I don't have a very good way of reproducing this behavior, other than to note that it only happens when I create a client and connect it with a down zookeeper, wait some number of seconds-minutes and start the Zookeeper.
In the absence of figuring out repro and/or root cause (which I will open as a separate issue), can you please change the various uses of throw
in the library to emit('error')
from the client? This would at least enable me to detect your client is in a bad state and move forward, short of relying on a process restart.
A node , for example ../node and it's data is {'key1':10, 'key2':20}, Can I only set key2's value to 30 leaving key1's unchanged.
I'm going to write types definition file of this lib to make the lib easier to be used in typescript.
There are 2 ways: ]
so, will this lib be willing to bundle the types?
In order to test client being disconnected from zookeeper server, I'd open a second client with same session id and password as the first client, and close the session from the second cient, causing the first client to receive connection loss and session expiration events.
Should be interesting to allow to list recursively (option on the command line) the nodes that we have under a parent node
Hello everyone,
I wan to set a dictionary on a node
node path = "/test/testNode1/"
node data = {
"a":"b",
"b":"c"
}
So I converted the dictionary to buffer in this way:
let buffer = Buffer.from(JSON.stringify(node_data));
But when I set a value to a node, i get this exception
client.setData(path, buffer, 19, function (error, stat) {
if (error) {
console.log(error.stack);
client.close()
return;
}
console.log('Data is set.');
});
Let me know if I have done something wrong.
Thanks in advance
After expired
state event emit, do we need to reconnect?
The readme section at https://github.com/alexguan/node-zookeeper-client/blob/master/README.md#state explains the various states.
I had a bug in my code where I called close()
before I was connected, causing a weird error in node-zookeeper-client. While not particularly graceful, my bad. But it's not documented during which states I am allowed to call close. This is the first problem.
The next problem is that the readme describes State.CONNECTED in the little bullet list, but in the example right below, it refers to State.SYNC_CONNECTED. So which is it? Should I expect both? Something needs to be fixed in the documentation here.
Hi Alex,
I have a question regarding how to use getChildren and getData together in Node.js? More details about my question can be found here
Could you please kindly help me? Thanks.
Regards,
RAnga
Hello Guys
I have been trying to connect to ZooKeeper znode which has sasl authentication scheme using nodejs.
I am not able to do so.
Can you please suggest a way to do this ?
Thanks
I have written a script:
var zookeeper = require('node-zookeeper-client');
let zk_addr = "localhost:2182"
var client = zookeeper.createClient(zk_addr, { sessionTimeout: 1000, retries: 0 });
client.connect()
In this example the zookeeper address is incorrect. So the script tries to reconnect and doesn't timeout.
I want my script to timeout after some time.
Help would be appreciated
Thanks
It's more appropriate to change the retires to the zookeeper server's count.
Hello. I am trying to connect my React app to the Kafka server. I am expecting my React app to act as a consumer. However, I have been getting the following error in console:
Uncaught Error: net.connect is not a function
@ConnectionManager.js in node-zookeeper-client
I am using [email protected], [email protected] and [email protected]
Please provide help/valuable guidance for tackling this issue.
Thanks in advance.
Client stays in SYNC_CONNECTED state when disconnected from the server for longer than its session timeout. I expected DISCONNECTED state after second 3 or 5 below:
$ node watch.js 127.0.0.1:2181 /blah/eph3 bomb& sleep 1; sudo disable_port 2181 10;
[1] 29292
[ '127.0.0.1:2181', '/blah/eph3' ]
0: client is DISCONNECTED[0]
"connected" to the server.
[ 'client state', { name: 'SYNC_CONNECTED', code: 3 } ]
Node: /blah/eph3 is successfully created.
zk.exists@/blah/eph3 [path, result, error, stat] ["/blah/eph3",0,null,{"specification":[{"name":"czxid","type":"long"},{"name":"mzxid","type":"long"},{"name":"ctime","type":"long"},{"name":"mtime","type":"long"},{"name":"version","type":"int"},{"name":"cversion","type":"int"},{"name":"aversion","type":"int"},{"name":"ephemeralOwner","type":"long"},{"name":"dataLength","type":"int"},{"name":"numChildren","type":"int"},{"name":"pzxid","type":"long"}],"czxid":[0,0,0,0,0,0,10,41],"mzxid":[0,0,0,0,0,0,10,41],"ctime":[0,0,1,65,127,31,233,184],"mtime":[0,0,1,65,127,31,233,184],"version":0,"cversion":0,"aversion":0,"ephemeralOwner":[1,65,96,50,23,73,0,51],"dataLength":4,"numChildren":0,"pzxid":[0,0,0,0,0,0,10,41]}]
1: client is SYNC_CONNECTED[3]
2: client is SYNC_CONNECTED[3]
3: client is SYNC_CONNECTED[3]
4: client is SYNC_CONNECTED[3]
5: client is SYNC_CONNECTED[3]
6: client is SYNC_CONNECTED[3]
7: client is SYNC_CONNECTED[3]
8: client is SYNC_CONNECTED[3]
9: client is SYNC_CONNECTED[3]
10: client is SYNC_CONNECTED[3]
11: client is SYNC_CONNECTED[3]
12: client is SYNC_CONNECTED[3]
zk watcher@/blah/eph3: NODE_DELETED[2]@/blah/eph3 state SYNC_CONNECTED[3]
zk.exists@/blah/eph3 [path, result, error, stat] ["/blah/eph3",-4,"Exception: CONNECTION_LOSS[-4]",null]
[ 'client state', { name: 'DISCONNECTED', code: 0 } ]
[ 'client state', { name: 'EXPIRED', code: -122 } ]
close1
[ 'client state', { name: 'DISCONNECTED', code: 0 } ]
13: client is DISCONNECTED[0]
14: client is DISCONNECTED[0]
15: client is DISCONNECTED[0]
$ cat watch.js
var zookeeper = require('node-zookeeper-client');
function inspect(o) { return require('util').inspect(o, false, null); }
console.dir([process.argv[2], process.argv[3]]);
var client = zookeeper.createClient(process.argv[2], {retries: 0, sessionTimeout: 5000});
var path = process.argv[3];
var data = new Buffer(process.argv[4]);
function exceptionObjectToCode(exceptionObject) { return exceptionObject ? exceptionObject.getCode() : zookeeper.Exception.OK; }
function exceptionObjectToString(exceptionObject) { return exceptionObject ? exceptionObject.toString() : null; }
client.on('state', function (s) {
console.dir(['client state', s]);
if (client.getState() === zookeeper.State.EXPIRED) {
console.log('close1');
client.close();
}
});
function curryExistsHandler(path, next) {
return function (exceptionObject, stat) {
var rc = exceptionObjectToCode(exceptionObject);
var error = exceptionObjectToString(exceptionObject);
console.log("zk.exists@" + path + " [path, result, error, stat] " + JSON.stringify([path, rc, error, stat]));
return next(null);
};
}
function watcherCallback(watcherEvent) {
var type = watcherEvent.getType();
var path = watcherEvent.getPath();
console.log('zk watcher@' + path + ': ' + watcherEvent.toString() + ' state ' + client.getState());
client.exists(path, watcherCallback,
curryExistsHandler(path,
function(nextError) {
if (nextError) {
console.log('exists callback throwing ' + inspect(nextError));
throw nextError;
}
}));
}
client.once('connected', function () {
console.log('"connected" to the server.');
client.create(path, new Buffer('blah'), zookeeper.ACL.OPEN_ACL_UNSAFE,
zookeeper.CreateMode.EPHEMERAL,
function (error) {
if (error) {
console.log('Failed to create node: %s due to: %s.', path, error);
} else {
console.log('Node: %s is successfully created.', path);
}
client.exists(path, watcherCallback, curryExistsHandler(path,
function(nextError) {
if (nextError) {
console.log('exists (top) callback throwing ' + inspect(nextError));
throw nextError;
}
}));
});
});
client.connect();
var startTime = Date.now();
function reportClientStateLoop() {
console.log(Math.round((Date.now()-startTime)/1000) + ': client is ' + client.getState());
setTimeout(reportClientStateLoop, 1000);
}
reportClientStateLoop();
$ cat disable_port
/bin/bash -c "/sbin/iptables -I INPUT 1 -p tcp --sport ${1} -j DROP; /sbin/iptables -I INPUT 2 -p tcp --dport ${1} -j DROP; sleep ${2}; /sbin/iptables -D INPUT 1; /sbin/iptables -D INPUT 1"
(node:9452) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 notification listeners added. Use emitter.setMaxListeners() to increase limit;
function listChildren(path, callback) {
client.getChildren(path, (event) => {
logger.debug('Got watcher event: ' + event);
if (event === Event.NODE_CHILDREN_CHANGED) {
listChildren(path, callback);
}
}, (error, children, stat) => {
let version = stat ? stat.version : 'UNKOWN';
if (error) {
logger.error('Failed to list children of node: ' + path + ' due to: ' + error + ' in version: ' + version);
if (error.code === Exception.NO_NODE) callback(null);
if (error.code === Exception.CONNECTION_LOSS) {
client.connect();
listChildren(path, callback);
}
callback([]);
} else {
logger.debug('Children of node: ' + path + ' in version ' + version + ' are: ' + children + ' .');
callback(children);
}
});
}
Hi,
I was playing around with this library and also some other libs for zookeeper support in nodejs, but none has the option to unwatch some node after setting a watcher. From the official docs it looks like zookeeper is able to remove alreday set-up watchers:
https://zookeeper.apache.org/doc/trunk/zookeeperProgrammers.html#sc_WatchRemoval
I'm wondering if there's any option to do this?
Hi, i am attempting to support deleting a large node tree in my ZK. Is it possible to recursively delete? I see this available in ZK 3.4.0+.
Thanks,
Matt
I see the error message Error: Cannot find module "."
when trying to run a react app which includes node-zookeeper-client
. The error occurs on line 776 when trying to require
the specification file, ./specification.json
. This file doesn't exist in the code bundled by webpack. I'm using node-zookeeper-client
as a transitive dependency of zoologist
.
A potential fix: if specification.json
was specification.js
and exported the json as a var then the import would work. I've tried this. Happy to put in a PR if you'd like me to. I could probably work around this with some webpack changes but: 1) at the moment the dependency fails when used from react and you probably don't want this and 2) I might have to 'eject' the app from the cradle of create-react-app
and I'd rather not do that for this reason.
Steps to reproduce (I'm using yarn here but the steps could easily be npm):
$ create-react-app node-zookeeper-client_react_test
create-react-app
version 1.3.1.$ cd node-zookeeper-client
$ yarn add node-zookeeper-client
src/App.js
and add var zookeeper = require('node-zookeeper-client');
below the imports. (Using an import produces the same error).$ yarn start
and let it open the app's root pageI'm using node-zookeeper-client version 0.2.1. I've got the following stacktrace:
Error: offset: 4 is out of buffer range.
at Record.deserialize (/opt/Autonomy/shared/zookeeper-ext/node_modules/node-zookeeper-client/lib/jute/index.js:465:15)
at ConnectionManager.onSocketData (/opt/Autonomy/shared/zookeeper-ext/node_modules/node-zookeeper-client/lib/ConnectionManager.js:456:34)
at ConnectionManager.onSocketData (/opt/Autonomy/shared/zookeeper-ext/node_modules/node-zookeeper-client/lib/ConnectionManager.js:580:14)
at Socket.EventEmitter.emit (events.js:95:17)
at Socket. (stream_readable.js:745:14)
at Socket.EventEmitter.emit (events.js:92:17)
at emitReadable (_stream_readable.js:407:10)
at emitReadable (_stream_readable.js:403:5)
at readableAddChunk (_stream_readable.js:165:9)
at Socket.Readable.push (_stream_readable.js:127:10)
at TCP.onread (net.js:528:21)
It seems zookeeper is sending an empty message which causes the client to throw. I don't think throwing an exception is the right thing to do just because a separate process gave you some data you didn't like. Especially since the exception is thrown from a handler and therefore can't be caught by the code that's calling node-zookeeper-client. And especially since (AFAICT) the interface with the server isn't documented.
This happened just after we briefly lost contact with the zookeeper server. I believe that https://issues.apache.org/jira/browse/ZOOKEEPER-1437 addresses the same server-side bug (but in a completely different client).
I've got this weird exception:
2017-12-24T14:15:37.021Z - error: uncaughtException Error: Unknown code: -1237188481
at validateCode (project-folder/node_modules/node-zookeeper-client/lib/Exception.js:51:15)
at Function.create (project-folder/node_modules/node-zookeeper-client/lib/Exception.js:140:5)
at ConnectionManager.Object.<anonymous>.ConnectionManager.onSocketData (project-folder/node_modules/node-zookeeper-client/lib/ConnectionManager.js:570:35)
Not sure if it's related to this package since I'm using kafka-node.
I'm also not sure what causes it; I'm just sending a message and listen to the message (straightforward use case).
Just curious whether we could run it on a browser. Thanks!
When they do not exist, mkdir runs the create with the data on all the nodes, unneccarily making duplicate data in the zookeeper (which for certain applications, could be a security issue as well).
The default state of the client is 'DISCONNECTED'. When the client has failed to connect, the state has not changed and the 'state' event cannot be triggered. As a result, the corresponding operation cannot be performed when the client is disconnected.
Loosing zookeper connection while watching a persistent znode emits DISCONNECTED and EXPIRED state events, which makes it easy to decide to re-establish the connection.
Loosing zookeper connection while watching an EPHEMERAL znode returns NODE_DELETED inside the watch callback prior to emitting DISCONNECTED and EXPIRED state events, which makes it hard to decide if connection needs to be re-established. Not sure if this is working as designed, but it would be nice if persistent and ephemeral znodes behaved consistently.
$ cat disable_port
/bin/bash -c "/sbin/iptables -I INPUT 1 -p tcp --sport ${1} -j DROP; /sbin/iptables -I INPUT 2 -p tcp --dport ${1} -j DROP; sleep ${2}; /sbin/iptables -D INPUT 1; /sbin/iptables -D INPUT 1"
$ cat watcher.js
var zookeeper = require('node-zookeeper-client');
function inspect(o) { return require('util').inspect(o, false, null); }
console.dir([process.argv[2], process.argv[3]]);
var client = zookeeper.createClient(process.argv[2], { retries : 0, sessionTimeout: 5000});
var path = process.argv[3];
var data = new Buffer(process.argv[4]);
var ZOK = zookeeper.Exception.OK;
var ZNONODE = zookeeper.Exception.NO_NODE;
function exceptionObjectToCode(exceptionObject) { return exceptionObject ? exceptionObject.getCode() : ZOK; }
function exceptionObjectToString(exceptionObject) { return exceptionObject ? exceptionObject.toString() : null; }
client.on('state', function (s) {
console.dir(['client state', s]);
if (client.getState() === zookeeper.State.EXPIRED) {
console.log('close1');
client.close();
}
});
function curryExistsHandler(path, next) {
return function (exceptionObject, stat) {
var rc = exceptionObjectToCode(exceptionObject);
var error = exceptionObjectToString(exceptionObject);
console.log("zk.exists@" + path + " [path, result, error, stat] " + JSON.stringify([path, rc, error, stat]));
if (rc === ZOK || rc === ZNONODE || rc === zookeeper.Exception.SESSION_EXPIRED) {
return next(null);
} else {
return next(new Error('zk.exists ' + path + ' returned [rc, error, stat, state] of ' + JSON.stringify([rc, error, stat, client.getState()])));
}
};
}
function watcherCallback(watcherEvent) {
var type = watcherEvent.getType();
var path = watcherEvent.getPath();
console.log('zk watcher@' + path + ': ' + watcherEvent.toString() + ' state ' + client.getState());
client.exists(path, watcherCallback,
curryExistsHandler(path,
function(nextError) {
if (nextError) {
console.log('exists callback throwing ' + inspect(nextError));
throw nextError;
}
}));
}
client.once('connected', function () {
console.log('"connected" to the server.');
client.create(path, new Buffer('blah'), zookeeper.ACL.OPEN_ACL_UNSAFE,
//zookeeper.CreateMode.EPHEMERAL,
zookeeper.CreateMode.PERSISTENT,
function (error) {
if (error) {
console.log('Failed to create node: %s due to: %s.', path, error);
} else {
console.log('Node: %s is successfully created.', path);
}
client.exists(path, watcherCallback, curryExistsHandler(path,
function(nextError) {
if (nextError) {
console.log('exists (top) callback throwing ' + inspect(nextError));
throw nextError;
}
}));
});
});
client.connect();
//////////////////////////// PERSISTENT:
$ node watch.js 127.0.0.1:2181 /blah/eph3 bomb& sleep 1; sudo disable_port 2181 4.6;
[2] 24023
[ '127.0.0.1:2181', '/blah/eph3' ]
"connected" to the server.
[ 'client state', { name: 'SYNC_CONNECTED', code: 3 } ]
Node: /blah/eph3 is successfully created.
zk.exists@/blah/eph3 [path, result, error, stat] ["/blah/eph3",0,null,{"specification":[{"name":"czxid","type":"long"},{"name":"mzxid","type":"long"},{"name":"ctime","type":"long"},{"name":"mtime","type":"long"},{"name":"version","type":"int"},{"name":"cversion","type":"int"},{"name":"aversion","type":"int"},{"name":"ephemeralOwner","type":"long"},{"name":"dataLength","type":"int"},{"name":"numChildren","type":"int"},{"name":"pzxid","type":"long"}],"czxid":[0,0,0,0,0,0,9,232],"mzxid":[0,0,0,0,0,0,9,232],"ctime":[0,0,1,65,118,183,110,173],"mtime":[0,0,1,65,118,183,110,173],"version":0,"cversion":0,"aversion":0,"ephemeralOwner":[0,0,0,0,0,0,0,0],"dataLength":4,"numChildren":0,"pzxid":[0,0,0,0,0,0,9,232]}]
$ [ 'client state', { name: 'DISCONNECTED', code: 0 } ]
[ 'client state', { name: 'EXPIRED', code: -122 } ]
close1
[ 'client state', { name: 'DISCONNECTED', code: 0 } ]
[2]- Done node watch.js 127.0.0.1:2181 /blah/eph3 bomb
//////////////////////////// EPHEMERAL:
// after interrupting connection for 4.6 seconds we get watch callback is called with NODE_DELETED,
// then exists returns CONNECTION_LOSS. I don't want to introduce the complexity of tracking this state to see if
// connection gets re-established, and would prefer if client threw DISCONNECTED, and EXPIRED as in the PERSISTENT example
$ node watch.js 127.0.0.1:2181 /blah/eph3 bomb& sleep 1; sudo disable_port 2181 4.6;
[2] 24002
[ '127.0.0.1:2181', '/blah/eph3' ]
"connected" to the server.
[ 'client state', { name: 'SYNC_CONNECTED', code: 3 } ]
Node: /blah/eph3 is successfully created.
zk.exists@/blah/eph3 [path, result, error, stat] ["/blah/eph3",0,null,{"specification":[{"name":"czxid","type":"long"},{"name":"mzxid","type":"long"},{"name":"ctime","type":"long"},{"name":"mtime","type":"long"},{"name":"version","type":"int"},{"name":"cversion","type":"int"},{"name":"aversion","type":"int"},{"name":"ephemeralOwner","type":"long"},{"name":"dataLength","type":"int"},{"name":"numChildren","type":"int"},{"name":"pzxid","type":"long"}],"czxid":[0,0,0,0,0,0,9,229],"mzxid":[0,0,0,0,0,0,9,229],"ctime":[0,0,1,65,118,182,135,112],"mtime":[0,0,1,65,118,182,135,112],"version":0,"cversion":0,"aversion":0,"ephemeralOwner":[1,65,96,50,23,73,0,29],"dataLength":4,"numChildren":0,"pzxid":[0,0,0,0,0,0,9,229]}]
$ zk watcher@/blah/eph3: NODE_DELETED[2]@/blah/eph3 state SYNC_CONNECTED[3]
zk.exists@/blah/eph3 [path, result, error, stat] ["/blah/eph3",-4,"Exception: CONNECTION_LOSS[-4]",null]
exists callback throwing [Error: zk.exists /blah/eph3 returned [rc, error, stat, state] of [-4,"Exception: CONNECTION_LOSS[-4]",null,{"name":"SYNC_CONNECTED","code":3}]]
/home/x/src/nucleus/node_modules/watch.js:44
throw nextError;
^
Error: zk.exists /blah/eph3 returned [rc, error, stat, state] of [-4,"Exception: CONNECTION_LOSS[-4]",null,{"name":"SYNC_CONNECTED","code":3}]
at /home/x/src/nucleus/node_modules/watch.js:31:19
at /home/x/src/nucleus/node_modules/node-zookeeper-client/index.js:165:26
at Object.async.whilst (/home/x/src/nucleus/node_modules/node-zookeeper-client/node_modules/async/lib/async.js:624:13)
at /home/x/src/nucleus/node_modules/node-zookeeper-client/node_modules/async/lib/async.js:620:23
at /home/x/src/nucleus/node_modules/node-zookeeper-client/index.js:145:21
at Object.callback (/home/x/src/nucleus/node_modules/node-zookeeper-client/index.js:732:21)
at ConnectionManager.cleanupPendingQueue (/home/x/src/nucleus/node_modules/node-zookeeper-client/lib/ConnectionManager.js:189:27)
at ConnectionManager.onSocketClosed (/home/x/src/nucleus/node_modules/node-zookeeper-client/lib/ConnectionManager.js:274:10)
at Socket.EventEmitter.emit (events.js:95:17)
at TCP.close (net.js:466:12)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.