Generating code and using it for HBase: 1.1.2 with Thrift compiler 0.10.0

[exposebox]

Generating code and using it for HBase: 1.1.2 with Thrift compiler 0.10.0

[moshewe]

Hi,

We are using the NodeJS code generated from the 0.98.4 HBase.thrift file with HBase running on version 1.1.2 in production without problems.

Here you can find the HBase 1.1.2 thrift definitions file you've requested. The main difference we see between the thrift definition files is that the 1.1.2 version allows you to pass the cacheBlocks flag to Scans.
Feel free to compile the newer definitions file and try the code out - we don't expect you'd have any problems. Please let us know how it goes!

[QuinnGao]

Hi,

Thanks for your prompt reply.
I'm trying to run any example for our HBase sever (1.1.2.2.5.3.0-37).

here's what I use in below,

var config = {
    hosts: ["master"],
    port: "9090",
};

var HBase = require('../src/service')(config);

var get = new HBase.Get('my_rowkey');  
get.addFamily('fl');

HBase.getAsync('my_table', get)
    .then(function (data) {
        console.log("Data for user with key 'row1':");
        console.log('==============================');
        _.each(data[0].columnValues, function (colVal, index) {
            console.log('Column value #', index);
            console.log('family:', colVal.family.toString());
            console.log('qualifier:', colVal.qualifier.toString());
            console.log('value:', colVal.value.readInt32BE(0, 4));
        });
    })
    .catch(function (err) {
        console.log('error:', err);
    });

In client side, I got the error

Thrift close connection error

In server side, I got error in hbase log

[main] mortbay.log: jetty-6.1.26.hwx
2017-08-31 19:14:52,781 INFO  [main] mortbay.log: Started [email protected]:8073
2017-08-31 19:14:52,795 INFO  [main] thrift.ThriftServerRunner: starting TBoundedThreadPoolServer on /0.0.0.0:8072; min worker threads=16, max worker threads=1000, max queued requests=1000
2017-08-31 19:15:02,795 ERROR [thrift-worker-0] thrift.TBoundedThreadPoolServer: Error occurred during processing of message.
java.lang.NullPointerException
	at org.apache.hadoop.hbase.util.Bytes.getBytes(Bytes.java:994)
	at org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.get(ThriftServerRunner.java:851)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.hbase.thrift.HbaseHandlerMetricsProxy.invoke(HbaseHandlerMetricsProxy.java:67)
	at com.sun.proxy.$Proxy11.get(Unknown Source)
	at org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$get.getResult(Hbase.java:4074)
	at org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$get.getResult(Hbase.java:4058)
	at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
	at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
	at org.apache.hadoop.hbase.thrift.TBoundedThreadPoolServer$ClientConnnection.run(TBoundedThreadPoolServer.java:289)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

Do you have any idea for this issue?

Thanks,

[Ran-P]

Are you running Thrift server on the your HBase or Thrift2?

P.s. in you code it should be:
HBase.getAsync('my_table', get, options)

[QuinnGao]

@Ran-P Thanks for your correct!
so the options should be as same as config?

var config = {
    hosts: ["master"],
    port: "9090",
};

var HBase = require('../src/service')(config);

var get = new HBase.Get('my_rowkey');  
get.addFamily('fl');

HBase.getAsync('my_table', get, config)
    .then(function (data) {
        console.log("Data for user with key 'row1':");
        console.log('==============================');
        _.each(data[0].columnValues, function (colVal, index) {
            console.log('Column value #', index);
            console.log('family:', colVal.family.toString());
            console.log('qualifier:', colVal.qualifier.toString());
            console.log('value:', colVal.value.readInt32BE(0, 4));
        });
    })
    .catch(function (err) {
        console.log('error:', err);
    });

Yes, here's my thrift server info

Attribute Name | Value
HBase Version | 1.1.2.2.5.3.0-37
HBase Compiled | Tue Nov 29 18:48:22 UTC 2016, jenkins
Thrift Server Start Time | Thu Aug 31 19:14:52 JST 2017
Thrift Impl Type | threadpool
Compact Protocol | false
Framed Transport | false

[moshewe]

The options passed to the Get object have nothing to do with the configuration passed to the library as a whole, please see the Get object code (maybe we should add this in the docs...) If you don't know what options to pass, or don't need to, just pass an empty options object ({}).

Please verify that the Thrift server you are running on your HBase server is in fact a thrift2 server.

[QuinnGao]

@moshewe, you are right, we are using the wrong thrift server.
It works now. Very thanks for all your kindly help!

[moshewe]

So now you are working with code generated from the 1.1.2 Thrift definition file? With what Thrift compiler did you generate the code?

[QuinnGao]

yes, I have generated 1.1.2 Thrift file using thrift-0.10.0

[moshewe]

OK, great! Please let us know if you have any more problems.