Comments (22)
@imjuni I solved the problem, because of my hive configuration. so funny.
Uh,
I am looking forward to your update to support the version of hive more than 2.0.****
from jshs2.
@xushuo Yes must be st NOSASL, because SASL setting require specific protocol handshake process in thrift. But jshs2 don't have a this handshake process. So jshs2 need NOSASL configuration. I will be add this feature in jshs2
from jshs2.
Are you run my test case? (use PromiseTest.js)
I estimate two issue. First, Your configuration invalid on hive environment. OR don't set NOSASL option in hive. See in https://cwiki.apache.org/confluence/display/Hive/Setting+Up+HiveServer2, Authentication/Security Configuration section. After set authentication option SASL -> NOSASL.
from jshs2.
@imjuni this is some error that i run your test:
E:\gitWorkspace\jshs2>mocha test
ThriftDriverTest
1) "before all" hook
2) "after all" hook
ThriftDriverTest
jshs2:OperationTestSuite Error caused from before task, ... +0ms
jshs2:OperationTestSuite Unexpected token / +2ms
jshs2:OperationTestSuite SyntaxError: Unexpected token /
at Object.parse (native)
at E:\gitWorkspace\jshs2\test\PromiseTest.js:26:25
at next (native)
at onFulfilled (E:\gitWorkspace\jshs2\node_modules\co\index.js:65:19)
at tryCallOne (E:\gitWorkspace\jshs2\node_modules\promise\lib\core.js:37:12)
at E:\gitWorkspace\jshs2\node_modules\promise\lib\core.js:123:15
at flush (E:\gitWorkspace\jshs2\node_modules\promise\node_modules\asap\raw.js:50:29)
at nextTickCallbackWith0Args (node.js:420:9)
at process._tickCallback (node.js:349:13) +1ms
3) "before all" hook
jshs2:OperationTestSuite Error caused from after task, ... +2s
jshs2:OperationTestSuite Cannot read property 'close' of undefined +0ms
jshs2:OperationTestSuite TypeError: Cannot read property 'close' of undefined
at E:\gitWorkspace\jshs2\test\PromiseTest.js:79:19
at next (native)
at onFulfilled (E:\gitWorkspace\jshs2\node_modules\co\index.js:65:19)
at E:\gitWorkspace\jshs2\node_modules\co\index.js:54:5
at co (E:\gitWorkspace\jshs2\node_modules\co\index.js:50:10)
at Context. (E:\gitWorkspace\jshs2\test\PromiseTest.js:78:5)
at callFnAsync (C:\Users\Administrator\AppData\Roaming\npm\node_modules\mocha\lib\runnable.js:357:21)
at Hook.Runnable.run (C:\Users\Administrator\AppData\Roaming\npm\node_modules\mocha\lib\runnable.js:309:7)
at next (C:\Users\Administrator\AppData\Roaming\npm\node_modules\mocha\lib\runner.js:299:10)
at Immediate._onImmediate (C:\Users\Administrator\AppData\Roaming\npm\node_modules\mocha\lib\runner.js:327:5)
at processImmediate as _immediateCallback +16ms
4) "after all" hook
0 passing (4s)
4 failing
-
ThriftDriverTest "before all" hook:
Uncaught SyntaxError: Unexpected token /
at Object.parse (native)
at E:\gitWorkspace\jshs2\test\CallbackTest.js:134:36
at FSReqWrap.readFileAfterClose as oncomplete -
ThriftDriverTest "after all" hook:
Uncaught TypeError: Cannot read property 'close' of undefined
at closeCursor (E:\gitWorkspace\jshs2\test\CallbackTest.js:197:15)
at fn (E:\gitWorkspace\jshs2\node_modules\async\lib\async.js:638:34)
at Immediate._onImmediate (E:\gitWorkspace\jshs2\node_modules\async\lib\async.js:554:34) -
ThriftDriverTest "before all" hook:
Error: timeout of 2000ms exceeded. Ensure the done() callback is being called in this test.
at null. (C:\Users\Administrator\AppData\Roaming\npm\node_modules\mocha\lib\runnable.js:230:19) -
ThriftDriverTest "after all" hook:
Error: timeout of 2000ms exceeded. Ensure the done() callback is being called in this test.
at null. (C:\Users\Administrator\AppData\Roaming\npm\node_modules\mocha\lib\runnable.js:230:19)
from jshs2.
@imjuni and this is my cluster.json:
{
"use": "ApacheHive", // Use Configuration
"HiveOnCDH": {
"auth": "NOSASL",
"host": "my-hive.com", // hive server hostname
"port": 13333, // port number
"timeout": 10000, // timeout number(important, timeout is must be integer value)
"username": "ironman", // username
"hiveType": 1, // hive type number, (hive 0, cdh hive 1 see below HS2Util.js)
"hiveVer": "0.13.1", // hive version
"cdhVer": "5.3.0", // cdh version
"thriftVer": "0.9.2", // thrift version
"maxRows": 5120, // max row
"nullStr": "NULL", // NULL field replace to this string
"i64ToString": true // If you use i64 value that set true
},
"ApacheHive": {
"auth": "NOSASL",
"host": "192.168.100.125",
"port": 10000,
"timeout": 10000,
"username": "hadoop",
"password": "hadoop",
"hiveType": 0,
"hiveVer": "1.2.0",
"thriftVer": "0.9.2",
"maxRows": 5120,
"nullStr": "NULL",
"i64ToString": true
},
"Query": {
"query": "show databases" // Enter your query
}
}
from jshs2.
@imjuni my HiveOnCDH is cdh5.7.1 that is too high
from jshs2.
I think that check your hive configuration. You need hive Authentication/Security Configuration, set NOSASL. Not only jshs2 but also hive configuration.
via: https://cwiki.apache.org/confluence/display/Hive/Setting+Up+HiveServer2,
Authentication mode:
hive.server2.authentication – Authentication mode, default NONE. Options are NONE (uses plain SASL), NOSASL, KERBEROS, LDAP, PAM and CUSTOM.
You set NOSASL mode on hive server2 configuration
from jshs2.
@imjuni ok , i will check my hive configuration as soon as possible .
from jshs2.
@imjuni i update my hive configuration .but the above problems will occur again,
from jshs2.
@xushuo I think your error caused from cluster.json. That is invalid json file, so parsing fail at JSON.parse. Check your cluster.json file using by http://jsonlint.com/.
or change code, PromiseTest.js line 26,
try {
var config = JSON.parse(yield new Promise(function (resolve, reject) {
fs.readFile('./cluster.json', function (err, buf) {
if (err) {
reject(err);
} else {
resolve(buf);
}
});
}));
} catch (err) {
console.log(err);
}
Change code after, you can see parse fail reason. thank, you
from jshs2.
@imjuni oh ~~ there are some Invalid format errors in my cluster.json.
but there have been some new error after i correct the error.
【connect refuse】
Is my hive configuration causing this error?
from jshs2.
Your hive-server version is 1.2.0 is valid? If you not install hive 1.2.0, hive server version is 1.1.0 is valid. See it, Cloudera Hive repo.
Also, my hive-server2 version is 1.1.0 and perfectly working. Because hive-server 1.1.0 client_protocol version is 6 but hive-server 1.2.0 client_protocol version is 7. Invalid client_protocol version cause error. Check your hive version.
from jshs2.
yes,my hive-server version is 1.2.0 .
Maybe i need to install hive-server 1.1.0 . but the latest hive has more than 2.0 and cdh 5.7. Do you plan to update the project?
from jshs2.
@imjuni Uh, sorry.I didn't see you update two days ago.
from jshs2.
@xushuo Sorry for late answer because Korea has a vacations. Can you explain that your problem?
from jshs2.
@imjuni My hive server doesn't start properly.so connect refuse......I reset the hive, and restart,and finished...
I see you have updated the project and add the hive 2.0.
from jshs2.
@xushuo I see, thank you. I was updated idl for hive 2.0.x. And I recommand that use thrift 0.9.3 because 0.9.3 ver more than faster(1 ~ 3%)
from jshs2.
@imjuni I have a question that the "hive.server2.authentication" must be 'NOSASL' ?
now , if i add the config,hive doesn`t work well .
from jshs2.
@imjuni OK!My project work well. But i find that if i use lower idl such as hiveVer = 1.1.0 , the project looks work well . I can also get the right result. why? Speed is the difference between them?
from jshs2.
Hive version affect query execution time. Higher version better query performance than lower version. But Thrift version have affect communication time. Fetch result download time decrease, So Higher version mostly cause better performance.
from jshs2.
@imjuni Thank you !I get it . Yahoo!
from jshs2.
@imjuni hi! do you have other example of the jshs2 ? Because I need the parameters in the process of query (such as time-consuming, status, progress).
from jshs2.
Related Issues (18)
- getLog problem solve HOT 1
- How can I know the query progress? HOT 2
- can this support spark sql? HOT 2
- can it not set 'NOSASL '? HOT 1
- org.apache.thrift.transport.TTransportException: Invalid status -128 HOT 1
- I am facing an issue with timed out when i run test run HOT 1
- try cursor.execute(query), but execute is not a function? HOT 4
- HiveQL question on SO HOT 1
- test script is not working. HOT 2
- Cannot get test to pass HOT 1
- example code in "Simple Usage" HOT 2
- Not able to connect to db HOT 3
- Only get 1024 data from hive server
- Vulnerable version of lodash HOT 1
- error in connection HOT 1
- The thrift dependency is bloated HOT 6
- question: is this library used for querying or for management? HOT 9
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from jshs2.