Comments (11)
Original comment by Xu Meng (Bitbucket: mengxumx, GitHub: dmabupt).
v1.0.6 fixed it.
from nodejs-idb-connector.
Awesome!
from nodejs-idb-connector.
Original comment by Kristopher Baehr (Bitbucket: krisbaehr, GitHub: krisbaehr).
@dmabupt, We switched over to version 1.0.6 last week and ran some tests. There was a significant performance improvement! Before db2a was performing about the same as JDBC at 500 rows, after our various tweaks, including OS tweaks for improved multi-threaded processing. Now, 1.0.6 is equal with JDBC at 1000 rows. At 2000 rows, it's still performing pretty well and flat out smokes db2a (I'm not sure what version that was). Thanks for everything.
from nodejs-idb-connector.
Original comment by Xu Meng (Bitbucket: mengxumx, GitHub: dmabupt).
@krisbaehr Does version 1.0.6 fix the problem? I forgot to mention that in the fix there is no more environment variable to set the column width. The accurate column width is calculated automatically to save memory usage.
from nodejs-idb-connector.
Original comment by Xu Meng (Bitbucket: mengxumx, GitHub: dmabupt).
@krisbaehr Commit b56ab92 and c97e353 fixed the problem. Now the column width is accurate. You can upgrade idb-connector to v1.0.6 to verify that.
from nodejs-idb-connector.
Original comment by Jesse G (Bitbucket: ThePrez, GitHub: ThePrez).
@krisbaehr, any luck? I'm hoping this shows proof of concept while we work on a more permanent fix.
from nodejs-idb-connector.
Original comment by Kristopher Baehr (Bitbucket: krisbaehr, GitHub: krisbaehr).
@dmabupt Thank you for this! I appreciate your diligence in resolving this performance issue. I will give it a shot soon.
from nodejs-idb-connector.
Original comment by Xu Meng (Bitbucket: mengxumx, GitHub: dmabupt).
Hello @krisbaehr ,
I have rebuilt the idb-connector to allow flexible column width to save memory usages. You can reinstall idb-connector and use the new environment value MAXCOLWIDTH to change the default value 32766. (Note: smaller MAXCOLWIDTH value costs less memory but may trim long text)
My test --
#!javascript
for(var colwidth = 128; colwidth < 65535; colwidth *= 2) {
process.env.MAXCOLWIDTH = colwidth;
var dbconn = new db.dbconn();
// fetch / exec ......
}
#!shell
bash-4.4$ node bench.js 8000
fetchAll took 2782.625468 ms -- Row Count: 8000 -- Col Width: 128
fetchAll took 1727.906687 ms -- Row Count: 8000 -- Col Width: 256
fetchAll took 1963.632875 ms -- Row Count: 8000 -- Col Width: 512
fetchAll took 1600.714552 ms -- Row Count: 8000 -- Col Width: 1024
fetchAll took 1616.5393869999998 ms -- Row Count: 8000 -- Col Width: 2048
fetchAll took 2070.021549 ms -- Row Count: 8000 -- Col Width: 4096
fetchAll took 3308.1328750000002 ms -- Row Count: 8000 -- Col Width: 8192
fetchAll took 4757.888971 ms -- Row Count: 8000 -- Col Width: 16384
fetchAll took 7919.675255 ms -- Row Count: 8000 -- Col Width: 32768
from nodejs-idb-connector.
Original comment by Kristopher Baehr (Bitbucket: krisbaehr, GitHub: krisbaehr).
@aaronbartell Be forewarned, I'm pretty new to this Node.js thing!
https://bitbucket.org/snippets/krisbaehr/keBa4E/node-sql-sp-call-for-performance-testing
from nodejs-idb-connector.
Original comment by Aaron Bartell (Bitbucket: aaronbartell, GitHub: aaronbartell).
Hi @krisbaehr,
Could you share the Node.js code you're using to calculate the Node.js stats? If the code is more than 100 lines then maybe create a Bitbucket Snippet.
from nodejs-idb-connector.
Original comment by Kristopher Baehr (Bitbucket: krisbaehr, GitHub: krisbaehr).
Node - SQL is doing the same thing the RPG SP is doing.
Node - RPG is calling an SP with the idb connector.
Java - RPG is Java calling the same SP as node using jdbc.
from nodejs-idb-connector.
Related Issues (20)
- Error SQLSTATE=02000 SQLCODE=100 HOT 4
- Error SQLSTATE=38501 SQLCODE=-443 does not return the true message HOT 1
- idb-connector not compatible with Node.js v16.x HOT 30
- Error SQLSTATE=02000 SQLCODE=100 HOT 1
- Error SQLSTATE=22504 SQLCODE=-191 Mixed data or UTF-8 data not properly formed. HOT 22
- Use of BINARY Parameter w/Stored Procedure Call Corrupts the Node Process HOT 3
- Handling permanent JOB... HOT 1
- Some issues when calling SP with dbstmt.bindParameters() HOT 1
- Error: SQLSTATE=22504 SQLCODE=-191 Mixed data or UTF-8 data not properly formed. HOT 1
- Not receiving output from Stored Procedure and no clear example..
- temporary memory of job increases massive when using CLOB/JSON field in SQL-request HOT 3
- Connection Pool HOT 3
- Add package-lock.json? HOT 1
- the sql state error seems to be fixed, but we have a different issue HOT 4
- idb-connector *SP calls returns invalid numeric data -- Node 18, IDB-Connector 1.2.18 HOT 21
- test: cleanup test infrastructure HOT 1
- CLOB: truncation of the last char ! HOT 6
- Fix license headers on source files
- Reference idb-pconnector for promise / connection pooling support
- Build from source is failing since release 1.2.19 HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nodejs-idb-connector.