tdebarochez / connect-couchdb Goto Github PK
View Code? Open in Web Editor NEWSession management middleware for connect
License: MIT License
Session management middleware for connect
License: MIT License
Been trying this out on a VPS with a simple app with very low traffic (still in test).
I noticed when using this for session store my cpu usage jumps up 5% (5 minute average) and was then climbing at an approx rate of 2% per week.
Similar with my IO rate, jumped to 50 blocks/sec and then climbed at a rate of approx 50 blocks average per week (i.e. after running for a week my IO rate had increased by an average of 100 blocks per second).
I've gone back to using a memorystore and my IO rate and CPU usage has dropped back down to before I used this middleware.
My app is using couchdb elsewhere (seperate DB from session and using cradle).
While I can understand an increase in load due to periodic updating/cleaning, I think there must be a problem somewhere causing the load to increase at such a steady rate.
I tried investigating what could be causing the increase in IO and iotop pointed the finger at kjournald.
Also I'm not actually using the session to store anything yet!
Running on Lindode VPS 512 (just running, a single instance of this node.js app with couchdb).
Ubuntu Server 10.10 with [email protected]
├─┬ [email protected]
│ ├─┬ [email protected]
│ │ ├── [email protected]
│ │ ├── [email protected]
│ │ └── [email protected]
│ └── [email protected]
├─┬ [email protected]
│ ├── [email protected]
│ └─┬ [email protected]
│ └── [email protected]
├─┬ [email protected]
│ ├─┬ [email protected]
│ │ └── [email protected]
│ ├── [email protected]
│ ├── [email protected]
│ └── [email protected]
├─┬ [email protected]
│ ├─┬ [email protected]
│ │ └── [email protected]
│ └── [email protected]
├─┬ [email protected]
│ ├── [email protected]
│ └── [email protected]
└─┬ [email protected]
└── [email protected]
If I can help debug/test then just shout.
Connect 3.0.0 breaks with the following error when trying to add the design docs:
> $ node tools/put_design_docs.js _sessions 1500 admin admin ⬡ 0.10.28 [±master ●]
/Users/mikehostetler/Source/connect-couchdb/lib/connect-couchdb.js:4
var Store = connect.session.Store;
^
TypeError: Cannot read property 'Store' of undefined
at module.exports (/Users/mikehostetler/Source/connect-couchdb/lib/connect-couchdb.js:4:32)
at Object.<anonymous> (/Users/mikehostetler/Source/connect-couchdb/tools/put_design_docs.js:16:77)
at Module._compile (module.js:456:26)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Function.Module.runMain (module.js:497:10)
at startup (node.js:119:16)
at node.js:906:3
The current release doesn't contain 234c54b (which raises warnings when writing tests with Mocha). Can you please roll a new release of connect-couchdb?
Hello. Are you still maintaining this module? I am getting a significant number of document conflict errors from your underlying YACW. I'll be looking into this further you but you have nothing there to manage conflicts.
Error: Document update conflict.
... node_modules/connect-couchdb/node_modules/yacw/lib/couch.js:34:15
Seems to occur mostly when logging out or destroying a session.
There is no need to have such a complicated view that I'm aware of. Is there a use case you have not yet covered in your code but intend to that would explain the current implementation ? Otherwise I would recommend to get to a simpler view that only does:
emit(doc.expires, null)
You can then request it with
endkey=(+new Date)
instead of
endkey='[' + (+new Date()) + ', {}]'
You don't need to emit the id as data as you do now because it is already included as a side data in your view (the id is returned with each row in a view if you do not use a reduce on it). You can even go further and use the include_docs=true parameter that will return to you the whole documents you selected in the view. That might look too much but it will prevent you from requesting each objects separately after that (you will to since you have to retrieve the rev field to DELETE them).
I also believe you should add the reduce=false parameter when requesting your expires view just in case someone adds for some strange reason a reduce function to your view. That will prevent it from breaking your code.
If you don't have time to do those optimizations (and agree with the spirit), let me know and I might try to tackle them and provide you with a corresponding pull request.
You can find the doc for most of what I described on the CouchDB wiki here : http://wiki.apache.org/couchdb/HTTP_view_API#Querying_Options
In the case where two concurrent commits are made on the same session, a conflict might occurs. I see three ways to deals with it :
When you feel the new work is stable we would appreciate a new tag and npm package. Maybe a 0.2.0
given the number of changes?
Hi, first I want to congratulate you for the work you've done.
I ask you: is it possible to access request params?
Let me explain:
I need to access a remote database that is with Unsigned and request ssl allows me only if I specify "request_defaults": {"strictSSL": false} in the options, but not how to do it in this library.
Thank you !
PS: My apologies for my English, I'm not native English speaker and Google Translate did most of the work.
Morning killer module, two issues/questions. First I found that the Connect-CouchDB store fails with older version of Express. The last three verisons of express 3.0.0rc1 - 3 appears to solve a session bug. Is this a known issue? For newbies, this might be helpful to document.
Second, can you clarify the how a session store will retire or clear our expired sessions? Does the dev need to write a function that will query the expired session view and purge documents?
Thanks, once we got the version/cookie issue solved with express, this module helped us move our app to multi-server EC setup.
Hi,
I have my application hosted in bluemix. I am using connect-couchdb module for session management.
Frequently I am facing the below errors. Please help me on this. I have provided the error logs below.
App/0Error: null2016-10-26T02:55:58.716+0530
App/0 at process._tickDomainCallback (node.js:492:13)2016-10-26T02:55:58.716+0530
App/0 at _stream_readable.js:944:162016-10-26T02:55:53.187+0530
App/0 at IncomingMessage. (/home/vcap/app/node_modules/request/request.js:965:12)2016-10-26T02:55:53.187+0530
App/0 at process._tickDomainCallback (node.js:492:13)2016-10-26T02:55:53.187+0530
App/0Error: null2016-10-26T02:55:53.187+0530
App/0 at Request.emit (events.js:95:17)2016-10-26T02:55:53.187+0530
App/0 at Request._callback (/home/vcap/app/node_modules/connect-couchdb/node_modules/yacw/lib/couch.js:39:15)2016-10-26T02:55:33.135+0530
App/0 at process._tickDomainCallback (node.js:492:13)2016-10-26T02:55:33.135+0530
App/0 at process._tickDomainCallback (node.js:492:13)2016-10-26T02:55:22.750+0530
App/0 at Request.self.callback (/home/vcap/app/node_modules/request/request.js:187:22)2016-10-26T02:55:22.750+0530
App/0 at IncomingMessage.emit (events.js:117:20)2016-10-26T02:55:22.750+0530
App/0 at Request. (/home/vcap/app/node_modules/request/request.js:1044:10)2016-10-26T02:55:22.750+0530
App/0 at IncomingMessage. (/home/vcap/app/node_modules/request/request.js:965:12)2016-10-26T02:55:22.750+0530
App/0 at IncomingMessage.emit (events.js:117:20)2016-10-26T02:54:58.465+0530
App/0 at Request.self.callback (/home/vcap/app/node_modules/request/request.js:187:22)2016-10-26T02:54:58.465+0530
App/0 at Request.self.callback (/home/vcap/app/node_modules/request/request.js:187:22)2016-10-26T02:54:03.012+0530
App/0 at process._tickDomainCallback (node.js:492:13)2016-10-26T02:54:03.012+0530
App/0 at Request. (/home/vcap/app/node_modules/request/request.js:1044:10)2016-10-26T02:54:03.012+0530
App/0 at Request.emit (events.js:98:17)2016-10-26T02:54:03.012+0530
App/0Error: null2016-10-26T02:54:03.012+0530
App/0 at _stream_readable.js:944:162016-10-26T02:54:03.012+0530
App/0 at Request.emit (events.js:98:17)2016-10-26T02:53:00.533+0530
App/0 at Request._callback (/home/vcap/app/node_modules/connect-couchdb/node_modules/yacw/lib/couch.js:39:15)2016-10-26T02:53:00.533+0530
App/0 at IncomingMessage.emit (events.js:117:20)2016-10-26T02:53:00.533+0530
App/0 at IncomingMessage.emit (events.js:117:20)2016-10-26T02:52:01.945+0530
App/0 at Request.emit (events.js:98:17)2016-10-26T02:52:01.945+0530
App/0 at _stream_readable.js:944:162016-10-26T02:52:01.945+0530
App/0Error: null2016-10-26T02:52:01.944+0530
App/0 at Request.emit (events.js:98:17)2016-10-26T02:51:59.052+0530
App/0 at Request.emit (events.js:95:17)2016-10-26T02:51:59.052+0530
App/0 at process._tickDomainCallback (node.js:492:13)2016-10-26T02:51:59.052+0530
App/0 at Request._callback (/home/vcap/app/node_modules/connect-couchdb/node_modules/yacw/lib/couch.js:39:15)2016-10-26T02:51:59.052+0530
App/0 at IncomingMessage.emit (events.js:117:20)2016-10-26T02:51:59.052+0530
App/0 at _stream_readable.js:944:162016-10-26T02:51:59.052+0530
App/0 at Request. (/home/vcap/app/node_modules/request/request.js:1044:10)2016-10-26T02:51:01.468+0530
App/0 at Request.self.callback (/home/vcap/app/node_modules/request/request.js:187:22)2016-10-26T02:51:01.468+0530
App/0 at process._tickDomainCallback (node.js:492:13)2016-10-26T02:51:01.468+0530
App/0 at _stream_readable.js:944:162016-10-26T02:51:01.468+0530
App/0 at IncomingMessage.emit (events.js:117:20)2016-10-26T02:51:01.468+0530
App/0Error: null2016-10-26T02:51:01.468+0530
App/0 at Request.emit (events.js:95:17)2016-10-26T02:51:00.330+0530
App/0 at process._tickDomainCallback (node.js:492:13)2016-10-26T02:51:00.330+0530
App/0 at _stream_readable.js:944:162016-10-26T02:51:00.330+0530
App/0 at IncomingMessage. (/home/vcap/app/node_modules/request/request.js:965:12)2016-10-26T02:51:00.330+0530
App/0 at _stream_readable.js:944:162016-10-26T02:50:53.920+0530
App/0 at IncomingMessage. (/home/vcap/app/node_modules/request/request.js:965:12)2016-10-26T02:50:53.920+0530
App/0Error: null2016-10-26T02:50:53.920+0530
App/0 at Request.emit (events.js:98:17)2016-10-26T02:50:53.920+0530
App/0 at Request._callback (/home/vcap/app/node_modules/connect-couchdb/node_modules/yacw/lib/couch.js:39:15)2016-10-26T02:50:53.920+0530
App/0 at _stream_readable.js:944:162016-10-26T02:50:51.496+0530
App/0 at process._tickDomainCallback (node.js:492:13)2016-10-26T02:50:51.496+0530
App/0 at Request. (/home/vcap/app/node_modules/request/request.js:1044:10)2016-10-26T02:50:44.603+0530
App/0 at Request.emit (events.js:98:17)2016-10-26T02:50:44.603+0530
App/0 at Request.emit (events.js:95:17)2016-10-26T02:50:44.603+0530
App/0 at Request._callback (/home/vcap/app/node_modules/connect-couchdb/node_modules/yacw/lib/couch.js:39:15)2016-10-26T02:50:44.603+0530
App/0 at Request.emit (events.js:95:17)2016-10-26T02:50:35.964+0530
App/0 at IncomingMessage.emit (events.js:117:20)2016-10-26T02:50:35.964+0530
App/0 at Request._callback (/home/vcap/app/node_modules/connect-couchdb/node_modules/yacw/lib/couch.js:39:15)2016-10-26T02:50:35.964+0530
App/0Error: null2016-10-26T02:50:35.964+0530
App/0 at IncomingMessage. (/home/vcap/app/node_modules/request/request.js:965:12)2016-10-26T02:50:35.964+0530
App/0 at Request.self.callback (/home/vcap/app/node_modules/request/request.js:187:22)2016-10-26T02:50:32.480+0530
App/0 at Request. (/home/vcap/app/node_modules/request/request.js:1044:10)2016-10-26T02:50:32.480+0530
App/0 at process._tickDomainCallback (node.js:492:13)2016-10-26T02:50:32.480+0530
App/0 at IncomingMessage. (/home/vcap/app/node_modules/request/request.js:965:12)2016-10-26T02:50:32.480+0530
App/0 at Request._callback (/home/vcap/app/node_modules/connect-couchdb/node_modules/yacw/lib/couch.js:39:15)2016-10-26T02:50:32.480+0530
App/0 at Request.emit (events.js:95:17)2016-10-26T02:50:31.317+0530
App/0 at Request. (/home/vcap/app/node_modules/request/request.js:1044:10)2016-10-26T02:50:31.317+0530
App/0 at Request._callback (/home/vcap/app/node_modules/connect-couchdb/node_modules/yacw/lib/couch.js:39:15)2016-10-26T02:50:31.317+0530
App/0 at Request.emit (events.js:95:17)2016-10-26T02:50:29.317+0530
App/0 at Request.self.callback (/home/vcap/app/node_modules/request/request.js:187:22)2016-10-26T02:50:29.317+0530
App/0Error: null2016-10-26T02:50:29.316+0530
App/0 at IncomingMessage.emit (events.js:117:20)
NPM install log:
npm ERR! error installing [email protected] Error: No compatible version found: connect@'>=1.5.0- <1.6.0-'
npm ERR! error installing [email protected] Valid install targets:
npm ERR! error installing [email protected] ["0.0.1","0.0.2","0.0.3","0.0.4","0.0.5","0.0.6","0.1.0","0.2.0","0.2.1","0.2.2","0.2.3","0.2.4","0.2.5","0.2.6","0.2.7","0.3.0","0.4.0","0.5.0","0.5.1","0.5.2","0.5.3","0.5.4","0.5.5","0.5.6","0.5.7","0.5.8","0.5.9","0.5.10","1.0.0","1.0.1","1.7.2"]
npm ERR! error installing [email protected] at installTargetsError (/usr/local/lib/node_modules/npm/lib/cache.js:424:10)
npm ERR! error installing [email protected] at /usr/local/lib/node_modules/npm/lib/cache.js:406:17
npm ERR! error installing [email protected] at saved (/usr/local/lib/node_modules/npm/lib/utils/npm-registry-client/get.js:136:7)
npm ERR! error installing [email protected] at Object.cb [as oncomplete] (/usr/local/lib/node_modules/npm/node_modules/graceful-fs/graceful-fs.js:36:9)
npm ERR! Error: No compatible version found: connect@'>=1.5.0- <1.6.0-'
npm ERR! Valid install targets:
npm ERR! ["0.0.1","0.0.2","0.0.3","0.0.4","0.0.5","0.0.6","0.1.0","0.2.0","0.2.1","0.2.2","0.2.3","0.2.4","0.2.5","0.2.6","0.2.7","0.3.0","0.4.0","0.5.0","0.5.1","0.5.2","0.5.3","0.5.4","0.5.5","0.5.6","0.5.7","0.5.8","0.5.9","0.5.10","1.0.0","1.0.1","1.7.2"]
npm ERR! at installTargetsError (/usr/local/lib/node_modules/npm/lib/cache.js:424:10)
npm ERR! at /usr/local/lib/node_modules/npm/lib/cache.js:406:17
npm ERR! at saved (/usr/local/lib/node_modules/npm/lib/utils/npm-registry-client/get.js:136:7)
npm ERR! at Object.cb [as oncomplete] (/usr/local/lib/node_modules/npm/node_modules/graceful-fs/graceful-fs.js:36:9)
npm ERR! Report this *entire* log at:
npm ERR! <http://github.com/isaacs/npm/issues>
npm ERR! or email it to:
npm ERR! <[email protected]>
npm ERR!
npm ERR! System Linux 2.6.38-12-generic
npm ERR! command "node" "/usr/local/bin/npm" "install" "api-easy" "async" "bcrypt" "cluster" "cluster.exception" "cradle" "connect" "connect-couchdb" "connect-form" "email" "express" "eyes" "formidable" "imagemagick" "jade" "JSV" "log" "mime" "qs" "rbytes" "request" "validator" "vows" "underscore"
npm ERR! cwd /home/diversario/Documents/Projects/stuff/node
npm ERR! node -v v0.6.0
npm ERR! npm -v 1.0.105
Session reaping can generate a lot of separate delete requests, which can spike socket usage (since node by default does not use keepalive connections longer than a tick through the event loop) when sessions are reaped. The bulk docs api could be used to delete all reaped sessions in one request.
Hello Thomas, I've got a fork of connect-couchdb here https://github.com/developmentseed/connect-couchdb w/ the largest change being a swap out of cradle for a much simpler couch library. I've also added things like limiting the revisions on docs, since every page load may generate multiple revisions of a user's session document, and added compaction to keep the db size in check. Are these changes you're interested in merging back into your branch?
var express = require('express'),
app = express.createServer(),
ConnectCouchDB = require('connect-couchdb')(express),
store = new ConnectCouchDB({name: 'user_sessions', username: 'user', password: 'pass', host: '123.45.67.8'});
app.use(express.session({ store : store, secret : 'secret!'}));
produces
connect-couchdb error:
{ stack: [Getter/Setter],
arguments: undefined,
type: undefined,
message: 'Authentication required.',
error: 'unauthorized',
reason: 'Authentication required.',
statusCode: 401 }
Error
at _check_error (/node/node_modules/connect-couchdb/lib/connect-couchdb.js:29:25)
at ConnectCouchDB.<anonymous> (/node/node_modules/connect-couchdb/lib/connect-couchdb.js:64:17)
at native
at [object Object].callback (/node/node_modules/connect-couchdb/lib/couch.js:31:21)
at [object Object].<anonymous> (/node/node_modules/connect-couchdb/node_modules/request/main.js:290:21)
at [object Object].emit (events.js:64:17)
at IncomingMessage.<anonymous> (/node/node_modules/connect-couchdb/node_modules/request/main.js:278:54)
at IncomingMessage.emit (events.js:81:20)
at HTTPParser.onMessageComplete (http.js:133:23)
at Socket.ondata (http.js:1228:22)
Username and password are tested to be correct. Am I doing something wrong here?
I'm having trouble getting a very standard rolling session scenario working with this adapter. As far as I can tell, the rolling option in express doesn't trigger any mutation of the session getting saved. This means no document is getting flushed when set() is called.
Is there something obvious I'm missing here? The rolling option means you should not need to mutate the session object with random data. I also have save=true set.
Cheers,
Eddie
When using this module, I experience the problem that flashed messages and the passport user are not always available until the next but one request on my local machine (e.g. flashed message only appears after a subsequent reload). It works fine on the production server (same node version and npm modules). Disabling the store or switching to sessionstore fixes the issue.
Saw this in my error output when running docpad
Error: Only reserved document ids may start with underscore.
at Request._callback (/home/docpad/node_modules/connect-couchdb/node_modules/yacw/lib/couch.js:31:15)
at Request.self.callback (/home/docpad/node_modules/connect-couchdb/node_modules/request/index.js:142:22)
at Request.EventEmitter.emit (events.js:98:17)
at Request. (/home/docpad/node_modules/connect-couchdb/node_modules/request/index.js:856:14)
at Request.EventEmitter.emit (events.js:117:20)
at IncomingMessage. (/home/docpad/node_modules/connect-couchdb/node_modules/request/index.js:808:12)
at IncomingMessage.EventEmitter.emit (events.js:117:20)
at _stream_readable.js:883:14
at process._tickCallback (node.js:415:13)
Changing ...
Couch = module.exports = function(config) {
... to ...
var Couch = module.exports = function(config) {
... should do the trick. At least, it doesn't leak anymore and still seems to work as expected...
By the way.. Does anyone use this module at a larger scale? Let's say ~2000 active/concurrent user sessions?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.