mikemintz / rethinkdb-websocket-server Goto Github PK
View Code? Open in Web Editor NEWNode.js WebSocket server that proxies to RethinkDB. Supports query validation.
License: MIT License
Node.js WebSocket server that proxies to RethinkDB. Supports query validation.
License: MIT License
In index.js, perMessageDeflate is set to false (here). There's a accompanying comment indicating that this was because of a bug in websockets (here). This bug has been fixed and it would be great if compression was supported in this package, potentially even by default.
I did some testing in my environment and everything seems to work perfectly when compression is enabled, and delivers significant reductions in network traffic.
When unsafelyAllowAnyQuery
is on, rethinkdb-websocket-server prints every query with all of its data to the console. This includes r.binary()
stuff like uploaded files and arrays with binary data (say, when storing UInt8Arrays in the DB as a part of the object). A couple of megabytes sent to stdout can really cause havoc. I think the data should be trimmed in the output after an X number of characters, plus there should be an option to turn off showing queries altogether. There are legitimate use cases for not using any validation and running the server, e.g. when running unit tests that should bypass validation or when running the server locally and just wanting to connect to the server instance from a web page, with full control.
It would be great if a client could do something like r.table('turtles').filter({name: 'Ralph'}), and the server would automatically inject the herdId filter (from your example) in between the .table and the .filter provided by the client, so that from a client's perspective, the only data in the DB is what is available to them. This could ease the burden not only on user-level security, but help to support multi-tenancy (e.g. filtering every table on tenantId).
Let's open
Since React Native support isn't there. If it is please create an issue to document it with a clear example. Currently getting "Unhandled rejection ReqlTimeoutError: Could not connect to localhost:3030 operation timed out".
This error takes about 30 seconds to show up. If we turn off the server it shows up right away, so it seems that it kind of works at first but then fails to reconnect.
If someone could help me make this work because it actually can work, I'd be happy to write the documentation.
Will you be updating the client and server to support 2.3.x in the future? What would need to be done to update?
I think the following change in the 2.3 driver is breaking it at the moment.
Disallowed calling r.wait, r.rebalance and r.reconfigure on the global scope to avoid confusing semantics. (#4382)
TypeError: _rethinkdb2.default.wait is not a function at Object.<anonymous> (/home/tersiusk/WebstormProjects/pinkdot/node_modules/rethinkdb-websocket-server/dist/ReqlTermExamples.js:109:32)
urlQueryParams seems extremely dependent on authentication with the same server. If I want to authenticate via another server (to include third-party) and then have a JWT for example, I need access to the request headers. Passing the request object as a second parameter would allow this and then the JWT could be authenticated.
It might be nice to express queries in the whitelist using the natural JS reql syntax, rather than the protocol AST syntax we currently use. Instead of this:
RQ(
RQ.INSERT(
RQ.TABLE("messages"),
{body: RQ.ref("body"), userId: RQ.ref("userId"), createdAt: RQ.NOW()}
)
).opt("db", RQ.DB(cfg.dbName))
.validate(...)
we'd allow writing this:
r.db(cfg.dbName)
.table('messages')
.insert({body: RP.ref("body"), userId: RP.ref("userId"), createdAt: r.now()})
.validate(...)
The biggest challenge is being able to log rejected queries in this format, so the application developer can copy/paste into their whitelist. The reason we have the current whitelist syntax is because it's derived directly from the AST sent over the wire protocol.
My hypothesis is that we can go from AST to JS reql syntax with the following algorithm:
(1) Start at the inner-most 0th arg of the nested arrays.
let x = query;
while (Array.isArray(x.args[0])) {
x = x.args[0];
}
(2) Work our way from the inside to the outside. Each time we go out one level, append the term name of the outer term to the current query, and use arguments after the 0th as its arguments, and options as an object for the last argument. For each argument, recursively apply this algorithm, making special cases for terms like FUNC and MAKE_ARRAY.
x = x.parentTerm;
result += '.' + termNames[x.termId] + '(';
result += x.arguments.slice(1).map(formatTerm).join(', ');
if (x.options) result += ', ' + formatOptions(x.options);
result += ')';
We've been experiencing a weird bug off and on over the past few months, and lately it seems to be getting worse.
Error: Uncaught SyntaxError: Unexpected token , in JSON at position 289587
I'm going to keep looking into this but I don't have many ideas about where to start. Does anything jump out?
Is there much involved in bumping the ws version to >1.1.1? For now I manually bumped the version in my projects and it's working fine.
I've been seeing this bug a lot: websockets/ws#778
Hi, it seems that it is currently not possible to connect to rethinkdb providing username and password. Since authKey is not present in the latest version of rethinkdb, the only way to get this proxy working with db is leaving db unsecured.
Great work on this! I'm excited to try it out.
I haven't yet grokked the queryWhiteList bit - I think I need to read the code a couple more times. I was wondering if you had any thoughts about the feasibility of an auth API similar to the allow/deny functions of Meteor? I can't tell yet how much effort that would take. I also like Firebase's approach.
In both cases there is a clean, easy-to-read mapping of data paths to functions which allow or deny writes and reads, given user/session data as well as snapshots of the current state of database when relevant.
I have a situation where the user sends a token from client via query params and I use the same to validate the session and store the the userId in the RP session.
options.sessionCreator = function (params) {
let token = params.token;
let user = params.user;
if (!token || token == "" || !user || user == "") {
return Promise.reject('Invalid auth token');
} else {
// use teade to validate the user from user service
const clients = {
users: {
host: process.env.SERVICE_RPC_HOST,
port: process.env.CXI_USER_PORT
}
}
const data = {
token: token,
user: user
}
const authenticator = require('./middlewares/authenticator');
return Promise.promisify(authenticator.auth)(clients, data)
.then(function(result){
return Promise.resolve({userId: result.data.id});
}).catch(function(err){
return Promise.reject(err.message);
})
}
};
now I want to add this userId in the filter of a query but I dont have this id in the frontend.
So I am looking for a way to either
// whitelisted query
r.table("trades").orderBy({index: allowSpecificFieldOrderBy(RP, "datetime")})
.filter(r.row("currency").eq(allowAnyString(RP)))
.filter(r.row("isExecuted").eq(true))
.filter(r.row('userId').eq(
/* what can I use here? */
))
.pluck("datetime", "rate", "id")
.changes({"includeInitial": true}).opt("db", r.db("cxihub"))
.validate(function (refs, session) {
// console.log(refs, session)
// { userId: '' } { userId: 1 }
return true;
})
If the connection to the database gets interrupted (e.g. database is restarted), the server seems to crash permanently.
It would be great if the server could reconnect when the connection is available again.
For example, this stacktrace is what happens to the Chat example when the database is restarted (and in perpetuity until the websocket server is restarted).
22:21:18.676 172.17.0.5:54391 Error in sessionCreator
ReqlDriverError: Connection is closed.
at ReqlDriverError.ReqlError [as constructor] (/app/node_modules/rethinkdb/errors.js:23:13)
at new ReqlDriverError (/app/node_modules/rethinkdb/errors.js:68:50)
at TcpConnection.Connection._start (/app/node_modules/rethinkdb/net.js:413:13)
at /app/node_modules/rethinkdb/ast.js:142:29
at Get.TermBase.run (/app/node_modules/rethinkdb/ast.js:131:12)
at AuthManager.js:17:47
at tryCatcher (/app/node_modules/bluebird/js/release/util.js:16:23)
at Promise._settlePromiseFromHandler (/app/node_modules/bluebird/js/release/promise.js:497:31)
at Promise._settlePromise (/app/node_modules/bluebird/js/release/promise.js:555:18)
at Promise._settlePromiseCtx (/app/node_modules/bluebird/js/release/promise.js:592:10)
at Async._drainQueue (/app/node_modules/bluebird/js/release/async.js:130:12)
at Async._drainQueues (/app/node_modules/bluebird/js/release/async.js:135:10)
at Immediate.Async.drainQueues [as _onImmediate] (/app/node_modules/bluebird/js/release/async.js:16:14)
at processImmediate [as _immediateCallback] (timers.js:383:17)
We have some basic security on our RethinkDB instances with the built-in authKey support for securing drivers. However, this library doesn't seem to support that.
I'm using 0.3.2 and now I'm getting this error
/app/node_modules/rethinkdb-websocket-server/dist/ReqlTermExamples.js:109
WAIT: _rethinkdb2['default'].wait(),
^
TypeError: undefined is not a function
I haven't had any problems with that before..
Hi, I'm trying to use the library but It never connects with rethinkdb server, log just prints:
04:14:52.518 ::ffff:192.168.99.1:54361 Connect
04:15:12.548 ::ffff:192.168.99.1:54361 webSocket closed
04:15:12.549 ::ffff:192.168.99.1:54361 dbSocket closed
Code on nodejs:
console.log('Server online!');
//First Part
var r = require('rethinkdb');
r.connect({host: "rethinkdb", port: '28015'}, function (err, connection) {
r.db('db').table('users').insert({
"id": 3,
"title": "Lorem ipsum",
"content": "Dolor sit amet"
}).run(connection, function (err, result) {
console.log(result);
connection.close();
});
});
//Second Part
var express = require('express');
var http = require('http');
var wsListen = require('rethinkdb-websocket-server').listen;
var options = {};
var httpServer = http.createServer();
options.dbHost = 'rethinkdb';
options.dbPort = '28015';
options.unsafelyAllowAnyQuery = true;
options.httpServer = httpServer;
wsListen(options);
httpServer.listen(3000);
As you can see on the first part I'm testing the connection directly, second part is not responding
Thanks for your help
When looking around this code I noticed a duplicate
https://github.com/mikemintz/rethinkdb-websocket-server/blob/master/src/Connection.js#L29
https://github.com/mikemintz/rethinkdb-websocket-server/blob/master/src/Connection.js#L45
Perhaps I'm missing something here?
I'd like to use this with an existing Express application that is already listening on a port.
Is it possible to do that, i.e. plug this in to a specific route on an existing web server, instead of listening on its own port?
cant query white list using filter
r.table('table')
.filter(r.row("amount").ge(100))
.run(conn, (err, cursor) => {
cursor.toArray((err, results) => {
})
//queryWhiteList
r.table('table')
.filter(
r.row("amount").ge(RP.check((actual, refs, session) => actual === 100))
)
.opt("db", r.db("dice"))
It would be nice if developers could add before-execute and after-execute callback functions to their queries in the whitelist. Some use cases:
Technically, before-execute is already possible using the .validate()
method. Are there use cases where we'd want a separate before-execute callback with different semantics? Some thoughts:
.validate()
function must return a truthy value or the query will be rejected. Having non-validation callbacks end in return true
might clutter things or lead to bugs.For after-execute, there are two possible variants: one that gets called right after the query is sent to RethinkDB, and another that gets called right after the query response is received from RethinkDB. The former is trivial to implement, and the latter requires parsing RethinkDB responses and tracking query tokens.
What's the best way to validate a query like this? For example, to ensure id > 0
r.table('test').insert([{id: 1}, {id: -1}]) // Query
r.table('test').insert(RP.check(arr => arr.every(e => e.id > 0))) // Whitelist entry
Whitelisting like this fails, I think because arr is a parsed query as opposed to a js array. I'm not sure how to loop through it.
I've been having a lot of trouble deploying this app on heroku and using it in conjunction with the client. What is the proper way to securely run this in production, with security? I get immediate timeouts when I try to use TLS.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.