mysql/mysql-js

NoSQL node.js connector for MySQL Cluster

mysql-js's Issues

Closing adapter mysql DBConnectionPool doesn't close all mysql connections

Undefined symbol __ZN15NdbQueryBuilder7prepareEv

Beginning with MySQL Cluster 7.6.3 (wl#10234) a private SPJ API that jones-ndb uses has changed, which will result in an undefined symbol error in the dynamic loader.

lost exception in promise.then()

In general, if a test case throws an exception, the harness catches it and the test case fails.

However, if a test case throws an exception inside promise.then(), the exception is lost and the test times out.

Allow querying on fields that are mapped to sparse column

Currently, query only allows fields that are mapped to their own columns. Fields that are mapped to sparse columns cannot be used in query expressions. This restriction is ok for now, since there is no way to push the query to the back end.

But in future, it will be possible to create query expressions that include fields from the column with the sparse fields, by using the JSON functions in the server. It will even be possible to index some of the fields from the sparse columns.

NdbRecordObject does not handle sparse fields

If use_mapped_ndb_record = false, the ndb adapter correctly handles sparse fields.

But if use_mapped_ndb_record = true, SPARSE_FIELDS itself shows up as a property:

{ SPARSE_FIELDS: '{"profile_text":"I am your little boot!"}',
tweet_count: 0,
full_name: 'Gaius Julius Casear Germanicus',
user_name: 'caligula',
profile_text: 'I am your little boot!' }

jones-ndb 7.6 cannot connect to NDB 8.x

When Jones is built with ndbclient from 7.6, attempts to connect to an 8.x cluster fail with an error from wait_until_ready

jones-ndb: V8 API deprecation warnings with node 6.3.1

At runtime, node 6.3.1 reports:

(node) v8::ObjectTemplate::Set() with non-primitive values is deprecated
(node) and will stop working in the next major release.

Support more than one relationship in Projection

Currently only one relationship can be specified for a Projection. To fully represent complex nested objects that are stored in multiple related tables, multiple relationships should be possible.

are join queries supported?

Are join queries supported?

If yes, an example would be great.

Thanks :)

TableMapping documentation omits relationship mapping

Callbacks for connection up/down

It should be possible to register a callback on SessionFactory to receive a notification when the database becomes unavailable.

Error object returned from connect where mapped tables do not exist needs more information

openSession() and connect() can be used to validate mappings.

1. If the mappings are not valid, the error returned by connect() is strange. For instance, if five mappings were to be validated, and all five received errors, the error object seems to be an array of five elements, but the prototype of the array is Error. I tried using a variety of console.log(), util.inspect(), etc. to view the five members of this structure but I was not able to.

2. The docs in API-documentation/Jones are not clear about whether a failure to validate the mappings causes openSession() or connect() to fail.

One way to reproduce this:
cd samples/tweet

neglect to create the tables used by tweet.js

node tweet.js get tweets-at nero

Unknown table mapping: Issue with TableMapping and Converter

Trying to map a table to an object, so that I can also use a Converter, but even doing basic TableMapping is not working.

All attempts produce a flavor of "Unknown table mapping"

Please see the code examples below.

Example of error message:

unknown table mapping {table: employees, fields; []}

Example of standard 1 line syntax:

The following works, and we can use session.find and session.executeQuery. However, it doesn't allow us to override the mapping with a converter for the gender database field that is an ENUM.

var mapping = new nosql.TableMapping('employees').applyToClass(Employee);

Examples of TableMapping with just tablename defined:

var mapping = new nosql.TableMapping({
  table: 'employees'
});
mapping.applyToClass(Employee);

Example Converter for ENUM of 'M', 'F':

function Converter() {};
Converter.prototype = {
  toDB : function(obj) { return obj},
  fromDB : function(val) { 
    console.log('fromDB' + val);
    if (val === '\u0001')
      return 'M';
    else
      return 'F';
  }
};

Example using Converter in one statement:

var mapping = new nosql.TableMapping({
  table: 'employees',
  database: 'employees',
  mapAllColumns: true,
  fields: {
    fieldName: 'gender',
    columnName: 'gender',
    converter: new Converter() //tried Converter and Converter() also
  }
});

mapping.applyToClass(Employee);

Example of TableMapping and adding custom field mapping after:

var mapping = new nosql.TableMapping('employees');
mapping.mapField('gender','gender',Converter,true);
mapping.applyToClass(Employee);

Better error message from UserContext with missing callback

If you write:
persist("a", row) [with no callback]
you get the error:
UserContext.js:37675: Uncaught TypeError: Cannot call method 'apply' of undefined

Complete with the wrong line number (a Node bug, not ours).
Can we have a beter error here?

Segmentation Fault with Projection API

Hey guys,

I believe I have set everything up correctly, but the only response I'm getting from the console when I run the script is "Segmentation fault". Here are the versions I'm running:

Software Versioning

mysql-js: up-to-date as of August 5th, 2016
node version: v0.12.2
MySQL Cluster version: 7.5.3

MySql Cluster Table Set-up

CREATE TABLE 'graph'.'objects' (  'id_company' INT NOT NULL COMMENT '',  'id' BIGINT(20) NOT NULL COMMENT '',  'otype' VARCHAR(45) NOT NULL COMMENT '',  'data' JSON, 'data2' JSON,  PRIMARY KEY ('id_company','id', 'otype')  COMMENT '')  ENGINE=NDBCLUSTER  PARTITION BY KEY(id_company);

CREATE TABLE 'graph'.'associations' (  'id_company' INT NOT NULL COMMENT '',  'id1' BIGINT(20) NOT NULL COMMENT '',  'atype' BIGINT(20) NOT NULL COMMENT '',  'id2' BIGINT(20) NOT NULL COMMENT '',  'time' DATETIME NULL DEFAULT NOW() COMMENT '',  'data' JSON ,  PRIMARY KEY ('id_company', 'id1', 'atype', 'id2')  COMMENT '')  ENGINE=NDBCLUSTER  PARTITION BY KEY(id_company);

Objects has a one-to-many relationship with associations.

App.js Code

"use strict";
var jones = require("/api_tests/mysql-js/database-jones/");
var connectionProperties = new jones.ConnectionProperties("ndb");


function Objects(id, id_company, otype, data, data2) {
        this.id = id;
        this.id_company = id_company;
        this.otype = otype;
        this.data = data;
        this.data2 = data2;
}

function Associations(id_company, id1, atype, id2, time, data) {
        this.id_company = id_company;
        this.id1 = id1;
        this.atype = atype;
        this.id2 = id2;
        this.time = time;
        this.data = data;
}

function mapObjects() {
    var objectMapping = new jones.TableMapping("objects");
        objectMapping.mapField("id");
        objectMapping.mapField("id_company");
        objectMapping.mapField("otype");
        objectMapping.mapField("data");
        objectMapping.mapField("data2");

        objectMapping.mapOneToMany({
            fieldName: "assoc",
            target: Associations,
            targetField: "obj_assoc"
        });

    objectMapping.applyToClass(Objects);
}

function mapAssociations() {
    var associationMapping = new jones.TableMapping("associations");
        associationMapping.mapField("id_company");
        associationMapping.mapField("id1");
        associationMapping.mapField("atype");
        associationMapping.mapField("id2");
        associationMapping.mapField("time");
        associationMapping.mapField("data");


        associationMapping.mapManyToOne({
            fieldName: "obj_assoc",
            target: Objects,
            foreignKey: "fk_id1"
        });

    associationMapping.applyToClass(Associations);
}

function mapShop() {
    mapObjects();
    mapAssociations();
}

mapShop();

var associationProject = new jones.Projection(Associations)
.addFields("id_company", "id1", "atype", "id2", "time", "data");

var objectProjection = new jones.Projection(Objects)
.addFields("id", "id_company", "otype", "data")
.addRelationship("assoc", associationProject);

var onFind = function(err, result) {
    if (err) { console.log(err);} else {
        console.log(result);
    }
};

jones.openSession(connectionProperties).then(
    function(session) {
        session.find(objectProjection, {id: 1}, onFind);
    }
);

Support projections for query

Currently projections are only supported for find operations which return exactly one (or zero) row. Projections should be supported for query as well.

createTable should work without a constructor

If createTable is called with a table mapping that does not include a constructor, the call fails.

Error: User exception: constructor for must have been annotated (call TableMapping.applyToClass).

If a constructor is not mapped, then createTable should create the table and allow plain objects to be persisted.

NoSQL into MySQL

Hi all,

Let me introduce my problem (hoping that someone can help me):
I'm a univeristy student and i'm doing some research about the product "mysql cluster" which provides some APIs for "NoSQL access". I'm using nodejs for testing my code and, with "mysql-js" , i realize some access to my database (test) and table (user).

My table is like this: (username=varchar , fullname=varchar,age=int).
I tried to insert an object like (username,fullname,age) and obviously works fine...then tried to insert another object like (username,fullname,age,email,...) and works too except losing the other parameter but is not a problem for my case ... the question is:

can i use this to insert an object like (username, (value1, value2,...), age) ? In particular...can i insert a nested object??

If not, why define Mysql cluster as SQL and NoSQL ?? Where is the NoSQL component??

I‘m sorry for my bad english and this inappropriate question ... hope that you can help me!

Stefano

Sort order by index?

I am selecting objects by some criteria, and want to return them in a reverse chronological order. How to do that? I don't see how I can specify a field or an index for that.

openSession should work without mappings

The openSession method takes properties, mappings to load, and an optional callback.

If openSession is called with properties and a callback, the callback is treated as mappings and the callback will never be called.

The behavior of openSession should change so that if the second parameter is a function, it is treated as a callback.

Promises

Here is our current thinking about changing the API to support promises in addition to the callback-driven coding style.

Consider session.find(). find() already has two variants -- one that takes a table name, and another that takes a domain object constructor. The constructor version looks like find(constructor, keys, callback, ...), where any optional arguments that are given after the callback are then passed on to the callback as-is.

Now we propose another variant, which does not take a callback or any extra arguments, and returns a promise:

  var promise = session.find(constructor, keys);

The returned promise will be one of our own implementation, conforming to the Promises/A+ spec

All of the async operations in the API will have similar variants that return promises.

Support table mapping for table name function variants

If the table name variant is used for session functions, the default table mapping is always used.

The default table mapping maps each column to a like-named field. If a table is created with a different mapping, the mapping is never used.

In order to allow different mappings to be used, the user should be able to specify a table mapping to be used for the table name variants. The table mapping could be specified either when the SessionFactory is created or when the Session is created. A table mapping specified at the SessionFactory would be used unless the user specified a table mapping when the Session was created.

The connect and openSession functions on Jones would be extended to allow table mappings in addition to constructor functions.

Query with qdt.key.eq(literal) fails with operation error

Query with qdt.key.eq(qdt.param('p1') succeeds but qdt.key.eq(literal) fails. The getKeys method does not return literals that are part of the query tree but only returns parameter values.

A new collection of tests based on numerictypes/QueryFindTest should use literals instead of query parameters.

setLockMode is not implemented

setLockMode is documented but not implemented in jones-ndb or jones-mysql.

Trouble with Projection API

Hey guys,

I replicated my issue using 2 simple tables running on MySQL Cluster. Here is the SQL I used to create the tables, along with the table mappings code and the result I get from node.

Background Info

• Running the latest version of mysql-js
• Node Version v0.12.2
• MySQL Cluster Version 7.5.3

MySQL Cluster

Customers Table
CREATE TABLE 'graph' . 'customers' ( 'id' INT NOT NULL COMMENT '', 'name' VARCHAR(45) NOT NULL COMMENT '', 'age' INT NOT NULL COMMENT '', PRIMARY KEY('id') COMMENT '') ENGINE=NDBCLUSTER PARTITION BY KEY(id)

Orders Table
CREATE TABLE 'graph' . 'orders' ( 'id_user' INT NOT NULL COMMENT '', 'id_order' VARCHAR(45) NOT NULL COMMENT '', 'cost' INT NOT NULL COMMENT '', PRIMARY KEY('id_user', 'id_order') COMMENT '', FOREIGN KEY (id_user) REFERENCES customers(id)) ENGINE=NDBCLUSTER PARTITION BY KEY(id_user);

Customers has a one-to-many relationship with Orders

App.js

"use strict";
var jones = require("/api_tests/mysql-js/database-jones/");
var connectionProperties = new jones.ConnectionProperties("ndb");


function Customers(id, name, age) {
    if(id !== undefined) {
        this.id = id;
        this.name = name;
        this.age = age;
    }
}

function Orders(id_order, cost) {
    if(id_order !== undefined) {
        this.id_order = id_order;
        this.cost = cost;
    }
}

function mapCustomers() {
    var customerMapping = new jones.TableMapping("customers");
        customerMapping.mapField("id");
        customerMapping.mapField("name");
        customerMapping.mapField("age");

        customerMapping.mapOneToMany({
            fieldName: "orders",
            target: Orders,
            targetField: "id_user"
        });

    customerMapping.applyToClass(Customers);
}

function mapOrders() {
    var orderMapping = new jones.TableMapping("orders");
        orderMapping.mapField("id_order");
        orderMapping.mapField("cost");


        orderMapping.mapManyToOne({
            fieldName: "id_user",
            target: Customers,
            foreignKey: "fk_id"
        });

    orderMapping.applyToClass(Orders);
}

function mapShop() {
    mapCustomers();
    mapOrders();
}

mapShop();

var orderProjection = new jones.Projection(Orders)
.addField("cost");

var customerProjection = new jones.Projection(Customers)
.addField("name")
.addRelationship("orders", orderProjection);

var onFind = function(err, result) {
    if (err) { console.log(err);} else {
        console.log(result);
    }
};

jones.openSession(connectionProperties).then(
    function(session) {
        session.find(customerProjection, {id: 4}, onFind);
    }
);

When I run this inside node, I get the following error:
this.keyRecord = indexHandler.dbIndex.record;
^
TypeError: Cannot read property 'dbIndex' of null at new NdbProjection (/api_tests/mysql-js/jones-ndb/impl/ndb/NdbProjection.js:76:37)

Might be worth mentioning that I can pass in a table name inside session.find and have the correct result returned to me. However, no matter what I try when I pass in a projection, I always end up with the error above. Please let me know if you need any more information. Thanks!

Find by table name incorrectly returns domain object

If a domain object is mapped to a table, the constructor is stored in the table mapping and then the table mapping incorrectly replaces the existing table mapping for the table name.

Parameterized test cases

This is a generalized version of the work to allow running the test suite with multiple adapters. It relates to removing all use of globals in test cases, by providing a dictionary of test parameters.

A test driver can set parameters with jones_test.driver.setParameter(); an individual test case, instantiated from e.g. new SmokeTest(), could retrieve them via test.getParameter().

In a single test run, it should be possible to run a test suite multiple times with different sets of parameters.

Limit of 4711 Ndbs per connection

The NDB API imposes a limit on the number of Ndb objects that can be used in one Ndb_cluster_connection. This limit is currently 4711 (MAX_NO_THREADS in TransporterFacade.hpp).

Node.JS is designed to allow tens of thousands of concurrent clients, and each client session uses one Ndb object, so we may start to encounter this limit. I am opening this issue preemptively so people are aware of the situation.

Deoptimizations in NDB SPI

From node --trace_deopt jscrund --adapter=ndb --spi we see a list of deoptimizations in NDB SPI code:

For Indy:

• DBOperation
• DBTableHandler.getFields
• DBTableHandler.getMappedFieldCount
• execute
• stats_incr
• DBTableHandler.getColumnMetadata
• DBOperation.prepare
• allocateRowBuffer
• completeExecutedOps
• releaseKeyBuffer

For Each:

• NdbSession.getTransactionHandler
• NdbSession.buildInsertOperation
• proto.execute
• encodeRowBuffer
• DBTableHandler.get
• onExecute
• getExecIdForOperationList
• run
• attachErrorToTransaction
• exports.closeActiveTransaction
• begin

For bulk:

• QueuedAsyncCall.enqueue

add more examples

examples for doing queries with where clause could be more than helpful.

I saw api documentation but that dint help much

Deprecation warnings building jones-ndb with node 6.9.1

../impl/include/common/JsWrapper.h: In constructor ‘Envelope::Envelope(const char*)’:
../impl/include/common/JsWrapper.h:173:55: warning: ‘static v8::Localv8::ObjectTemplate v8::ObjectTemplate::New()’ is deprecated (declared at /Users/jdd/.node-gyp/6.9.1/include/node/v8.h:4664): Use isolate version [-Wdeprecated-declarations]
Local proto = ObjectTemplate::New();

Poor NDB error handling when table does not exist

See issue#16.
"Table Not Found" should be a clear error with SQLState 42S02. Instead it appears as an empty error object from a listIndexes call that returns -1.

remote ndb cluster

I seem to have issues connecting to our remote NDB Cluster 7.2 SQL node (NDB Connect failed 1. Can't find extra info on the error. Piped process.stderr to a writable stream (appended file) wich stays empty). So I am a bit stuck there...

However, with the mysql adapter everything works fine.

What would be the impact of connecting with the mysql adapter vs. the ndb adapter with regards to performance and functionality?

Create UUID columns from meta.char(XX).generated()

I think we intended that if the user says meta.char(XX).generated() with an appropriate value of XX then the field will be generated as a UUID. However we have not implemented this.

jones-ndb: assertion fails on getNoOfBlobColumns

In part of the issue#37 we see:
Assertion `nblobs == record->getNoOfBlobColumns()' failed

With a default TableMapping, persistent fields are stored in SPARSE_FIELDS on insert

With this code

authorMapping = new jones.TableMapping('author');
// authorMapping.mapField("user_name");
// authorMapping.mapField("full_name");
// authorMapping.mapField("tweet_count");
authorMapping.mapSparseFields("SPARSE_FIELDS");

all fields (including user_name, tweet_count, and full_name) are stored in JSON format in the SPARSE_FIELDS column on insert. If you uncomment the explicit mapFields() lines, so that the mapping is no longer a default mapping, then the behavior is correct and only sparse properties are stored in SPARSE_FIELDS.

Add query capability

Here are some ideas on how to incorporate query into the mysql-js connector:

The idea is to map queries into one of:

primary key lookup
unique key lookup
index scan
table scan

The main object for query is the query domain type. A QD is obtained from a session, and is bound to that session.

A QD can be created by the createQuery function on session. The required parameter is one of: a table name; a constructor function; an instance of a domain object; or a query from a previous createQuery. A query created from another query is bound to the current session but retains all of the properties of the original query, including the table or constructor and filter. The parameter query must be obtained from a session obtained from the same session factory (to make sure the underlying table is the same).

function createQuery(tableNameOrConstructor, function(err, qd))

The QD has properties corresponding to the field names of the mapped constructor. Each field has immediate methods used to construct query predicates:

function eq(valueOrParameterMarker)
function ge(valueOrParameterMarker)
function gt(valueOrParameterMarker)
function lt(valueOrParameterMarker)
function le(valueOrParameterMarker)
function ne(valueOrParameterMarker)
function between(valueOrParameterMarker, valueOrParameterMarker)
function in(value, value, ...)
function in(parameterMarker)
function like(valueOrParameterMarker)

The query predicate has immediate methods to combine multiple query predicates and return a new query predicate:
function and(queryPredicate)
function or(queryPredicate)
function not(queryPredicate)

Parameter markers are named, and are created by the QD

function param(name)

The filter is specified by calling the immediate function where, passing a query predicate. The where function returns the QD.

function where(queryPredicate)

Queries are executed by calling the execute function with parameters. The parameters is an object with properties corresponding to each named parameter plus keyword properties that govern the execution of the query. The following keywords may not be used as named parameters:

skip: the number of elements to skip from the results
limit: the number of elements to return after skipping
asc: if true, results are ordered in the database and returned in ascending order
desc: if true, results are ordered in the database and returned in descending order
specifying both asc and desc true is an error

Best practice is probably to name the parameters with some distinguishing prefix, such as:

var p_id = qemp.param('p_id');

function execute(parameters, function(err, result))

The result of a query is an object that behaves similar to an array, with operator/functions:

length: the number of elements returned by the query
[ i ] the i'th object from the query
forEach(function(element))

Other array methods can be added as required. It's not obvious whether the result should be read-only or allow all array methods including push, pop, shift, unshift, etc.

Queries also can be used to delete and count objects.

function delete(parameters, function(err, numberDeleted))

function count(parameters, function(err, numberCounted))

Example with a single condition

session.createQuery(Employee, function(err, qemp) {
if (err) throw...
var p_name = qemp.param('p_name'); // creates a named parameter
qemp.where(qemp.name.eq(p_name))
.execute({asc: true, limit: 10, p_name: 'Craig'}, function(err, result) {
if (err) throw...
result.forEach(function(emp) {
console.log(emp.name);
}
}
}

Example with multiple conditions joined with "and"

session.createQuery(Employee, function(err, qemp) {
if (err) throw...
var p_age = qemp.param('p_age'); // creates a named parameter p_age
var p_salary = qemp.parameter('p_salary'); // creates a named parameter p_salary
qemp.where(
qemp.age.ge(p_age)
.and(qemp.salary.lt(p_salary)))
.execute({asc: true, limit: 10, p_age: 50, p_salary: 10000}, function(err, result) {
if (err) throw...
result.forEach(function(emp) {
console.log(emp.name);
}
}
}

For convenience, query filters can be constructed as literals. Each property is either a keyword or the name of a field; the named fields are used to construct a filter using operator "eq" and joined with "and".

If no properties are needed, the first parameter of execute can be omitted.

session.createQuery('t_employee', function(err, qemp) {
if (err) throw...
qemp.where({name: 'Craig'})
.execute(function(err, result) {
if (err) throw...
result.forEach(function(emp) {
console.log(emp.name);
}
}
}

Extra credit: Literal queries could be extended beyond equality but it's not obvious that the result is worth the effort:

session.createQuery('t_employee', function(err, qemp) {
if (err) throw...
qemp.where({name: 'Craig', age: {ge: 20, le: 55}, salary: {gt: 20000, le: 50000}})
.execute(function(err, result) {
if (err) throw...
result.forEach(function(emp) {
console.log(emp.name);
}
}
}

session.createQuery('t_employee', function(err, qemp) {
if (err) throw...
qemp.count(function(err, result) {
if (err) throw...
console.log('counted', result, 'rows in t_employee');
}
}
}

Session should support oneShot operation

In order to clean up resources associated with a Session, the Session must be closed. Some applications will perform just one action and regardless of the outcome, clean up the Session.

With either the callback model or promises model, there is extra work to do to close the Session.

This proposal adds a function to Session. Session.oneShot() will set the Session into a mode where it will be closed after executing a single function. When the promise is fulfilled or the callback is called, the Session is no longer usable.

unecessary check for mysql-test directory while building mysql-js

configure.js checks for mysql-test directory while building the ndb adapter.
However only include and lib folders are required for building the adapter.

All we need to check is include and lib folders are present in mysql directory or not.

Confused over mapping in openSession()

This code does not work:

// Map SQL Tables to JS Constructors using default mappings
mappings = [];
mappings.push(new nosql.TableMapping('tweet').applyToClass(Tweet));
mappings.push(new nosql.TableMapping('author').applyToClass(Author));
mappings.push(new nosql.TableMapping('hashtag').applyToClass(HashtagEntry));

// uncomment this line to make it work as intended:
// mappings = [ Tweet, Author, HashtagEntry ];
...
nosql.openSession(dbProperties, mappings, runOperationInSession, operation);

The API docs are for Mynode are not clear about the type of "mappings" in openSession(). Strangely enough, it seems that it cannot be an array of TableMapping.

Split jones-ndb into two modules

The compiled part of jones-ndb (ndb_adapter.node) could be maintained separately from the javascript part, making it safer and easier to obtain ndb_adapter.node file from a cluster binary but then install the rest of jones using npm.

Poor error handling from openSession()

If you call mynode.openSession() with a connectionProperties literal, and the literal does not have an "implementation" member, you get this exception:

path.js:360
        throw new TypeError('Arguments to path.join must be strings');
              ^
TypeError: Arguments to path.join must be strings
    at path.js:360:15
    at Array.filter (native)
    at Object.exports.join (path.js:358:36)
    at Object.exports.getDBServiceProvider (/Users/jdd/bzr-repo/nodejs-adapter/Adapter/impl/SPI.js:41:29)
    at Object.exports.getConnectionKey (/Users/jdd/bzr-repo/nodejs-adapter/Adapter/api/mynode.js:80:16)
    at getSessionFactory (/Users/jdd/bzr-repo/nodejs-adapter/Adapter/api/UserContext.js:751:26)
    at exports.UserContext.openSession (/Users/jdd/bzr-repo/nodejs-adapter/Adapter/api/UserContext.js:2131:5)
    at Object.exports.openSession (/Users/jdd/bzr-repo/nodejs-adapter/Adapter/api/mynode.js:96:18)
    at Object.<anonymous> (/Users/jdd/bzr-repo/nodejs-adapter/samples/intro.js:12:7)
    at Module._compile (module.js:456:26)

Compound index

The documentation hints that it is possible to specify a compound index depending on two or more fields. Is it implemented? If so, how to do it correctly?

XA

Here's a sketch for how distributed transactions would be supported.

Session.beginDistributedTransaction(xid) is a new immediate method returning a DistributedTransaction with XA transaction identifier "xid," where xid is a value assigned by some distributed transaction manager. The session's current transaction must be Idle. The effect is like getCurrentTransaction(), plus an operation to transform the current transaction into a distributed transaction with id xid, plus begin(). The transaction that is returned is Active and is not auto-commit.

The mysql adapter implements this using the SQL statement "XA BEGIN xid". The ndb adapter implements it as a simple begin.

Transaction.prepare() is a new async method available only on distributed transactions. The mysql adapter implements prepare() as as the SQL statements "XA END xid", signaling the end of SQL DML, followed by "XA PREPARE xid". The ndb adapter implements prepare() by (1) issuing an executeNoCommit() on the current transaction and (2) storing the transaction in the Local Prepared List.

Transaction.commit() is executed by mysql using "XA COMMIT xid". By ndb as: executeCommit(), and, on success, remove the transaction from the Local Prepared List.

Transaction.rollback() is executed by mysql using of "XA ROLLBACK xid". By ndb as: rollback() and, on success, removal of transaction from the Local Prepared List.

Also has part of this support, SessionFactory must provide a DistributedRecoveryManager, which supports three methods listPreparedTransactions(), commit(xid), and rollback(xid). These are used for a recovery scenario that assumes NDB and mysql-js have kept running while the Transaction Manager restarted. They rely on a list of transactions that have been prepared, but not committed or rolled back. In the mysql case, this list is on the mysql server at the other end of the connection, but for the NDB case it is stored locally in mysql-js.

SessionFactory.getDistributedRecoveryManager() returns the recovery manager. (Or, aternately, the methods are available on SessionFactory itself).

ListPreparedTransactions() returns the list of prepared transactions. For mysql it is implemented as an "XA RECOVER" statement. For Ndb it is implemented by returning the xids of the transactions on the Local Prepared List.
commit(xid) is calls commit() on the transaction matching xid.
rollback(xid) calls rollback() on the transaction matching xid.

Reference material for MySQL: http://dev.mysql.com/doc/refman/5.6/en/xa-statements.html and http://dev.mysql.com/doc/refman/5.6/en/xa-states.html

join.js sample does not work in jones-ndb

The example script samples/tweet/join.js runs correctly with mysql but not with ndb.

Intermediate Types

We are making some changes to the initial design of converters, and, before implementing them, I will sketch them out here.

For every MySQL data type, there will be a corresponding JavaScript intermediate type. The intermediate type will be lossless, vis-a-vis the MySQL type. For instance, the intermediate type for DateTime will be capable of microsecond precision, and the intermediate type for BIGINT will be capable of representing all 64-bit numbers.

The ToDB() method of a Converter class will be expected to write the intermediate type for a column, and the FromDB() method will expect to read the intermediate type.

In some cases, the intermediate types are the same as the default presentation of the types, but not always. For example, the default mapping of TIME is to a MySQL-style HH:MM:SS string, but the intermediate type is a MySQLTime object.

Proposed Intermediate Types

• JS Number
  • MySQL INT (signed and unsigned inc. tiny, small, medium)
  • MySQL FLOAT and DOUBLE
  • MySQL ENUM, SET, and YEAR
  • MySQL BIT(1) through BIT(52)
  • MySQL TIMESTAMP expressed as microseconds since the Unix epoch
• JS String
  • MySQL CHAR, VARCHAR and Text
  • MySQL BIGINT and DECIMAL
• Node::Buffer
  • MySQL BINARY, VARBINARY, and BLOB
• JS MySQLTime object
  • MySQL TIME, DATE, and DATETIME

Failure to install on Mac

For some reasons I cannot install mysql-js on Mac (abbreviated):

$ npm install --save https://github.com/mysql/mysql-js/archive/2014-10-06.tar.gz
/
> [email protected] preinstall /Users/elazutkin/work/kangaroo/server-poc/node_modules/mysql-js
> node configure && node-gyp rebuild -d

#
#                 MySQL Cluster NoSQL API for Node.JS
#  April, 2013
[...snipped...]
MySQL Install Path> /usr/local/Cellar/mysql/5.6.24

Now run this command:
    node-gyp configure build -d
  CXX(target) Debug/obj.target/ndb_adapter/Adapter/impl/common/src/async_common.o
  CXX(target) Debug/obj.target/ndb_adapter/Adapter/impl/common/src/unified_debug.o
  CXX(target) Debug/obj.target/ndb_adapter/Adapter/impl/common/src/common_v8_values.o
  CXX(target) Debug/obj.target/ndb_adapter/Adapter/impl/ndb/src/AsyncNdbContext_wrapper.o
In file included from ../Adapter/impl/ndb/src/AsyncNdbContext_wrapper.cpp:21:
../Adapter/impl/ndb/include/AsyncNdbContext.h:21:10: fatal error: 'NdbApi.hpp' file not found
#include <NdbApi.hpp>
         ^
1 error generated.
make: *** [Debug/obj.target/ndb_adapter/Adapter/impl/ndb/src/AsyncNdbContext_wrapper.o] Error 1
[...snipped...]

The problem appears to be in the adapter code (?), which pulls in a non-existent NdbApi.hpp, which probably belongs to an ndb-cluster installation, while I have a MySQL Server 5.6.24 installed with brew.

The same bug exists when trying a current version of the repo.

The above is a failure with node.js 0.10. The current stable version of node.js is 0.12, which fails with numerous bugs related to different internals. It looks like the code of mysql-js was not updated to support new V8.

Just to stave off possible questions: I do not intend to use Mac as a deployment platform, but I want to use it as a development environment, and as such I expect that I can develop mysql-js-based web applications. If certain configurations are purposely not supported, please list them explicitly.

jones-ndb should support partition key distinct from primary key

In part of issue#37 we see a table where the NDB partition key is not the same as the primary key.

Hinting in startTransaction() should always use the partition key.

Cannot build jones-ndb with node 7.x

The compiler says:
no template named 'WeakCallbackData'; did you mean 'v8::WeakCallbackInfo'?

Relationship mapping should return an error if relationship name is a field name

If a relationship name is also a field name, both cannot be mapped in the same Projection.

To avoid confusion, we should disallow creating a relationship that has the same name as an existing field.