Giter Site home page Giter Site logo

loopbackio / loopback-connector-mongodb Goto Github PK

View Code? Open in Web Editor NEW
187.0 61.0 234.0 3.01 MB

The official MongoDB connector for the LoopBack framework.

Home Page: https://loopback.io/doc/en/lb4/MongoDB-connector

License: Other

JavaScript 98.91% Makefile 0.28% Shell 0.82%
mongodb nodejs loopback loopback4 hacktoberfest

loopback-connector-mongodb's Introduction

loopback-connector-mongodb

The official MongoDB connector for the LoopBack framework.

Installation

In your application root directory, enter this command to install the connector:

npm install loopback-connector-mongodb --save

This installs the module from npm and adds it as a dependency to the application's package.json file.

If you create a MongoDB data source using the data source generator as described below, you don't have to do this, since the generator will run npm install for you.

Supported versions

Starting from the version 6.0.0, this connector is no longer compatible with LoopBack 3. Please use the latest 5.x version in your LoopBack 3 applications.

This module adopts the Module Long Term Support (LTS) policy, with the following End Of Life (EOL) dates:

Version Status Published EOL LoopBack Juggler
6.x Current Mar 2021 Apr 2025 (minimum) 4 4.x
5.x Active LTS Jun 2019 Apr 2023 3, 4 3.x, 4.x
4.x Maintenance LTS Nov 2018 Apr 2021 3, 4 3.x, 4.x

Creating a MongoDB data source

For LoopBack 4 users, use the LB4 Command-line interface to generate a DataSource with MongoDB connector to your LB4 application. Run lb4 datasource, it will prompt for configurations such as host, post, etc. that are required to connect to a MongoDB database.

After setting it up, the configuration can be found under src/datasources/<DataSourceName>.datasource.ts, which would look like this:

const config = {
  name: 'db',
  connector: 'mongodb',
  url: '',
  host: 'localhost',
  port: 27017,
  user: '',
  password: '',
  database: 'testdb',
};

If your username or password contains special characters like @, $ etc, encode the whole username or password using encodeURIComponent.

Eg: pa$$wd would become pa%24%24wd.

Connection properties

Property Type   Description
connector String Connector name, either "loopback-connector-mongodb" or "mongodb".
database String Database name
host String Database host name
name String Name of the datasource in the app
password String Password to connect to database
port Number Database TCP port
url String Connection URL of form mongodb://user:password@host/db. Overrides other connection settings (see below).
user String Username to connect to database
authSource String Optional. Authentification database name. Usually "admin" value.

If you run a MongoDB with authentification (Docker's example here), you need to specify which database to authenticate against. More details can be found in MongoDB documentation on Authentification Methods. The default value is usually "admin", like in the official docker image.

NOTE: In addition to these properties, you can use additional Single Server Connection parameters supported by node-mongodb-native.

Additional properties

Property Type Default Description
allowExtendedOperators Boolean false Set to true to enable using MongoDB operators such as $currentDate, $inc, $max, $min, $mul, $rename, $setOnInsert, $set, $unset, $addToSet, $pop, $pullAll, $pull, $push, and $bit. See Update Operators section below
enableGeoIndexing Boolean false Set to true to enable 2d sphere indexing for model properties of type GeoPoint. This allows for indexed near queries.
lazyConnect Boolean false When set to true, the database instance will not be attached to the datasource and the connection is deferred. It will try to establish the connection automatically once users hit the endpoint. If the MongoDB server is offline, the app will start, however, the endpoints will not work.
disableDefaultSort Boolean false Set to true to disable the default sorting behavior on id column, this will help performance using indexed columns available in MongoDB.
collation String N/A Specify language-specific rules for string comparison, such as rules for letter-case and accent marks. See MongoDB documentation for details. It can also be used to create case insensitive indexes.

Setting the url property in datasource.ts

You can set the url property to a connection URL in <datasourceName>.datasources.ts to override individual connection parameters such as host, user, and password. E.g loopback:pa55w0rd@localhost:27017/testdb.

Using the mongodb+srv protocol

MongoDB supports a protocol called mongodb+srv for connecting to replica sets without having to give the hostname of every server in the replica set. To use mongodb+srv as the protocol set the protocol connection property in the datasource.json to mongodb+srv. For example:

const config = {
  name: 'db',
  connector: 'mongodb',
  host: 'myserver',
  database: 'testdb',
  protocol: 'mongodb+srv',
};

Note: the port is not specified when using the mongodb+srv protocol and will be ignored if given.

TLS/SSL Connections

Note: SSL options deprecated since MongoDB 4.2

const config = {
  name: 'db',
  connector: 'mongodb',
  url: '',
  host: 'localhost',
  port: 27017,
  user: '',
  password: '',
  database: 'testdb',
  tls: true,
  tlsCertificateKeyFile: '/local/path/to/pem-file',
  tlsCAFile: '/local/path/to/ca-file',
};

Security Considerations

MongoDB Driver allows the $where operator to pass in JavaScript to execute on the Driver which can be used for NoSQL Injection. See MongoDB: Server-side JavaScript for more on this MongoDB feature.

To protect users against this potential vulnerability, LoopBack will automatically remove the $where and mapReduce operators from a query before it's passed to the MongoDB Driver. If you need to use these properties from within LoopBack programmatically, you can disable the sanitization by passing in an options object with disableSanitization property set to true.

Example:

await PostRepository.find(
  { where: { $where: "function() { /*JS function here*/}" } },
  { disableSanitization: true }
);

Type mappings

See LoopBack 4 types (or LoopBack 3 types) for details on LoopBack's data types.

LoopBack to MongoDB types

Type conversion is mainly handled by MongoDB. See 'node-mongodb-native' for details.

Update Operators

Except the comparison and logical operators LoopBack supports in the operator list of Where filter, you can also enable MongoDB update operators for update* methods by setting the flag allowExtendedOperators to true in the datasource configuration.

Here is an example of updating the price for all the products under category furniture if their current price is lower than 100:

await productRepo.updateAll({ $max: { price: 100 }}, { category: {eq: 'furniture'} // where clause goes in here });

{% include tip.html content="you will not need the dollar sign '$' for operators in the Where clause." %}

Handling ObjectId

MongoDB uses ObjectId for its primary key, which is an object instead of a string. In queries, string values must be cast to ObjectId, otherwise they are not considered as the same value. Therefore, you might want to specify the data type of properties to enforce ObjectId coercion. Such coercion would make sure the property value converts from ObjectId-like string to ObjectId when it accesses to the database and converts ObjectId to ObjectId-like string when the app gets back the value. (An ObjectId-like string is a string that has length 12 or 24 and has the format of an ObjectId i.e /^[0-9a-fA-F]{24}$/.)

LoopBack provides two scopes to handle such coercion: per model or per property. Please check the following to see which configuration meets your requirements.

{% include important.html content="please make sure you are using loopback-connector-mongodb package version 5.2.1 or above to handle ObjectId properly." %}

  • No ObjectId coercion: CRUD operations can be operated with non-ObjectId-like string or ObjectId-like string ids.

  • Enforce ObjectId coercion: the property value can only be ObjectId or ObjectId-like string, otherwise it will be rejected.

Enforcing ObjectId coercion can be done by setting the flag strictObjectIDCoercion in the model definition or by specifying dataType: ObjecId in the property definition.

Model scope

This scope would do the conversion for all properties in the model.

@model({settings: {
  strictObjectIDCoercion: true
}})
export class User extends Entity {
@property({
    type: 'string',
    id: true,
  })
  id: string;
...
}

Property scope

This scope would only convert an ObjectId-like string to ObjectId with a certain property in the model.

@property({
    type: 'string',
    id: true,
    mongodb: {dataType: 'ObjectId'}
  }
  id: string;

Also notice that for RELATIONS, if the primary key/source key has set to enforce ObjectId coercion (no matter by strictObjectIDCoercion: true or dataType: 'ObjectId'). The corresponding foreign key will need to have it set as well to make sure relations work smoothly.

@model()
export class User extends Entity {
// source key
@property({
    type: 'string',
    id: true,
    mongodb: {dataType: 'ObjectId'}
  })
  id: string;
...
}

@model(// )
export class Address extends Entity {
  ...
  // foreign key
  @belongsTo(() => User,
   {}, //relation metadata goes in here
   {// property definition goes in here
    mongodb: {dataType: 'ObjectId'}
  })
  UserId: string;
}

Customize collection/field names

loopback-connector-mongodb allows you to have different collection and field names from the models. Such configurations can be added to the model definition and the property definition respectively as mongodb:{ <field>: <customValue>}. For example, the following setting would define a collection with custom name Custom_Collection_User, and it has a custom field name Custom_Name in the database:

{% include code-caption.html content="/src/models/User.model.ts" %}

@model({
  settings: {
    // model definition goes in here
    mongodb: { collection: "Custom_Collection_User" },
  },
})
export class User extends Entity {
  @property({
    type: "string",
    id: true,
    generated: true,
  })
  id: string;

  @property({
    type: "string",
    mongodb: {
      fieldName: "Custom_Name",
    },
  })
  name?: string;
}

{% include important.html content="Since in MongoDB _id is reserved for the primary key, LoopBack does not allow customization of the field name for the id property. Please use id as is. Customizing the id property would cause errors." %}

Running tests

Own instance

If you have a local or remote MongoDB instance and would like to use that to run the test suite, use the following command:

  • Linux
MONGODB_HOST=<HOST> MONGODB_PORT=<PORT> MONGODB_DATABASE=<DATABASE> CI=true npm test
  • Windows
SET MONGODB_HOST=<HOST> SET MONGODB_PORT=<PORT> SET MONGODB_DATABASE=<DATABASE> SET CI=true npm test

Docker

If you do not have a local MongoDB instance, you can also run the test suite with very minimal requirements.

  • Assuming you have Docker installed, run the following script which would spawn a MongoDB instance on your local:
source setup.sh <HOST> <PORT> <DATABASE>

where <HOST>, <PORT> and <DATABASE> are optional parameters. The default values are localhost, 27017 and testdb respectively.

  • Run the test:
npm test

Leak detection

Tests run for 100 iterations by default, but can be increased by setting the env var ITERATIONS.

make leak-detection # run 100 iterations (default)

or

ITERATIONS=1000 make leak-detection # run 1000 iterations

Running benchmarks

Benchmarks must be run on a Unix-like operating system.

make benchmarks

The results will be output in ./benchmarks/results.md.

Release notes

  • 1.1.7 - Do not return MongoDB-specific _id to client API, except if specifically specified in the model definition

loopback-connector-mongodb's People

Contributors

0candy avatar 1602 avatar achrinza avatar agnes512 avatar alexandrusavin avatar amir-61 avatar b-admike avatar bajtos avatar cgole avatar crandmck avatar dhmlau avatar emonddr avatar fabien avatar hugopoi avatar jannyhou avatar loay avatar mitsos1os avatar mrbatista avatar pandaiolo avatar phairow avatar raymondfeng avatar renovate[bot] avatar ritch avatar rmg avatar shimks avatar siddhipai avatar smartmouse avatar ssh24 avatar superkhau avatar virkt25 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

loopback-connector-mongodb's Issues

Issue with Array format while using MongoDB connector using REST API

ISSUE is how result of type array is inserted. See the value of result under "2. What was inserted finally." to understand the issue.

  1. My input:
    { "cvsName" : "cvs_OrderVolume_StrongLoopTry",
    "database" : "pst",
    "description" : "Orders Previous Year to Date by Month",
    "detailedDescription" : "",
    "displayRank" : 89,
    "graphTitle" : "Previous Year to Date",
    "graphType" : "line",
    "linearResult" : ["2012-01", "2012-02"],
    "link" : [],
    "name" : "OrderVolumePreviousYear2DateByMonth",
    "pageId" : "newCustomerByMonth",
    "query" : "this is a complex select query which I cant print here.",
    "result" : [22644, 21752]
    }
  2. What was inserted finally:
    {
    "cvsName" : "cvs_OrderVolume_StrongLoopTry",
    "database" : "pst",
    "description" : "Orders Previous Year to Date by Month",
    "detailedDescription" : "",
    "displayRank" : 89,
    "graphTitle" : "Previous Year to Date",
    "graphType" : "line",
    "linearResult" : [["2012-01"], ["2012-02"]],
    "link" : "",
    "name" : "OrderVolumePreviousYear2DateByMonth",
    "pageId" : "newCustomerByMonth",
    "query" : "this is a complex select query which I cant print here.",
    "result" : [[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, "SO MANY MORE NULL THAT I EXCEEDED THE MAX OF 65535 characters. lol"]],
    "_id" : ObjectId("53150cc8cd26342c05e3d72b")
    }
  3. My model defined in model.json
    "devil": {
    "properties": {
    "cvsName": {
    "type": "String"
    },
    "database": {
    "type": "String"
    },
    "description": {
    "type": "String"
    },
    "detailedDescription": {
    "type": "String"
    },
    "displayRank": {
    "type": "Number"
    },
    "graphTitle": {
    "type": "String"
    },
    "graphType": {
    "type": "String"
    },
    "linearResult": {
    "type": "Array"
    },
    "link": {
    "type": "String"
    },
    "name": {
    "type": "String"
    },
    "pageId": {
    "type": "String"
    },
    "query": {
    "type": "String"
    },
    "result": {
    "type": "Array"
    }
    },
    "public": true,
    "dataSource": "mongoDev",
    "plural": "devils"
    }

how could I use MongoDB.prototype.buildWhere() function?

I'd like to use aggregate but I have to convert queries like { "upload_time": {"gt": "2014-12-01T00:00:00.000Z"}} to { "upload_time": {$gt: new Date("2014-12-01T00:00:00.000Z")} for $match.
I found MongoDB.prototype.buildWhere function, how could I use it?
Thank you very much.

//json
{
    "acn":"jason",
    "email":"[email protected]",
    "where": {
        "and": [
            { "upload_time": {"gt": "2014-12-01T00:00:00.000Z"}}
        ]
    }
}
//numessage.js
var numessageCollection = Numessage.getDataSource().connector.collection(Numessage.modelName);
numessageCollection.aggregate([
                {$match: { $and:[ { key: { $in: siteIds } }, { "upload_time": {$gt: new Date("2014-12-01T00:00:00.000Z")}} ] } },
                {$group: { _id: {owner:"$owner"}, total: { $sum: 1 } } }
          ], function(err, groupByRecords) {
                ...
            });

Data conversion for properties of type Buffer

For a model as follows:

"employee": {
  "properties": {
    "name": {"type": "String"},
    "testValue": {"type": "Buffer"}
  }
}
  1. When the ‘employee’ is first created, the response is original object with the returned id from MongoDB.
{
  "name": "X",
  "testValue": [
    42
  ],
  "id": "533adc6528543b2e59d1657a”
}

The test value is the Buffer.

  1. MongoDB stores testValue as Binary:
{
  "name": "X",
  "testValue": { "$binary" : "Kg==", "$type" : "0" },
  "_id": { "$oid" : "533adc6528543b2e59d1657a" }
}
  1. When the query is run, MongoDB returns testValue as Binary. But it is not converted back to Buffer. And Binary.toJSON() returns the base64 encoded value.

1.5.0 breaking JSON Schema documents

The upgrade from 1.4.5 -> 1.5.0 prevents the creation of models that include JSON Schema documents that contain an embedded $ref. For example (simplified),

{
   "name": "example",
   "type": "object",
   "properties": {
     "billing_address": { "$ref": "/definitions/address" },
     "shipping_address": {
       "allOf": [
         { "$ref": "/definitions/address" },
         { "properties":
           { "type": { "enum": [ "residential", "business" ] } },
           "required": ["type"]
         }
       ]
     }
   }
}

Attempting to create a Model with such a property included produces the error:

{ [MongoError: key $ref must not start with '$']
  name: 'MongoError',
  message: 'key $ref must not start with \'$\'' }

Is this something wrong with loopback-connector-mongodb? Is it produced by a deprecation in the mongo 2.0 driver?

Don't output null values in JSON - don't store NULL values by default

It's a waste of bandwidth, and non-standard behavior (which probably is the result of BSON handling null, whereas JSON doesn't). Aparently NULL values are indeed inserted into the MongoDB documents.

An option to disable this somehow (on the input as well as output side of things) seems to be in order.

how to use write concern

Hi, This is how i am defining my mongo connection details:- in my loopback config file,
My mongo is a replica set.

"mongo": {
"connector": "mongodb",
"url": "mongodb://90.40.26.46,90.40.26.47/ugc-prod?readPreference=secondary"
},

Now I want to add the "write concern" true here, can anyone help me how to accomplish that?

RangeError when querying using 'inq' and 'nin' operators using angular sdk

Using a mongodb datasource, mongodb connector and angular sdk, queries using 'inq' and 'nin' fail with a RangeError message. Queries without these filters work fine. An example query:

MyModel.find({ filter: { where: { myField: { inq: ['val1', 'val2', 'val3']}}}}).$promise

Fails with the error message:

{"error": {"name":"RangeError", "status":500,"message":"attempt to write outside buffer bounds", "stack":"RangeError: attempt to write outside buffer bounds\n at Buffer.write (buffer.js:371:11)\n at packElement (/home/rmd/workspace/qb/node_modules/loopback-connector-mongodb/node_modules/mongodb/node_modules/bson/lib/bson/bson.js:877:60)\n at serializeObject (/home/rmd/workspace/qb/node_modules/loopback-connector-mongodb/node_modules/mongodb/node_modules/bson/lib/bson/bson.js:377:15)\n at Function.serializeWithBufferAndIndex (/home/rmd/workspace/qb/node_modules/loopback-connector-mongodb/node_modules/mongodb/node_modules/bson/lib/bson/bson.js:343:10)\n at BSON.serializeWithBufferAndIndex (/home/rmd/workspace/qb/node_modules/loopback-connector-mongodb/node_modules/mongodb/node_modules/bson/lib/bson/bson.js:1556:15)\n at QueryCommand.toBinary (/home/rmd/workspace/qb/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/commands/query_command.js:236:35)\n at Connection.write (/home/rmd/workspace/qb/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/connection/connection.js:265:37)\n at __executeQueryCommand (/home/rmd/workspace/qb/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/db.js:1721:16)\n at Db._executeQueryCommand (/home/rmd/workspace/qb/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/db.js:1871:7)\n at Cursor.nextObject (/home/rmd/workspace/qb/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/cursor.js:749:13)\n at Cursor.toArray (/home/rmd/workspace/qb/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/cursor.js:168:10)"}
}

Subsequent queries result in the connection being closed.

How I can run a groupby query, a find distinct too.

which is the properly way to run queries like: GROUP BY , or find DISTINCT documents based on a key in a document's collection through Mongodb connector, also mongodb has $agregation and $mapReduce methods, please help me with this. Thank for answers.

Unable to update to MongoDB using "updateAttributes"

Hi,

When trying to update the details to db(mongodb), I am getting error.

/Users/username/JavaScriptWorkSpace/myproject/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/connection/base.js:245
throw message;
^
TypeError: Object function ModelConstructor(data, options) {
if (!(this instanceof ModelConstructor)) {
return new ModelConstructor(data, options);
}
if (ModelClass.settings.unresolved) {
throw new Error('Model ' + ModelClass.modelName + ' is not defined.');
}
ModelBaseClass.apply(this, arguments);
} has no method 'updateAttributes'

at /Users/username/JavaScriptWorkSpace/myproject/models/UserWithPrivileges.js:49:24

Pasting the code I have wrote;

var loopback = require('loopback');
var app = require('../app');
var privilegeModel=app.models.privileges;
var Orginaluser=app.models.User;

privilegeModel.updateUserWithPrivilege = function (UserDetail,include, fn) {
var usersId=UserDetail.user_id;
Orginaluser.findOne({where: {"user_id": usersId}}, function(err, Orginaluser) {

    privilegeModel.updateAttributes({privilege_user_management: 'C,R,E'}, function(err, privilegeModel){
        console.log(privilegeModel);
    });
});

Please help let me know whether i am doing anything wrong here.

If a value is 'undefined' for a 'where' condition, the condition is skipped

I upgraded to version 1.7.0 and a lot of my queries are not working. When an undefined value is passed to a condition, this condition is just skipped.

For example, if I want to get a post with authorId:1 and groupId:undefined, the result is that I get all of the post for author with id 1, instead of bringing the posts for this author that do not have a groupId defined.

var group;

Post.findOne({
        where: {
          and: [
            {groupId: group},
            {authorId: ctx.req.accessToken.userId}
          ]
        }}, function (err, post) { ...

makes sense?

How can I use $inc

If I try to use $inc in the data field of update method I get "The dollar ($) prefixed field '$inc' in '$inc' is not valid for storage.". When I dug deeper I found that the only operation that update method is supporting is "$set" (see mongodb.js:643:43). Am I wrong? Is there no way of incrementing fields using this connector?

v1.6.0 causes MongoException in UpdateOrCreate/findAndModify

I upgraded to loopback-connector-mongodb 1.6.0 and I am getting the following error after calling updateOrCreate several times on the same doc object. I do not get this error if I revert only loopback-connector-mongodb back to 1.4.5. We're using mongo version 2.6.7.

 Assertion: 10307:Client Error: bad object in message: bson length doesn't match what we found in object with unknown _id

I don't have an isolated reproducible case yet as that would take me a full day most likely, but I wanted to post it to let you know.

Here is the verbose MongoDB log.

23T16:36:16.669-0500 [conn26] command scm_ops_tst.$cmd {
    findandmodify : "JobInstanceLog",
    query : {
        _id : ObjectId('54c2bed00f992bf8214288f1')
    },
    sort : {
        _id : 1
    },
    new : true,
    remove : false,
    upsert : true,
    update : {
        $set : {
            jobInstanceId : "512610624747220410000",
            jobId : "5474e123550d9b34260f05c7",
            jobName : "Pricing",
            startTime : new Date(1422048976768),
            hostname : "temp",
            logs: [...edited out...]
        }
    }
} keyUpdates:0 numYields:0 locks(micros) w:2575 reslen:8889 2ms
2015-01-23T16:36:17.144-0500 [conn26] Assertion: 10307:Client Error: bad object in message: bson length doesn't match what we found in object with unknown _id
2015-01-23T16:36:19.753-0500 [conn26] mongod.exe    ...\src\mongo\util\stacktrace.cpp(169)                           mongo::printStackTrace+0x43
2015-01-23T16:36:19.753-0500 [conn26] mongod.exe    ...\src\mongo\util\log.cpp(127)                                  mongo::logContext+0x9c
2015-01-23T16:36:19.753-0500 [conn26] mongod.exe    ...\src\mongo\util\assert_util.cpp(183)                          mongo::msgasserted+0xfb
2015-01-23T16:36:19.753-0500 [conn26] mongod.exe    ...\src\mongo\util\assert_util.cpp(174)                          mongo::msgasserted+0x13
2015-01-23T16:36:19.753-0500 [conn26] mongod.exe    ...\src\mongo\db\dbmessage.cpp(116)                              mongo::DbMessage::nextJsObj+0x11f
2015-01-23T16:36:19.753-0500 [conn26] mongod.exe    ...\src\mongo\db\dbmessage.h(213)                                mongo::QueryMessage::QueryMessage+0x74
2015-01-23T16:36:19.753-0500 [conn26] mongod.exe    ...\src\mongo\db\instance.cpp(253)                               mongo::receivedQuery+0xcc
2015-01-23T16:36:19.753-0500 [conn26] mongod.exe    ...\src\mongo\db\instance.cpp(437)                               mongo::assembleResponse+0x2f4
2015-01-23T16:36:19.753-0500 [conn26] mongod.exe    ...\src\mongo\db\db.cpp(203)                                     mongo::MyMessageHandler::process+0x111
2015-01-23T16:36:19.753-0500 [conn26] mongod.exe    ...\src\mongo\util\net\message_server_port.cpp(210)              mongo::PortMessageServer::handleIncomingMsg+0x671
2015-01-23T16:36:19.753-0500 [conn26] mongod.exe    ...\src\third_party\boost\libs\thread\src\win32\thread.cpp(185)  boost::`anonymous namespace'::thread_start_function+0x21
2015-01-23T16:36:19.753-0500 [conn26] mongod.exe    f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(314)      _callthreadstartex+0x17
2015-01-23T16:36:19.753-0500 [conn26] mongod.exe    f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(292)      _threadstartex+0x7f
2015-01-23T16:36:19.753-0500 [conn26] kernel32.dll                                                                   BaseThreadInitThunk+0xd
2015-01-23T16:36:19.753-0500 [conn26] 
2015-01-23T16:36:19.753-0500 [conn26] warning: DBException thrown :: caused by :: 10307 Client Error: bad object in message: bson length doesn't match what we found in object with unknown _id
2015-01-23T16:36:21.245-0500 [conn26] mongod.exe    ...\src\mongo\util\stacktrace.cpp(169)                           mongo::printStackTrace+0x43
2015-01-23T16:36:21.245-0500 [conn26] mongod.exe    ...\src\mongo\util\assert_util.cpp(66)                           mongo::DBException::traceIfNeeded+0xec
2015-01-23T16:36:21.245-0500 [conn26] mongod.exe    ...\src\mongo\util\assert_util.cpp(183)                          mongo::msgasserted+0x1a7
2015-01-23T16:36:21.245-0500 [conn26] mongod.exe    ...\src\mongo\util\assert_util.cpp(174)                          mongo::msgasserted+0x13
2015-01-23T16:36:21.245-0500 [conn26] mongod.exe    ...\src\mongo\db\dbmessage.cpp(116)                              mongo::DbMessage::nextJsObj+0x11f
2015-01-23T16:36:21.245-0500 [conn26] mongod.exe    ...\src\mongo\db\dbmessage.h(213)                                mongo::QueryMessage::QueryMessage+0x74
2015-01-23T16:36:21.245-0500 [conn26] mongod.exe    ...\src\mongo\db\instance.cpp(253)                               mongo::receivedQuery+0xcc
2015-01-23T16:36:21.245-0500 [conn26] mongod.exe    ...\src\mongo\db\instance.cpp(437)                               mongo::assembleResponse+0x2f4
2015-01-23T16:36:21.245-0500 [conn26] mongod.exe    ...\src\mongo\db\db.cpp(203)                                     mongo::MyMessageHandler::process+0x111
2015-01-23T16:36:21.245-0500 [conn26] mongod.exe    ...\src\mongo\util\net\message_server_port.cpp(210)              mongo::PortMessageServer::handleIncomingMsg+0x671
2015-01-23T16:36:21.245-0500 [conn26] mongod.exe    ...\src\third_party\boost\libs\thread\src\win32\thread.cpp(185)  boost::`anonymous namespace'::thread_start_function+0x21
2015-01-23T16:36:21.245-0500 [conn26] mongod.exe    f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(314)      _callthreadstartex+0x17
2015-01-23T16:36:21.246-0500 [conn26] mongod.exe    f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(292)      _threadstartex+0x7f
2015-01-23T16:36:21.246-0500 [conn26] kernel32.dll                                                                   BaseThreadInitThunk+0xd
2015-01-23T16:36:21.246-0500 [conn26] 
2015-01-23T16:36:21.246-0500 [conn26] AssertionException handling request, closing client connection: 10307 Client Error: bad object in message: bson length doesn't match what we found in object with unknown _id
Note: loopback-connector-mongodb dependency changes:

loopback-connector-mongodb 1.6.0 = mongodb 2.0.14 (Breaks)
loopback-connector-mongodb 1.4.5 = mongodb ~1.4.7 (Works)

When comparing the json sent, 1.4.5(LEFT) and 1.6.0 (RIGHT)
image
Only the 1,0 and true, false appear to be different which I would guess should not matter.

Also, because of this error, #81 was noticed as well.

Adding reference to #77 as I assume it's related.

MongoDB.prototype.buildWhere doesn't $ nested specs

It doesn't because it isn't fully correctly recursive. Yet nested specs are needed to query embedded documents. They almost work, with workarounds.

Workaround is at top level to use LoopBack syntax, at lower levels MongoDB syntax. Solves many cases.

Suggested low cost fix: Allow MongoDB syntax at any level.

Example given here. To achieve MongoDB getting

find({"permissions":{$elemMatch:{"user":{$in:["one","other"]},"code":4}}})

had to have a where like

{"permissions":{elemMatch:{"user":{$in:["one","other"]},"code":4}}}

To be clear what is wrong with this picture: Had to have $in because inq wouldn't be replaced. But couldn't make it both $in and $elemMatch because that would have become $$elemMatch.

LoopBack syntax

{"permissions":{elemMatch:{"user":{inq:["one","other"]},"code":4}}}

would result in MongoDB dysfunctional

{"permissions":{$elemMatch:{"user":{inq:["one","other"]},"code":4}}}

A simple fix I'm going to send a pull request for will allow a "pure MongoDB" syntax. Such as

{"permissions":{$elemMatch:{"user":{$in:["one","other"]},"code":4}}}

Seems much easier to implement than complete correct handling.

Some may appreciate the ability to "think and write MongoDB queries" all the way.

Connector fails to find models with ids of type Number

When attempting to set the id type to be Number, the MyModel.find method fails to find by id.

Try configuration:

{
  "name": "MyModel",
  "properties": {
    "id": {
      "type": "Number",
      "required": true,
      "id": true
    }
  }
}

I have noticed that the objects will be saved in MongoDB with an _id of type number (not String or ObjectID).

Why converting string to ObjectId in 'inq' filter?

I get the idea behind doing so is to allow querying by _id, but the problem it's causing is any string of length 12 or 24 will be converted too, even worser if those strings matches ObjectID format, they will be converted to ObjectID type when they should remain String type.

In short, this kind of "forced" conversion is kind of a surprise and smell like a hack?

} else if (spec === 'inq') {
        query[k] = { $in: cond.map(function (x) {
          if ('string' !== typeof x) return x;
          return ObjectID(x);
        })};
      } 

See full code here:
http://goo.gl/Xwhlj4

Config to native mongodb driver

We should have a way of passing certain config option directly to the underlying native mongodb driver. I am specifically looking for ReadPreference config option.

Error when querying mongodb with AND as well as near

When using a mongo db datasource and trying to query a model by using the and operator it fails when using "near" as well.

Query:
{"where": {"and": [{"location": {"near": "52.335421, 13.096018"}},{"name": {"like": "aus"}}]}}

Error:
"message": "Can't canonicalize query: BadValue not handled: $near", "stack": "MongoError: Can't canonicalize query: BadValue not handled: $near\n at Object.toError (/home/codio/workspace/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/utils.js:114:11)\n at /home/codio/workspace/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/cursor.js:689:54\n at Cursor.close (/home/codio/workspace/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/cursor.js:972:5)\n at commandHandler (/home/codio/workspace/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/cursor.js:689:21)\n at /home/codio/workspace/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/db.js:1847:9\n at Server.Base._callHandler (/home/codio/workspace/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/connection/base.js:445:41)\n at /home/codio/workspace/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/connection/server.js:478:18\n at MongoReply.parseBody (/home/codio/workspace/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/responses/mongo_reply.js:68:5)\n at null.<anonymous> (/home/codio/workspace/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/connection/server.js:436:20)\n at EventEmitter.emit (events.js:95:17)"

Switching the model to in memory this query works just fine.

'order' field added to query object in MongoDB.prototype.all and never removed

This causes all of my queries with updateAll (I suspect not the only method affected) to fail.

For example...

MongoDB.prototype.all(model, filter, callback)

If the filter parameter passed into the all method is { "username" : "Terekhov" }, then by the time it gets to

MongoDB.prototype.updateAll(model, where, data, cb)

...the where object looks like this:

{
    "username": "Terekhov",
    order:
    [
        "id"
    ]
}

My solution was to replace the use of filter.order with a tempOrder variable, like so:

  var order = {};
  var tempOrder = '';
  if (!tempOrder) {
    var idNames = this.idNames(model);
    if (idNames && idNames.length) {
      tempOrder = idNames;
    }
  }
  if (tempOrder) {
    var keys = tempOrder;
    if (typeof keys === 'string') {
      keys = keys.split(',');
    }
    for (var index = 0, len = keys.length; index < len; index++) {
      var m = keys[index].match(/\s+(A|DE)SC$/);
      var key = keys[index];
      key = key.replace(/\s+(A|DE)SC$/, '').trim();
      if(key === idName) {
        key = '_id';
      }
      if (m && m[1] === 'DE') {
        order[key] = -1;
      } else {
        order[key] = 1;
      }
    }
  } else {
    // order by _id by default
    order = {_id: 1};
  }
  cursor.sort(order);

However, it's entirely possible I just don't understand the reason to include the order property in the query object.

  1. If this is so, what is the reason and how can I avoid it polluting my searches in updateAll?
  2. If not, thoughts on the above fix, seem reasonable?

Mod on _id not allowed when trying to update my model in mongdb

I'm unable to update an existing document in my mongodb. From what I have been able to read about so far it looks like a conflict between mongodb storing _id as an object conflicting with me passing a string value but I'm not sure how to change that. Should I have an explicit '_id' property in my model definition?

I have tried: Event.upsert, Event.updateOrCreate, and Event.save and they all fail

I also tried removing the '_id' property from the model before 'PUT'ing it back to the server:

delete saveEventObj._id;

but that ended up creating a copy of the original document with null for both 'id' and '_id' properties in the new value forcing me to delete the instance manually.

My model code looks like this:

  "event": {
    "properties": {
      "date": {
        "type": "date"
      },
      "type": {
        "type": "string"
      },
      "nearestCity": {
        "type": "string"
      },
      "stateProv": {
        "type": "string"
      },
      "country": {
        "type": "string"
      },
      "blurb": {
        "type": "string"
      },
      "notes": {
        "type": "string"
      },
      "createdDate": {
        "type": "date"
      },
      "lastUpdate": {
        "type": "date"
      },
      "material": {
        "type": "string"
      },
      "tags": {
        "type": "array"
      }
    },
    "public": true,
    "dataSource": "mongodb",
    "plural": "events"
  },

I'm also using the loopback-angular auto-generated lb-services code and calling:

          Event.upsert(saveEventObj,
              function(response){
                console.log('Saved the event: ' + JSON.stringify(response));
              },
              function(response){
                console.log('bad update event: ' + JSON.stringify(response));
              }
            );

my browser console has the following:

PUT http://0.0.0.0:3001/api/events?id=52e8a43c36354025bd0c1144 500 (Internal Server Error)

bad update event: {"data":{"error":{"name":"MongoError","status":500,"message":"Mod on _id not allowed","lastErrorObject":{"err":"Mod on _id not allowed","code":10148,"n":0,"connectionId":204,"ok":1},"ok":0,"errmsg":"Mod on _id not allowed","stack":"MongoError: Mod on _id not allowed\n    
at Object.toError (/Users/seanbrookes/_stacks/projects/oilwreck/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/utils.js:110:11)\n    
at /Users/seanbrookes/_stacks/projects/oilwreck/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/db.js:1097:29\n    
at /Users/seanbrookes/_stacks/projects/oilwreck/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/db.js:1806:9\n    
at Server.Base._callHandler (/Users/seanbrookes/_stacks/projects/oilwreck/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/connection/base.js:442:41)\n   
 at /Users/seanbrookes/_stacks/projects/oilwreck/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/connection/server.js:485:18\n    
at MongoReply.parseBody (/Users/seanbrookes/_stacks/projects/oilwreck/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/responses/mongo_reply.js:68:5)\n    
at null.<anonymous> (/Users/seanbrookes/_stacks/projects/oilwreck/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/connection/server.js:443:20)\n    
at EventEmitter.emit (events.js:95:17)\n    at null.<anonymous> (/Users/seanbrookes/_stacks/projects/oilwreck/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/connection/connection_pool.js:191:13)\n   
 at EventEmitter.emit (events.js:98:17)"}},
"status":500,"config":{"transformRequest":[null],"transformResponse":[null],"url":"/api/events","method":"PUT",
"data":{"date":"2013-10-17T07:00:00.000Z","type":"train","nearestCity":"Sexsmith","stateProv":"Alberta","country":"Canada","blurb":"<p> Residents in the northwestern Alberta town of Sexsmith were forced from their homes after four CN rail cars carrying anhydrous ammonia left the rails. The cars remained upright and there were no leaks.</p>",
"notes":null,"createdDate":null,"lastUpdate":1391530836262,"material":null,
"tags":[],
"id":"52e8a43c36354025bd0c1144","_id":"52e8a43c36354025bd0c1144",
"$$hashKey":"006"},"params":{"id":"52e8a43c36354025bd0c1144"},"headers":{"Accept":"application/json, text/plain, */*","Content-Type":"application/json;charset=utf-8"}}} 

Boolean fields always true sent to POST via REST

Hi there,
I made a POST call to rest with jQuery like this:

var obj = {};
obj.isFree = false;
obj.yob = 1981;
obj.name = "PlayerName";
$.post("/api/players?access_token="+getAccessToken(), obj, "json").done(function(result){
    ...
}).fail(function(){
    ...
});

but when I go through my DB the isFree field is set to true.

Here's my model:

{
  "name": "player",
  "plural": "players",
  "base": "PersistedModel",
  "idInjection": true,
  "properties": {
    "name": {
      "type": "string",
      "required": true
    },
    "yob": {
      "type": "number"
    },
    "photo": {
      "type": "string"
    },
    "isFree": {
      "type": "boolean"
    }
  },
  "validations": [],
  "relations": {
    "playerRole": {
      "type": "hasOne",
      "model": "playerRole",
      "foreignKey": ""
    }
  },
  "acls": [
    {
      "accessType": "*",
      "principalType": "ROLE",
      "principalId": "$everyone",
      "permission": "DENY"
    },
    {
      "accessType": "*",
      "principalType": "ROLE",
      "principalId": "$authenticated",
      "permission": "ALLOW"
    }
  ],
  "methods": []
}

Here the validation of the model in player.js:

module.exports = function(Player) {
    Player.validatesUniquenessOf('name', { scopedTo: ['yob'] });
};

Here the final result my DB mongo:

{
    "_id" : ObjectId("54d0e2f3074fbc9c97a6606f"),
    "name" : "PlayerName",
    "yob" : 1981,
    "photo" : null,
    "isFree" : true
}

I think this could be a bug in mongo connector.
Anyone with this issue?

How to save file to gridfs?

How to save file to gridfs as below for example?
Thank you very much.

def save(inputStream, contentType, originalFilename) {
        def inputFile = gridFS.createFile(inputStream)
        String extension = originalFilename.split('\\.')[-1]
        String gridFSFilename=inputFile.id.toString() + '.' + extension
        inputFile.setFilename(gridFSFilename)
        inputFile.save()
        return inputFile.filename
    }

id string converted to large number after save

Few ids are converted to hex numbers when saved to mongodb.

The id "line-by-line" is interpreted as ObjectId("6c696e652d62792d6c696e65") in mongo and
for id "line-stream" the mongo id is "line-stream"

Duplicate Key error when trackChanges is enabled

With trackChanges enabled, I post new content to my model and the server dies shortly after with MongoError: insertDocument :: caused by :: 11000 E11000 duplicate key error index: mydb.mymodel-change.$_id_ dup key: { : "XXXXXX" }; When I restart the server, I get the same error. Is there a step I am missing after enabling trachChanges

Related:
strongloop/loopback#566

Object Properties in model.json

I've combed through the documentation, and searched everywhere for an answer to no avail. My question is how would I define the following model in model.json? Specifically how would I define the emails object property? I basically want to create detailed schemas for my MongoDB models to ensure consistent data. Any help is appreciated.

 var Customer = dataSource.createModel('customer', {seq: {type: Number, id: true}, name: String, emails: [{label: String, email: String}], age: Number});

npm test results in make eror ( even after ensuring mocha is installed.)

Got the latest build and ran npm test. I am seeing some make errors posted below.

Build error

./node_modules/.bin/mocha -G --timeout 10000 test/.test.js
/bin/sh: ./node_modules/.bin/mocha: No such file or directory
make: *
* [test] Error 127
npm ERR! Test failed. See above for more details.

updateattributes.

Hi,

I am facing issue in updating db using updateattributes.

Getting id related error.

Orginaluser.prototype.updateAttributes(UserDetail.user_details, function(err, theOrginaluser){

});

/Users/company/JavaScriptWorkSpace/myproject/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/connection/base.js:245
throw message;
^
TypeError: Cannot set property 'user_id' of undefined
at /Users/company/JavaScriptWorkSpace/myproject/node_modules/loopback-datasource-juggler/lib/dao.js:888:37

Please help as soon as possible.

https://groups.google.com/forum/#!searchin/loopbackjs/updateattribute/loopbackjs/f2yx26ZBHGI/Cl7RZBoMvHYJ

Composite ID component stored in database as _id

When storing entries into the database using composite IDs, it appears that one of the composite IDs is used as the internal _id property.

This is creating an issue with having composite IDs used in a many-to-many relationship manner.

This is accomplished by building a model entity with a foreign key to two other model entities, along with a 3rd ID as its own. It appears to use the first item of the composite id definition as defined by the LDL is used for the _id.

ECONNREFUSED when deploy on heroku

Hi,

I have a strongloop app working pefectly on local even connecting to a remote mongo instance
But when I deploy on heroku or modulus.io I get ECONNREFUSED error doesnt matter where is the mongo instance,

Somebody have experience on StrongLoop on Heroku or Modulus.io?

Thank in advance!

Antonio

ping never returns on ECONNREFUSED

When the connection string points to a port where nobody is listening on, the ping method never returns. A message is logged instead:

Connection fails:  [Error: failed to connect to [localhost:4]]
It will be retried for the next request.

Test cases (the first one passes, the second one fails):

// test/mongodb.test.js
  describe('.ping(cb)', function() {
    it('should return true for valid connection', function(done) {
      db.ping(done);
    });

    it('should return descriptive error on ECONNREFUSED', function(done) {
      var ds = getDataSource({
        host: 'localhost',
        port: 4 // unassigned by IANA
      });
      ds.ping(function(err) {
        (!!err).should.be.true;
        err.code.should.equal('ECONNREFUSED');
        done();
      });
    });
  });

// patch for test/init.js
-global.getDataSource = global.getSchema = function () {
-  var db = new DataSource(require('../'), config);
+global.getDataSource = global.getSchema = function (customConfig) {
+  var db = new DataSource(require('../'), customConfig || config);

See also strongloop/strong-arc#135.

@raymondfeng

Connection closed

Attempting to grab a large data set from mongodb via REST API [ get : http://localhost:3000/api/someModel ] - should return a roughly 8mb string of JSON. This causes an error in mongodb connection pool, for which I'm not sure why --

{
  "error": {
    "name": "Error",
    "status": 500,
    "message": "parseError occured",
    "stack": "Error: parseError occured\n    at null.<anonymous> (/Users/Eric/Code/spry/vbox/projects/loopback/poleApp/server/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/connection/connection_pool.js:192:34)\n    at EventEmitter.emit (events.js:98:17)\n    at Socket.<anonymous> (/Users/Eric/Code/spry/vbox/projects/loopback/poleApp/server/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/connection/connection.js:410:20)\n    at Socket.EventEmitter.emit (events.js:95:17)\n    at Socket.<anonymous> (_stream_readable.js:745:14)\n    at Socket.EventEmitter.emit (events.js:92:17)\n    at emitReadable_ (_stream_readable.js:407:10)\n    at emitReadable (_stream_readable.js:403:5)\n    at readableAddChunk (_stream_readable.js:165:9)\n    at Socket.Readable.push (_stream_readable.js:127:10)"
  }
}

The problem is the node.js server stays responsive, but I can no longer query the mongodb -- there's actually no error posted to the console. Any ideas?

loopback-connector-mongodb version: "1.3.0",
Loopback-connector version : 1.0.0

Loopback models and "date" property type issue.

Hi massive, i'm pretty new in nodejs, so probably i just missed something.

There is model defined:

  "name": "landingUser",
  "base": "PersistedModel",
  "strict": false,
  "idInjection": false,
  "properties": {
    "email": {
      "type": "string",
      "required": true,
      "index": true
    },
    "created": {
      "type": "date"
    }
  },
...

I use AngularJS on front end to save record into mongo:

                var apiResponse = LandingUser.create({email: $scope.user.email});
                apiResponse.$promise.then(
                    function(value, responseHeaders) { // if success

And i want to save current time into database (like default=datetime.utcnow in SQLAlchemy) so a added this hook into model:

LandingUser.definition.properties.created.default = Date.now;

Date.now or Timestamp it does not matter, in both cases see:

{ "_id" : ObjectId("54d65c97be1c481a79552dd1"), "email" : "[email protected]", "created" : 1423334551689 }
{ "_id" : ObjectId("54d65f4abe1c481a79552dd2"), "email" : "[email protected]", "created" : 1423335242802 }
{ "_id" : ObjectId("54d65fb2be1c481a79552dd3"), "email" : "[email protected]", "created" : 1423335346493 }
{ "_id" : ObjectId("54d6600cbe1c481a79552dd4"), "email" : "[email protected]", "created" : 1423335436174 }

It's not a dates it's just a numbers. I tried to google but with little success. After i even checked MongoDB driver docs (https://www.npmjs.com/package/mongodb#data-types) and tried to resolve using this:

    var mongo = require('mongodb');
    ....

    LandingUser.beforeCreate = function(next, data)
    {
        data.created = new mongo.Timestamp();
        next();
    };

Miracle happened i got ISODate() which is correct native representation of dates:

{ "_id" : ObjectId("54d670db32ad6cc826e610da"), "email" : "[email protected]", "created" : ISODate("1999-12-31T17:00:00Z") }

Is it a bug or feature? Can loopback-connector-mongodb serialize objects to native mongodb format or i should handle this manually?

$setOnInsert for default model values

When using default values in a model and creating/upserting a model instance without specifying these values, the mongodb connector stores the document as-is, and does not fetch the default values to use it with $setOnInsert.
http://docs.mongodb.org/manual/reference/operator/update/setOnInsert/#up._S_setOnInsert

Exemple:
My model defined as:

{
  "name": "Device",
  "plural": "Devices",
  "base": "PersistedModel",
  "properties": {
    "deviceId": {
      "type": "string",
      "id": true,
      "required": true
    },
    "culture": {
      "type": "string"
    },
    "phoneNumber": {
      "type": "string"
    },
    "imei": {
      "type": "string"
    },
    "createdAt": {
      "type": "date"
    },
    "updatedAt": {
      "type": "date"
    }
  }
}

And using default value for createdAt with:

Device.definition.rawProperties.createdAt.default =
Device.definition.properties.createdAt.default = function() {
    return new Date();
};

And creating a model instance with:

this.upsert({
            'deviceId': 'PHONE_UNIQUE_ID',
            'culture': 'EN_UK',
            'imei': 'UNIQUE_IMEI',
            'phoneNumber': '+44940395039',
            'updatedAt': new Date()
        });

The resulting data in the mongodb database is:

{
    "_id": "PHONE_UNIQUE_ID",
    "culture": "EN_UK",
    "phoneNumber": "+44940395039",
    "imei": "UNIQUE_IMEI",
    "updatedAt": {
        "$date": "2014-11-26T13:53:43.783Z"
    }
}

As you can see, the createdAt value is missing. If I setup the memory connector, createdAt is here as expected.

Getting wrong filter result with MongoDB relationship

Hi,

I have created a model class using the command .

slc lb model privileges -i --data-source mongo --connecor mongodb

Then I have edited the models.json file as shown below.

"privileges": {
"options": {
"relations": {
"user": {
"type": "belongsTo",
"model": "user",
"foreignKey": "user_id"
}
}
},
"properties": {
"user_id": {"type": "string"},
"privilege_user_management": { "type": "string"},
"privilege_country_management": {"type": "string"},
"privilege_menu_management": { "type": "string"}
},
"public": true,
"dataSource": "mongo"

}

I am able to insert data to db using user API as well as privileges API. But I am getting wrong response for filter result.

http://0.0.0.0:3000/api/privileges/findOne?filter={%22user_id%22:%22536b5c0b6e2af02903dd0697%22}

{
"user_id": "536b526a808f9cd902aae749",
"privilege_user_management": "C,R,U,D",
"privilege_country_management": "C,R,U,D",
"privilege_menu_management": "C,R,U,D",
"id": "536b56c56e2af02903dd0694"

}

The response getting back from API corresponds to the first entry in the privilege table. I am getting same result for all the user_id in DB.

Please let me know, whether I am doing something wrong? Do I need to do anything else to implement this method?

I have not written any logic for the model class I have created.

Property value `undefined` is converted to `null`

When a model property defined in the model schema with the value undefined is persisted to MongoDB, the value is converted to null.

This is inconsistent with other connectors like MySQL, as they preserve undefined property values as `undefined.

The fix will IMO break backwards compatibility, thus it will need a new major version.

Test cases for test/mongodb.test.js to reproduce the problem. We may want to move these tests to the juggler's shared suite run by all connectors:

it('create should preserve `undefined` fields', function(done) {
  PostWithStringId.create(
    { title: 'a-title', content: undefined },
    function(err, created) {
      if (err) return done(err);

      created.toObject().should.eql({
        id: created.id,
        title: 'a-title',
        content: undefined
      });

      PostWithStringId.findById(created.id, function(err, found) {
        if (err) return done(err);
        found.toObject().should.eql({
          id: created.id,
          title: 'a-title',
          content: undefined
        });
        done();
      });
    });
});

it('updateOrCreate should preserve `undefined` fields', function(done) {
  PostWithStringId.create(
    { title: 'a-title', content: undefined },
    function(err, instance) {
      if (err) return done(err);
      instance.toObject().should.eql({
        id: instance.id,
        title: 'a-title',
        content: undefined
      });

      PostWithStringId.updateOrCreate(
        { id: instance.id, title: 'updated title' },
        function(err, updated) {
          if (err) return done(err);
          updated.toObject().should.eql({
            id: instance.id,
            title: 'updated title',
            content: undefined
          });
          done();
        });
    });
});

@raymondfeng You have mentioned that we should strip undefined properties from the data. AFAIK, juggler's lib/dao.js is already doing that, see the method removeUndefined in lib/utils.js. This makes the problem even more puzzling.

include.test.js not imported (commented out) - fails

For some reason, the include.test.js is commented out, and it turns out that some of these tests fail.

Also, common.batch.js doesn't run all tests, manipulation.test.js is missing for example (but will run fine when added).

'order' field added to query object in MongoDB.prototype.all and never removed

This causes all of my queries with (for example) updateAll to fail.

For example...

MongoDB.prototype.all(model, filter, callback)

If the filter parameter passed into the all method is { "username" : "Terekhov" }, then by the time it gets to

MongoDB.prototype.updateAll(model, where, data, cb)

...the object looks like this:

{
    "username": "Terekhov",
    order:
    [
        "id"
    ]
}

My solution was to replace the use of filter.order with a tempOrder variable, like so:

  var order = {};
  var tempOrder = '';
  if (!tempOrder) {
    var idNames = this.idNames(model);
    if (idNames && idNames.length) {
      tempOrder = idNames;
    }
  }
  if (tempOrder) {
    var keys = tempOrder;
    if (typeof keys === 'string') {
      keys = keys.split(',');
    }
    for (var index = 0, len = keys.length; index < len; index++) {
      var m = keys[index].match(/\s+(A|DE)SC$/);
      var key = keys[index];
      key = key.replace(/\s+(A|DE)SC$/, '').trim();
      if(key === idName) {
        key = '_id';
      }
      if (m && m[1] === 'DE') {
        order[key] = -1;
      } else {
        order[key] = 1;
      }
    }
  } else {
    // order by _id by default
    order = {_id: 1};
  }
  cursor.sort(order);

However, it's entirely possible I just don't understand the reason to include the order property in the query object.

  1. If this is so, what is the reason and how can I avoid it polluting my searches in updateAll?
  2. If not, thoughts on the above fix, seem reasonable?

Default sort order may impact performance on some queries

If a sort order is not specified a default one by _id is used. This may impact the performance of some queries where a sort is neither needed nor specified upfront. For example, using model.findOne results in the MongoDB connector calling db.collection.find(query).sort({_id:1}).limit(1)
If the sort part is skipped the result is returned almost immediately using the appropriate indices. However, having the sort in there causes the entire query to slow down. In my case there is only one record matching the given criteria and the sort is not needed.

Would you consider making the sort order explicit?

Default timeout value too low

Hi,

I am occasionally receiving "Connection fails: { [MongoError: no valid seed servers in list] name: 'MongoError', message: 'no valid seed servers in list' } " when instances bootup (app instances and mongo cluster are in different datacentres). Mongo is hosted at Compose.io

I suspect it's a timeout on the initial connection.

It would be nice to be able to give connection arguments directly to the driver through config.json

Pleas advise and many thanks!

Cannot read property value of null

On line 382 of mongodb.js, result could be null if err is returned in callback from mongodb.

this.collection(model).findAndModify({
    _id: oid
  }, [
    ['_id', 'asc']
  ], data, {upsert: true, new: true}, function (err, result) {
    if (self.debug) {
      debug('updateOrCreate.callback', model, id, err, result);
    }
    var object = result.value;
    if (!err && !object) {
      // No result
      err = 'No ' + model + ' found for id ' + id;
    }
    self.setIdValue(model, object, id);
    object && idName !== '_id' && delete object._id;
    callback && callback(err, object);
  });

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.