sebelga / gstore-node Goto Github PK
View Code? Open in Web Editor NEWGoogle Datastore Entities Modeling for Node.js
License: Apache License 2.0
Google Datastore Entities Modeling for Node.js
License: Apache License 2.0
With the new gcloud-node update, they changed the whole way transactions are handled.
This means the runInTransaction alias is no longer valid, and the internal transaction handling should be updated to support gcloud v0.37.0
I've been trying to query and re-save a bunch of entities lately, I noticed something that causes a little friction.
When doing data validation for an entity property which has a type of gcloud.datastore.geoPoint
, it will fail, even if the structure of the date is completely correct. Such as:
{
latitude: 37.305885314941406,
longitude: -89.51815032958984
}
It would be great if it allowed for both of these use cases- or perhaps auto-converted the data to the correct gcloud.datastore.geoPoint
object automatically if the structure is fine. As the data returned from a query is just plain and it doesn't convert back easily.
Because of this function:
function parseId(id) {
return isFinite(id) ? parseInt(id, 10) : id;
}
The call User.get("1")
tries to find an entity with a numeric ID instead of a String ID.
This is great for a codebase written from scratch, but causes problems when dealing with legacy.
I'm reading the gstore-node docs & I'm not clear on how the __override property on discussed here https://sebelga.gitbooks.io/gstore-node/content/middleware-hooks/pre-hooks.html
works (and what is it for)?
can you elaborate & give an example?
Im playing around with GAE, datastore and came across this package. Taking this along lines of mongoose is really good.
One of the major complaints of GAE datastore seems to be you will be locked into google-eco-system and cant migrate to other database engines.
So what do you think about having a 'new or this' package which will act as an abstraction layer on top of datastore, mongoose and mysql. One package - talk to mongoose or datastore or mysql
This means an application 'migration' is easy and more users.
I havent used mongoose or datastore intensively - so wanted to know what you feel about this proposal ?
By running the following code:
db
.ProjectModel
.query()
.select(['name', 'updateOn']);
You get the following error
Error: no matching index found. recommended index is:<br/>- kind: Project<br/> properties:<br/> - name: name<br/> - name: updateOn<br/>
at /Users/cristyansepulveda/visualive_website/node_modules/grpc/src/node/src/client.js:569:15 code: 412, metadata: Metadata { _internal_repr: {} } }
model:
const ProjectSchema = new Schema({
name: {type: 'string'},
updateOn: {type: 'datetime', default: gstore.defaultValues.NOW},
});
For one of my kinds, I store custom string as the ID in the format of "user1:user2" (where user1 and user2 are numerical IDs) for easy and fast retrieving. To create this string, I'm using util.format
and passing it to the second parameter of a new model instance. When I save this to the datastore and check to see how it saved, it shows only "user1" as a numerical ID. How can I force gstore to save the ID as a string even though one is passed to the new instance?
For now, I'm saving through gcloud which works fine.
const studentSchema = new Schema({
name: { joi: Joi.string().required() },
scores:{ joi: Joi.array().required(), excludeFromIndexes: true }
})
I get this error
{ Error: Exclude from indexes cannot be set on a list value
at /Users/Mike/dev/current/datastore/node_modules/grpc/src/node/src/client.js:554:15 code: 400, metadata: Metadata { _internal_repr: {} } }
When giving a non-existent key to model.get()
, I get
UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): TypeError: Cannot read property 'Symbol(KEY)' of undefined
and I cannot catch
the rejection of Promise, like
User.get('NON-EXISTENT-KEY')
.then((entities) => {
// an entity found
})
.catch((err) => {
// never happen
})
I am using gstore-node 0.8.0
, google-cloud 0.44.2
and node v6.9.1
. I guess something happening here?
https://github.com/sebelga/gstore-node/blob/master/lib/model.js#L110
It would be awesome to be able to plug in a custom Promise library (i.e. bluebird), just like you would with mongoose.
I know you can set a custom lib for google-cloud/datastore, but it would be even cooler to have this feature for gstore-node
STR
const gstore = require('gstore-node');
const datastore1 = require('@google-cloud/datastore')({
projectId: 'my-google-project-id',
namespace: `1`
});
// Then connect gstore to the datastore
gstore.connect(datastore1);
const datastore2 = require('@google-cloud/datastore')({
projectId: 'my-google-project-id',
namespace: `2`
});
gstore.connect(datastore2);
The second datastore datastore2
overwrites the first one and therefore calls to delete
operation, for example, on datastore1
are done on namespace2
instead and therefore fails. The issue is that we export new Gstore
at https://github.com/sebelga/gstore-node/blob/master/lib/index.js#L135 instead of ust Gstore
so there is no way to create two separate instances of Gstore
I have been investigating a failing test in our test suite and am struggling with a test that basically validates some params and calls Model.update()
. The call to Model.update()
ends up throwing an unhandled rejection:
UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): TypeError: Cannot read property '0' of undefined
After some debugging I found the error being thrown at model.js:258 (onEntityUpdated)
, where updateData
is undefined
. This sort of makes sense as model.save()
doesn't return the created entity, but this seems like a common use case and I figured the tests would cover a case like this.
I made a bare test that has this issue. DevApp has just 2 fields: appName: string
, type: number
:
const newName = 'New Test App Name';
const fApp = new DevApp({appName: 'f-app', type: 0});
return fApp.save(function (err) {
assert.ifError(err);
assert.equal(fApp.appName, 'f-app');
return DevApp.update(fApp.entityKey.id, {appName: newName})
.then((retApp) => {
assert.equal(retApp.appname, newName);
done();
});
This is a bit contrived as I couldn't get a sane, promise-only version working either:
return fApp.save()
.then(() => {
return DevApp.update(fApp.entityKey.id, {appName: newName});
})
.then((retApp) => {
assert.equal(retApp.appname, newName);
done();
})
.catch(console.error);
This errors with: { code: 400, metadata: Metadata { _internal_repr: {} } }
I would like to be able to save a model, update it, and see the changes updated (all in a promise chain).
Am I doing something unreasonable or wrong? Any help would be appreciated.
First off sweet project cant wait to get it working!
I am trying to get the minimal setup working and when I run my seed function to add a page to Datastore I get: cannot read property 'key' of undefined.
I have tried every different variation I could from going through the docs, and not sure if there a way I can define the key on a field in the model.
My create function
Note
/* eslint no-unused-vars: 0 */
const gstore = require('gstore-node');
const Page = require('../../../models/datastore/pageSchema');
const createPage = (req, res, next) => {
const entityData = Page.sanitize(req);
const page = new Page(entityData, entityData._id);
page.save()
.then(entity => res.json(entity.plain()))
.catch(err => next(err));
};
And the full error:
H:\Coding\Projects\react-starter-kit\node_modules\gstore-node\lib\entity.js:158
return namespace ? self.gstore.ds.key({ namespace, path }) : self.gstore.ds.key(path);
^
TypeError: Cannot read property 'key' of undefined
at createKey (H:\Coding\Projects\react-starter-kit\node_modules\gstore-node\lib\entity.js:158:80)
at ModelInstance.Entity (H:\Coding\Projects\react-starter-kit\node_modules\gstore-node\lib\entity.js:25:30)
at Model (H:\Coding\Projects\react-starter-kit\node_modules\gstore-node\lib\model.js:21:1)
at ModelInstance (H:\Coding\Projects\react-starter-kit\node_modules\gstore-node\lib\model.js:23:31)
at createPage (H:/Coding/Projects/react-starter-kit/src/data/queries/googleDatastore/page/controller.js:23:19)
at Object.<anonymous> (H:/Coding/Projects/react-starter-kit/src/data/seed/index.js:20:1)
at Module._compile (module.js:570:32)
at loader (H:\Coding\Projects\react-starter-kit\node_modules\babel-register\lib\node.js:144:5)
at Object.require.extensions.(anonymous function) [as .js] (H:\Coding\Projects\react-starter-kit\node_modules\babel-register\lib\node.js:154:7)
at Module.load (module.js:487:32)
error Command failed with exit code 1.
My model
const gstore = require('gstore-node');
const Schema = gstore.Schema;
const pageSchema = new Schema({
_id: {
type: 'string',
required: true
},
path: {
type: 'string',
optional: true
},
public: {
type: 'boolean',
required: true,
default: false,
excludeFromIndexes: true
},
... rest of fields
});
const listSettings = {
limit: 25,
order: { property: '_id' },
};
pageSchema.queries('list', listSettings);
module.exports = gstore.model('Page', pageSchema);
As google-cloud datastore lib has changed their entity response format from
{ key: someKey, data: someData }
to
{ entityData }
in @google-cloud/datastore 0.5.0 (The change is actually in this commit ), this clearly will breaks model creation. For example, trying to fetch an entity with a key would result in the following error as entity.key in the response is no longer presence.
/MyProject/node_modules/gstore-node/lib/serializers/datastore.js:23
data.id = entity.key.path[entity.key.path.length - 1];
^
TypeError: Cannot read property 'path' of undefined
May I ask whether there is any plan to support datastore 0.5.0 soon?
Hey there! I think this is a bug, as the documentation does not mention having to create indexes for queries.
I have an example model that looks like this:
const gstore = require('gstore-node')();
const Schema = gstore.Schema;
const itemSchema = new Schema({
name: { type: 'string', required: true },
arrOfObjs: {
type: 'array',
validate: {
rule: validateArrOfObjs,
}
},
createdOn: { type: 'string', default: gstore.defaultValues.NOW, write: false, read: false }
});
function validateArrOfObjs(obj, validator) {
if (!Array.isArray(obj)) {
return false;
}
var allObjs = obj.every((item) => {
return item !== null && typeof item === 'object';
});
return allObjs;
};
const itemQuerySettings = {
order : { property: 'createdOn', descending: true }, // descending defaults to false and is optional
select : ['name', 'createdOn']
};
// Add settings to schema
itemSchema.queries('list', itemQuerySettings)
const Item = gstore.model('Item', itemSchema);
export default Item;
If I create a few items and then try to list them with the following endpoint:
const listItems = (req, res) => {
Item.list()
.then((response) => {
res.json({
entities: response.entities,
nextPageCursor: response.nextPageCursor
});
});
};
I get the following error:
(node:14987) UnhandledPromiseRejectionWarning: Error: no matching index found. recommended index is:
- kind: Item
properties:
- name: createdOn
direction: desc
- name: name
at ./node_modules/grpc/src/client.js:554:15
On node v9.3.0, Yarn package.json:
{
"name": "gstore-poc",
"version": "1.0.0",
"description": "POC for gstore library",
"main": "index.js",
"scripts": {
"api": "babel-node server.js"
},
"author": "",
"license": "ISC",
"dependencies": {
"babel-preset-env": "^1.6.1",
"body-parser": "^1.18.2",
"express": "^4.16.2",
"gstore-node": "^2.1.4",
"mongoose": "^5.0.0-rc1",
"morgan": "^1.9.0"
},
"devDependencies": {
"babel-cli": "^6.26.0",
"babel-core": "^6.26.0"
},
"babel": {
"presets": [
"env"
]
}
}
I get an error below when deleting a table with more than 500 entities. This is because the deleteAll
function seems to be fetching the entries first and then trying to delete them in one go instead of breaking them down to a batch of 500. Workaround is do the same by the caller but that makes deleteAll
function unnecessary.
cannot write more than 500 entities in a single call
Hello,
while looking at the code to investigate why Gstore was returning only the first error as opposed to all of them (like ORMs normally do), I found this piece of code inside validateEntityData()
in model.js:
if (!validate.success) {
delete validate.success;
return validate.errors[Object.keys(validate.errors)[0]];
}
As a result, Model.save()
on validation failure returns only the first error.
Is there a reason for that? Couldn't we just return validate.errors
in its entirety?
Hello,
First of all thank you for your work.
I would have liked to know if you will soon migrate to typescript?
Thank you in advance
Hi there,
I see you've got the equivalent of the datastore.save( entity )
method, which we can access in this library the other way round with entity.save();
- returning a promise or passing in a callback if we like to save a single element.
I'm wondering if there is anything which would be the equivalent of datastore.save( entities[] )
For example, in my code:
async function resaveEntities(model, cursor = null) {
const query = model.query().limit(50);
if (cursor) {
query.start(cursor);
}
const result = await query.run();
const response = result[0];
log.info(`fetched ${response.entities.length} entities of ${model.entityKind}`);
// CODE HERE TO RE-SAVE ALL THE FETCHED ENTITIES
if (response.nextPageCursor) {
resaveEntities(model, response.nextPageCursor);
} else {
log.info(`END of re-saving of [${model.entityKind}]`);
}
}
The reason I ask is because if I'm not mistaken, batch saving large groups of entities (as I'm trying to do here, basically running through all the entities of a kind and re-saving them- in order to re-index their properties if I changed the schema) can be optimised when its done in groups instead of individually every time.
I suppose I could use the google-cloud-node
library to re-save them, but would I not lose the schema then? When I look at the data that is being returned in the query, it appears that it's just a bunch of plain data- no extra schema information or even functions so I'm unsure if entity.save()
would actually even work either.
I'm going to play around with it for a bit. But let me know if there is a solution I may be overlooking.
hello, i have an entity with the ff model.
model
first_name: { type: 'string' },
last_name: { type: 'string' },
address: { type: 'object' },
and as I save, I save the address as
{ number: 001, street: example street, City: example City }
AFTER I SAVE IT, i will update the City only...
how should i do that? because as i update it, it will overwrite all the values with
street: { City: overwriteValuehere }
i want it instead street: { number: 001, street: example street, City: overwriteValuehere }
tnx!
Are there any plans to add support for named keys? It seems that createKey
is always expecting ID to either be a numeric string or number, but I would like to be able to create an entity that builds keys using the string without parsing it. It would be nice to define the type of key when defining a schema.
On a related note, since the ID is being parsed with parseInt
, IDs that are larger than 253 -1 will not parse properly ๐ข . This could be a problem for anyone looking to interoperate with IDs generated from languages that properly support 64-bit integers. Instead of parsing as JS numbers, they should be wrapped with datastore.int()
.
From the code it looks like the global save https://github.com/sebelga/gstore-node/blob/master/lib/index.js#L100 is a wrapper around google-cloud/datastore's save method and doesn't use transactions.
If I have a array
"test": [{"organizationName":"strge","familyName":"raju"},{"organizationName":"sand","familyName":"sand"}]
want to filter in organizationName=strge and familyName=sand and get only if both filters match in particular object not in different object of a array. #
Isn't this inconsistent?
Model.findOne({foo: 'bar'}, (err, entity) => {
// entity != Array
})
versus
Model.findOne({foo: 'bar'}).then((entities) => {
// entites == Array
})
Datastore refuses 'string' properties greater than 1500 characters. It will accept 'text' properties of any length. How do we set the property type to 'text'?
> var Test = require('./server/apis/test.model.js')
datastore.js
shioh.productArray.model.js
undefined
> var str = ''
undefined
> for (let i = 0; i<1502; i++){str = str.concat('a')}
'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa'
> var entity = new Test({text : str})
undefined
> entity
ModelInstance {
className: 'Entity',
schema:
Schema {
instanceOfSchema: true,
methods: {},
statics: {},
virtuals: {},
shortcutQueries: {},
paths: { text: {} },
callQueue: { model: {}, entity: {} },
options: { validateBeforeSave: true, queries: [Object] },
__meta: {} },
excludeFromIndexes: [],
entityKey: Key { namespace: undefined, kind: 'test', path: [Getter] },
entityData: { text: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaa' },
pre: [Function: pre],
post: [Function: post],
hook: [Function: hook],
__setupHooks: [Function: __setupHooks] }
> entity.save()
Promise { <pending> }
> (node:12000) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error: The value of property "text" is lo
nger than 1500 bytes.
Hi,
I'm having some issues working with the library. Maybe I'm initializing in a bad way the gstore-node?
Saving is not working, or nothing happens at all when this method is executed:
var data = ClientApp.sanitize(req.body);
console.log(data);
var clientApp = new ClientApp(data,data.appName);
clientApp.save().then(function() {
this.datastoreEntity().then((data) => {
var entity = data[0];
console.log(entity);
});
}).catch((err) => {
console.log(err);
});
Here are the files:
clientApp.controller.js
var ClientApp = require('../models/clientApp');
function create(req, res) {
var data = ClientApp.sanitize(req.body);
console.log(data);
// {
// appName: 'Yikes',
// androidBundleId: 't.t.t',
// iosBundleId: 'a.a.a'
// }
var clientApp = new ClientApp(data, data.appName);
var data = ClientApp.sanitize(req.body);
console.log(data);
var clientApp = new ClientApp(data,data.appName);
clientApp.save().then(function() {
this.datastoreEntity().then((data) => {
var entity = data[0];
console.log(entity);
});
}).catch((err) => {
console.log(err);
});
}
module.exports = {
create: create
};
(Model)
clientApp.js
var uuid = require('uuid');
var datastore = require('@google-cloud/datastore')({
projectId: 'tttttttt',
keyFilename: './keyfile.json'
});
var gstore = require('gstore-node');
gstore.connect(datastore);
var Schema = gstore.Schema;
var schema = new Schema({
appName: {
type: 'string',
required: true,
read: false
},
androidBundleId: {
type: 'string',
required: true,
read: false
},
iosBundleId: {
type: 'string',
required: true,
read: false
},
modified: {
type: 'boolean',
default: false,
read: false,
write: false
},
createdOn: {
type: 'datetime',
default: gstore.defaultValues.NOW,
write: false,
read: false
},
appKey: {
type: 'string',
default: uuid.v4(),
write: false
}
});
schema.virtual('timestamp').get(function() {
// the scope (this) is the entityData object
return this.createdOn;
});
var ClientApp = gstore.model('ClientApp', schema);
module.exports = ClientApp;
If you call the Model.get
method with an array consisting of only one ID, nothing seems to be returned.
For example, calling the following returns an array of 2 entities:
MyModel.get([10, 15])
However calling the following doesn't return anything:
MyModel.get([10])
I'm trying to use a custom validator to verify that a field is an array containing one or more objects. However, the custom validator function is getting passed a string as input instead of the original array being passed in. It seems as if the array is having .toString() called on it before it's passed into the validator. However, if I remove the custom validator function then the correct values are saved to the database.
Code, output, package, and node version information below.
Schema and validator:
const itemSchema = new Schema({
name: { type: 'string', required: true },
arrOfObjs: {
type: 'array',
validate: {
rule: validateArrOfObjs,
}
},
createdOn: { type: 'string', default: gstore.defaultValues.NOW, write: false, read: false }
});
function validateArrOfObjs(obj, validator) {
console.log("Input type:");
console.log(typeof obj);
console.log("Input is array:");
console.log(Array.isArray(obj));
console.log("Array:");
console.log(obj);
console.log("First item:");
console.log(obj[0]);
console.log("First item type:");
console.log(typeof obj[0]);
if (!Array.isArray(obj)) {
return false;
}
var allObjs = obj.every((item) => {
return item !== null && typeof item === 'object';
});
return allObjs;
};
const Item = gstore.model('Item', itemSchema);
Testing code:
var arr = [
{ name: 'foo' },
{ name: 'bar' },
{ name: 'baz' }
];
var item = new Item( {name: 'test123', arrOfObjs: arr} );
console.log("Array:");
console.log(item.arrOfObjs);
console.log("Array first item:");
console.log(item.arrOfObjs[0]);
console.log("Array first item type:");
console.log(typeof item.arrOfObjs[0]);
console.log("Validating!");
const { error } = item.validate();
if (error !== null) throw error;
item.save();
Output:
Array:
[ { name: 'foo' }, { name: 'bar' }, { name: 'baz' } ]
Array first item:
{ name: 'foo' }
Array first item type:
object
Validating!
Input type:
string
Input is array:
false
Array:
[object Object],[object Object],[object Object]
First item:
[
First item type:
string
yarn package.json
{
"name": "gstore-poc",
"version": "1.0.0",
"description": "POC for gstore library",
"main": "index.js",
"scripts": {
"api": "babel-node server.js"
},
"author": "",
"license": "ISC",
"dependencies": {
"babel-preset-env": "^1.6.1",
"body-parser": "^1.18.2",
"express": "^4.16.2",
"gstore-node": "^2.1.2",
"mongoose": "^5.0.0-rc1",
"morgan": "^1.9.0"
},
"devDependencies": {
"babel-cli": "^6.26.0",
"babel-core": "^6.26.0"
},
"babel": {
"presets": [
"env"
]
}
}
Node version:
node --version
v9.3.0
Hello. I have the following schema for a Pose
model:
{
name: {type: 'string', required: true},
skeleton: {values: ['HumanBody', 'HumanHand'], required: true},
...
}
When I try to create a new entity without skeleton
attribute, it's automatically picking HumanBody
as the default value:
console.log(new Pose({name: 'Whatever'}))
// will output: {name: 'Whatever', skeleton: 'HumanBody'}
Thus, the required: true
option has no effect because it's setting a default value.
This behavior is confusing, and also undesired in my case, as I need the attribute to be explicitly defined, raising a ValidationError when not given. How can I achieve this?
Thanks.
As a user of mongoose, wanting to start switching to google cloud datastore, this library is ideal for me, thank you so much for making it.
So I created a connection:
var gstore = require('gstore-node');
var gcloud = require('google-cloud')(config.googlecloud);
gstore.connect(gcloud.datastore());
Then I created a schema and made it into a model:
var gstore = require('gstore-node');
var Schema = gstore.Schema;
var urlSchema= new Schema({
name: { type: 'String', excludeFromIndexes: true },
subdomain: { type: 'String' }
});
module.exports = gstore.model('url', urlSchema);
I don't know what step I might be missing in the docs, but when I try to query my new model:
var Url = require('./url.model');
Url.query().run(function (error, results) {
console.error(error);
console.log(results);
});
I get the above error (it doesn't even make it to the console.error
line. Specifically, it points to line 924
of model.js
in gstore-node:
return self.ds.createQuery.apply(self.ds, createQueryArgs);
For some reason, the datastore is supposed to be passed in when creating the model, but is not. Given that there is a line 924 of model.js in the lib/ folder or gstore-node, at this point I'm pretty sure I'm in over my head and could use some guidance.
Did I do something wrong in the process of creating my model? Note that if I dump the Url
object, it has a ds property with a datastore. object on it. In fact, if I dump the self
object in model.js, It contains the Url
complete with the attached datastore, but for some reason self.ds
is undefined.
Hi @sebelga,
First, thanks for this beautiful project. Eases the implementation by 400%. You're awesome, keep up the good work (and have a Happy New Year ๐).
I read that you use the validator.js
- we can even see it in action in this line - but that implementation only supports simple validation rules that don't expect additional parameters (e.g. the isIn rule).
I suggest either to pass something extra on the object like:
{
validate: 'isIn',
validation_args: [ /* an array of multiple args */ ],
// ...
}
or change the validate
prop to support both string, object/array:
// simple case
{
validate: 'isEmail'
}
// complex case
{
validate: {
rule: 'isIn',
args: [ /* ... */ ]
}
// another approach
{
validate: ['isIn', arg2, arg3, arg4, ..., argN]
}
Or just say, "No we only support simple cases" :)
order
does not work in a datetime
field
const ProjectSchema = new Schema({
name: {type: 'string'},
members: {type: 'object'},
createOn: {type: 'datetime', default: gstore.defaultValues.NOW},
updateOn: {type: 'datetime', default: gstore.defaultValues.NOW},
files: {type: 'object'}
});
I tried the following query
const query =
db
.ProjectModel
.query()
.order('updateOn', {
descending: true
})
.limit(20);
It is not delivered in the expected order.
It had taken me while to figure out how to get it working and was thinking a example with returning objects they can use would be very helpful to new comers. In the Gitbook it is on the models/methods/GET documentation, but I think that adding a example to map out the entities and use plain() would be a very common requirement for people. It would also be good to add it to the readme where the other function calls are. Me a year ago probably would have never figured it out :)
My function for fetching based on a array of _id's and returning objects
If there is a more performant way to do this please let me know!
export async function getPagesBy_id(_ids) {
let response = null;
try {
response = await Page.get(_ids, null, null, null, { preserveOrder: true })
.then((entities) => {
return entities.map((entity)=> {
return entity.plain();
});
});
} catch (err) {
console.log('getPagesBy_id err', err);
}
return response;
};
I am new to both datastore
and gstore-node
. Having a hard time figuring out how to write the schema
for one-to-many
and many-to-many
relationships without using hierarchal relationships.
If we take a small example of a User
having many Address
es, how should the schema look ?
const userSchema = new Schema({
name: { type: 'string' }
});
const addressSchema = new Schema({
city: { type: 'string' }
});
I apologize for posting a question here.
The line:
For info on how to configure gcloud read the docs here.
has a broken link, thanks!
Hi!
I guess this is more of a question than an issue, but is there any easy way to convert these to JSON objects that can be returned by express routes? Thank you!
I'd like to send documentation improvements, but cannot find where it is stored in this repo.
The code for // from inside a Transaction
in https://sebelga.gitbooks.io/gstore-node/content/model/update-method.html is missing an {
.
As far as I can see now, results from a query are the raw results from the query executed by the underlying gcloud-node library? What would be the preferred way to convert these entities back to gstore-node Model objects?
I.e my use case is the following roughly:
query(..., (entities) => { // do something with entities, update some properties entity.save(); });
Hello!
Does all the "save" operations works as upsert(which will overwrite an entity if it already exists in Cloud Datastore) or is there any way to use insert(which requires that the entity key not already exist in Cloud Datastore)?
Thanks in advance.
Sorry to be blowing up your issues list but when I launch my app with these two lines:
var gcloud = require('gcloud')(configGcloud);
var datastore = gcloud.datastore({"apiEndpoint": "127.0.0.1:8343"});
It loads in about 2 seconds, if I add:
var gstore = require('gstore-node');
it goes up to 10-15 seconds, any idea why it takes so long to import or what I'm doing wrong? Thanks!
Seems like it should validate for "required: true" in the schema. validate.js doesn't accept nulls or zero-lengths so maybe those checks should be at the top of model.validate() and flip skip if value is null or zero-length after trimming.
I have upgraded my Node.js from 4.x.x to 6.11.0 and got a problem now. The entity method save()
doesn't seem to work for me anymore. The code below throws this message in console: (node:7184) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): TypeError: Cannot read property 'then' of undefined
.
const faker = require('faker');
const User = require('./../models/user.model');
const entity = new User({
displayName: faker.name.firstName() + ' ' + faker.name.lastName(),
username: faker.internet.userName()
});
entity.save()
.then(() => {
res.send('OK');
})
.catch((err) => {
res.send('Not OK');
})
;
Looks like the method is not returning a promise, so the entity doesn't get saved in the datastore. On the other side if I use the general gstore method for saving, it works perfectly.
...
var gstore = require('gstore-node');
gstore.save(entity)
.then(() => {
res.send('OK');
})
.catch((err) => {
res.send('Not OK');
})
;
Is there a problem with my code or the library needs to be fixed? Thanks.
how do I apply filter in key ?
Today I found super strange behavior
When I start the application with pm2, the save
method returns undefined
Any idea that it may be happening?
Similar to #1, but with getting instead. Getting an object by number ID works fine, so does a regular string name (e.g: 'testobject'). However, in my case, a string name of "12345:12345" returns 404 every time through gstore-node. Tested in gcloud works successfully.
Thank you
Hi,
Love what's been done here, this is just the kind of thing I've been looking for after moving from Java (using the objectify library for Google datastore) into Javascript Node development and finding the implementation a bit bare-bones in the base google-cloud library. The schema and model concepts really make it easy to work with.
The only thing I'd really love to see added is the option of having Promises returned instead of using callbacks (on the relevant such methods), such as the Model methods get, updated, delete etc. It could be added using a different name such as BlogPost.getAsync(1234)
. I like to make use of async functions and combined with promises it makes things amazingly easy to reason about.
For now I'll have to wrap all the base methods in Promises myself, but would be nice to have it out the box.
In model.js:824, the type of the property is validated. However, if the value of said property is equal to undefined, the property validation fails.
For an entity like below?
id: string
results: {
"success_rate": 90.0,
"failure_rate": 50.5
}
Can't find any effect of this. Model shows excludeFromIndexes
The field shows that it is indexed. (No other properties falsely show it)
But then its values all state not indexed. (Note the array content type)
Not sure if its anything to pay attention to as it is not stored in the indexs (n0deEdge).
The note above showed that the content type of n0deEdge is an array (which is correct), and below it shows it is a string.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.