can't save embedded entity with string length > 1500

[kanethal]

can't save embedded entity with string length > 1500

[kanethal]

I confirmed that for un-nested properties, the

excludeFromIndexes: true

property parameter allows long strings to be saved. How do we pass that goodness to nested properties? Is there such a thing as subdocs/childschema here ala mongoose.js?

var childSchema = new Schema({ name: 'string' });

var parentSchema = new Schema({
  // Array of subdocuments
  children: [childSchema],
  // Single nested subdocuments. Caveat: single nested subdocs only work
  // in mongoose >= 4.2.0
  child: childSchema
});

[sebelga]

Hello,
Yes the long string cannot be indexes in Google Datastore that's why you need to set 'excludeFromIndexes' to true.

For what I remember embeded entities are not indexed by default. Have you tried to set 'excludeFromIndexes' on the Schema to those properties?

thanks

[kanethal]

I did, I attempted a long string on a non indexed property and it stored just fine. I then put a nested object with a long string on that non indexed property and it came back with an error that the string length was too long.

…

On Aug 2, 2017 11:22 AM, "Sébastien Loix" ***@***.***> wrote: Hello, Yes the long string cannot be indexes in Google Datastore that's why you need to set 'excludeFromIndexes' to true. For what I remember embeded entities are not indexed by default. Have you tried to set 'excludeFromIndexes' on the Schema to those properties? thanks — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#71 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AIBzWUPRAY7eZD8R7hjA5OjMBEQGs_gmks5sULBUgaJpZM4Oiw4Y> .

[kanethal]

I tried this demonstrating that nested properties are subject to the 1500 character limit as 'string' properties. If these can be classified as 'text' properties then the limit would not apply perhaps? Not sure:

./server/apis/test.model.js :
const gstore = require('./datastore.js');
const Schema = gstore.Schema;
const testArray = new Schema({
    texts: {type: 'object', excludeFromIndexes: true}
    
let Model = gstore.model('test', testArray);
module.exports = Model
});

> var model = require('./server/apis/test.model.js')
datastore.js
shioh.productArray.model.js
undefined
> var obj = {a:'hello',b:{c:'hello'},d:{e:{f:(function(){var i = ''; for(let j=0;j<2000;j++){i=i.concat('a') }; return i } )()}}}
undefined
> obj
{ a: 'hello',
  b: { c: 'hello' },
  d: { e: { f: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' } } }
> ent = new model({texts:obj})
ModelInstance {
  className: 'Entity',
  schema:
   Schema {
     instanceOfSchema: true,
     methods: {},
     statics: {},
     virtuals: {},
     shortcutQueries: {},
     paths: { texts: [Object] },
     callQueue: { model: {}, entity: {} },
     options: { validateBeforeSave: true, queries: [Object] },
     __meta: {} },
  excludeFromIndexes: [ 'texts' ],
  entityKey: Key { namespace: undefined, kind: 'test', path: [Getter] },
  entityData: { texts: { a: 'hello', b: [Object], d: [Object] } },
  pre: [Function: pre],
  post: [Function: post],
  hook: [Function: hook],
  __setupHooks: [Function: __setupHooks] }
> ent.save()
Promise { <pending> }
> (node:21492) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error: The value of property "f" is longer than 1500 bytes.

[kanethal]

This appears to be an ongoing concern. It is not clear that nested properties are excluded from indexing on the basis of exclusion of the parent property.

googleapis/google-cloud-node#1916

[sebelga]

Sorry for the late answer. It seems that it is a "hot" topic from the issue you pass :) I didn't bump into this problem so I was not aware of it. As embedded entities are not indexed I thought that their properties were not indexed either and thus the 1500 length limit did not exist.

As gstore node is just a wrapper on top of google-cloud, I will follow the issue and according to the fix/solution they come with I will update the lib.

thanks for reporting! 👍

[sebelga]

Hello,
I realised yesterday that google cloud made an update to the API for embedded entities. I just published a new version of gstore node (v1.5.0) that should allow the exclusion of embedded entities properties in the indexes.

You can see the update in the docs
https://sebelga.gitbooks.io/gstore-node/content/schema/other-paremeters.html