Comments (14)
Thank you for this demo, there are two nuances:
- circular references
- HUGE amount of inherited properties provided by mongoose
So yes, it's a good idea to convert document to plain object before going deep.
Here is an illustrated version of your code:
const { eachDeep } = require('deepdash/standalone')
const mongoose = require('mongoose')
const RandomSchema = mongoose.Schema({ items: [mongoose.Schema()] })
const RandomModel = mongoose.model('random', RandomSchema)
function demo () {
const myDocument = new RandomModel({
items: [{}]
})
const has = Object.prototype.hasOwnProperty
eachDeep(myDocument.items, (v, k, p, x) => {
console.log(`${x.depth}: ${x.path}`)
if (x.isCircular) {
console.log(` - ${k} is a circular ref`)
}
if (!has.call(p, k)) {
console.log(` - ${k} is inherited property`)
return false // skip inherited props otherwize the whole universe will fall on you
}
}, { checkCircular: true })
}
demo()
So as you see even with inherited props skipped it's still pretty big.
Since v5.2 for performance reasons I switched to for .. in
way of iteration over objects instead of Lodash forOwn
and decided to not check internally if property is own.
So it was my mistake to bump only minor version, not major, because it's undocumented but breaking change.
I think I will add an option to iterate over own properties only, true by default,
so it will get previous expected behavior back.
Thank you guys @TeleMediaCC @Ilshidur for your patience and help!
from deepdash.
In v5.3.0 new option widely added ownPropertiesOnly
and it's true by default, so now all the methods should behave as they used to before v5.2.0
if ownPropertiesOnly
is false - then deepdash will also go over prototype props.
No time for docs 😢
from deepdash.
Hi, could you please provide some details how to reproduce this?
from deepdash.
Also make sure your data have no circular references or activate checkCircular option.
from deepdash.
Can't create reproduce link. Its part of a big nuxt.js projekt.
I upgraded 5.1.2 to 5.2.0 and after start immediately crashed the app. When i downgrade to 5.1.2, everything works.
from deepdash.
- Try to set checkCircular option: It probably started visiting some paths, ignored previously, and stuck in some circular ref.
- Share your usage examples: which deepdash methods, what options, amount, and structure of data?
from deepdash.
Let me know if any news please, closing for now because of no way to reproduce.
from deepdash.
I had this issue while doing a recursive lookup in a Mongoose document. I had to prevent this lib to use the internal fields like so :
const deepdash = require('deepdash/standalone');
const myMongooseObject = Model.find(...);
// `myMongooseObject.randomArray` is a Mongoose sub-schema.
deepdash.eachDeep(myMongooseObject.randomArray, ...); // Crashes.
deepdash.eachDeep(myMongooseObject.toJSON().randomArray, ...); // No crash.
I hope it can help whoever steps on this issue.
from deepdash.
Are you able to check if this works?
deepdash.eachDeep(myMongooseObject.randomArray, ()=>{}, { checkCircular: true });
from deepdash.
Oh, my apologies, I did not provide the options in my last message.
{
leavesOnly: false,
checkCircular: true,
pathFormat: 'string',
children: ['main', 'sub.main'],
}
It works with these options.
from deepdash.
Hi, thanks, but could you please check it works with these options but without converting mongo doc to json?
from deepdash.
Hi. I was able to reproduce the issue. However, I don't know where this could be from, neither whether this is an issue on my end or not.
Here's the most minimalist way I found to reproduce this behavior :
const { eachDeep } = require('deepdash/standalone');
const mongoose = require('mongoose');
const RandomSchema = mongoose.Schema({ items: [mongoose.Schema()] });
const RandomModel = mongoose.model('random', RandomSchema);
function demo() {
const myDocument = new RandomModel({
items: [{}],
});
eachDeep(myDocument.items);
}
demo();
Here's my console afterwards :
<--- Last few GCs --->
e [15516:0x45eb700] 6027 ms: Mark-sweep 2038.1 (2070.2) -> 2037.9 (2074.7) MB, 68.1 / 0.0 ms (+ 257.0 ms in 62 steps since start of marking, biggest step 13.6 ms, walltime since start of marking 332 ms) (average mu = 0.103, current mu = 0.021) finalize[15516:0x45eb700] 6416 ms: Mark-sweep 2042.7 (2074.7) -> 2042.5 (2078.2) MB, 78.0 / 0.0 ms (+ 300.9 ms in 58 steps since start of marking, biggest step 14.5 ms, walltime since start of marking 388 ms) (average mu = 0.064, current mu = 0.026) finalize
<--- JS stacktrace --->
==== JS stack trace =========================================
0: ExitFrame [pc: 0x1a5c1a2]
Security context: 0x2a489939a299 <JSObject>
1: push [0x2a489938bf11](this=0x2fc204050611 <JSArray[59]>,0x09ae6197e8f9 <Object map = 0x2971eab8cde9>)
2: getOwnChildren(aka getOwnChildren) [0x2da1d8ea3289] [/home/ncoutin/Documents/projects/roma-carts/node_modules/deepdash/private/getIterate.js:~211] [pc=0x7720ae9625d](this=0x058209e804d1 <undefined>,0x09ae6197c331 <Object map = 0x2971eab8d069>,0x0fa6...
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
1: 0x97a1c0 node::Abort() [node]
2: 0x97b189 node::OnFatalError(char const*, char const*) [node]
3: 0xaf2d0e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
4: 0xaf3089 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
5: 0xefe235 [node]
6: 0xf08b9b v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [node]
7: 0xf098b7 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
8: 0xf0af9f v8::internal::Heap::HandleGCRequest() [node]
9: 0xea5890 v8::internal::StackGuard::HandleInterrupts() [node]
10: 0x11a90b5 v8::internal::Runtime_StackGuard(int, unsigned long*, v8::internal::Isolate*) [node]
11: 0x1a5c1a2 [node]
[1] 15516 abort (core dumped) node index.js
Anyway, thank you very much for this lib. I use it at work and it really saved some of my time.
from deepdash.
I think the problem when the object contains prop name: $type
from deepdash.
I think the problem on the Mongoose.Types values.
I have the same problem with mapValuesDeep.
from deepdash.
Related Issues (20)
- omitDeep changes input on empty array HOT 1
- omitDeep doesn't omit objects HOT 7
- ERR_PACKAGE_PATH_NOT_EXPORTED HOT 3
- how to remove values from json if they exist
- Feature Request Parent path
- filterDeep not working on deeply nested object HOT 2
- Working with promises HOT 2
- Frustrating that things like `pickDeep` and `filterDeep` can only return a nested structure HOT 3
- deepdash-es not working with jest-resolve HOT 9
- deepFilter keep parents
- ESM: IIterateeContext typescript interface not exposed
- findPathDeep doesn't return first Value path
- findDeep or findDeepValue - Return object that contains the key & value
- eachDeep with childrenPath == wrong parent, parent path etc
- Bug in documentation _.findDeep*
- findDeep return types problem HOT 3
- Add contributing docs HOT 1
- findDeep is not a function HOT 1
- explicit return values, why and how does this work HOT 1
- mapValuesDeep mutates the original object if `leavesOnly: false` HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from deepdash.