Giter Site home page Giter Site logo

Comments (9)

mrmlnc avatar mrmlnc commented on July 22, 2024 1

Thanks for clarifying the problem! So, we well work on the solution here:

Also you can follow to #42.

from fast-glob.

mrmlnc avatar mrmlnc commented on July 22, 2024

Hello, @robrichard,

I will try to reproduce right now.

from fast-glob.

mrmlnc avatar mrmlnc commented on July 22, 2024

Confirm. This is related to fast-glob.

Steps to reproduce:

  1. Install very «big» packages
$ npm i fast-glob iconv-lite iconv browserify autoprefixer eslint tslint jake xo xterm webpack ava jest babel babel-cli rollup relay relay-compiler
  1. Run script from topic.
Node.js 4.8.6
/Users/mrmlnc/Documents/Tests/test-big-repo/node_modules/fast-glob/out/providers/readdir.js:53
        throw err;
        ^

RangeError: Maximum call stack size exceeded
    at Object.fs.lstatSync (fs.js:839:18)
    at Object.exports.lstat (/Users/mrmlnc/Documents/Tests/test-big-repo/node_modules/readdir-enhanced/lib/sync/fs.js:58:20)
    at stat (/Users/mrmlnc/Documents/Tests/test-big-repo/node_modules/readdir-enhanced/lib/stat.js:20:6)
    at DirectoryReader.processItem (/Users/mrmlnc/Documents/Tests/test-big-repo/node_modules/readdir-enhanced/lib/directory-reader.js:168:3)
    at /Users/mrmlnc/Documents/Tests/test-big-repo/node_modules/readdir-enhanced/lib/sync/for-each.js:14:5
    at Array.forEach (native)
    at Object.syncForEach [as forEach] (/Users/mrmlnc/Documents/Tests/test-big-repo/node_modules/readdir-enhanced/lib/sync/for-each.js:13:9)
    at /Users/mrmlnc/Documents/Tests/test-big-repo/node_modules/readdir-enhanced/lib/directory-reader.js:80:12
    at once (/Users/mrmlnc/Documents/Tests/test-big-repo/node_modules/readdir-enhanced/lib/once.js:17:17)
    at Object.exports.readdir (/Users/mrmlnc/Documents/Tests/test-big-repo/node_modules/readdir-enhanced/lib/sync/fs.js:19:5)
Node.js 6.12.3, 8.9.0, 9.4.0
FATAL ERROR: v8::ToLocalChecked Empty MaybeLocal.
 1: node::Abort() [/usr/local/nvs/node/9.4.0/x64/bin/node]
 2: node::FatalTryCatch::~FatalTryCatch() [/usr/local/nvs/node/9.4.0/x64/bin/node]
 3: v8::V8::ToLocalEmpty() [/usr/local/nvs/node/9.4.0/x64/bin/node]
 4: node::ReadDir(v8::FunctionCallbackInfo<v8::Value> const&) [/usr/local/nvs/node/9.4.0/x64/bin/node]
 5: v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo<v8::Value> const&)) [/usr/local/nvs/node/9.4.0/x64/bin/node]
 6: v8::internal::MaybeHandle<v8::internal::Object> v8::internal::(anonymous namespace)::HandleApiCallHelper<false>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::FunctionTemplateInfo>, v8::internal::Handle<v8::internal::Object>, v8::internal::BuiltinArguments) [/usr/local/nvs/node/9.4.0/x64/bin/node]
 7: v8::internal::Builtin_Impl_HandleApiCall(v8::internal::BuiltinArguments, v8::internal::Isolate*) [/usr/local/nvs/node/9.4.0/x64/bin/node]
 8: 0x3633e07042fd
Abort trap: 6

from fast-glob.

mrmlnc avatar mrmlnc commented on July 22, 2024

Closed in favor of facebook/relay#2295.

from fast-glob.

jrylan avatar jrylan commented on July 22, 2024

I'm also hitting this exact error using [email protected] and [email protected].

from fast-glob.

mrmlnc avatar mrmlnc commented on July 22, 2024

Hello, @jrylan,

You can provide find . | wc -l and tree -L 1 output for your directory? And please file answer to #42 (comment) question :)

from fast-glob.

jrylan avatar jrylan commented on July 22, 2024

@mrmlnc Thank you so much for all the work with this project and the quick response!

Answers to your questions:

Result of find . | wc -l:
118369

Result of tree -L 1:

├── LICENSE
├── README.md
├── apps
├── commands
├── config.ts
├── data.json
├── dev
├── node_modules
├── package-lock.json
├── package.json
├── packages
├── projectPaths.ts
├── schema.json
├── setup.js
├── sites
├── tsconfig.json
└── tslint.json

6 directories, 11 files

I also have 16GB of RAM on my machine and at the time 9GB of RAM was available.

In this particular case, the pattern I'm providing to fast-glob is not even recursive, I'm basically doing the following:

const packages = fastGlob.sync<string>('[a-z]*', {cwd: PACKAGES_DIR, onlyDirectories: true})

from fast-glob.

mrmlnc avatar mrmlnc commented on July 22, 2024

@jrylan, @robrichard,

Please, try to use npm i fast-glob@next --force for your broken cases. I tried to fix the problem with large directories, temporarily switching to the readdir-enhanced fork.

from fast-glob.

Papi3232 avatar Papi3232 commented on July 22, 2024

`root@Synology:/opt/iobroker# npm i fast-glob@next --force
npm WARN using --force I sure hope you know what you are doing.
npm WARN read-shrinkwrap This version of npm is compatible with lockfileVersion@1, but package-lock.json was generated for lockfileVersion@2. I'll try to do my best with it!
[..................] \ fetchMetadata: sill removeObsoleteDep removing [email protected] from the tree as its been replaced by a newer version or is no longer required
npm ERR! Error while executing:
npm ERR! /usr/bin/git ls-remote -h -t ssh://[email protected]/mrmlnc/readdir-enhanced.git
npm ERR!
npm ERR! error: cannot run ssh: No such file or directory
npm ERR! fatal: unable to fork
npm ERR!
npm ERR! exited with error code: 128

npm ERR! A complete log of this run can be found in:
npm ERR! /opt/iobroker/.npm/_logs/2023-01-15T06_52_42_319Z-debug.log
root@Synology:/opt/iobroker#
root@Synology:/opt/iobroker# npm i fast-glob@next --force`

I'm afraid that won't work

from fast-glob.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.