package.json expects binaries in the global PATH

[verdaccio]

package.json expects binaries in the global PATH

I have some other issues with package.json. For example, why is it YAML and not JSON? And on my work Windows 7 laptop, npm install --dev seems to

• run forever and consume masses of disk space (cancelled it after 15 minutes and 1.1GB)
• produce huge amounts of output about failing to build crypt3 on Windows at a whole bunch of different filesystem levels within node_modules
• produce more warnings about deprecated and unsupported things than I have ever seen in my life in any install

I can't help thinking there is significant room for improvement. I don't know what the consensus about warnings is, but my tolerance level is low. They just cause noise, and indicate something that should have been addressed, IMO.

At the top of package.yaml there is this comment

# use "yapm install ." if you're installing this from git repository

yapm is another abandoned project by the same author. It seems to be intended to replace npm, but npm is actively developed, so why would you replace it anyway?

Also, yet more warnings:

$ npm install yapm
npm WARN installMany yapm-progress was bundled with [email protected], but bundled package wasn't found in unpacked tree
npm WARN installMany npmlog was bundled with [email protected], but bundled package wasn't found in unpacked tree
[email protected] node_modules\yapm

I vote to eliminate any dependency on yapm, and convert the package file to a conventional json format.

I got a bit further into attempting to declutter and remove some unnecessary non-standard bits, including

• removed yapm
• replaced package.yaml with standard package.json
• removed JSON5 body parser (four year old proposed extensions to JSON which have not yet happened) and reinstated standard Express body parser
• reverted Express 5 alpha to Express 4 for maximum stability
• replaced all out of date version dependencies with the latest versions to clean up the warnings

It's actually working with all these changes, but I can't run the tests yet (because Windows), so I consider it WIP for the moment.

[trentearl]

This was actually the number one change I wanted to see in this project, so I really appreciate you putting the work in 👍

I think we're probably after much the same thing, as I think you said your company relies on it, and so does mine. I would like to end up with something that's maybe a tiny bit more appropriate for corporate use in some ways. Much as I respect the original author's hard work, there are some areas where there's not really much of an upside to using non-standard components, and plenty of long term downsides. For one thing, it can scare off contributors. For another, it can make it hard to layer additional functionality on top due to compatibility issues.

One thing I hope to figure out (which is not necessarily on topic) is why it sometimes fails to retrieve some packages, but an immediate re-fetch does succeed. It is particularly flaky when being hit by multiple users at once. It would be great to sort that out. I'd quite like to get it to authenticate users who can publish locally against LDAP/AD, too. Now the gloves are off, so to speak...

[trentearl]

@steve-p-com yes this is my use case as well, and I also would like to see this project working well for internal business use.

I have also noticed the package failing to install intermittently, it seemed that it was sinopia/verdaccio failing when talking to npmjs.org and not sinopia/verdaccio talking to the client. Unfortuanetly I dont remember how I came to that conclusion. If I see it happen again I will open up an issue about it.

Out of curiosity are you using the npm 3 client?

I am not using version 3 (yet). I am actually still using 2.x that came with Node of various versions, both on my development machines, and on some linux servers running Node.js around the globe. Some of my colleagues are using 3.x though, and they haven't reported any specific issues with it.

At some point soon, I guess node will change and start to ship with version 3. It has some useful features such as automatically flattening the dependency tree. There may be 3.x things that could use extra Verdaccio support as well.

[trentearl]

At my company we exclusively use npm 3, I assumed that the networking problems were atleast somewhat related to that. I'm glad you're using npm 2, we can get better reporting since were both on different versions.

Just a progress update on this.

What is done so far

• removed yapm
• replaced package.yaml with standard package.json
• removed post-install reconstruction of npm-shrinkwrap file, which defeats the purpose of the file
• removed JSON5 body parser (four year old proposed extensions to JSON which have not yet happened) and reinstated standard Express body parser
• reverted Express 5 alpha to Express 4 for maximum stability
• replaced all out of date version dependencies in the new package.json with the latest versions to clean up the warnings
• tested standard (non-dev) installation on OS X
• confirmed all tests work
• confirmed caching works
• confirmed local publishing works
• I'm using it myself as my main registry to fully test it

There are two optional dependencies which are both problematic for different reasons.

fs-ext

• used for file-level locking
• produces deprecation warnings which I'm not optimistic will be addressed
• see baudehlo/node-fs-ext#54
• does not build on Windows, meaning you can install on Windows but not suitable for production

crypt3

• used for checking passwords
• does not build on Windows
• used in a test, so tests can't all be run on Windows
• don't know what impact it not being installed has on password authentication on Windows, but probably not good

IMO we should address these two as a separate issue. There must be alternatives which can be used instead.

Now moving on to the dev dependencies. Installing with the dev dependencies is still a horror show of a gazillion warnings, huge disk space consumption and time taken of unknown proportions (because I've never let it finish). Some of them indicate the packages won't work on modern Node.js and others have dependencies that won't be satisfied by npm 3. To be clear, this is with specifying the latest version of each of these packages in package.json. So it's now the packages that are questionable, rather than the package.json for Verdaccio, That's what I'm going to look at next.

OK, ignore that last paragraph.

Some more digging tells me that the dev dependencies were getting installed anyway, just with a straight npm install

Adding the --dev flag did not do what I thought it did. It makes npm try to install every package's dev dependencies, and it's recursive, which is where all the bad stuff happens.

So, I'm going to commit this shortly and raise a merge request.

[nwhite89]

@steve-p-com have you tried npm install --production, that's what should be run if you don't want devDependencies to be installed, unless I have misunderstood the issue

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.