Not everything may be being cached

[verdaccio]

Not everything may be being cached

I don't think verdaccio can fix the problem. It would require it to unpack the tarball, modify the shinkwrap file, and repack it. That would change the checksum which would have further consequences. Packing the tarball has to be the responsibility of the client.

Subsequent to what I posted above, we are moving onto using yarn for npm package deployment. See

https://www.npmjs.com/package/yarn
https://code.facebook.com/posts/1840075619545360

It still uses npm behind the scenes, but it doesn't use the shrinkwrap file. Instead it uses a new file called yarn.lock which does not have these problems. Further, it appears that after running yarn to create the yarn.lock file, running npm shrinkrap creates a shrinkwrap file that doesn't have any resolved elements, so there's no longer any need to run the gulp task (so far anyway). And npm publish with a package which has both yarn.lock and npm-shrinkwrap.json files works very nicely. The package can then be deployed either using yarn add packagename or npm install packagename depending on what the user has installed. And with this workflow, it works fine with verdaccio.

Just to add the yarn.lock file does contain references to the address of the verdaccio server, which works for us because we are always deploying from the verdaccio server and never directly. The shrinkwrap file generated afterwards doesn't contain references to the server, though (no resolved elements).

In either case, i.e. installing using either yarn or npm, everything is now properly cached. Even so, installing using yarn takes a fraction of the time that npm install takes. For example one project takes 120 seconds with npm and only 25 seconds with yarn. So it's a big efficiency improvement.

Just to bring this up to date: I'm not sure this is something that should be addressed within Verdaccio. The npm client is in charge of the shrinkwrap file. Any time you install or update anything with --save set, if there's a shrinkwrap file, it will get updated. The safest option if you need network-free installs is to update the shrinkwrap file before publishing. I use a gulp task for it which looks like this:

  gulp.task('rewrap', function (cb) {
    // remove resolved elements from npm-shrinkwrap
    var shrinkwrapFile = path.join(__dirname, 'npm-shrinkwrap.json'),
      fs = require('fs'),
      shrinkwrap = require('./npm-shrinkwrap.json')

    gutil.log('unwrap: removing resolved elements from ' + shrinkwrapFile)

    function replacer(key, value) {

      if (!this.version) {
        return value
      }

      switch (key) {

        case 'resolved':
        case 'from':
          return undefined

        default:
          return value
      }
    }

    fs.writeFile(shrinkwrapFile, JSON.stringify(shrinkwrap, replacer, 2), function (err) {
      cb(err)
    })

  })

All it does is remove the resolved elements, which means an npm install of the published package will fetch everything from the default registry (so if that's Verdaccio, there's no Internet access required).

[jsumners]

I'm seeing this while trying to install nodemailer on an internal only server that uses a sinopia cache as a proxy.

[trentearl]

I have recently noticed this as well, thanks for looking into it.

[mhverbakel]

I think I found the cause of this problem, or actually two of them.

1. When a package that you install is still in your cache, downloaded directly (not through the cache), it will show up in the shrinkwrap as resolved from the npmjs server (which is true). This should not be resolved by verdaccio, as it is a user fault.
2. If a package contains a npm-shrinkwrap.json file, it will contain direct urls to the original repository. In my opinion, verdaccio should scan the packages for these files, and update the resolved tags there if that url points to the upstream repository.

[juanpicado]

I think @steve-p-com cleared that up. Thanks. Closing

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.