pnpm/cmd-shim

The cmd-shim used in pnpm

简体中文 | 日本語 | 한국어 | Italiano | Português Brasileiro

Fast, disk space efficient package manager:

• Fast. Up to 2x faster than the alternatives (see benchmark).
• Efficient. Files inside node_modules are linked from a single content-addressable storage.
• Great for monorepos.
• Strict. A package can access only dependencies that are specified in its package.json.
• Deterministic. Has a lockfile called pnpm-lock.yaml.
• Works as a Node.js version manager. See pnpm env use.
• Works everywhere. Supports Windows, Linux, and macOS.
• Battle-tested. Used in production by teams of all sizes since 2016.
• See the full feature comparison with npm and Yarn.

To quote the Rush team:

Microsoft uses pnpm in Rush repos with hundreds of projects and hundreds of PRs per day, and we’ve found it to be very fast and reliable.

Platinum Sponsors

Gold Sponsors

Silver Sponsors

Support this project by becoming a sponsor.

Background

pnpm uses a content-addressable filesystem to store all files from all module directories on a disk. When using npm, if you have 100 projects using lodash, you will have 100 copies of lodash on disk. With pnpm, lodash will be stored in a content-addressable storage, so:

1. If you depend on different versions of lodash, only the files that differ are added to the store. If lodash has 100 files, and a new version has a change only in one of those files, pnpm update will only add 1 new file to the storage.
2. All the files are saved in a single place on the disk. When packages are installed, their files are linked from that single place consuming no additional disk space. Linking is performed using either hard-links or reflinks (copy-on-write).

As a result, you save gigabytes of space on your disk and you have a lot faster installations! If you'd like more details about the unique node_modules structure that pnpm creates and why it works fine with the Node.js ecosystem, read this small article: Flat node_modules is not the only way.

💖 Like this project? Let people know with a tweet

Getting Started

• Installation
• Usage
• Frequently Asked Questions
• Chat
• Twitter

Benchmark

pnpm is up to 2x faster than npm and Yarn classic. See all benchmarks here.

Benchmarks on an app with lots of dependencies:

Usage Trend

Usage Trend of pnpm

Backers

Thank you to all our backers! Become a backer

Contributors

This project exists thanks to all the people who contribute. Contribute.

License

MIT

cmd-shim's Issues

Does cmdShim() support sh / batch (cmd) / pwsh script as src?

Related:

• yarnpkg/yarn#6902
• yarnpkg/yarn#6913

Yarn for Windows calls cmdShim() with the following src and to when yarn global add create-react-app is executed:

src	to
[Yarn global folder]\node_modules\.bin\create-react-app (for sh)	[prefix]\bin\create-react-app
[Yarn global folder]\node_modules\.bin\create-react-app.cmd	[prefix]\bin\create-react-app.cmd
[Yarn global folder]\node_modules\.bin\create-react-app.ps1	[prefix]\bin\create-react-app.ps1

https://github.com/yarnpkg/yarn/blob/1af4c5f50c4c146a833c0c2b20d10be1f5a61803/src/cli/commands/global.js#L168
https://github.com/yarnpkg/yarn/blob/1af4c5f50c4c146a833c0c2b20d10be1f5a61803/src/cli/commands/global.js#L168

As the result, this module creates:

• Junk named create-react-app.cmd and create-react-app.ps1, which looks like a batch or PowerShell script but in fact are sh scripts; They can't execute in CMD or PowerShell in spite of their extensions.
• 4 scripts with strange names create-react-app.{ps1,cmd}.{cmd,ps1}; To make matters worse, create-react-app.ps1.* can't be executed when pwsh (for .NET Core) is not installed.

This issue was suppressed in Yarn until this module suppored PowerShell shims because the following code of Yarn, which replace create-react-app.cmd (junk sh script) with create-react-app.cmd.cmd:
https://github.com/yarnpkg/yarn/blob/1af4c5f50c4c146a833c0c2b20d10be1f5a61803/src/cli/commands/global.js#L169-L171

Anyway, this module must:

• Sense that src is a shell (sh, CMD, or PowerShell) script (maybe created by this module).
• Supress creating unwanted scripts.
• Create shims to PowerShell scripts that properly works even if pwsh is unavailable.

Question about / possible bug in `normalizePathEnvVar`: /mnt/ prefix typically does not exist in WSL

https://github.com/pnpm/cmd-shim/blob/master/index.js#L464

This is a question first, but the answer might point to a bug in normalizePathEnvVar

On Windows, it looks like each path in NODE_PATH is prefixed with /mnt/. I'm unfamiliar with Cygwin, but I use WSL a lot. WSL initially mounted Windows drives as /mnt/c/ but these days the /mnt/ prefix is usually omitted, so it's mounted at /c. Might that cause problems with cmd-shim?

I don't know how NODE_PATH is being used, so I'm not sure.

Support target and src being on different drives (for example, E:, C:)

Users of npm-pwsh reported cmd-shim can't link across drives.
cspotcode/npm-pwsh#18
npm-pwsh is currently using cmd-shim, but I'd like to switch to @zkochan/cmd-shim. Unfortunately both libraries exhibit this limitation.

The problem can be generalized to handling when the target path must be absolute instead of relative.

This can be accomplished by modifying these 2 lines:
https://github.com/pnpm/cmd-shim/blob/master/index.js#L252
https://github.com/pnpm/cmd-shim/blob/master/index.js#L257

... to check if target is an absolute path. If it is, it means we're linking to a different drive letter, so use target verbatim. If not, prepend with %~dp0 just like the current behavior.

Related: npm#21

The value of NODE_PATH should be in double quotes

There are issues on Windows because the value of NODE_PATH is not in double quotes

Blows away user-defined NODE_PATH

I have a module that acts as a kind of proxy for node, in that it spawns node as a child process and passes along the environment variables of the parent. This works fine in most cases, just not when the module expects a user-defined NODE_PATH value.

In case that description was unclear, here’s some psuedocode to recreate the issue I’m having:

#!/usr/bin/env node
console.log(process.env.NODE_PATH)

$ pnpm i some-module

# expected 
$ NODE_PATH=./some/path ./node_modules/.bin/some-module
> ./some/path

# actual 
$ NODE_PATH=./some/path ./node_modules/.bin/some-module
> /path/to/node_modules:/path/to/node_modules:/path/to/node_modules:/path/to/node_modules:/path/to/node_modules:/path/to/node_modules:/path/to/node_modules:/path/to/node_modules:/path/to/node_modules:/path/to/node_modules:/path/to/node_modules

v3.1.0 dependency deprecation:mkdirp-promise

[email protected]: This package is broken and no longer maintained. 'mkdirp' itself supports promises now, please switch to that.

Workaround `Terminate batch job` prompts

A workaround for cmd files has been uncovered to avoid the Terminate batch job prompts when terminating scripts.
We merged the workaround in Yarn in yarnpkg/berry#1939 and cmd-shim applied it in npm@c039646. It would be great to have this fix applied here as well so we can get rid of the prompt for the yarn, pnpm, and npm shims created by corepack (https://github.com/nodejs/corepack/blob/00c2ab1e2c4e606ab1c3a12535fab399789d2814/sources/commands/Enable.ts#L96-L98)

The PowerShell script doesn’t include the required $env:NODE_PATH change

Turns out that the PowerShell script I added in #5 doesn’t include the $env:NODE_PATH change from 68dc8ce, as I just ported the commit from npm#27, without paying much thought to all the changes that needed to be done, and it slipped by testing, as it worked in all scenarios I threw at (except for the case when depending on dependencies which are using pluggable packages with plugins, which I discovered only after pnpm 2.11 was released), then they may break in odd ways.

The issue with this is that the PowerShell script may be run on either Windows or Linux (where Windows uses ; as PATH separators, but Unix uses :, so that will need to be handled from inside the PowerShell script)

Shims created for batch scripts on Windows don't run the batch script

Shims created for .cmd and .bat files don't call cmd.exe with the required flags to actually run the script. To run a script, the script path needs to be preceded with the /C (or /K) flags. I have attached a sample batch file shim generated by pnpm, but the issue also affects the PowerShell and shell scripts.

@SETLOCAL
@IF EXIST "%~dp0\cmd.exe" (
  "%~dp0\cmd.exe"  "%~dp0\global\5\node_modules\sass-embedded-win32-x64\dart-sass\sass.bat" %*
) ELSE (
  @SET PATHEXT=%PATHEXT:;.JS;=;%
  cmd  "%~dp0\global\5\node_modules\sass-embedded-win32-x64\dart-sass\sass.bat" %*
)

The cmd call in line 6 (as well as line 3) above needs to include the /C flag, as below:

cmd /C "%~dp0\global\5\node_modules\sass-embedded-win32-x64\dart-sass\sass.bat" %*

Document opts.prog and opts.args

Hi

Looking at the README it seemed like there was no way to customise the args passed (needed for yarnpkg/yarn#6493), however there are some additional undocumented options that would be good to add to the README:

cmd-shim/index.js

Line 82 in b3c57c0

let prog = opts.prog

cmd-shim/index.js

Line 88 in b3c57c0

let args = opts.args || ''

Implement read-cmd-shim via a command-based directive instead of regexp parsing

Over on npm/cmd-shim, it was mentioned that changing the shim to implement #17 will require a change to read-cmd-shim to parse the new implementation. I guess read-cmd-shim uses a regexp to parse the target path from the shim.

I think it'd be better going forward to embed the target path in a way that can be parsed out more easily. For example, at the top of every shim is a comment like this:

@REM __CMD_SHIM_TARGET__=../foo/bin/bar.js

...or...

# __CMD_SHIM_TARGET__=../foo/bin/bar.js

read-cmd-shim can check for this directive. If found, that's the target path. If not, it'll fall-back to legacy regexp matching. This is a more future-proof approach to parsing the target out of a shim because the shims are free to change in the future. As long as they include that directive, they can be read unambiguously.

...of course I'm not sure who uses read-cmd-shim or what they use it for. So maybe this is not worth the effort. I still want to share the idea.

I also think, if read-cmd-shim is a useful feature, it makes more sense to include it in cmd-shim than to implement as a separate module. It's not much code and putting them in 2 places is just lots of duplicated boilerplate and configuration.