google/zx

A tool for writing better scripts

zx

#!/usr/bin/env zx

await $`cat package.json | grep name`

let branch = await $`git branch --show-current`
await $`dep deploy --branch=${branch}`

await Promise.all([
  $`sleep 1; echo 1`,
  $`sleep 2; echo 2`,
  $`sleep 3; echo 3`,
])

let name = 'foo bar'
await $`mkdir /tmp/${name}`

Bash is great, but when it comes to writing more complex scripts, many people prefer a more convenient programming language. JavaScript is a perfect choice, but the Node.js standard library requires additional hassle before using. The zx package provides useful wrappers around child_process, escapes arguments and gives sensible defaults.

Install

npm install zx

Documentation

Read documentation on google.github.io/zx.

License

Apache-2.0

Disclaimer: This is not an officially supported Google product.

zx's Issues

How to use commands stored in variables?

Hi,
Tried this package, looks really nice but i can't seem to understand how i run commands stored in a variable.
Maybe i'm just a noob or is it not supported.?

Example:

`
let url = 'https://example.com'
let testCommand = 'wget ' + url; // a string containing the command.

Now.. how do i use $"runsomecommand"

`

Any help would be very much appreciated, looking forward to use zx in my projects but i need this basic functionallity.

Cheers

`$.prefix` not work as expected

#!/usr/bin/env zx

$.shell = '/usr/local/bin/zsh'
$.prefix = 'echo 122;source ~/.zshrc;'

await $`z test;  pwd` # use on-my-zsh z plugin

got

zsh:1: command not found: z

expect

1. echo 123 first
2. pwd print somedir of which name contains test

Feature suggestion: Typescript support.

Write and execute scripts in typescript .ts files

ج

Not woriking with node v12.22.1

Specifications

Failed to load script and throws this error
` zx https://medv.io/example-script.mjs
file:///home/dubey_aditya/.nvm/versions/node/v12.22.1/lib/node_modules/zx/zx.mjs:43
let ok = await scriptFromStdin()
^^^^^

SyntaxError: Unexpected reserved word
at Loader.moduleStrategy (internal/modules/esm/translators.js:140:18)
at async link (internal/modules/esm/module_job.js:42:21)
`

• Version: 12.22.1
• Platform: Ubuntu 21,04

cmd.exe support

Expected Behavior

zx works with any common shell on windows OS.

Actual Behavior

In cmd, powershell, and git bash, a typical usage of zx fails.

Output from running test in cmd.exe

    
C:\Users\macke>npm i -g -f zx
npm WARN using --force I sure hope you know what you are doing.
C:\Users\macke\AppData\Roaming\npm\zx -> C:\Users\macke\AppData\Roaming\npm\node_modules\zx\zx.mjs
+ [email protected]
updated 1 package in 0.63s
C:\Users\macke>zx -h
'command' is not recognized as an internal or external command,
operable program or batch file.
'which' is not recognized as an internal or external command,
operable program or batch file.
The system cannot find the file specified.
Error occurred while processing: -p.
The system cannot find the file specified.
Error occurred while processing: bash.
child_process.js:616
    err = new Error(msg);
          ^
Error: Command failed: command -v bash || which bash || type -p bash
'command' is not recognized as an internal or external command,
operable program or batch file.
'which' is not recognized as an internal or external command,
operable program or batch file.
The system cannot find the file specified.
Error occurred while processing: -p.
The system cannot find the file specified.
Error occurred while processing: bash.
    at checkExecSyncError (child_process.js:616:11)
    at execSync (child_process.js:652:15)
    at file:///C:/Users/macke/AppData/Roaming/npm/node_modules/zx/index.mjs:75:14
    at ModuleJob.run (internal/modules/esm/module_job.js:152:23)
    at async Loader.import (internal/modules/esm/loader.js:166:24)
    at async Object.loadESM (internal/process/esm_loader.js:68:5) {
  status: 1,
  signal: null,
  output: [
    null,
    Buffer(0) [Uint8Array] [],
    Buffer(360) [Uint8Array] [
       39,  99, 111, 109, 109,  97, 110, 100,  39,  32, 105, 115,
       32, 110, 111, 116,  32, 114, 101,  99, 111, 103, 110, 105,
      122, 101, 100,  32,  97, 115,  32,  97, 110,  32, 105, 110,
      116, 101, 114, 110,  97, 108,  32, 111, 114,  32, 101, 120,
      116, 101, 114, 110,  97, 108,  32,  99, 111, 109, 109,  97,
      110, 100,  44,  13,  10, 111, 112, 101, 114,  97,  98, 108,
      101,  32, 112, 114, 111, 103, 114,  97, 109,  32, 111, 114,
       32,  98,  97, 116,  99, 104,  32, 102, 105, 108, 101,  46,
       13,  10,  39, 119,
      ... 260 more items
    ]
  ],
  pid: 2628,
  stdout: Buffer(0) [Uint8Array] [],
  stderr: Buffer(360) [Uint8Array] [
     39,  99, 111, 109, 109,  97, 110, 100,  39,  32, 105, 115,
     32, 110, 111, 116,  32, 114, 101,  99, 111, 103, 110, 105,
    122, 101, 100,  32,  97, 115,  32,  97, 110,  32, 105, 110,
    116, 101, 114, 110,  97, 108,  32, 111, 114,  32, 101, 120,
    116, 101, 114, 110,  97, 108,  32,  99, 111, 109, 109,  97,
    110, 100,  44,  13,  10, 111, 112, 101, 114,  97,  98, 108,
    101,  32, 112, 114, 111, 103, 114,  97, 109,  32, 111, 114,
     32,  98,  97, 116,  99, 104,  32, 102, 105, 108, 101,  46,
     13,  10,  39, 119,
    ... 260 more items
  ]
}
    
  

Output from running test in pwsh.exe (powershell core 7)

    
PS C:\Users\macke> npm i -g -f zx
npm WARN using --force I sure hope you know what you are doing.
C:\Users\macke\AppData\Roaming\npm\zx -> C:\Users\macke\AppData\Roaming\npm\node_modules\zx\zx.mjs
+ [email protected]
updated 1 package in 0.594s
PS C:\Users\macke> zx -h
'command' is not recognized as an internal or external command,
operable program or batch file.
'which' is not recognized as an internal or external command,
operable program or batch file.
The system cannot find the file specified.
Error occurred while processing: -p.
The system cannot find the file specified.
Error occurred while processing: bash.
child_process.js:616
    err = new Error(msg);
          ^
Error: Command failed: command -v bash || which bash || type -p bash
'command' is not recognized as an internal or external command,
operable program or batch file.
'which' is not recognized as an internal or external command,
operable program or batch file.
The system cannot find the file specified.
Error occurred while processing: -p.
The system cannot find the file specified.
Error occurred while processing: bash.
    at checkExecSyncError (child_process.js:616:11)
    at execSync (child_process.js:652:15)
    at file:///C:/Users/macke/AppData/Roaming/npm/node_modules/zx/index.mjs:75:14
    at ModuleJob.run (internal/modules/esm/module_job.js:152:23)
    at async Loader.import (internal/modules/esm/loader.js:166:24)
    at async Object.loadESM (internal/process/esm_loader.js:68:5) {
  status: 1,
  signal: null,
  output: [
    null,
    Buffer(0) [Uint8Array] [],
    Buffer(360) [Uint8Array] [
       39,  99, 111, 109, 109,  97, 110, 100,  39,  32, 105, 115,
       32, 110, 111, 116,  32, 114, 101,  99, 111, 103, 110, 105,
      122, 101, 100,  32,  97, 115,  32,  97, 110,  32, 105, 110,
      116, 101, 114, 110,  97, 108,  32, 111, 114,  32, 101, 120,
      116, 101, 114, 110,  97, 108,  32,  99, 111, 109, 109,  97,
      110, 100,  44,  13,  10, 111, 112, 101, 114,  97,  98, 108,
      101,  32, 112, 114, 111, 103, 114,  97, 109,  32, 111, 114,
       32,  98,  97, 116,  99, 104,  32, 102, 105, 108, 101,  46,
       13,  10,  39, 119,
      ... 260 more items
    ]
  ],
  pid: 21584,
  stdout: Buffer(0) [Uint8Array] [],
  stderr: Buffer(360) [Uint8Array] [
     39,  99, 111, 109, 109,  97, 110, 100,  39,  32, 105, 115,
     32, 110, 111, 116,  32, 114, 101,  99, 111, 103, 110, 105,
    122, 101, 100,  32,  97, 115,  32,  97, 110,  32, 105, 110,
    116, 101, 114, 110,  97, 108,  32, 111, 114,  32, 101, 120,
    116, 101, 114, 110,  97, 108,  32,  99, 111, 109, 109,  97,
    110, 100,  44,  13,  10, 111, 112, 101, 114,  97,  98, 108,
    101,  32, 112, 114, 111, 103, 114,  97, 109,  32, 111, 114,
     32,  98,  97, 116,  99, 104,  32, 102, 105, 108, 101,  46,
     13,  10,  39, 119,
    ... 260 more items
  ]
}
    
  

Output from running test in bash.exe (git bash)

    
macke@DESKTOP-B1AO5HR MINGW64 ~
$ npm i -g -f zx
npm WARN using --force I sure hope you know what you are doing.
C:\Users\macke\AppData\Roaming\npm\zx -> C:\Users\macke\AppData\Roaming\npm\node_modules\zx\zx.mjs
+ [email protected]
updated 1 package in 0.726s
macke@DESKTOP-B1AO5HR MINGW64 ~
$ zx -h
'command' is not recognized as an internal or external command,
operable program or batch file.
internal/process/esm_loader.js:74
    internalBinding('errors').triggerUncaughtException(
                              ^
Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only file and data URLs are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'c:'
    at Loader.defaultResolve [as _resolve] (internal/modules/esm/resolve.js:782:11)
    at Loader.resolve (internal/modules/esm/loader.js:86:40)
    at Loader.getModuleJob (internal/modules/esm/loader.js:230:28)
    at Loader.import (internal/modules/esm/loader.js:165:28)
    at importModuleDynamically (internal/modules/esm/translators.js:114:35)
    at exports.importModuleDynamicallyCallback (internal/process/esm_loader.js:30:14)
    at file:///C:/Users/macke/AppData/Roaming/npm/node_modules/zx/zx.mjs:57:5
    at ModuleJob.run (internal/modules/esm/module_job.js:152:23)
    at async Loader.import (internal/modules/esm/loader.js:166:24)
    at async Object.loadESM (internal/process/esm_loader.js:68:5) {
  code: 'ERR_UNSUPPORTED_ESM_URL_SCHEME'
}
    
  

Steps to Reproduce the Problem

Into any common windows shell:

1. type npm i -g -f zx
2. type xz -h

Specifications

• Version: 1.2.2
• Platform: Windows 10 OS

Idea to include `global.argv` / `minimist`?

This library is a timesaver! What do you think of the idea to also add passed-in arguments to the global, using something like minimist?

The change:

global.argv = require('minimist')(process.argv.slice(2));

Run script:

$ zx sync.js --username smeijer

Usage:

let username = argv.username || await question('What is your username? ')

Add GitHub release?

It'd make it easier to find the current version, etc. if GitHub release were to be used.

Even better would be to add an automated GitHub actions workflow to handle this.

makefile like

is there good way to work with this like a makefile ?

something that would let me define a file like

test:
   let x = $.args.whatever
   await $`do some ${x}`
build:
  await $`some else`

The cli arguments would be parsed automatically, so if i call zx test --whatever I can access this value in $.args.whatever.

Escape args in command?

Expected Behavior

In shell:

#!/bin/bash

message='; ls'
echo ${message}
# Output: '; ls'

So I think:

#!/usr/bin/env zx

const message = '; ls'
await $`echo ${message}`
// Expect Output: '; ls'

Actual Behavior

• Actual Output: result of ls

Steps to Reproduce the Problem

1. Execute above commands

Consideration

• On migrate from ShellScript to zx, The above case are likely to occur.
• And scripts has potential of Arbitrary Command Execution.
• To prevent, escape args or provide methods for escape variables.

Specifications

• Version: 1.0.2
• Platform: macOS

Use undici-fetch rather then node-fetch

when it goes stable

It is more performant then node-fetch

Port to Deno :)

Make bash-specifics customizable

Test script

$.shell = '/usr/local/bin/fish'
let bar = "\\\\a'a"
assert((await $`echo ${bar}`).stdout.trim() === bar)

Expected Behavior

Assert passess

Actual Behavior

Fails at bash-specific set command. Removing that command causes fail due to quoting.

Steps to Reproduce the Problem

1. Add test script to test.mjs.
2. yarn test (get prepend problem)
3. Remove set command.
4. Remove the cat /dev/not_found | sort test by prepending if (0), as it depends on the set thing.
5. yarn test (get quote problem)

Specifications

• Version: current git HEAD (1.1.1)
• Platform: macOS 11

Suggested fix

• Turn the prepended part into $.prepend.
• Turn the quoting function into $.quote, as in #10's README change.

Otherwise there really isn't much point in $.shell, except for using a newer version of bash. (Is there a point in using another shell? I think so! Just think about why we are using bash instead of splitting the words ourselves and doing a spawn. Also consider shells with "special power", such as how spawn cannot handle *.bat and how powershell commands poke into the system.)

Feature suggestion: real time command output using streams

It would be really useful if one could get the output of commands in real time as they are written to stdout with a readable stream instead of awaiting for the command to exit. For example:

const pingStream = $.stream`ping 8.8.8.8`

pingStream.on('data', () => {
  console.log('ping wrote to stdout');
});

pingStream.on('end', () => {
  console.log('ping terminated with exit code 0');
});

pingStream.on('error', () => {
  console.log('ping terminated with an error');
});

/bin/sh: 1: set: Illegal option -o pipefail

Expected Behavior

The example script works correctly.

#!/usr/bin/env zx

// Copyright 2021 Google LLC
// 
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
// 
//     https://www.apache.org/licenses/LICENSE-2.0
// 
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

await $`ls -1 | wc -l`

let branch = await $`git branch --show-current`
await $`printf ${branch} | wc`

await $`test -f package.json`

Actual Behavior

The example script doen't work.

$ ./test.mjs 
$ ls -1 | wc -l  # Note: This line is zx output.
/bin/sh: 1: set: Illegal option -o pipefail
  at file:///home/hata6502/zx-test/test.mjs:17:8

I think this error is occured by using sh instead of bash.
set -euo pipefail; seems not work in sh.

$ bash
$ set -euo pipefail; echo "test"
test
$ sh
$ set -euo pipefail; echo "test"
sh: 1: set: Illegal option -o pipefail

Steps to Reproduce the Problem

1. Make test.mjs.
2. Run chmod +x test.mjs
3. Run ./test.mjs

Specifications

• Version: 1.1.0
• Platform:
  • OS: Ubuntu 20.04

Bundle zx package

Currently installing zx package, costs ~125 files and 832K of disk space (401K reported by packagephobia). Also, it is making zx dependant on node_modules.

By bundling zx package:

• Install cost will be reduced to few files (basically one zx.mjs) and ~92KB as of v1.2.2
• zx script will be standalone. Can possibly help #25 for embedded solutions or selfupdate
• Using rollup plugins, we can have a chance for deno build from same source (#24)
• Memory footprint will be reduced as a benefit from tree-shaking unused imports (quick test max RSS from 44580kb to 37628kb for test.mjs
• Easier dist analyzes
• Possibility to use typescript
• Removing need of cjs hack for importing version from package.json

PR: #51

Spelling mistake in README

Under the fs section, y'all put fx in the link.

question not working with choices

Expected Behavior

The choices are displayed when asking a question that contains choices.

Actual Behavior

No choices are displayed: the user can enter any value.

Steps to Reproduce the Problem

#!/usr/bin/env zx

const number = await question("Pick a number: ", {
  choices: ["1", "2"],
});

1. Run the above.
2. See that no choices are displayed.

Specifications

• Version: 1.6.0
• Platform: macOS 10.14.6

unexpected result

Expected Behavior

"jsToSh"

Actual Behavior

Administrator@WIN-JH5VDQS6BP7 MINGW64 /f/codeTest/jsToSh/src
$ zx file:///F:/codeTest/jsToSh/src/index.mjs
$ cat ../package.json | grep name | cut -d":" -f 2 | cut -d',' -f1

Administrator@WIN-JH5VDQS6BP7 MINGW64 /f/codeTest/jsToSh/src
$

Sh run

Administrator@WIN-JH5VDQS6BP7 MINGW64 /f/codeTest/jsToSh/src
$ cat ../package.json | grep name | cut -d":" -f 2 | cut -d',' -f1
 "jsToSh"

Administrator@WIN-JH5VDQS6BP7 MINGW64 /f/codeTest/jsToSh/src
$

Package.json

{
  "name": "jsToSh",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "devDependencies": {
    "zx": "^1.1.1"
  }
}

Steps to Reproduce the Problem

1. touch index.mjs
2. vim index.mjs

#!/usr/bin/env zx

let res = await $`cat ../package.json | grep name | cut -d":" -f 2 | cut -d',' -f1`;

console.log(res);

3. zx file:///F:/codeTest/jsToSh/src/index.mjs

Specifications

• node: v14.16.0

• Platform: Windows 10 Professional

Smarter array handling?

Expected Behavior

Arrays is passed as individual arguments. (I mean, anyone wishing to stringify it are free to do a join)

Actual Behavior

Array is stringified

Steps to Reproduce the Problem

if (hasBash) {
  let a = ["a", "b"]
  let result = await $`printf '%s|' ${a}`
  assert(result.stdout  == 'a|b|')
}

Specifications

• Version: HEAD -- 1.2.2 apparently
• Platform: macOS 11

P...S

Maybe we shouldn't let shq handle the stringification at all, but cause failures on unknown types immediately? This would save zx a few future semver bumps -- just in the case someone actually end up relying on some odd String() constructor behavior.

set: Illegal option -o pipefail

$ cat package.json | grep esm
/bin/sh: 1: set: Illegal option -o pipefail
  at file:///tmp/skrypt.mjs:3:8

after changing file index.mjs

let child = exec('set -euo pipefail;' + cmd, options)

to

    let child = exec('set -eu;' + cmd, options)

it works as expected.
My default shell is bash, I'm using Konsole 19.12.3

Unexpected token {

Taking any .mjs script, or trying to run $ zx I'm having following error:

$ zx ./skrypt.mjs 
/usr/local/lib/node_modules/zx/zx.mjs:17
import {join, basename} from 'path'
       ^
SyntaxError: Unexpected token {
    at Module._compile (internal/modules/cjs/loader.js:723:23)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
    at Module.load (internal/modules/cjs/loader.js:653:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
    at Function.Module._load (internal/modules/cjs/loader.js:585:3)
    at Function.Module.runMain (internal/modules/cjs/loader.js:831:12)
    at startup (internal/bootstrap/node.js:283:19)
    at bootstrapNodeJSCore (internal/bootstrap/node.js:623:3)

The only steps I peformed were to install zx using npm, as stated in README.

Feature suggestion: Interactive Shell

I am talking about a proper Bash substitute a marriage of good ideas between Node.js REPL
and https://www.npmjs.com/package/vorpal (or "Tiny CLI")

It will make you famous, a bit more handsome, and you'll be the talk of the town.

Yes.

readline() is not defined

Expected Behavior

Take Input from User

Actual Behavior

ReferenceError: `readline` is not defined
    at ModuleJob.run (node:internal/modules/esm/module_job:175:25)
    at async Loader.import (node:internal/modules/esm/loader:178:24)
    at async file:///usr/local/lib/node_modules/zx/zx.mjs:58:5

Steps to Reproduce the Problem

• While Running this Script

#!/usr/bin/env zx

const a = readline();
const b = readline();
const sum = a + b;
console.log("Sum is: " + sum);

I have created a zx-examples for all uses cases of this amazing script

Specifications

• Version: 1.4.0
• Platform: VSCode, Ubuntu

Template literals function turn numeric 0 into ''

Hi, all. I found this project very inspiring and tried it out right when I saw it. However, I met some issues when I was trying to create a script to arrange files into folders, like this:

const prefix = './test/'
const parts = 5
for (let i = 0; i < parts; ++i) {
  $`mkdir -p ${prefix}${i}`
}

console.log(chalk.red("Done!"))

Expected Behavior

$ mkdir -p ./test/0
$ mkdir -p ./test/1
$ mkdir -p ./test/2
$ mkdir -p ./test/3
$ mkdir -p ./test/4
Done!

Actual Behavior

$ mkdir -p ./test/''
$ mkdir -p ./test/1
$ mkdir -p ./test/2
$ mkdir -p ./test/3
$ mkdir -p ./test/4
Done!

Steps to Reproduce the Problem

1. just create a script using the code above.
2. run it.

Further Description

When I look into the code, the $ defined in index.mjs calls $.quote which is an alias of shq that turns numeric 0 to ''. I know it is how shq handles the non-string variable, but I do not think it behaves commonly (or maybe logically) here. If we need to make sure the variables passed into the template literals to be all strings, maybe we need to throw the bad cases. Otherwise, to turn 0 into '' might not be a good idea?

Thanks.

Specifications

• Version: 1.5.0
• Platform: Node v16.1.0 on Ubuntu 20.04

better output

Expected Behavior

Actual Behavior

Steps to Reproduce the Problem

Specifications

• Version:
• Platform:

Auto-install packages

first of all, thank you very much. zx is a very nice tool.

As far as I can see, people want to use the packages they are used to with zx.
I've browsed some forks. People create their own packages by adding their own dependencies.

I am not sure about the side effects.
However, it seems like a good idea to install and use packages in the runtime.

Maybe something like this.. await use('package-name')
If this package does not exist in node_modules in the directory where zx is installed, it is installed with npm install and added to global with Object.assing

#!/usr/bin/env zx

await use('shelljs')

shelljs.rm('-rf', 'out/Release');
shelljs.cp('-R', 'stuff/', 'out/Release');

I implemented this idea experimentally as follows.

zx.mjs

import { join, basename, dirname } from 'path'
import os, { tmpdir } from 'os'
import { promises as fs } from 'fs'
import url from 'url'
import { v4 as uuid } from 'uuid'
import { $, cd, question, fetch, chalk, ProcessOutput } from './index.mjs'
import { version } from './version.js'

const use = async (packageName, packageNameGlobal = '') => {
  if (packageNameGlobal == '') {
    packageNameGlobal = packageName
  }

  const __dirname = dirname(url.fileURLToPath(import.meta.url))

  if (global[packageNameGlobal]) {
    return global[packageNameGlobal]
  }

  let isExist = false
  try {
    isExist = (
      await fs.lstat(join(__dirname, 'node_modules', packageName))
    ).isDirectory()
  } catch (e) {}

  if (isExist) {
    Object.assign(global, {
      [packageNameGlobal]: import(packageName),
    })
  } else {
    const cwdPrev = $.cwd
    $.cwd = __dirname

    await $`npm install ${packageName}`

    $.cwd = cwdPrev

    Object.assign(global, {
      [packageNameGlobal]: import(packageName),
    })
  }

  return global[packageNameGlobal]
}

Object.assign(global, {
  $,
  cd,
  fetch,
  question,
  chalk,
  fs,
  os,
  use,
})

....

Testing scripts?

Great module, and thank you for this neat tool!

I'm curious how I might go about testing scripts created for zx? Ideally I'd want to import them into a jest test suite and run assertions.

I didn't see a clean way to import a script without running it immediately. Perhaps zx could export a testing harness that understands how to run the imported script in a sandboxed way and provide ways to make assertions on the commands that were run and their results

Removal of the test function

Hi,

I want to know why the test function was removed 4c8d602 ?

Instead of this:

if (test('-f test.js')) {
    // do stuff
}

I now need to write:

try {
    await $`test -f test.js`;
    // do stuff
} catch (e) {}

I think this looks way more ugly.

I understand that the test function wasn't really written well, but you could refactor it to:

export async function test(cmd) {
    try {
        await $`test ${cmd}`;
        return true;
    } catch (e) {
        return false;
    }
}

Better error output

Will be cool to add colors to error output of uncatched errors.

Fatal error in , line 0

Running zx from shell produces the following error. Also tried running without zx bin, as documented in README:

• Platform: macOS M1

#
# Fatal error in , line 0
# Check failed: allocator->SetPermissions(reinterpret_cast<void*>(region.begin()), region.size(), PageAllocator::kNoAccess).
#
#
#
#FailureMessage Object: 0x16d3ad5f8
 1: 0x102b4d758 node::NodePlatform::GetStackTracePrinter()::$_3::__invoke() [/Users/user/.nvm/versions/node/v15.6.0/bin/node]
 2: 0x103895d50 V8_Fatal(char const*, ...) [/Users/user/.nvm/versions/node/v15.6.0/bin/node]
 3: 0x1031e7b98 v8::internal::wasm::WasmCodeManager::Decommit(v8::base::AddressRegion) [/Users/user/.nvm/versions/node/v15.6.0/bin/node]
 4: 0x1031eb218 v8::internal::wasm::NativeModule::FreeCode(v8::internal::Vector<v8::internal::wasm::WasmCode* const>) [/Users/user/.nvm/versions/node/v15.6.0/bin/node]
 5: 0x1031fa400 v8::internal::wasm::WasmEngine::FreeDeadCodeLocked(std::__1::unordered_map<v8::internal::wasm::NativeModule*, std::__1::vector<v8::internal::wasm::WasmCode*, std::__1::allocator<v8::internal::wasm::WasmCode*> >, std::__1::hash<v8::internal::wasm::NativeModule*>, std::__1::equal_to<v8::internal::wasm::NativeModule*>, std::__1::allocator<std::__1::pair<v8::internal::wasm::NativeModule* const, std::__1::vector<v8::internal::wasm::WasmCode*, std::__1::allocator<v8::internal::wasm::WasmCode*> > > > > const&) [/Users/user/.nvm/versions/node/v15.6.0/bin/node]
 6: 0x1031f8108 v8::internal::wasm::WasmEngine::PotentiallyFinishCurrentGC() [/Users/user/.nvm/versions/node/v15.6.0/bin/node]
 7: 0x1031f957c v8::internal::wasm::WasmEngine::ReportLiveCodeForGC(v8::internal::Isolate*, v8::internal::Vector<v8::internal::wasm::WasmCode*>) [/Users/user/.nvm/versions/node/v15.6.0/bin/node]
 8: 0x1031f992c v8::internal::wasm::WasmEngine::ReportLiveCodeFromStackForGC(v8::internal::Isolate*) [/Users/user/.nvm/versions/node/v15.6.0/bin/node]
 9: 0x102d83348 v8::internal::StackGuard::HandleInterrupts() [/Users/user/.nvm/versions/node/v15.6.0/bin/node]
10: 0x1030be830 v8::internal::Runtime_StackGuard(int, unsigned long*, v8::internal::Isolate*) [/Users/user/.nvm/versions/node/v15.6.0/bin/node]
11: 0x1033afb6c Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_NoBuiltinExit [/Users/user/.nvm/versions/node/v15.6.0/bin/node]
12: 0x103348910 Builtins_InterpreterEntryTrampoline [/Users/user/.nvm/versions/node/v15.6.0/bin/node]
13: 0x103348874 Builtins_InterpreterEntryTrampoline [/Users/user/.nvm/versions/node/v15.6.0/bin/node]
14: 0x10333fea4 Builtins_ArgumentsAdaptorTrampoline [/Users/user/.nvm/versions/node/v15.6.0/bin/node]
15: 0x103348874 Builtins_InterpreterEntryTrampoline [/Users/user/.nvm/versions/node/v15.6.0/bin/node]
16: 0x103348874 Builtins_InterpreterEntryTrampoline [/Users/user/.nvm/versions/node/v15.6.0/bin/node]
17: 0x103348874 Builtins_InterpreterEntryTrampoline [/Users/user/.nvm/versions/node/v15.6.0/bin/node]
18: 0x103348874 Builtins_InterpreterEntryTrampoline [/Users/user/.nvm/versions/node/v15.6.0/bin/node]
19: 0x103374c74 Builtins_AsyncFunctionAwaitResolveClosure [/Users/user/.nvm/versions/node/v15.6.0/bin/node]
20: 0x1033fbdfc Builtins_PromiseFulfillReactionJob [/Users/user/.nvm/versions/node/v15.6.0/bin/node]
21: 0x1033679e8 Builtins_RunMicrotasks [/Users/user/.nvm/versions/node/v15.6.0/bin/node]
22: 0x103346008 Builtins_JSRunMicrotasksEntry [/Users/user/.nvm/versions/node/v15.6.0/bin/node]
23: 0x130008000 
zsh: trace trap  zx ./script.mjs

Auto await?

Adding ‘await’ to everything is what you want most of the time, but is quite tedious to have to write out.

Have you considered automatically transforming the imported JavaScript to automatically insert await statements? This is quite simple to do with Babel… see https://github.com/ziolko/babel-plugin-auto-await.

The downside is that sometimes you may actually want async behavior, so maybe it can be combined with some escape hatch (a decorator, or perhaps a special comment like ‘ // @ZX:no-auto-await. ‘)

Error on script with shebang but without file extension

Expected Behavior

When including #!/usr/bin/env zx at the top of a file, it just runs.

Actual Behavior

If the script has no file extension this error occurs:

node:internal/process/esm_loader:74
    internalBinding('errors').triggerUncaughtException(
                              ^

TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension "" for /Users/xxx/.git/hooks/pre-push
    at new NodeError (node:internal/errors:329:5)
    at Loader.defaultGetFormat [as _getFormat] (node:internal/modules/esm/get_format:71:15)
    at Loader.getFormat (node:internal/modules/esm/loader:102:42)
    at Loader.getModuleJob (node:internal/modules/esm/loader:231:31)
    at async Loader.import (node:internal/modules/esm/loader:165:17)
    at async file:///Users/blp/.fnm/node-versions/v15.12.0/installation/lib/node_modules/zx/zx.mjs:57:5 {
  code: 'ERR_UNKNOWN_FILE_EXTENSION'
}

I expected any script with the shebang to work, as any other shell script would. Requiring a file extension restricts where zx scripts can be run from.

Is this a limitation of ESM?

Steps to Reproduce the Problem

1. create zx script in .git/hooks/pre-push
2. push to repo

Specifications

• Version: 1.2.2
• Platform: darwin

Variable List of Command Line Args

Context

I'm making a small tar shortcut script so that I don't have to remember the right flags when I use it. One script is tools-tar and another is tools-untar. I need to pass in a variable number of file and folder names to put in the archive and can't make it work with the $`` syntax.

Usage of the tool: zx tools-tar.mjs output.tar.gz testfile1 testfile2 testdir1

Expected Behavior

An output.tar.gz archive containing the files.

Actual Behavior

$ tar czvf output.tar.gz 'testfile1 testfile2 testdir1'
tar: testfile1 testfile2 testdir1: Cannot stat: No such file or directory

Script to Reproduce

const args = process.argv.slice(3);
const outputPath = args[0];
const inputs = args.slice(1);

await $`tar czvf ${outputPath} ${inputs.join(" ")}`

Actual script is longer with some validation, etc. but that should repro the above output.

The variables in the inputs list are all getting put into 1 string and then I think zx is escaping it (notice the single quotes in the error message around the files/dirs). Any guidance on a different approach is welcome as well.

Question function is throwing the value you type in (node <= v15.7.0)

Example:

This example usage of question always throws with the value you type.

test.mjs

#!/usr/bin/env zx

const answer = await question("Type something here: ");
console.log({ answer });

Expected Behavior

The output should look like this:

❯ ./test.mjs
Type something here: hello
{ answer: 'hello' }

Actual Behavior

❯ ./test.mjs

Type something here: hello

internal/process/esm_loader.js:74
    internalBinding('errors').triggerUncaughtException(
                              ^
hello
(Use `node --trace-uncaught ...` to show where the exception was thrown)

Steps to Reproduce the Problem

1. create the example test.mjs above
2. npm install -g zx
3. chmod a+x ./test.mjs
4. ./test.mjs

Specifications

• Version: [email protected]
• Platform: mac OS Big Sur (Intel)
• Node: v14.15.5

Current workaround

#!/usr/bin/env zx

const answer = await question("Type something here: ").catch((e) => e);
console.log({ answer });

Fetch automatically logging to console

When using the fetch command, $ fetch and the parameters, (e.g the URL and options like method, body, headers) are logged to the console automatically. This is causing issues for some scripts where the output needs to be clean as it gets parsed by another programme. A hacky workaround I've been using is to redefine console.log to a blank function before using fetch and restore afterwards. Is there another way to prevent the logging?

Specifications

• Version: 1.6.0
• Platform: macOS Big Sur

$.verbose = false; is not working as expected when using zx as a module

Expected Behavior

When $.verbose = false then only output of the command should be displayed.

Example:

#!/usr/local/bin/node
import { $ } from 'zx'
$.verbose = false
await ($`uptime`)

Expected output:

9:42  up 16 days, 21 mins, 1 user, load averages: 1.63 2.00 3.2

Actual Behavior

Based on the example above:

$ uptime
9:42  up 16 days, 21 mins, 1 user, load averages: 1.63 2.00 3.2

Steps to Reproduce the Problem

1. Create a js file with the code mentioned above
2. $ node index.js

Specifications

• Version: zx version 1.2.2
• Platform: Darwin 19.6.0 Darwin Kernel Version 19.6.0

Suggested patch:

--- index.mjs
+++ index.mjs.original
@@ -53,12 +53,12 @@
     let child = exec($.prefix + cmd, options)
     let stdout = '', stderr = '', combined = ''
     child.stdout.on('data', data => {
-      process.stdout.write(data)
+      if ($.verbose) process.stdout.write(data)
       stdout += data
       combined += data
     })
     child.stderr.on('data', data => {
-      process.stderr.write(data)
+      if ($.verbose) process.stderr.write(data)
       stderr += data
       combined += data
     })

Example script doesn't show proper usage

Expected Behavior

The example script shows this example:

#!/usr/bin/env zx

let branch = await $`git branch --show-current`

Which leads one to assume you could just use the branch name now, as it's been returned from the command.

Actual Behavior

Despite my expectation, branch is now a ProcessOutput object which must be used as such:

branch.toString()

and that's assuming the command executed correctly!

---

Ideally a successful execution could just return the stdout, rather than the entire ProcessOutput object. Or at the very least, the example script should probably indicate that you cannot just use the stdout results as you would in a bash script. I can certainly see the argument for returning the standard ProcessOutput object every time, I'm just thinking of useful shortcuts here.

---

Steps to Reproduce the Problem

Exactly as described above. Paste into a .mjs file and execute with zx

Specifications

• Version: zx version 1.0.2
• Platform: MacOS 10.13.6

Set verbosity and shell via command line flags

Expected Behavior

• -V to turn off verbose output
• -s SHELL to set which shell to use.

and --help and --version while we're at it lol

Actual Behavior

$ zx -h
internal/process/esm_loader.js:74
    internalBinding('errors').triggerUncaughtException(
                              ^

Error [ERR_MODULE_NOT_FOUND]: Cannot find module '/Users/maxtimkovich/notes/-h' imported from /Users/maxtimkovich/.asdf/installs/nodejs/14.16.0/.npm/lib/node_modules/zx/zx.mjs
    at finalizeResolution (internal/modules/esm/resolve.js:276:11)
    at moduleResolve (internal/modules/esm/resolve.js:699:10)
    at Loader.defaultResolve [as _resolve] (internal/modules/esm/resolve.js:810:11)
    at Loader.resolve (internal/modules/esm/loader.js:86:40)
    at Loader.getModuleJob (internal/modules/esm/loader.js:230:28)
    at Loader.import (internal/modules/esm/loader.js:165:28)
    at importModuleDynamically (internal/modules/esm/translators.js:114:35)
    at exports.importModuleDynamicallyCallback (internal/process/esm_loader.js:30:14)
    at file:///Users/maxtimkovich/.asdf/installs/nodejs/14.16.0/.npm/lib/node_modules/zx/zx.mjs:51:18
    at ModuleJob.run (internal/modules/esm/module_job.js:152:23) {
  code: 'ERR_MODULE_NOT_FOUND'
}

On Windows execution failed

Expected Behavior

success

Actual Behavior

Administrator@WIN-JH5VDQS6BP7 MINGW64 /f/codeTest/jsToSh/src
$ zx file:///F:/codeTest/jsToSh/src/index.js
internal/process/esm_loader.js:74
    internalBinding('errors').triggerUncaughtException(
                              ^

Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only file and data URLs are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'f:'
    at Loader.defaultResolve [as _resolve] (internal/modules/esm/resolve.js:782:11)
    at Loader.resolve (internal/modules/esm/loader.js:86:40)
    at Loader.getModuleJob (internal/modules/esm/loader.js:230:28)
    at Loader.import (internal/modules/esm/loader.js:165:28)
    at importModuleDynamically (internal/modules/esm/translators.js:114:35)
    at exports.importModuleDynamicallyCallback (internal/process/esm_loader.js:30:14)
    at file:///C:/Users/Administrator/AppData/Roaming/npm/node_modules/zx/zx.mjs:51:18
    at ModuleJob.run (internal/modules/esm/module_job.js:152:23)
    at async Loader.import (internal/modules/esm/loader.js:166:24)
    at async Object.loadESM (internal/process/esm_loader.js:68:5) {
  code: 'ERR_UNSUPPORTED_ESM_URL_SCHEME'
}

Administrator@WIN-JH5VDQS6BP7 MINGW64 /f/codeTest/jsToSh/src
$ zx ./index.js
internal/process/esm_loader.js:74
    internalBinding('errors').triggerUncaughtException(
                              ^

Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only file and data URLs are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'f:'
    at Loader.defaultResolve [as _resolve] (internal/modules/esm/resolve.js:782:11)
    at Loader.resolve (internal/modules/esm/loader.js:86:40)
    at Loader.getModuleJob (internal/modules/esm/loader.js:230:28)
    at Loader.import (internal/modules/esm/loader.js:165:28)
    at importModuleDynamically (internal/modules/esm/translators.js:114:35)
    at exports.importModuleDynamicallyCallback (internal/process/esm_loader.js:30:14)
    at file:///C:/Users/Administrator/AppData/Roaming/npm/node_modules/zx/zx.mjs:51:18
    at ModuleJob.run (internal/modules/esm/module_job.js:152:23)
    at async Loader.import (internal/modules/esm/loader.js:166:24)
    at async Object.loadESM (internal/process/esm_loader.js:68:5) {
  code: 'ERR_UNSUPPORTED_ESM_URL_SCHEME'
}

Steps to Reproduce the Problem

1. touch index.js
2. vim index.js

#!/usr/bin/env zx

void (async function () {
	let count = parseInt(await $`ls -1 | wc -l`);
	console.log(`Files count: ${count}`);
})();

3. ./index.js or zx ./index.js or zx file:///F:/codeTest/jsToSh/src/index.js

Specifications

• node: v14.16.0

• Platform: Windows 10 Professional

Allow use without a file extension

I'd love to be able to use zx scripts without a file extension (like $ ./dev instead of $ ./dev.mjs), still with top-level await. I'm not sure what would be required to make that happen, but the readme today says you need the extension.

I'm also not sure whether common editors like VS Code or Github recognize the zx shebang for syntax highlighting as .mjs javascript.

Installed minimum required node version 14.8 but version.js throws error

Expected Behavior

Wrote test.mjs script and followed README -- should execute

Actual Behavior

import {version} from './version.js'
^^^^^^^
SyntaxError: The requested module './version.js' is expected to be of type CommonJS, which does not support named exports. CommonJS modules can be imported by importing the default export.
For example:
import pkg from './version.js';
const {version} = pkg;
at ModuleJob._instantiate (internal/modules/esm/module_job.js:98:21)
at async ModuleJob.run (internal/modules/esm/module_job.js:137:5)
at async Loader.import (internal/modules/esm/loader.js:165:24)
at async Object.loadESM (internal/process/esm_loader.js:68:5)

Steps to Reproduce the Problem

1. Install node 14.8.0
2. Write test.mjs
3. Copy script sample from README

Specifications

• Version: 1.1
• Platform: Mac OSX Big Sur

Support for passing environment variables

👋🏼 Thanks for the library. Such a job to use!

I was wondering if there is any support to extend or pass environment variable when executing a command. Node.js spawn function has support for it as an option. Something which libraries such as execa support. I was wondering how this would or is supported by zx.

🙇🏼 Thanks.

Add hint for required Node version regarding top-level await usage

Expected Behavior

There should be a hint in the README regarding the required Node version for the usage of top-level await.

Actual Behavior

Currently, this is not in the README.

Feature suggestion: include shelljs

I really love that you include a bunch of handy, commonly-used packages without any need to require() them!

I think it'd also be extremely handy to include all of shelljs too, so you can write this:

if (!which('git')) {
  echo('Sorry, this script requires git');
  exit(1);
}

// Copy files to release dir
rm('-rf', 'out/Release');
cp('-R', 'stuff/', 'out/Release');

// Replace macros in each .js file
cd('lib');
ls('*.js').forEach(function (file) {
  sed('-i', 'BUILD_VERSION', 'v0.1.2', file);
  sed('-i', /^.*REMOVE_THIS_LINE.*$/, '', file);
  sed('-i', /.*REPLACE_LINE_WITH_MACRO.*\n/, cat('macro.js'), file);
});
cd('..');

// Run external tool synchronously
if (shell.exec('git commit -am "Auto-commit"').code !== 0) {
  echo('Error: Git commit failed');
  exit(1);
}

Instead of this:

var shell = require('shelljs');

if (!shell.which('git')) {
  shell.echo('Sorry, this script requires git');
  shell.exit(1);
}

// Copy files to release dir
shell.rm('-rf', 'out/Release');
shell.cp('-R', 'stuff/', 'out/Release');

// Replace macros in each .js file
shell.cd('lib');
shell.ls('*.js').forEach(function (file) {
  shell.sed('-i', 'BUILD_VERSION', 'v0.1.2', file);
  shell.sed('-i', /^.*REMOVE_THIS_LINE.*$/, '', file);
  shell.sed('-i', /.*REPLACE_LINE_WITH_MACRO.*\n/, shell.cat('macro.js'), file);
});
shell.cd('..');

// Run external tool synchronously
if (shell.exec('git commit -am "Auto-commit"').code !== 0) {
  shell.echo('Error: Git commit failed');
  shell.exit(1);
}

The await $`foo ${arg}` syntax is definitely great too and I'm excited to use it as well, but for many of these commonly-used commands, synchronous js-native functions would be handy too – especially with escaping/quoting challenges (eg, I love that you can use a JS regex with sed in shelljs, and I'm not sure how that'd translate directly into $`cmd` syntax).

using colorette instead of chalk

Chalk, however, is quite mature and most widely used package, is adding more Node dependencies, ~100KB install size, memory footprint, and ~36KB to bundle size (#50, #51) which is ~40% of the entire bundle! One suggestion would be using colorette which is basically few lines of code, doing most of the formatting possibilities of chalk, supporting NO_COLOR and possibly making Deno support (#24) easier.

Feature suggestion: per command verbosity

Being able to turn verbosity on/off for individual command.

Use case

let [_, result2] = await Promise.all([
    $`command-with-progress-bar`,
    $`command-to-be-muted`,
])
// do something else with `result2`

I want to monitor some commands' output (e.g. it has a progress bar) and not display the others' (while still keep them for further calculation). If the commands are executed synchronously, I can change the global $.verbose setting in between. However, for parallel async executions, it is currently impossible to have different setting for each command.

Command parsing

If a command simple enough, only programs with args and pipes → parse the command and use spawn with an array of args and use Node.js to do all the piping.

Transform this:

find ${dir} -type f -print0 | xargs -0 grep foo | wc -l

Into this:

pipe(
  spawn('find', [dir, '-type', 'f', '-print0']),
  spawn('xargs', ['-0', 'grep', 'foo']),
  spawn('wc', ['-l']),
)

Idea: Self-installing runtime to run scripts anywhere

Maybe it could be a good idea to include a compact line at the top of zx scripts that installs the required runtime if it is not present on the machine. Something like executing a sh that reads from curl hosted on the official github repo or similar. This would allow the scripts to be executed at any time without installing anything.

Maybe it could even be possible to install things (node, zx) locally if the user does not have enough permissions and/or the execution is unattended.

See how gradle does a similar thing with gradle wrapper:
https://docs.gradle.org/current/userguide/gradle_wrapper.html#sec:using_wrapper

can not run sample of README.md first one

Expected Behavior

print output to console

Actual Behavior

file:///Users/zq-jhon/.nvm/versions/node/v10.2.0/lib/node_modules/zx/zx.mjs:43
let ok = await scriptFromStdin()
^^^^^

Steps to Reproduce the Problem

1. npm i zx -g
2. cd ~/desktop && vim test.mjs
3. paste the code and :wq save and quit:

#!/usr/bin/env zx

await $`cat package.json | grep name`

let branch = await $`git branch --show-current`
await $`dep deploy --branch=${branch}`

await Promise.all([
  $`sleep 1; echo 1`,
  $`sleep 2; echo 2`,
  $`sleep 3; echo 3`,
])

let name = 'foo bar'
await $`mkdir /tmp/${name}`

4. zx ./test.mjs

Specifications

• Version:

1. node version: v12.21.0
2. nvm version: v0.37.0

• Platform: macOS Catalina v10.15.7