Comments (12)
@biels Yes. It would be a tree of streams that efficiently pipe data from stdout to stdin. That is what Node Streams is patterned after, pipes, pehaps even UNIX pipes. There is this wonderful opportunity here to parse the command line and make it come to life within node.
The main argument against the brilliant $`command1 | command2` is the same argument that is made for Node Streams, read chunk at a time, instead of cramming the whole thing into memory.
I think antonmedv did an amazing job, this is not a critique, I must admit, I think this is outside of the scope of this program.
I am simply saying parse the commands with peg.js build a child_process stream tree and allow for incredible complexity. Complexity and functionality currently obscured by node just sending a string to /bin/sh and then sitting there and waiting until a potentially massive chunk of text is returned.
So it is not just a single child_proces that wraps the entire command, it is a thing that interprets the command line, and creates a child_proces tree. It will work for simple commands like $`tar -cvf bork.tar bork` but also for freakishly complex things that reroute output streams, run as a daemon, respond to something like UNIX's nice(1) command, and enhance the operating system with node.
await $`cat package.json | grep name`
Will actually start two
child_processes one is cat and the other grep, cat will send things to grep chunk at a time, by relying on Node Streams.
The next step to this would be /bin/zxsh a zx based Bash replacement. Which will not be possible to make if we just send strings at /bin/sh
And the step after that would be using Electron as a replacement for X Windows, perhaps with https://nextapps-de.github.io/winbox/
and then antonmedv could release a https://www.linuxfromscratch.org/ called ANTONUX
And we would all end up using it.
My instinct says that
let name = await $`cat package.json | grep name`;
obscures powerful things, that if we can get access too we can create an alternative to https://www.npmjs.com/package/pm2 alternative to Bash, and even mess around with electron as a GUI something that could potentially be as tiny as http://www.damnsmalllinux.org/
The sending a string to /bin/sh and waiting for one back obscures powerful functionality.
antonmedv your work is perfect, zx is awesome, this is not a critique, but a discussion of a potential future of this program.
You started something, that could develop legs.
Way to go!
from zx.
@catpea What about:
await $`cat package.json`.pipe($`grep name`)
let name = await $`cat package.json`.pipe($`grep name`); // works since `.pipe` returns its argument
The main idea is to transform Promise<ProcessOutput>
into DuplexStream & PromiseLike<ProcessOutput>
and rely on .pipe
. This is significantly easier to implement than using pegjs, while increasing the awesomeness of pipes:
const r = fs.createReadStream('file.txt');
const w = fs.createWriteStream('file.txt.gz');
await r.pipe($`gzip`).pipe(w);
Pros:
- Easy to implement (redirect duplex input to stdin and stdout to duplex output, and add a
.then
function forawait
to work) - User-friendly
- Might not break the traditional
|
(await $`cat package.json | grep name`;
) - Seamless integration with any node library using streams.
- Improve support for Windows
Cons:
- May affect performance (but the user can always fall back on
|
in this case) - Node specific feature (but
fs
andos
are already node only)
from zx.
This is excellent! A lot of other great things have been added since I opened this issue as well. Thank you for bringing this library to such a high standard.
from zx.
I think this is out of the scope of the zx project. Closing.
UPDATE 2022-05-31
Now zx supports real time output as well.
Pipes can be used to show real-time output of programs:
$.verbose = false
await $`echo 1; sleep 1; echo 2; sleep 1; echo 3;`
.pipe(process.stdout)
Or like this:
let { stdin, stdout } = $`npm init`
for await (let chunk of stdout) {
if (chunk.includes('package name:')) stdin.write('test\n')
}
from zx.
Now zx support real time output!
from zx.
@catpea What you are suggesting would be quite involved to implement but I think you are on to something.
from zx.
Why not just use nodejs API directly for this?
from zx.
Sorry if I'm misunderstanding the way this works. I thought the promise of $some-command
only resolved after the command exited? In this way it would not be possible to get the output of a command as it is happening before it terminated.
from zx.
Yes, only then the command finished.
from zx.
I am worried about this.
zx is a great idea and $`` is wonderfully lightweight - you are wonderful!
But my feline instincts, are saying, get https://pegjs.org/ in here, and instead of shelling everything out to /bin/sh get node to take over.
This will not be lightweight, but it will unfurl what is going on between $`a | b` and grant access to all the output streams of all the commands being executed.
Now you will be able to get the stream of a's stderr for example, as it stdout into b's stdin. This is what programming is about, beautiful great things that keep you awake at night... that rustle your jimmies... that use node to make running commands even more powerful...
Instead of being clever and lightweight, which is very noble, you become a leader, you become legendary, which is even better.
I think we forget not to $`` all to often, here's Brian to remind us, what it means to hold a legendary idea:
https://www.youtube.com/watch?v=tc4ROCJYbm0&t=249s
Make flow control a first class citizen via https://www.sweetjs.org/ and instead of $`a | b` which is "what it is", allow for something even more brilliant like $ cat purr.txt | wc -l
(I leave the DSL sweetness up to you as you are excellent at it)
Make $`commands` into real commands, not something that is just thrown at /bin/sh, but something that comes to life in node.
@yvesnrb is on the right path, we want to keep getting the data, we want to $.stream`tail -f logs/flarp` or read sockets.
pingStream.on('data', () => {
console.log('ping wrote to stdout');
});
This is an opportunity to change the world,
you made something special, and @yvesnrb has the right idea to make it better.
from zx.
@catpea If I understand correctly are you talking about executing executables using child_process directly instead of executing sh commands?
from zx.
@catpea For those looking for the pipeline feature, I made bazx
For instance (#35 (comment))
console.log(await collect($`find ${dir} -type f -print0`.pipe($`xargs -0 grep foo`).pipe($`wc -l`)))
// $ find /tmp/bazx/test -type f -print0 | xargs -0 grep foo | wc -l
// 3
// { success: true, code: 0, stdout: "3", stderr: "" }
https://github.com/Minigugus/bazx/blob/023496cae0a53c0ed9b9c4f72f799a3ca36a6703/test/debug.js#L5-L13
from zx.
Related Issues (20)
- Setting stdio/err to inherit throws error HOT 1
- Typo in 8.0 release description HOT 1
- Feature request: automatically add a newline after commands that don't end their output with one HOT 4
- The type for `glob` is `any` after 8.0.0 HOT 5
- Port listening doesn't stop when using `CTRL+C` HOT 11
- Feature request: add `usePwsh` helper for PowerShell 7+ HOT 3
- Why zx instead of Python ? HOT 2
- Markdown script didn't work for CRLF file
- Is Node.js 12.17.0 supported for zx as a package ? HOT 7
- Single file executable?
- Docs: Typo in setup docs
- fetch's document link is wrong HOT 4
- bun compatibility: get rid of async_hooks.createHook call HOT 3
- cannot concatenate strings with ${} HOT 4
- Understanding `zx` roadmap and idea 🙏 HOT 6
- YAML.parseAllDocuments is no longer accessible HOT 8
- Provide better out of the box TypeScript support HOT 2
- Could not resolve "zx/globals.cjs" HOT 4
- Since 8.1.4 YAML.parseAllDocuments is not allowed from zx HOT 9
- Importing this package into CJS project triggers false-positive TypeScript error HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from zx.