Giter Site home page Giter Site logo

cmd's Introduction

go-cmd/Cmd

Go Report Card Coverage Status Go Reference

This package is a small but very useful wrapper around os/exec.Cmd that makes it safe and simple to run external commands in highly concurrent, asynchronous, real-time applications. It works on Linux, macOS, and Windows. Here's the basic usage:

import (
	"fmt"
	"time"
	"github.com/go-cmd/cmd"
)

func main() {
	// Start a long-running process, capture stdout and stderr
	findCmd := cmd.NewCmd("find", "/", "--name", "needle")
	statusChan := findCmd.Start() // non-blocking

	ticker := time.NewTicker(2 * time.Second)

	// Print last line of stdout every 2s
	go func() {
		for range ticker.C {
			status := findCmd.Status()
			n := len(status.Stdout)
			fmt.Println(status.Stdout[n-1])
		}
	}()

	// Stop command after 1 hour
	go func() {
		<-time.After(1 * time.Hour)
		findCmd.Stop()
	}()

	// Check if command is done
	select {
	case finalStatus := <-statusChan:
		// done
	default:
		// no, still running
	}

	// Block waiting for command to exit, be stopped, or be killed
	finalStatus := <-statusChan
}

That's it, only three methods: Start, Stop, and Status. When possible, it's better to use go-cmd/Cmd than os/exec.Cmd because go-cmd/Cmd provides:

  1. Channel-based fire and forget
  2. Real-time stdout and stderr
  3. Real-time status
  4. Complete and consolidated return
  5. Proper process termination
  6. 100% test coverage, no race conditions

Channel-based fire and forget

As the example above shows, starting a command immediately returns a channel to which the final status is sent when the command exits for any reason. So by default commands run asynchronously, but running synchronously is possible and easy, too:

// Run foo and block waiting for it to exit
c := cmd.NewCmd("foo")
s := <-c.Start()

To achieve similar with os/exec.Cmd requires everything this package already does.

Real-time stdout and stderr

It's common to want to read stdout or stderr while the command is running. The common approach is to call StdoutPipe and read from the provided io.ReadCloser. This works but it's wrong because it causes a race condition (that go test -race detects) and the docs say it's wrong:

It is thus incorrect to call Wait before all reads from the pipe have completed. For the same reason, it is incorrect to call Run when using StdoutPipe.

The proper solution is to set the io.Writer of Stdout. To be thread-safe and non-racey, this requires further work to write while possibly N-many goroutines read. go-cmd/Cmd has done this work.

Real-time status

Similar to real-time stdout and stderr, it's nice to see, for example, elapsed runtime. This package allows that: Status can be called any time by any goroutine, and it returns this struct:

type Status struct {
    Cmd      string
    PID      int
    Complete bool
    Exit     int
    Error    error
    Runtime  float64 // seconds
    Stdout   []string
    Stderr   []string
}

Complete and consolidated return

Speaking of that struct above, Go built-in Cmd does not put all the return information in one place, which is fine because Go is awesome! But to save some time, go-cmd/Cmd uses the Status struct above to convey all information about the command. Even when the command finishes, calling Status returns the final status, the same final status sent to the status channel returned by the call to Start.

Proper process termination

os/exec/Cmd.Wait can block even after the command is killed. That can be surprising and cause problems. But go-cmd/Cmd.Stop reliably terminates the command, no surprises. The issue has to do with process group IDs. It's common to kill the command PID, but usually one needs to kill its process group ID instead. go-cmd/Cmd.Stop implements the necessary low-level magic to make this happen.

100% test coverage, no race conditions

In addition to 100% test coverage and no race conditions, this package is actively used in production environments.


Acknowledgements

Brian Ip wrote the original code to get the exit status. Strangely, Go doesn't just provide this, it requires magic like exiterr.Sys().(syscall.WaitStatus) and more.


License

MIT © go-Cmd.

cmd's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cmd's Issues

Appropriate way to handle STDIN?

Howdy.

We are in process of evaluating this package to help streamline running external processes however it is already plausible in our use cases that writing to STDIN would be of value. What route is recommended for writing to STDIN with this package? Seems we will have to modify it to support this but perhaps there are lessons learned that could be discussed here.

If we end up going with this go-cmd, would a PR for STDIN support be in line with the vision of this package, @daniel-nichter ?

Thank you for your time.

Stdout empty when using Streaming

when i use Options

package main

import (
	"fmt"
	"github.com/go-cmd/cmd"
	"os/exec"
	"syscall"
)

func main() {

	opt := cmd.Options{
		Streaming: true,
		BeforeExec: []func(cmd *exec.Cmd){
			func(cmd *exec.Cmd) { cmd.SysProcAttr = &syscall.SysProcAttr{HideWindow: true} },
		},
	}


	newCmd := cmd.NewCmdOptions(opt, "cmd.exe", "/c", "whoami")
	//newCmd := cmd.NewCmd("cmd.exe", "/c", "whoami")

	gotStatus := <-newCmd.Start()

	fmt.Println(gotStatus.Stdout)

}

have no Stdout
===>output : [ ]

if not use Options it have a Stdout

package main

import (
	"fmt"
	"github.com/go-cmd/cmd"
)

func main() {

	//opt := cmd.Options{
	//	Streaming: true,
	//	BeforeExec: []func(cmd *exec.Cmd){
	//		func(cmd *exec.Cmd) { cmd.SysProcAttr = &syscall.SysProcAttr{HideWindow: true} },
	//	},
	//}

	//newCmd := cmd.NewCmdOptions(opt, "cmd.exe", "/c", "whoami")
	newCmd := cmd.NewCmd("cmd.exe", "/c", "whoami")

	gotStatus := <-newCmd.Start()

	fmt.Println(gotStatus.Stdout)

}

===>output : [desktop-123\nobody]

go get does not work on Windows

Windows 10

When I type go get github.com/go-cmd/cmd

I get the following error

github.com/go-cmd/cmd

go\src\github.com\go-cmd\cmd\cmd.go:123:9: undefined: syscall.Kill
go\src\github.com\go-cmd\cmd\cmd.go:173:48: unknown field 'Setpgid' in struct literal of type syscall.SysProcAttr

Process not terminating

I am on Mac.

When I run the streaking example but calling my long running server ( a golang server ), and I use control c key my golang server is still hanging around..

??

Something in the example is missing for the streaking example ?

Or my golang server is not sending out the correct termination signals ?

Command is still executed if Stop() is called during BeforeExec hooks

If for some reason a BeforeExec takes some time (to delay a command for example) and Stop() is called while BeforeExec hasn't finished, the command is still executed.

Example:

package main

import (
	"fmt"
	"os/exec"
	"time"

	"github.com/go-cmd/cmd"
)

func main() {
	cmdOptions := cmd.Options{
		Buffered:  true,
		Streaming: true,
		BeforeExec: []func(_c *exec.Cmd){
			func(_c *exec.Cmd) {
				for i := 0; i < 4; i++ {
					fmt.Printf("Waiting %d\n", i)
					time.Sleep(1 * time.Second)
				}
			},
		},
	}

	c := cmd.NewCmdOptions(cmdOptions, "ls", "-l")

	fmt.Printf("Starting\n")
	c.Start()
	time.Sleep(2 * time.Second)

	fmt.Printf("Stopping\n")
	err := c.Stop()
	if err != nil {
		fmt.Printf("E: %s\n", err)
	}

	fmt.Printf("Waiting process to end\n")
	<-c.Done()
	fmt.Printf("Done\n")
	fmt.Printf("Output: %s\n", c.Status().Stdout)
}

Output:

Starting
Waiting 0
Waiting 1
Waiting 2
Stopping
E: command not running
Waiting process to end
Waiting 3
Done
Output: [total 8 -rw-r--r--  1 renard  staff  612 Sep 13 22:24 main.go]

Stop() has no effect if command is not started because of

cmd/cmd.go

Lines 290 to 297 in fa11d77

// c.statusChan is created in StartWithStdin()/Start(), so if nil the caller
// hasn't started the command yet. c.started is set true in run() only after
// the underlying os/exec.Cmd.Start() has returned without an error, so we're
// sure the command has started (although it might exit immediately after,
// we at least know it started).
if c.statusChan == nil || !c.started {
return ErrNotStarted
}

run() runs all BeforeExec functions and starts the command (

cmd/cmd.go

Lines 431 to 448 in fa11d77

// Run all optional commands to customize underlying os/exe.Cmd.
for _, f := range c.beforeExecFuncs {
f(cmd)
}
// //////////////////////////////////////////////////////////////////////
// Start command
// //////////////////////////////////////////////////////////////////////
now := time.Now()
if err := cmd.Start(); err != nil {
c.Lock()
c.status.Error = err
c.status.StartTs = now.UnixNano()
c.status.StopTs = time.Now().UnixNano()
c.done = true
c.Unlock()
return
}
).

Would it make sense to add a ForceStop() function like:

// ForceStop forces a command stop by calling Stop(). If the command hasn't
// started yet flags it as stopped to prevent Start() from starting it.
//
// See Stop() for further details
func (c *Cmd) ForceStop() error {
	err := c.Stop()
	// If command hasn't started, return error (or nil) from Stop()
	if err != ErrNotStarted {
		return err
	}
	// flag command as stopped to prevent it from being started later.
	c.stopped = true
	return nil
}

Thanks in advance

How to test executed commands

Any tip on how to mock a command execution for testing?

I tried the trick from here: https://npf.io/2015/06/testing-exec-command/ but it's not working properly

// cmd.go tweak

var ComandExecuter = exec.Command

func (c *Cmd) run() {
	defer func() {
		c.statusChan <- c.Status() // unblocks Start if caller is waiting
		close(c.doneChan)
	}()

	// //////////////////////////////////////////////////////////////////////
	// Setup command
	// //////////////////////////////////////////////////////////////////////
	cmd := ComandExecuter(c.Name, c.Args...)

Child process not killed with various signals

I have found that when I’m executing my go tool that uses a cmd and kill the go tool with ctrl C the process that was created by the NewCmd is still running. So I used a channel for signals which worked for that case. However when I tested with sigkill I was not able to handle it and the child process was still running. Any suggestions?

Edit: Go noob here but I understand this https://medium.com/@felixge/killing-a-child-process-and-all-of-its-children-in-go-54079af94773 but not how to handle if I send a kill -9 from terminal

Edit2: I think I have just realized that kill -9 is by definition not expected to be caught and is not something that will be used often, so I should never expect to be able to handle this case. #theMoreYouKnow I'm closing the issue :-)

stop() error on windows

os: Windows 10 21H1
go: 1.17.1

this code:

c := cmd.NewCmd("sleep", "10")
c.Start()
time.Sleep(5000 * time.Millisecond)
if err := s.Stop(); err != nil {
   fmt.Printf("%v", err)
}

output:
DuplicateHandle: Descripteur non valide

is it possible to change password?

is it possible to change linux user password ?
$ passwd mostain

i have tried but shown the following error:

Current password: passwd: Authentication token manipulation error
passwd: password unchanged
Changing password for mostain.

after investigation it seems program terminating by the os with exit status 10

Compile error on windows

.\cmd.go:192:9: undefined: syscall.Kill
.\cmd.go:244:48: unknown field 'Setpgid' in struct literal of type syscall.SysProcAttr

Maintaining the project

Hi @daniel-nichter , I would like to maintain this project, if you agree.

This is the initial list of things I would like to do:

  • keep the test coverage above 95% or 100% if possible
  • simplify the tests by using a more advanced testing library?
  • implement a Clone function to duplicate a Cmd to be able to start it again, fresh
  • implement sending signals to the Cmd
  • implement a consistent state of the Cmd, instead of multiple variables (started, stopped, final, etc)
  • implement watching the state changes via channels
  • more love to the readme file
  • find a nice logo? :)
  • support only the 3 most recent Go-lang versions (ex: currently 1.12, 1.11 and 1.10)
  • I understand there are projects using this lib, so I'll work on a separate development branch and create tags for every major change

I am building a process manager on top of this project and that project is used in another project so it's in my best interest to have this lib stable and up to date.
You can see some of the changes I mentioned above in here: https://github.com/ShinyTrinkets/overseer but I'll take a safer approach with this lib.

ErrLineBufferOverflow error when a command uses return (\r) to overwrite previous output line

Some commands use a return character at the end of each line in order to overwrite the previous output line with new data. This is often used to show progress of the running command. An example is the dc3dd command. This command outputs the progress of the command to one line that consistently gets overwritten with the new progress stats. If a long running command does this the output will continually be saved to the buffer and not passed to the go channel in the Write function of OutputStream because this function only checks for \n or \r\n at the end of a line. Because a command like this will never have a \n at the end of an output line the buffer will overflow every time the command is run.

Mixed output

I took the code from examples/blocking-streaming and replaced the print-some-lines with something that prints (and flushes) 0 to 9 randomly to stdout or stderr:

#!/usr/bin/env python3

import sys
import time
import random

for i in range(10):
    r = random.random()
    if r > 0.5:
        sys.stdout.write("o: %d\n" % i)
        sys.stdout.flush()
    else:
        sys.stderr.write("e: %d\n" % i)
        sys.stderr.flush()
    time.sleep(0.000035)

One example can be:

e: 0
e: 1
o: 2
o: 3
e: 4
o: 5
e: 6
e: 7
o: 8
e: 9

Numbers should always be in order but the output channel (o or e) can change.

I ran the following test (It grabs the numbers, computes the md5sum, and see if there are any diverting results):

for i in $(seq 100); do ./cmd-test . 2>&1| awk '{print $2}'| md5; done | sort | uniq -c

The expected output is:

 100 e20b902b49a98b1a05ed62804c757f94

If the sleep time is too low (0.000035 something on my computer) I start to get some weird results such as:

   2 6d0cfa70985c6d01424ec5c2cfb50c4d
  98 e20b902b49a98b1a05ed62804c757f94

Meaning a mix in result lines. I start to get:

e: 0
e: 2
e: 4
o: 1
o: 3
o: 6
o: 9
e: 5
e: 7
e: 8

Did I missed something? How can I manage to get lines in correct order?

(I can provide further details if needed)

Thanks in advance.

Why status.Stdout is empty in my code

func main() {
	findCmd := cmd.NewCmd("dir")
	statusChan := findCmd.Start()
	go func() {
		for {
			fmt.Println("-------")
			status := findCmd.Status()
			fmt.Println(status.Stdout)
		}
	}()
	<-statusChan
	fmt.Println("=======")
}

I run this program on Win10, and get something like that:

-------
[]
-------
[]
-------
[]
-------
[]
-------
[]
-------
[]
-------
[]

So, what's the problem of it.

Proposal: Add details to cli error message

Hello there!
I would like to add the grpc status details in the error output of the CLI, I simply want to display the response as google designed it, see here

Currently the CLI print the error like this: (cmd/grpcurl/grpcurl.go)

if h.Status.Code() != codes.OK {
	fmt.Fprintf(os.Stderr, "ERROR:\n  Code: %s\n  Message: %s\n", 
		h.Status.Code().String(), h.Status.Message())
	exit(1)
}

I propose to add the details:

if h.Status.Code() != codes.OK {
	fmt.Fprintf(os.Stderr, "ERROR:\n  Code: %s\n  Message: %s\n Details: %+v\n", 
		h.Status.Code().String(), h.Status.Message(), h.Status.Details())
	exit(1)
}

I will be happy to do the PR, so what do you think?

P.S: I was wondering if we could add an option to output this in JSON too, since the CLI already outputs in JSON for OK responses.

python command in streaming mode not streaming the stdout

Hey, thank you so much for the hard work building this package!

I have created a function that takes command and run it, and prints the stdout/stderr live.
I have 1 issue when I run the below code, the prints start only after the command is finished, but when I add -u to the command its starts printing and I don't want to use the -u flag on python. this is not happening if I use the exec go build-in, so I will appreciate any insight here.

	cmdOptions := cmd.Options{
		Buffered:  false,
		Streaming: true,
	}
	envCmd := cmd.NewCmdOptions(cmdOptions, "python3", "train.py")
	
	for envCmd.Stdout != nil || envCmd.Stderr != nil {
			select {
			case line, open := <-envCmd.Stdout:
				if !open {
					envCmd.Stdout = nil
					continue
				}
				fmt.Println(line)
			case line, open := <-envCmd.Stderr:
				if !open {
					envCmd.Stderr = nil
					continue
				}
				fmt.Fprintln(os.Stderr, line)
			}
		}

Command never returns if streaming std not read

I have a slightly modified version of the blocking-streaming example:

func (bs *BashExecutor) ExecuteCommand(commandString string) int {
	// Disable output buffering, enable streaming
	cmdOptions := cmd.Options{
		Buffered:  false,
		Streaming: true,
	}

	mphCmd := cmd.NewCmdOptions(cmdOptions, "bash", "-c", commandString)

	// Print to STDOUT
	if bs.verbose {
		go func() {
			for line := range mphCmd.Stdout {
				fmt.Println(line)
			}
		}()
	}

	// Run the command
	cmdStatus := <-mphCmd.Start()

	// Cmd has finished but wait for goroutine to print all lines
	for len(mphCmd.Stdout) > 0 {
		time.Sleep(10 * time.Millisecond)
	}

	if cmdStatus.Exit != 0 {
		glog.Errorf("Exit code: %d\n", cmdStatus.Exit)
		glog.Errorf("Executed command: %s\n", cmdStatus.Cmd)
		glog.Errorf("Error: %+v", cmdStatus.Error)
		for _, line := range cmdStatus.Stderr {
			fmt.Println(line)
		}
		return cmdStatus.Exit
	}

	glog.Infof("Command took %f seconds to complete", cmdStatus.Runtime)
	return cmdStatus.Exit
}

When bs.verbose is true, everything works fine, the command out is printed to the STDOUT.

When bs.verbose is false, the command std out is never read and the cmdStatus := <-mphCmd.Start() never returns.

I tried to change the method to:

if bs.verbose {
		go func() {
			for line := range mphCmd.Stdout {
				fmt.Println(line)
			}
		}()
	} else {
		go func() {
			for range mphCmd.Stdout {
				// do nothing
			}
		}()
	}

and things start working again. Is there something I am supposed to do when the output is not read?

Hacking go-cmd

Hi !

Is it Ok if I embed the cmd.go file in my process manager library? Obviously, I'll credit the authors and provide links to the original repo.
This is the link: https://github.com/ShinyTrinkets/overseer.go (also MIT licensed)

I'm already using a fork, but I need to make more drastic changes, like removing the OutputBuffer completely, because I'm not using it and defining more detailed steps for the running process, inspired by supervisord. But if I'm wrapping on top of what's already in cmd.go, I have to duplicate some functionality.

I'm pretty new to Golang, so I'll definitely experiment a lot before reaching a decent design 😅

Thank you!

Process not being killed

I'm working on wrapping cloudflared with a HTTP interface, but I'm experiencing issues wherein the command isn't being killed and my application is hanging trying to read from the 'done' channel (<-command.Done()).

Amongst other bits, I have a function along the following lines that is called when I attempt to 'close' a tunnel:

func stopTunnel(tun *tunnelCmd) {
	log.Infof("Stopping tunnel for %s", tun.Hostname)

	var complete = tun.Command.Status().Complete
	if !complete {
		if err := tun.Command.Stop(); err != nil {
			log.Error(err)
		}

		// Added for debugging
		log.Warn("Waiting for Done()")
		go func() {
			for {
				time.Sleep(2 * time.Second)
				log.Infof("-- %+v", tun.Command.Status())
			}
		}()
		// end

		<-tun.Command.Done()
		log.Infof("Existing tunnel for %s stopped with code %d", tun.Hostname, tun.Command.Status().Exit)
	}

	log.Infof("Stopped tunnel for %s", tun.Hostname)
}

In the above example, I've added in a goroutine to poll, so I can observe the state of the Command and I can see that it's still running. If I manually pkill -9 <pid> it will unblock the Done channel and all is as expected.

I'm looking at possibly forking and adding some logging because I'm not sure what's causing this issue, but I'm also concerned that perhaps the actual SIGTERM needs to be better enforced? Any ideas off that bat?

Thanks for your work on this library!

Converting Stdout []string back to []byte?

Hi there,

While using this library, I wanted to take the stdout and convert it back into []byte array. However, as Stdout returns a string using the default buffer, I'm having trouble with that.

I tried something along the line of

tfCmd := cmd.NewCmd("/export/content/lid/terraform/bins/0.12.20/terraform", "show", "-no-color", "-json", "tfaasgenerated/plan.tf.out")
// tfCmd := cmd.NewCmd("/export/content/lid/terraform/bins/0.12.20/terraform", "show", "-no-color", "-json")
tfCmd.Dir = "/export/content/lid/data/tfaas-worker/fbe47527-6dd2-4283-95ce-298bdd2e12d9/iac/lit-zscus-1/test-unmanaged"
result := <-tfCmd.Start()
b, err := terraform.NewJSONProcessor(strings.Join(result.Stdout, "\n"), terraform.WithFilter("prior_state")).Bytes()
assert.Nil(t, err)
fmt.Println(fmt.Sprintf("%s", b))

However, parts of the values are missing in the conversion back to []byte! I'm not sure why this is the case. Is there any advice on the way to handle this?

not work on windows

.\api-service-platform-master-fbddfc271a9e5a3d8ac1e03ac6305e24a7325521\pkg\mod\github.com\go-cmd\[email protected]\cmd.go:298:41: unknown field 'Setpgid' in struct literal of type syscall.SysP
rocAttr

Long running commands with a lot of output

Hi,
Is there a way to avoid the output accumulation in the string arrays, and get the output data in real time in a channel, or with a callback ?

For instance to handle a long running tail -f on a fast growing file.

Thank you for the great work.

Part of #11517

Part of #11517

(take 2 - other branch got accidentally messed up when I tried to update it...)

Description

This is to position the quick view card relative to the the position of the pr list item. It will align top if there is height enough and otherwise align bottom if there is height enough. Otherwise, it will attempt to center the quick view card on the pr list view item with a min top position of the top of the list and a min bottom position of visible on screen.

Screenshots

Screen.Recording.2021-12-09.at.2.19.27.PM.mov

Release notes

Notes: no-notes

Originally posted by @tidy-dev in desktop/desktop#13552

Pseduo code in README

There are some issues in your README - example, which prevents users to copy and paste it to test.

Mainly:

status := c.Status() -> c never declared. 
finalStatus -> never used

and main function missing. Wouldn't it be better to show off a running code snippet instead of a pseudo code?

Race condition on short Timeout

There is a race condition if you try to stop the process after a short timeout. I think the problem, is that c.run() is called in an own go routine:

go c.run()
return c.statusChan

The example program may call Stop before the process even started and returns nil. The external program is not killed. One solution would be to select on status and check started before defining any timeout.

package main

import (
	"fmt"
	"time"

	"github.com/go-cmd/cmd"
)

func main() {
	// Start a long-running process, capture stdout and stderr
	findCmd := cmd.NewCmd("find", "/")
	statusChan := findCmd.Start() // non-blocking

	ticker := time.NewTicker(2 * time.Second)

	// Print last line of stdout every 2s
	go func() {
		for range ticker.C {
			status := findCmd.Status()
			n := len(status.Stdout)
			fmt.Println(status.Stdout[n-1])
		}
	}()

	// Stop command after short time.
	go func() {
		<-time.After(1 * time.Millisecond)
		fmt.Printf("Timeout reached.")
		findCmd.Stop()
	}()

	// Block waiting for command to exit, be stopped, or be killed
	finalStatus := <-statusChan
	fmt.Printf("%+v", finalStatus)
}

window get

github.com/go-cmd/cmd

..\src\github.com\go-cmd\cmd\cmd.go:123: undefined: syscall.Kill
..\src\github.com\go-cmd\cmd\cmd.go:173: unknown field 'Setpgid' in struct literal of type syscall.SysProcAttr

How to Fix it?

window10 x64
my environment:
set GOARCH=amd64
set GOBIN=D:\Go\bin
set GOEXE=.exe
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=E:\Works\GoLang
set GORACE=
set GOROOT=D:\Go
set GOTOOLDIR=D:\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\jeffery\AppData\Local\Temp\go-build544971735=/tmp/go-build -gno-record-gcc-switches
set CXX=g++
set CGO_ENABLED=1
set PKG_CONFIG=pkg-config
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2

Is there a way to have stderr buffered into stdout

I know the standard os/exec lib has a function called CombinedOutput. Can go-cmd do something similar? My use case is after an executable has run, I am writing the stdout / stderr buffers to a log file. If the executable's output produces both stdout and stderr output, I don't have a way to produce a consolidated log in the order the output was received. I know this can be handled by streaming and the select, but was wanting to use the buffer since I don't have a need for real time inspection.

New release after go.mod changes

Thanks for accepting PR #37 .

As a logical extension to the same, it would be useful to have a 1.0.6 or a 1.1.0 release as you see appropriate. This makes projects dependent on go-cmd have a more readable go.mod as opposed to yyyymmdd-hash kind of version in their projects.

make terminating process group optional

I've got a use case where I don't necessarily want to terminate an entire process group (i.e. having children outlive the parent is desired).

If you're open to such a patch, I can look into implementing it. The default behavior should be the current behavior: SIGTERM the process group.

When launching the same binary, one instance closes the other

Hi,

so this is the snipper.

package main

import (
	"log"
	"time"

	"github.com/go-cmd/cmd"
)

func main() {
	p1 := cmd.NewCmd("chromium")
	p1done := p1.Start()
	go func() {
		select {
		case pp1 := <-p1done:
			log.Println(pp1)
		}
	}()

	p2 := cmd.NewCmd("chromium")
	p2done := p2.Start()
	go func() {
		select {
		case pp2 := <-p2done:
			log.Println(pp2)
		}
	}()

	time.Sleep(time.Second * 20)
}

When running the code, there is p1 exit or p2 exit in the output, but the windows are still running.
I believe it has got something to do with how chromium handles multiple windows. But I also think that a message should be sent over the channel, if the process is still running.
Same thing happens for other GUI apps as well.

Am I missing something?

example cmd.NewCmdOptions not working

example of examples/blocking-streaming/main.go directory does not produce any output.
But print-some-lines produces output when ran using the following command:

$ bash print-some-lines

Line 2
Line 3
Line 4
Line 5
One more...
Last line

Stdout empty when line is 65k or more

Hi, thank you for writing this package. I'm using it to run a command that outputs a long JSON string all on one line. I discovered that when the output reaches 2^16 chars on a line the Stdout is empty. An example program is below:

package main

import ( 
    "fmt"
    "log"

    "github.com/go-cmd/cmd"
)

func main() { 
    for i := 65534; i < 65540; i++ {
        bashScript := fmt.Sprintf("for i in {1..%d} ; do echo -n X ; done", i)
        c := cmd.NewCmd("/bin/bash", "-c", bashScript)
        s := <-c.Start()
        if s.Error != nil {
            log.Fatal(s.Error)
        }
        fmt.Printf("%d output chars - %d Stdout lines\n", i, len(s.Stdout))
    } 
}

My output (on go version go1.11.5 linux/amd64) is:

65534 output chars - 1 Stdout lines
65535 output chars - 1 Stdout lines
65536 output chars - 0 Stdout lines
65537 output chars - 0 Stdout lines
65538 output chars - 0 Stdout lines
65539 output chars - 0 Stdout lines

If you need any more info from me please let me know. Thanks much.

Segmentation Violation Error

I am building a CLI tool for generating/building Go + Angular projects while also performing tasks like start development server, install dependencies, etc. Here's the sample code to run an external command (like ng new or go get) that I am using:

func runExternalCmd(name string, args []string) {
	c := cmd.NewCmd(name, args...)
	statusChan := c.Start()

	ticker := time.NewTicker(time.Nanosecond)

	var previousLine string
	var previousError string
	var previousStderr = []string{""}
	var previousStdout = []string{""}

	var color func(string) string

	if name == "ng" {
		color = ansi.ColorFunc("red+bh")
	} else if name == "go" || name == "gin" {
		color = ansi.ColorFunc("cyan+b")
	}

	var stderr bool

	go func() {
		for range ticker.C {
			status := c.Status()

			if status.Complete {
				c.Stop()
				break
			}

			if err := status.Error; err != nil {
				fmt.Errorf("error occurred: %s", err.Error())
				break
			}

			n := len(status.Stdout)
			n2 := len(status.Stderr)

			if n2 < 1 {
				stderr = false
			} else {
				stderr = true
			}

			var currentLine string
			var currentError string
			var currentStderr []string
			var currentStdout []string

			if n < 1 && n2 < 1 {
				continue
			}

			if n == 1 {
				currentLine = status.Stdout[n-1]
			}

			if n2 == 1 {
				currentError = status.Stderr[n2 - 1]
			}

			if n2 > 1 {
				currentStderr = status.Stderr
				if !reflect.DeepEqual(currentStderr, previousStderr) {
					for _, err := range status.Stderr {
						fmt.Println(color(err))
					}
				}
				previousStderr = currentStderr
			}

			if n > 1 {
				currentStdout = status.Stdout
				if !reflect.DeepEqual(currentStdout, previousStdout) {
					for _, err := range status.Stdout {
						fmt.Println(color(err))
					}
				}
				previousStdout = currentStdout
			}

			if n == 1 || n2 == 1 {
				if stderr && (previousError != currentError || previousError == "" && (currentError != "" && currentError != "\n")) {
					fmt.Println(color(currentError))
					previousError = currentError
				}

				if previousLine != currentLine || previousLine == "" && (currentLine != "" && currentLine != "\n") {
					fmt.Println(color(currentLine))
					previousLine = currentLine
				}

				continue
			} else {
				continue
			}
		}
	}()

	// Check if command is done
	select {
		case _ = <-statusChan:
			c.Stop()
		default:
			// no, still running
	}

	// Block waiting for command to exit, be stopped, or be killed
	_ = <-statusChan
}

When I use this for running ng new, go get, ng serve, and a few others, it works fine but throws the following error when I run npm install via this function:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x13831cb]

at the line containing fmt.Println(color(currentError)).

I can't seem to figure out what's the issue. Any help?

PS - I know the code is a lot messy than it should be.

channel stderr and stdout are not closed

Please close stderr and stdout after cmd is executed

for _,command := range task.Steps {
		// Create Cmd with options
		stepCmd := cmd.NewCmdOptions(cmdOptions,command.Cmd,command.Args...)
		task.Logger.Infof("Start executing %s %s ",command.Cmd, strings.Join(command.Args," "))
		flag := make(chan struct{},1)
		go func(flag chan struct{}) {
			for {
				select {
				case line,ok := <-stepCmd.Stdout:
					if ok {
						task.Logger.Infoln(line)
					} else {
						stepCmd.Stdout = nil
						task.Logger.Infof("Got all output from stdout")
					}
				case line,ok := <-stepCmd.Stderr:
					if ok {
						task.Logger.Errorln(line)
					} else {
						stepCmd.Stderr = nil
						task.Logger.Infof("Got all output from stderr")
					}
				}
				if stepCmd.Stderr == nil && stepCmd.Stdout == nil {
					task.Logger.Infof("Got all output")
					break
				}
			}
			flag <- struct{}{}
			close(flag)
		}(flag)
		// Run and wait for Cmd to return, discard Status
		status:= <-stepCmd.Start()
		task.Logger.Infof("Command '%s %s' is executed with return code %d",command.Cmd, strings.Join(command.Args," ") , status.Exit)
		<-flag

the code above will never return

StartWithStdin channel keeps blocking after process stops

Thanks for maintaining go-cmd.
I've got a C program that get's started with stdin.
This allows the Go code to send instructions over stdin to the C program.
However if the C program finishes, the cmd.Status chan does not seem to unblock.
Is there something I'm doing wrong here?

Go code:

//Dummy reader which does nothing
type TestReader struct {
}

//Read get's called by go-cmd
func (rt *TestReader) Read(p []byte) (n int, err error) {
	return 0, nil
}

func NewTestReader() *TestReader {
	rt := TestReader{}
	return &rt
}

func buffered() {
	testCmd := cmd.NewCmd("seg")
	rt := NewTestReader()
	statusChan := testCmd.StartWithStdin(rt)

	// statusChan := testCmd.Start()

	ticker := time.NewTicker(1 * time.Second)

	go func() {
		for range ticker.C {
			status := testCmd.Status()
			fmt.Println(status.Stdout)
			fmt.Println(status.Stderr)
		}
	}()

	// Block waiting for command to exit, be stopped, or be killed. 
	// This does not unblock
	state := <-statusChan

	fmt.Println("statusChan final state")
	fmt.Println(state.Stdout)
	fmt.Println(state.Stderr)
}

C code:

#include <unistd.h>

int main(int argc, char* argv[])
{
    setvbuf(stdout, NULL, _IOLBF, 0);
    setvbuf(stderr, NULL, _IOLBF, 0);

    printf("C test\n");
    sleep(1);
    printf("1\n");
    sleep(1);
    printf("done\n");
}```

Running a script to start an external application, when the main application terminates, the one started from the script crashes as it no longer has an Stdout & Stderr

Hi there, we're using your library to run a handful of external applications from our go application, and one of them keeps crashing as its Stdout & Stderr are no longer set after our go application finishes running.

In this example we are running a script that starts a process which can run indefinitely.

Here is the function we are using:

func RunScript(wait bool, scriptPath []string, parameters ...string) {
	var cmdExec *cmd.Cmd
	var tempBatchFilePath []string

	workingDirectory, _ := os.Getwd()
	scriptPathString := GetOsFormattedPath(scriptPath)
	err := os.Chmod(scriptPathString, 0777)
	if err != nil {
		log.Warn("Unable to set executable permissions to the executable")
	}

	fullArgs := fmt.Sprintf("cd %s && %s %s", workingDirectory, GetOsFormattedPath(scriptPath), strings.Join(parameters, " "))
	cmdExec = cmd.NewCmd("bash", "-c", fullArgs)
	
	if wait {
		log.Debugf("Launching script and waiting for command to finish...")
		<-cmdExec.Start()
	} else {
		log.Debug("Launching script...")
		cmdExec.Start()
		log.Debug("Waiting 5 seconds...")
		time.Sleep(5 * time.Second)
	}

	status := cmdExec.Status()
	out := status.Stdout
	if len(out) > 0 {
		logLines(out, log.Info)
	}
	err := status.Stderr
	if len(err) > 0 {
		logLines(err, log.Error)
	}

	if status.Exit > 0 {
		log.Warnf("Error reported (%v) running %s %v", status.Exit, GetOsFormattedPath(scriptPath), parameters)
	}
}

Is there any way to retain the Stdout & Stderr of the application we started with this script?

Many thanks.

Allowed customize SysProcAttr

e.g. windows need to set HideWindow

func hideWindow(cmd *exec.Cmd) {
	cmd.SysProcAttr.HideWindow = true
}

maybe cmd can accept option like this

type CmdOptions func(c *exec.Cmd)

cmd won't exit after nohup in script

For some reason when I run a bash script containing nohup from golang using cmd and blocking for it to exit the process launched with nohup won't detach and the go program will keep waiting for it.

I wrote a example code using both this library and stdlib. With stdlib the process launched with nohup detached correctly and the go program will exit while the launched process keeps running on background.

package main

import (
	"fmt"
	"os/exec"

	"github.com/go-cmd/cmd"
)

func main() {
	stdlibExec()
	//gocmdExec()
}

func gocmdExec() {
	p := cmd.NewCmd("/bin/bash", "./launch_me.sh")

	fmt.Println("start")
	<-p.Start()
	fmt.Println("waiting")
}

func stdlibExec() {
	cmd := exec.Command("/bin/bash", "./launch_me.sh")
	fmt.Println("start")
	err := cmd.Start()
	if err != nil {
		panic(err)
	}
	fmt.Println("waiting")
	cmd.Wait()
}

launch_me.sh

#!/bin/bash

nohup top &

Then you can check if top is running on background with pgrep top.

I ran this on OS X: Darwin Kernel Version 17.5.0.

add windows support

Please, please add windows support.. currently when i use it in window, it throws:

D:\Go\src\github.com\go-cmd\cmd\cmd.go:204:9: undefined: syscall.Kill
D:\Go\src\github.com\go-cmd\cmd\cmd.go:279:48: unknown field 'Setpgid' in struct literal of type syscall.SysProcAttr

unable to use with binary stdout

I'm using exec.Cmd() to run graphics magick to convert an image, reading from stdin and writing to stdout.

	stdout := bytes.Buffer{}
	stderr := bytes.Buffer{}
	gm := exec.Command("gm", "convert", args...)
	gm.Stdin = bytes.NewReader(img.Data)
	gm.Stdout = &stdout
	gm.Stderr = &stderr

	err = gm.Run()
	if err != nil {
		return nil, fmt.Errorf("gm: %s", stderr.String())
	}
	new := stdout.Bytes()

I've been using go-cmd for all my other commands, but for this use case, it doesn't seem possible, because go-cmd treats all its outputs as strings. Is that correct? I'd love to be doing other work while I wait for the conversion to finish.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.