mattn / go-shellwords Goto Github PK
View Code? Open in Web Editor NEWParse line as shell words
License: MIT License
Parse line as shell words
License: MIT License
I appreciate this project is old and may not be maintained anymore, but before forking it I thought I'd file my particular feature request in the event that it is still maintained.
It would be great if the Parser
had an option to ParseSmartQuotes
, which would:
“
and ”
as "
.‘
and ’
as '
.That's it. Thanks!
go-shellwords fails to parse sh -c "printf 'Hello\tworld\n'"
it get parsed as ["sh", "-c", "printf 'Hellotworldn'"]
It's not uncommon, in certain scenarios, to use parens inside args.
An example can be seen in the Traefik docs here (click CLI
and look at the example), where it's expected that, to set a specific value, the arg --providers.docker.constraints=Label('a.label.name','foo')
be used.
With the current implementation of go-shellwords
, parsing a string including an arg like that throws an error, even though this is a valid shell-word string.
Quoted whitespace-only arguments are ignored.
Please see this diff for added testcases:
index 2c47b06..f8c9741 100644
--- a/shellwords_test.go
+++ b/shellwords_test.go
@@ -23,6 +23,14 @@ var testcases = []struct {
{`var "--bar baz"`, []string{`var`, `--bar baz`}},
{`var --"bar baz"`, []string{`var`, `--bar baz`}},
{`var --"bar baz"`, []string{`var`, `--bar baz`}},
+
+ {`a "b"`, []string{`a`, `b`}},
+ {`a " b "`, []string{`a`, ` b `}},
+ {`a " "`, []string{`a`, ` `}}, // FAILS !
+
+ {`a 'b'`, []string{`a`, `b`}},
+ {`a ' b '`, []string{`a`, ` b `}},
+ {`a ' '`, []string{`a`, ` `}}, // FAILS !
}
func TestSimple(t *testing.T) {
Specifically, I was testing:
s := "pg_dump db -U usr -F c > bck.dump"
argv, _ := shellwords.Parse(s)
for _, v := range argv {
println(v)
}
Which returned:
pg_dump
db
-U
usr
-F
c
I noticed in the source code (https://github.com/mattn/go-shellwords/blob/master/shellwords.go#L246) that we're not accepting ">" as a token amongst other characters like "|". Since these characters are very important in certain contexts, I think it might make sense to add a flag which will recognize them as tokens (by flag I mean something similar to p.ParseBacktick = true
).
I would expect that
Parse(`foo "" bar ''`)
would give ["foo", "", "bar", ""] with the empty strings appearing as results, but this is not the case, you would only get ["foo","bar"]. Is this intended behavior?
When ParseEnv is true, variables like
line := "ssh 127.0.0.1 "echo $JAVA_HOME""
is parsed (e.g. shellwords.Parse(line)) into one element slice:
["ssh", "127.0.0.1", "echo", "/usr/local/jdk1.8.0_171"]
But the expected result should be a 3-element slice:
["ssh", "127.0.0.1", "echo /usr/local/jdk1.8.0_171"]
line := "ansible 127.0.0.1 -m shell -a "ps -ef | grep console""
is parsed (e.g. shellwords.Parse(line)) into one element slice:
["ansible", "127.0.0.1", " -m", "shell", "-a", "ps", "-ef"]
But the expected result should be a 6-element slice:
["ansible", "127.0.0.1", " -m", "shell", "-a", "ps -ef | grep console"]
Hello, thank you for this great library.
I've been using go-shellwords in my applications and it greatly helps me.
I've noticed go-shellwords's quote handling is different from general shells when ParseEnv
is true.
Is it possible to change that, or is this intentional behavior?
$ cat a.sh
echo 0:$1
echo 1:$2
echo 2:$3
$ bash a.sh a '' c
0:a
1:
2:c
parser := shellwords.Parser{ParseEnv: true}
result, _ := parser.Parse("a '' c")
for i, v := range result {
fmt.Printf("%d:%s\n", i, v)
}
// Result:
// 0:a
// 1:c
Input | go-shellwords(2c8720d) | Expected |
---|---|---|
a '' c | 0:a, 1:c | 0:a, 1:, 2:c |
'A B C' c | 0:A, 1:B, 2:C, 3:c | 0:A B C, 1:c |
'A "B' c | (Error) | 0:A "B, 1:c |
'${TEMP}' | 0:/tmp | 0:${TEMP} |
Hi, i suggest the ability to replace the os.Getenv
function with a generic function - to be more specific adding another field to parser (https://github.com/mattn/go-shellwords/blob/master/shellwords.go#L34):
type Parser struct {
ParseEnv bool
Getenv func(string) string
ParseBacktick bool
Position int
}
if it is nil
, os.Getenv is used (this is for backward compatibility). If it weren't for backward compatibility, I would suggest it replaces the ParseEnv variable entirely.
Would be happy to send out a PR if this makes sense.
I tried to parse curl http://httpbin.org/get?a=1&b=2
to shell word list as the following code
package main
import (
"fmt"
"log"
"github.com/mattn/go-shellwords"
)
func main() {
sw, err := shellwords.Parse("curl http://httpbin.org/get?a=1&b=2")
if err != nil {
log.Fatal(err)
}
fmt.Println(sw)
}
[curl http://httpbin.org/get?a=1&b=2]
[curl http://httpbin.org/get?a=1]
go: 1.16.5
go-shellwords: v1.0.12
In Windows, backslashes are used at the OS path separator. When I run this code, I get an incorrect result for the value of cmd
:
func main() {
sh := `upx c:\github.com\jftuga\test\test.exe`
cmd, err := shellwords.Parse(sh)
if err != nil {
fmt.Println("err:", err)
return
}
fmt.Println(cmd)
}
This outputs:
[upx c:github.comjftugatesttest.exe]
when it should output:
[upx c:\github.com\jftuga\test\test.exe]
Can this be fixed, please?
$ cat shellparse.go
package main
import ("github.com/mattn/go-shellwords"; "fmt")
func main() {
args, _ := shellwords.Parse("a > b")
fmt.Println(args)
}
yields in
$ go run shellparse.go
panic: runtime error: index out of range
goroutine 1 [running]:
github.com/mattn/go-shellwords.(*Parser).Parse(0xc420051ef8, 0x4f6e67, 0x5, 0x0, 0x0, 0xc420051f01, 0xc42000e230, 0xc420051f10)
/home/alex/.go/src/github.com/mattn/go-shellwords/shellwords.go:143 +0xdff
github.com/mattn/go-shellwords.Parse(0x4f6e67, 0x5, 0x403d8c, 0xc42001c0b8, 0x0, 0xc420051f68, 0x4bb063)
/home/alex/.go/src/github.com/mattn/go-shellwords/shellwords.go:177 +0x72
main.main()
/home/alex/lib/software/example/go/src/my/hello/shellparse.go:5 +0x3a
exit status 2
$ go version
go version go1.10.2 linux/amd64
Hi @mattn , I found a thing while trying to use go-shellwords
.
go test -v
=== RUN ExampleShellwords
--- FAIL: ExampleShellwords (0.02s)
got:
["-flag=val0" "val1" "-flag=val2"]
want:
["-flag=val0" "-flag=val1" "-flag=val2"]
FAIL
exit status 1
FAIL testsw 0.041s
cat testsw_test.go
package testsw
import (
"fmt"
"github.com/mattn/go-shellwords"
)
func ExampleShellwords() {
parser := shellwords.NewParser()
parser.ParseBacktick = true
args, err := parser.Parse("-flag=val0 -flag=`cat testdata/flag-val.txt` -flag=val2")
if err != nil {
panic(err)
}
fmt.Printf("%q\n", args)
// Output:
// ["-flag=val0" "-flag=val1" "-flag=val2"]
}
cat go.mod
module testsw
go 1.12
require github.com/mattn/go-shellwords v1.0.5
cat testdata/flag-val.txt
val1
I spent quite a bit of time trying to understand a weird problem with Hashicorp Vault Agent. The post-template-rendering command execution failed. The error was unhelpful:
failed parsing command: fork/exec : no such file or directory:
"
After much befuddlement, it eventually turned out that this only happened when the command contained backticks, and was due to SHELL not being set in the environment, leading to a failure in shellRun in util_posix.
It seems like it would be reasonable to default to /bin/sh here if the variable is not set.
Hi, I noticed shellwords.Parse
does not handle backslash-escaped whitespaces at the end of the argument.
words, err := shellwords.Parse("trailing escaped space\\ ")
// invalid command line string
This is due to strings.TrimSpace
at the beginning of the function.
When i do 'go get github.com/mattn/go-shellwords' on my vps which is Debian Jessie 8 32 bit i get the error eerr.Stderr undefined (type *exec.ExitError has no field or method Stderr)
Following commands throws errors:
$(echo xxx | xargs)
$(sleep 1 && sleep 1)
$(sleep 1; sleep 1)
Here is full runnable code: https://github.com/curusarn/go-shellwords-issue/blob/master/main.go
Output:
Commandline: $(sleep 1; sleep 1)
2019/08/10 22:19:46 invalid command line string
exit status 1
Hello, I'm trying to parse the following command:
command := "RUN /bin/sh -c echo hi --password "pdhjcrkCHJfSchjd3f837"/ # buildkit"
args, err := shellwords.Parse(command)
and I'm getting
"RUN", "/bin/sh", "-c", "echo", "hi", "--password", "pdhjcrkCHJfSchjd3f837/", "#", "buildkit"
so one of the arguments is "pdhjcrkCHJfSchjd3f837/" and it's wrong.
Do you have an idea what can I do in order to avoid it?
Thank you :)
When ParseEnv is true, shell variables like
export CMD='program -f abc.txt'
or
export CMD="program -f abc.txt"
is parsed (e.g. shellwords.Parse("${CMD}")
) into one element slice:
["program -f abc.txt"]
But the expected result should be a 3-element slice:
["program" "-f" "abc.txt"]
I just wanted to say thank you, I've made a very small utility because I'm lazy and your package is a central part of if!
Would you consider adding support for $(command)
besides supporting backticks for executing subcommands?
Docker is using this as vendor package. In docker we can check for checking copyright for all the vendor packages and expects any files in .*LICENSE.*' '.*COPYRIGHT.*' '.*COPYING.*'
pattern, currently while validating this package it is printing the warning message.
Please create explicit licence file to avoid any warnings while validation, so create a LICENCE file in parent directory with the MIT licence referred in the README file.
Hi,
It looks like the parsing of $()
is not supported. Any plans on adding this?
E.g.
"foo
Thank you!
My code uses |
pipe character, the @753a2322a99f87c0eff284980e77f53041555bc6 commit now captures this character and discards the remaining line.
Could you add an option to disable this? And the other characters now being captured by the library.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.