bblfsh/bash-driver

Bash driver for Babelfish

bash driver for babelfish.

Development Environment

Requirements:

• docker
• Go 1.13+

To initialize the build system execute: go test ./driver, at the root of the project. This will generate the Dockerfile for this driver.

To run the tests just execute go run test.go, this will start the test over the native and the Go components of the driver using Docker.

The build is done executing go run build.go. To evaluate the result using a docker container, execute: go run build.go test-driver && docker run -it test-driver.

If the project is located under $GOPATH, run all the above with GO111MODULE=on environment variable, or move the project to any other directory outside of $GOPATH.

License

GPLv3, see LICENSE

bash-driver's Issues

semantic: type missing prefix

The type Parameter_expansion_operator_':-' is missing the bash: prefix, unlike Parameter_expansion_operator_'!' or Parameter_expansion_operator_'-',in Semantic mode.

You can see this by parsing export foo=${foo:-bar}.

native: discrepancy in array_assignment_list

When parsing export foo=(bar), the array_assignment_list node has token value bar) instead of either bar or bar). This is also true if the list has multiple element, e.g. export foo=(bar baz) yields an array_assignment_list node with token value bar baz).

The underlying combined_word node(s) have the correct value(s) though, although the parentheses are not contained - I was expecting bash:LP or bash_LT nodes but maybe this is expected behavior ?

NoClassDefFoundError: org/apache/commons/lang/StringUtils

Running the following command (where ./utils/install_server.sh is from a fresh clone of Redis):

/target # bblfsh-cli -a localhost:9432 -l bash -m native --v2 ./utils/install_server.sh

Produces a UAST but also generates the following warning:

WARN: com.ansorgit.plugins.bash.lang.lexer._BashLexer
java.lang.NoClassDefFoundError: org/apache/commons/lang/StringUtils
  at com.ansorgit.plugins.bash.lang.util.HeredocSharedImpl.trimNewline(HeredocSharedImpl.java:56)
  at com.ansorgit.plugins.bash.lang.util.HeredocSharedImpl.cleanMarker(HeredocSharedImpl.java:29)
  at com.ansorgit.plugins.bash.lang.lexer.HeredocLexingState$HeredocMarkerInfo.<init>(HeredocLexingState.java:73)
  at com.ansorgit.plugins.bash.lang.lexer.HeredocLexingState.pushMarker(HeredocLexingState.java:54)
  at com.ansorgit.plugins.bash.lang.lexer._BashLexerBase.advance(_BashLexerBase.java:1642)
  at com.intellij.lexer.FlexAdapter.locateToken(FlexAdapter.java:110)
  at com.intellij.lexer.FlexAdapter.getTokenType(FlexAdapter.java:69)
  at com.intellij.lexer.MergingLexerAdapterBase.locateToken(MergingLexerAdapterBase.java:75)
  at com.intellij.lexer.MergingLexerAdapterBase.getTokenType(MergingLexerAdapterBase.java:48)
  at com.intellij.lang.impl.PsiBuilderImpl.a(PsiBuilderImpl.java:270)
  at com.intellij.lang.impl.PsiBuilderImpl.<init>(PsiBuilderImpl.java:191)
  at com.intellij.lang.impl.PsiBuilderImpl.<init>(PsiBuilderImpl.java:158)
  at com.intellij.lang.impl.PsiBuilderFactoryImpl.createBuilder(PsiBuilderFactoryImpl.java:78)
  at bblfsh.bash.Parser.builder(Parser.java:56)
  at bblfsh.bash.Parser.parse(Parser.java:30)
  at bblfsh.bash.Driver.processRequest(Driver.java:62)
  at bblfsh.bash.Driver.processOne(Driver.java:35)
  at bblfsh.bash.Driver.run(Driver.java:20)
  at bblfsh.bash.Main.main(Main.java:14)

Can this warning be fixed/silenced? Does the lack of this ClassDef adversely affect the generated UAST?

Bash driver doesn't generate tests from source files

Since last night I am trying to generate tests for the bash-driver unsuccessfully.

First, my source files where .sh, the bblfsh-sdk test was running without errors, but no output generated.

I changed my files to .bash and I see the followings.

TestBashDriver/semantic/bench_ethopian_multiplication (2.01s)
        	require.go:794: 
        			Error Trace:	fixtures.go:241
        			Error:      	Received unexpected error:
        			            	driver failure: read |0: i/o timeout
        			            	
        			            	github.com/bblfsh/bash-driver/vendor/gopkg.in/bblfsh/sdk.v2/driver/native.(*Driver).Parse
        			            		/go/src/github.com/bblfsh/bash-driver/vendor/gopkg.in/bblfsh/sdk.v2/driver/native/native.go:213
        			            	github.com/bblfsh/bash-driver/vendor/gopkg.in/bblfsh/sdk.v2/driver/fixtures.(*Suite).testFixturesUAST.func1
        			            		/go/src/github.com/bblfsh/bash-driver/vendor/gopkg.in/bblfsh/sdk.v2/driver/fixtures/fixtures.go:232
        			            	testing.tRunner
        			            		/usr/local/go/src/testing/testing.go:777
        			            	runtime.goexit
        			            		/usr/local/go/src/runtime/asm_amd64.s:2361
        			Test:       	TestBashDriver/semantic/bench_ethopian_multiplication
FAIL

If you think there might be something wrong with my files, I can make a PR with them.
From RosettaCode, the source code of the files I used is inUnix shell.

Bash driver is not a detected language

We use enry for language detection both on bblfshd side as well as on our applications. Enry, as well as github/linguist, do not detect Bash itself, but just Shell (this includes the whole family of POSIX-like shell languages).

This generates the following problem:

• A bash file will never be routed to bash-driver if using automatic language detection at bblfshd side.
• If we use enry in our applications, we'll never use bash-driver out of the box either.

Add annotations for identifiers

semantic: wrong escape sequence in string literals

As discovered in bblfsh/javascript-driver#81 (comment) and noted at bblfsh/javascript-driver#81 (comment) - bash driver does not handle string literals in semantic mode the same way as other drivers do.

On

bc = "b\nc"

//uast:String Path query results in

"@type":"uast:String","Format":"","Value":"b\\nc"

where the expected result for semantic mode would be b\ncq.