tree-sitter/tree-sitter-bash

Bash grammar for tree-sitter

tree-sitter-bash

Bash grammar for tree-sitter.

Development

Install the dependencies:

npm install

Build and run the tests:

npm run build
npm run test

Run the build and tests in watch mode:

npm run test:watch

References

• Bash man page
• Shell command language specification
• mvdnan/sh - a shell parser in go

tree-sitter-bash's Issues

Exclamation before subshell in conditions is not handled correctly

Example

#!/bin/bash

if ! (exit 1); then
    echo "error"
fi

Expected: The script above is valid and should not throw any errors

Actual: Failed to parse expression

Unable to parse indirect expansion with an ! in {!var}

I'm not using tree-sitter directly but I am using the Bash Language Server in a Visual Studio Code extension and any time I use an a ! character in a variable like {!var} I get an Failed to parse expression error

In an issue about this error the owner specified that the error was in tree-sitter and requested that an issue be opened here. bash-lsp/bash-language-server#43

||, &&, <, > in conditions are handled incorrectly

Example:

if [[ "$lsb_dist" != 'Ubuntu' || $(ver_to_int "$lsb_release") < $(ver_to_int '14.04') ]]; then
	return 1
fi

Double pipes, double ampersand less then and greater than signs will get "Failed to parse expression" errors.

Parse error for here document using exclamation mark

#!/bin/sh

cat <<!HEREDOC!
hello
this
is
smth..
!HEREDOC!

Originally reported in bash-language-server: bash-lsp/bash-language-server#126

Backslash does not require newline to be valid

Example

#!/usr/bin/env bash

echo 1 \ 2 \ 3

Expected: Example above is valid and outputs 1 2 3

Actual: failed to parse expression

parse error when negating a variable assignment in an if statement

I was sent over here after filing this issue: bash-lsp/bash-language-server#135

I'm getting a parse error when I have an if statement where I negate a variable assignement. For example:

#! /usr/bin/env bash

bar() {
  return 1
}

foo=''

if ! foo="$(bar)"; then # <-- Syntax error: expected "word" somewhere in the file
  echo "ok."
fi

The error message in the comment is coming from the language server that's implementing your library.

Hopefully this is enough info for a fix. thanks so much!

Can't find bash-language-server on your PATH

I have bash-language-server installed globally, and the global bin directory is in PATH, declared both in my shell (fish), and other installed shells .bashrc and .zshrc.

With all this done, I still get a "Can't find bash-language-server on your PATH" error in VSCode.

Could the path to the language server be configurable as a VSCode setting?

Error while parsing stream redirection inside dollar-parentheses

Snippet which fails:

function ff () {
    local xxxx="$(<aa)"
    case x in
      * ) echo "blabla"; return 1 ;;
    esac &> foo || { echo "ffff"; return 0; }
}

node.isMissing() returns unexpected position

I'm trying to use node.isMissing() for improving https://github.com/mads-hartmann/bash-language-server, but the startPosition and endPosition are off for the fi type below.

Example

if [[ $PATH_INPUT -nt $PATH_OUTPUT ]]; then
  babel --compact false ${PATH_INPUT} > ${PATH_OUTPUT}
f

echo "test"

The missing node will be shown after the echo statement, although I would expect it to fail at the position of f.

Any advice how to fix this?

Heredoc redirect to file is not handled properly

Example:

#!/usr/bin/env bash

tmpfile="/tmp/fix-smb.reg"

cat <<EOF > $tmpfile
Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management]
"LargeSystemCache"=dword:00000001

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\LanmanServer\Parameters]
"Size"=dword:00000003
EOF

The heredoc redirect above is valid and should not be reported as an issue.

Parse error for override redirection

Hey 👋,

The redirection operator >| overrides the specified file. See Bash Redirections for further details.

If the redirection operator is ‘>’, and the noclobber option to the set builtin has been enabled, the redirection will fail if the file whose name results from the expansion of word exists and is a regular file. If the redirection operator is ‘>|’, or the redirection operator is ‘>’ and the noclobber option is not enabled, the redirection is attempted even if the file named by word exists.

tree-sitter-bash is currently not able to parse this syntax. Here's an example:

echo "foo" >| foo.txt

Installation errors, too few arguments to function call, macOS 10.14.4, node 12.1.0

macOS 10.14.4

§ uname -a
Darwin EPUSPRIW1129 18.5.0 Darwin Kernel Version 18.5.0: Mon Mar 11 20:40:32 PDT 2019; root:xnu-4903.251.3~3/RELEASE_X86_64 x86_64

§ node --version
v12.1.0

§ npm i -g tree-sitter

> [email protected] install /usr/local/lib/node_modules/tree-sitter
> prebuild-install || node-gyp rebuild

prebuild-install WARN install No prebuilt binaries found (target=12.1.0 runtime=node arch=x64 libc= platform=darwin)
  CC(target) Release/obj.target/tree_sitter/vendor/tree-sitter/lib/src/lib.o
  LIBTOOL-STATIC Release/tree_sitter.a
  CXX(target) Release/obj.target/tree_sitter_runtime_binding/src/binding.o
  CXX(target) Release/obj.target/tree_sitter_runtime_binding/src/conversions.o
../src/conversions.cc:32:12: warning: 'Set' is deprecated: Use maybe version [-Wdeprecated-declarations]
  exports->Set(Nan::New("pointTransferArray").ToLocalChecked(), Uint32Array::New(js_point_transfer_buffer, 0, 2));
           ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8.h:3358:3: note: 'Set' has been explicitly marked deprecated here
  V8_DEPRECATE_SOON("Use maybe version",
  ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8config.h:322:29: note: expanded from macro 'V8_DEPRECATE_SOON'
  declarator __attribute__((deprecated(message)))
                            ^
../src/conversions.cc:42:11: warning: 'Set' is deprecated: Use maybe version [-Wdeprecated-declarations]
  result->Set(Nan::New(start_position_key), PointToJS(range.start_point));
          ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8.h:3358:3: note: 'Set' has been explicitly marked deprecated here
  V8_DEPRECATE_SOON("Use maybe version",
  ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8config.h:322:29: note: expanded from macro 'V8_DEPRECATE_SOON'
  declarator __attribute__((deprecated(message)))
                            ^
../src/conversions.cc:43:11: warning: 'Set' is deprecated: Use maybe version [-Wdeprecated-declarations]
  result->Set(Nan::New(start_index_key), ByteCountToJS(range.start_byte));
          ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8.h:3358:3: note: 'Set' has been explicitly marked deprecated here
  V8_DEPRECATE_SOON("Use maybe version",
  ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8config.h:322:29: note: expanded from macro 'V8_DEPRECATE_SOON'
  declarator __attribute__((deprecated(message)))
                            ^
../src/conversions.cc:44:11: warning: 'Set' is deprecated: Use maybe version [-Wdeprecated-declarations]
  result->Set(Nan::New(end_position_key), PointToJS(range.end_point));
          ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8.h:3358:3: note: 'Set' has been explicitly marked deprecated here
  V8_DEPRECATE_SOON("Use maybe version",
  ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8config.h:322:29: note: expanded from macro 'V8_DEPRECATE_SOON'
  declarator __attribute__((deprecated(message)))
                            ^
../src/conversions.cc:45:11: warning: 'Set' is deprecated: Use maybe version [-Wdeprecated-declarations]
  result->Set(Nan::New(end_index_key), ByteCountToJS(range.end_byte));
          ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8.h:3358:3: note: 'Set' has been explicitly marked deprecated here
  V8_DEPRECATE_SOON("Use maybe version",
  ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8config.h:322:29: note: expanded from macro 'V8_DEPRECATE_SOON'
  declarator __attribute__((deprecated(message)))
                            ^
../src/conversions.cc:68:3: warning: 'Get' is deprecated: Use maybe version [-Wdeprecated-declarations]
  INIT(start_point, start_position_key, PointFromJS);
  ^
../src/conversions.cc:60:33: note: expanded from macro 'INIT'
    auto field = Type(js_range->Get(Nan::New(key))); \
                                ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8.h:3412:3: note: 'Get' has been explicitly marked deprecated here
  V8_DEPRECATE_SOON("Use maybe version", Local<Value> Get(Local<Value> key));
  ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8config.h:322:29: note: expanded from macro 'V8_DEPRECATE_SOON'
  declarator __attribute__((deprecated(message)))
                            ^
../src/conversions.cc:69:3: warning: 'Get' is deprecated: Use maybe version [-Wdeprecated-declarations]
  INIT(end_point, end_position_key, PointFromJS);
  ^
../src/conversions.cc:60:33: note: expanded from macro 'INIT'
    auto field = Type(js_range->Get(Nan::New(key))); \
                                ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8.h:3412:3: note: 'Get' has been explicitly marked deprecated here
  V8_DEPRECATE_SOON("Use maybe version", Local<Value> Get(Local<Value> key));
  ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8config.h:322:29: note: expanded from macro 'V8_DEPRECATE_SOON'
  declarator __attribute__((deprecated(message)))
                            ^
../src/conversions.cc:70:3: warning: 'Get' is deprecated: Use maybe version [-Wdeprecated-declarations]
  INIT(start_byte, start_index_key, ByteCountFromJS);
  ^
../src/conversions.cc:60:33: note: expanded from macro 'INIT'
    auto field = Type(js_range->Get(Nan::New(key))); \
                                ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8.h:3412:3: note: 'Get' has been explicitly marked deprecated here
  V8_DEPRECATE_SOON("Use maybe version", Local<Value> Get(Local<Value> key));
  ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8config.h:322:29: note: expanded from macro 'V8_DEPRECATE_SOON'
  declarator __attribute__((deprecated(message)))
                            ^
../src/conversions.cc:71:3: warning: 'Get' is deprecated: Use maybe version [-Wdeprecated-declarations]
  INIT(end_byte, end_index_key, ByteCountFromJS);
  ^
../src/conversions.cc:60:33: note: expanded from macro 'INIT'
    auto field = Type(js_range->Get(Nan::New(key))); \
                                ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8.h:3412:3: note: 'Get' has been explicitly marked deprecated here
  V8_DEPRECATE_SOON("Use maybe version", Local<Value> Get(Local<Value> key));
  ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8config.h:322:29: note: expanded from macro 'V8_DEPRECATE_SOON'
  declarator __attribute__((deprecated(message)))
                            ^
../src/conversions.cc:80:11: warning: 'Set' is deprecated: Use maybe version [-Wdeprecated-declarations]
  result->Set(Nan::New(row_key), Nan::New<Number>(point.row));
          ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8.h:3358:3: note: 'Set' has been explicitly marked deprecated here
  V8_DEPRECATE_SOON("Use maybe version",
  ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8config.h:322:29: note: expanded from macro 'V8_DEPRECATE_SOON'
  declarator __attribute__((deprecated(message)))
                            ^
../src/conversions.cc:81:11: warning: 'Set' is deprecated: Use maybe version [-Wdeprecated-declarations]
  result->Set(Nan::New(column_key), ByteCountToJS(point.column));
          ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8.h:3358:3: note: 'Set' has been explicitly marked deprecated here
  V8_DEPRECATE_SOON("Use maybe version",
  ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8config.h:322:29: note: expanded from macro 'V8_DEPRECATE_SOON'
  declarator __attribute__((deprecated(message)))
                            ^
../src/conversions.cc:92:35: warning: 'Get' is deprecated: Use maybe version [-Wdeprecated-declarations]
  Local<Value> js_row = js_point->Get(Nan::New(row_key));
                                  ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8.h:3412:3: note: 'Get' has been explicitly marked deprecated here
  V8_DEPRECATE_SOON("Use maybe version", Local<Value> Get(Local<Value> key));
  ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8config.h:322:29: note: expanded from macro 'V8_DEPRECATE_SOON'
  declarator __attribute__((deprecated(message)))
                            ^
../src/conversions.cc:98:38: warning: 'Get' is deprecated: Use maybe version [-Wdeprecated-declarations]
  Local<Value> js_column = js_point->Get(Nan::New(column_key));
                                     ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8.h:3412:3: note: 'Get' has been explicitly marked deprecated here
  V8_DEPRECATE_SOON("Use maybe version", Local<Value> Get(Local<Value> key));
  ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8config.h:322:29: note: expanded from macro 'V8_DEPRECATE_SOON'
  declarator __attribute__((deprecated(message)))
                            ^
../src/conversions.cc:105:41: error: too few arguments to function call, single argument 'context' was not specified
  if (std::isfinite(js_row->NumberValue())) {
                    ~~~~~~~~~~~~~~~~~~~ ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8.h:2564:3: note: 'NumberValue' declared here
  V8_WARN_UNUSED_RESULT Maybe<double> NumberValue(Local<Context> context) const;
  ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8config.h:347:31: note: expanded from macro 'V8_WARN_UNUSED_RESULT'
#define V8_WARN_UNUSED_RESULT __attribute__((warn_unused_result))
                              ^
../src/conversions.cc:106:52: error: too few arguments to function call, single argument 'context' was not specified
    row = static_cast<uint32_t>(js_row->Int32Value());
                                ~~~~~~~~~~~~~~~~~~ ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8.h:2569:3: note: 'Int32Value' declared here
  V8_WARN_UNUSED_RESULT Maybe<int32_t> Int32Value(Local<Context> context) const;
  ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8config.h:347:31: note: expanded from macro 'V8_WARN_UNUSED_RESULT'
#define V8_WARN_UNUSED_RESULT __attribute__((warn_unused_result))
                              ^
../src/conversions.cc:111:44: error: too few arguments to function call, single argument 'context' was not specified
  if (std::isfinite(js_column->NumberValue())) {
                    ~~~~~~~~~~~~~~~~~~~~~~ ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8.h:2564:3: note: 'NumberValue' declared here
  V8_WARN_UNUSED_RESULT Maybe<double> NumberValue(Local<Context> context) const;
  ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8config.h:347:31: note: expanded from macro 'V8_WARN_UNUSED_RESULT'
#define V8_WARN_UNUSED_RESULT __attribute__((warn_unused_result))
                              ^
../src/conversions.cc:112:58: error: too few arguments to function call, single argument 'context' was not specified
    column = static_cast<uint32_t>(js_column->Int32Value()) * BYTES_PER_CHARACTER;
                                   ~~~~~~~~~~~~~~~~~~~~~ ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8.h:2569:3: note: 'Int32Value' declared here
  V8_WARN_UNUSED_RESULT Maybe<int32_t> Int32Value(Local<Context> context) const;
  ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8config.h:347:31: note: expanded from macro 'V8_WARN_UNUSED_RESULT'
#define V8_WARN_UNUSED_RESULT __attribute__((warn_unused_result))
                              ^
../src/conversions.cc:130:47: error: too few arguments to function call, single argument 'context' was not specified
  return Nan::Just<uint32_t>(arg->Uint32Value() * BYTES_PER_CHARACTER);
                             ~~~~~~~~~~~~~~~~ ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8.h:2567:3: note: 'Uint32Value' declared here
  V8_WARN_UNUSED_RESULT Maybe<uint32_t> Uint32Value(
  ^
/Users/oleksii_chekulaiev/.node-gyp/12.1.0/include/node/v8config.h:347:31: note: expanded from macro 'V8_WARN_UNUSED_RESULT'
#define V8_WARN_UNUSED_RESULT __attribute__((warn_unused_result))
                              ^
13 warnings and 5 errors generated.
make: *** [Release/obj.target/tree_sitter_runtime_binding/src/conversions.o] Error 1
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:262:23)
gyp ERR! stack     at ChildProcess.emit (events.js:196:13)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:256:12)
gyp ERR! System Darwin 18.5.0
gyp ERR! command "/usr/local/Cellar/node/12.1.0/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /usr/local/lib/node_modules/tree-sitter
gyp ERR! node -v v12.1.0
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok 
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: `prebuild-install || node-gyp rebuild`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/oleksii_chekulaiev/.npm/_logs/2019-05-01T15_52_34_814Z-debug.log

2019-05-01T15_52_34_814Z-debug.log

2019-05-01T15_52_34_814Z-debug.log

Support regular expression operator

Currently the parser will be unhappy with the following expression

[[ "35d8b" =~ ^[0-9a-fA-F]{5}$ ]] && echo "It's a hash"

My initial idea was to simple create $.regex rule and add that to $._expression but given that there aren't any marker characters for regular expressions (like /abc/ in javascript) in Bash I think we have to go another route. I'm not sure what the best approach is at the moment so I'll just leave the issue open 😄

Three-expression bash for loops syntax

failed to parse expression for :

for (( i=2 ; i <= n ; i++ )) {
	echo $i
}

bash-lsp/bash-language-server#66

Handle escaped backticks

Stumbled upon an escaped backtick in one my scripts which seems to trip up the parser

echo "Escape backtick\`"

Haven't investigated the issue yet so just adding it here for later.

Parse error for here document inside an if with variable

Parse fails for the entire document when you have a here document inside an if, that contains a variable as the first expression.

MWE:

if true; then
  a_string='some stuff to cat'
  cat <<XXX
${a_string}
XXX
fi

The issue does not occur if:

• not inside an if
• there is anything except newlines before ${a_string}, including whitespace

---

Origin: bash-lsp/bash-language-server#136

No newline after echo reported as an error

Example:

#!/usr/bin/env bash

echo 1

Expected: there is no error in having no new line here

Actual: reported as a parsing error

Issues parsing closing curly brace followed by symbols in conditions

Here are examples of conditional expressions that show errors but are actually valid.
With comments which part is highlighted in "red". All of them seem to stumble upon closing curly brace.

# from expression start till first closing curly brace highlighted
if [[ "$1" =~ ^${var}${var}*=..* ]]; then
  echo 1
fi

# closing curly brace highlighted
if [[ "$1" =~ ^\-${var}+ ]]; then
  echo 1
fi

# closing curly brace and * highlighted
if [[ ${var1} == *${var2}* ]]; then
  echo 1
fi

# from expression start till first closing curly brace highlighted
if [[ "$server" =~ [0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3} ]]; then
  echo 1
fi

# from expression start till first closing curly brace highlighted
if [[ "$primary_wins" =~ ([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}) ]]; then
  echo 1
fi

# dollar sign after closing curly brace highlighted
if ( grep ^${var}$ ); then
  echo 1
fi

p.s. Thanks for great work fixing other issues! These curly braces are the last errors left in our 6k+ lines bash code.

Subshell redirects are not handled properly

Example

#!/usr/bin/env bash

(echo "hello") >/dev/null

The code above is valid bash code but reports parsing issue right now.

Error parsing expressions containing non comment '#'

Found these corner cases while using bash-language-server. The parser fails on some expressions where '#' should not be interpreted as the start of a comment.
Examples:

• Interpret number in a certain base: dec=$((10#$x))
• Referring to the number of positional parameters : last=${!#}, n=${#}.

Arrays support

Currently this snippet fails to be parsed

if ((${#aaa[@]} == 0)); then
fi

Parse error in redirection inside conditional expression

Example

if [ ]; then
> aa
fi

Function body range is parsed incorrectly if there are curly brackets

STR:

fff () {
    ls || { echo "ffff"; return 0; }
    echo aa
    
}

1. Place cursor to echo aa
2. You are outside of fff

Parse every bash file from bash-it without error.

We now parse all the scripts from bash-it on CI, except for a known list of files that do not parse. Currently, we successfully parse about 3/4 of the files. We should continue fixing errors and removing files from the list of known failures until the list is empty.

Parsing issue in here doc

I have a script that produces another script using a here document, here's a snippet:

cat  << EOF > job.conf
...
DEV_NAME=\$(lsblk --nodeps -no name,serial | grep $SERIAL | awk '{printf \$1}')
...
EOF

The parser chokes on it but I can't see why.

heredoc single/double quoted : failed to parse expression

https://en.wikipedia.org/wiki/Here_document#Unix_shells
in this example i wouldn't variable to be replaced

#!/usr/bin/env bash

function f2() {

cat << 'EOF' > script.sh
#!/bin/bash
echo $PWD
EOF

}

f2

see the error

$! is not supported

cpid=$! - error

Case statements

It seems like the current grammar for case_item is a bit too strict. I've found two variants that are interesting

One line case lines

In this case it's okay for the statement itself not to be terminated.

case "a" in
  a) echo a ;;
  *) echo other ;;
esac

Optional terminator in case line

It seems that the last ;; can be omitted as it's inferred by the esac

case "a" in 
  a) echo a ;; 
  *) echo other
esac

I don't have any suggestions regarding how to fix it right now though 😉

Parameter substitution case

Not sure if this is covered by #9 but I found a parameter substitution case in one of my Bash files that the parser doesn't like. A minimal example is the following:

echo ${var1#*#}

It will remove everything from the beginning of var1 until it reaches an #. Here's a valid example

var1=foobar-#123
echo ${var1#*#}

Double Parens in for Loop is Unable to Be Parsed

  for ((index = 1; index < ${#expired_app_names[@]}; index++)); do

Gives the parsing error here:

Parse error in arithmetic expressions with braces

echo $(( (1 + 2)))
is OK, but
echo $(((1 + 2) ))
fails.
echo $(( (1 + 2) / 2))
also fails.

How to use this parser in the main tree-sitter?

Hello, documentation is really fuzzy regarding how to use specific language parser functions. Is describing example code, what is doing but not how to integrate a plugin and easily use it. Moreover, I use the suggestion to just add source code and header to my project, but I don't understand how I can do the same for the bash parser. I think it is really appreciated to have a doc describing this in details, also to spread the use of this nice project.
Thanks

Parse error for multi-line strings with eval string

main() {
    local foo="asd"`
        `"fgh"
}

Originally reported in bash-language-server: bash-lsp/bash-language-server#129

Parse error for compound expression while loops

When calling while a loop with multiple statements before the do, I get a parse error:

loop_assign() {
  while local name="$1" val="$2"; shift 2; do # <-- Failed to parse expression
    printf "%s (%s)\n" "$val" "$name"
  done
}

loop_assign foo bar baz bing thing thang thong

No 2x new line after last command get error "Failed to parse expression"

I'm not sure it's mandatory to have bash script ended by 2 newlines

The parser is KO with :

#!/usr/bin/env bash

exit 0<newline>

The parser is OK with :

#!/usr/bin/env bash

exit 0<newline>
<newline>

but script 1 run well so...

parse error string with pseudo inline comment

Example:

REPOS="
    main 
    restricted
    universe
    multiverse
    ppa:cubic-wizard/release `#needed for cubic`
"

The example variable assignment gives me a parse error.

Variables in functions missing type

First of all sorry if the report is not detailed enough. I've been asked to report it here by bash-language server author . So variables in functions like:

#!/bin/sh
# A simple script with a function...

add_a_user()
{
  USER=$1
  PASSWORD=$2
  shift; shift;
  # Having shifted twice, the rest is now comments ...
  COMMENTS=$@
  echo "Adding user $USER ..."
  echo useradd -c "$COMMENTS" $USER
  echo passwd $USER $PASSWORD
  echo "Added user $USER ($COMMENTS) with pass $PASSWORD"
}

###
# Main body of script starts here
###
echo "Start of script..."
add_a_user bob letmein Bob Holness the presenter
add_a_user fred badpassword Fred Durst the singer
add_a_user bilko worsepassword Sgt. Bilko the role model
echo "End of script..."

are missing something from grammar side as indicated to me in bash-lsp/bash-language-server#101 .
Please let me know how I can help in mediating the conversation between the two projects I'm not developer of so the issue gets resolved.

Curly braces following each other parse error

snippet:

function ff() {
    grep -o aaa xml.xml | xargs -I{} find xml/{} -type f 
}

0.13.2 missing parsing errors

0.13.1 error on a file like this with a missing fi:

PATH_INPUT=src/in.js
PATH_OUTPUT=src/out.js

if [[ $PATH_INPUT -nt $PATH_OUTPUT ]]; then
  babel --compact false ${PATH_INPUT} > ${PATH_OUTPUT}
f

I guess this is a regression from 0.13.1? Didn't investigate if this is due to 6ff7fe3

Parse error on bitshift operator

Code example:

MAX_SIZE=$((4 << 30)) # 4gb

Unable to parse ${#}

I am having similar issue as #21. Sample code:

if [[ ${#} -ne 3 ]]; then
    tpcp::utils::usage
    tpcp::utils::terminate
fi

Parse error for echo call

#!/usr/bin/env bash
function to_decimal () {
    echo "$(("$1"#"$2"))"
}
to_decimal 16 ff

The function to_decimal converts the number "$2" in base "$1" to decimal. The parser fails to recognize this instruction.

--

Origin: bash-lsp/bash-language-server#138

Variables inside [[ conditionals error

Example

if [[ $AAA == false && "${x:0:1}" != "#" && "${x:0:10}" != "#disabled#" ]] ; then
fi

Replacing ${x:0:1} with $x removed error.

Error parsing string with $ sign

#!/bin/bash

echo "s$"

Parse error for functions

This fails to parse:

foo() (
  true
)

Note that a function's body can be any Compound Command, so it's also legal to write:

bar() [[ -e foo ]]
baz() if [[ $a = $b ]]; then
  echo 1
else
  echo 2
fi

Originally from bash-lsp/bash-language-server#133

Grammar does not support case fallthrough terminators

I'm using tree-sitter-bash through bash-language-server => w0rp/ale on VIM and I have a case statement that uses a fallthrough terminator similar to the following.

case "$1" in
    init) echo "here is where I would do some init"
        exit;;

    --help | -h)
        ;;&
 
    *) print_usage
        exit;;
esac

Perhaps there is a better way to do the above but this is just a simple example to reproduce the issue. I should also note that the script runs fine when executed in the terminal.

The errors that are reported are:
Syntax error: expected "esac" somewhere in the file (on line ;;&) and
Failed to parse expression (on line *) print_usage)

I'm not sure if these are coming from bash-language-server/tree-sitter or are generic ALE errors (I ran :ALEInfo and do not see these errors printed anywhere)

Parse error for arithmetic expression and comma operator in for loop

In VS Code I get parse errors for a code snippet like this:

#!/bin/bash

f()
{
  local x wc=0 wcx=0 k=0 N=1 w cx c d=1 pf=1
  x=$(( ( $wcx - $k ) / $N ))
  for (( cx=0, w=1;  c=$cx/$pf, c<$wc-$k;  cx+=$d, w++ )); do
    echo "$cx"
  done
}

f

The code has been trimmed down to show the problem. See this screenshot for the squiggly lines. tree-sitter-bash version is 0.13.7, indirect dependency of the VS Code bash-ide extension.

Multi-line sub-shell throws an error

Example

#!/bin/bash

_path=$(
    while [[ ! -f $1 ]] && [[ ! -d $1 ]] && [[ $(pwd) != / ]]; do
        cd ".."
    done;
    if [[ -f $1 ]] || [[ -d $1 ]]; then echo $(pwd); exit; fi
)

Above script is valid and should not throw any errors.

Corpus of shell scripts

Hi I just learned of this project through a reddit comment [1]

I saw that you have some example shell scripts as test cases for the parser. FWIW I have a corpus of 11,568 shell scripts here:

http://www.oilshell.org/blob/wild/

Here's a report from running the Oil parser over this corpus (which I wrote):

http://www.oilshell.org/release/0.6.pre6/test/wild.wwz/index.html

There is a bunch of background on the blog, for example: http://www.oilshell.org/blog/2017/11/10.html

I put a link to your parser here, along with 3-4 other parsers (although it's not clear how complete they are.)

https://github.com/oilshell/oil/wiki/ExternalResources

[1] https://www.reddit.com/r/commandline/comments/9p6nb2/complete_command_line_flags_in_vimemacs_or/?utm_content=full_comments&utm_medium=message&utm_source=reddit&utm_name=frontpage

Failed to parse $(( 1 < 2 ? 1 : 2 ))

Initially reported here bash-lsp/bash-language-server#143

Failed to parse:

$(( 1 < 2 ? 1 : 2 ))