colis-anr / morbig Goto Github PK
View Code? Open in Web Editor NEWA static parser for POSIX Shell
License: Other
A static parser for POSIX Shell
License: Other
When running morbig foo
where foo
is a directory, morbig does not fail but produces an output file foo.json
containing only []
.
Moreover, when running morbig foo/
, morbig does not fail but produces an output file foo/.json
which can be pretty annoying.
This is probably what causes #10 to happen.
The 2016 edition of the posix standard says:
If the value of the alias replacing the word ends in a <blank>, the shell shall check the next command word for alias substitution; this process shall continue until a word is found that is not a valid alias or an alias value does not end in a <blank>.
I have absolutely no idea what does means (what is the "next command" ?), but there probably is a reason why this was written. We should try to find out what precisely this is supposed to mean.
there is a corner case of that weird rule of alias handling when the substitution word ends on a space:
alias x="echo "
x
x x
x x x
in the third line, morbig does not expand the third ocurrence of x. This does not happen when the
replacement word is single-quoted. Also, it happens only with at least three occurrences of the aliased
word in a row.
When morbig encounters a syntactically incorrect file, it prints poorly informative
error messages such as Fatal error: exception Libmorbig.Engine.ParseError
or (a bit better) Fatal error: exception Failure("Unclosed subshell (got '(').")
.
It should at least print the location of the error (line, character) and avoid talking about an « exception »
Hello,
I get a "Failure: lexing: empty token." on the following script :
try.txt
If you remove one of the two comment lines in the middle of the file, or the line between them, then it passes.
-Ralf.
There are slight modifications in the grammar since the new POSIX 2018 version.
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_10_02
In particular, the starting point of the grammar has now a name! (and it's "program")
program : linebreak complete_commands linebreak
| linebreak
;
complete_commands: complete_commands newline_list complete_command
| complete_command
;
complete_command : list separator_op
| list
;
...
For instance the following:
${parameter:-word}
${parameter-word}
is parsed just like:
${parameter-word}
${parameter-word}
see https://github.com/colis-anr/morbig/blob/master/tests/good/2.6-word-expansions/2.6.2-parameter-expansion/basic.sh for instance.
use ocamlfind to install everything, not only the META file. Make sure that the target directory can be controlled when invicong make (with an option or environment variable)
README.mde says that one needs ocaml <= 4.04.2 to compile. However, it seems to compile and run without problems with ocaml 4.05.0. I sugest to drop the upper limit on the version of ocaml, unless there is compelling reason to keep it.
I'm using the word_cst
s in an other project (Morsmall / CoLiS-language) and I'm getting quite confused while using them:
Lines 322 to 335 in 2faccf3
In particular:
What are the constructs that can appear in the word
of a WordDoubleQuoted
?
Same question for a WordSingleQuoted
. In particular, why not have simple WordSingleQuoted of string
? I guess it has to do with 3.
I'm still not convinced by these WordName
and WordAssignmentWord
constructors that are sub-cases of WordLiteral
. Wouldn't that be more practical to have only WordLiteral
and helpers like is_name : string -> bool
and to_assignment_word : string -> name * word
?
What kind of constructs can one find in a WordAssignmentWord
that occurs inside a WordDoubleQuoted
(because I guess it can happen)? And inside a WordSingleQuoted
?
I assume that WordEmpty
is the same thing as WordLiteral ''
?
Do we still need WordOther
?
Also, I think we should document all this. I can take care of that, we just need to decide where.
when parsing the shell script containing the single line
'a'
that is letter a in single quotes, the simple json output contains
"CmdName_Word", [ "Word", "'"a"", (rest omitted)
with only one single quote at the beginning, and spurious double quotes.
Hi there,
In e9c9070 you populated the VERSION
file with "0.1
". Does that mean this commit is the release 0.1? If so, would you mind adding a git tag?
$ git tag 0.1 e9c90708fc64790f2abeb6021d289daa6fe2acdc
$ git push --tags origin master
GitHub automatically generates tarballs for tags, so it’s easier to download a stable version instead of relying on the relative stableness of HEAD
.
Thanks!
there seems to be a problem with the way alias expansion is integrated in the engine. If alias expansion has an impact on the grammatical structure of the script then this is not recognized properly by morbig, at least not in the following example:
#!/bin/sh
alias si='if'
alias alors='then'
alias sinon='else'
alias cesttout='fi'
alias blabla='echo'
si [ -n a ]
alors
blabla oui
sinon
blabla non
cesttout
Morbig recognizes this as a sequence of simple commands insetad of an if-then-else.
According to POSIX (see "alias" and "unalias" in the Chapter "4: Utilities" of the volume "Shell & Utilities"), both alias and unalias can take multiple arguments, like this :
alias show=echo list=ls
unalias show list
but morbig does not accept this
morbig --help
promises a dot output fotrmat. However,
% morbig --as dot postinst
Failure: Not me!
.
Unquoting is applied to the delimiter of a here document. However, morbig does not treat correctly the case of a backslash inside double quotes. Standard 2.2.3 says about the text inside double quotes:
The shall retain its special meaning as an escape character (see Escape Character (Backslash)) only when followed by one of the following characters when considered special: $ ` " \
That is, the following script should be correct as the \O inside the double quotes shall not be replaced by O but remain intact:
if true
then cat <<"E\OF"
hello
E\OF
fi
dash does this correctly, but morbig thinks that the here document is not completed.
I have added test cases to tests/good/ 2.7-redirection/2.7.4-here-document:
delimiter-backslash-in-double-quotes-todrop.sh
delimiter-backslash-in-double-quotes-tokeep.sh
Discussion open! @Niols, shall you start?
when the delimiter word of a here document is not quoted, the symbol behaves as inside
double quotes (standard 2.7.4). That is, indicates a continuation line. For instance
(test case continuation-line-in-here-documents):
if true
then
cat << EOF
first line
second line
third line with continuation\
EOF
forth and last line ((
EOF
else echo "baeh"
fi
the here document is ended in line 9, not line 7.
When using Libmorbig and parsing a incorrect file, I get:
Fatal error: exception Libmorbig.Errors.ParseError(_)
but when I try to try ... match Libmorbig.Errors.ParseError _ -> ...
, I get Error: Unbound module Libmorbig.Errors
.
revisit the mlpack file and make sure that types are correctly exported
The dependencies in the README file seem to be a bit strong. I can compile morbig on
debian stable, which has older versions than mentioned in the README for the following:
libyojson-ocaml 1.3.2-1+b1
libyojson-ocaml-dev 1.3.2-1+b1
libppx-visitors-ocaml-dev 20170404-1
-Ralf.
Morbig miscalculates positions in the input file. For instance, on the input file contining only one line:
true
Morbig reports the word "true" starting at column 4 and ending and columns 5.
With morbig 20180209 I get an Unclosed Subshell error on the following input:
#!/bin/sh
f() {
tempdir=`mktemp --directory --suffix=.aptlc`
}
Seems that morbig does not like the "=" sign in the second option". -Ralf.
add a doc target to the makefile to generate ocamldoc
Morbig gives an exception Engine.ParseError on the following file:
alias rename=alias
rename endif=fi
if true; then echo "hello"; endif
I cannot see anything in the posix specification that excludes this kind of alias. dash (debian 0.5.8-2.5) executes this correctly, and prints "hello". Interestingly, bash (debian 4.4-5) does not, and gives an error message "rename: command not found".
When parsing something like:
{ cat;} 9<<EOF
a
EOF
Morbig complains:
line 1, character 11: Syntax error.
It does not happen when there is no brace group or when there is no file descriptor. It seems to me that it should be accepted by the grammar. dash
, bash
and zsh
do accept the script.
I added a test (68a1cd5) for this particular case.
from an anonymous reviewer:
Consider this shell script:
cat <<'EOT'
abc ` def
ghi \
jkl
EOT
The delimiting word is quoted, so that the here-document lines should not be
expanded (§2.7.4). Morbig gives me a here-document containing "abc ` def\nghi
jkl\n". The backslash should have been included verbatim, not treated as
escaping the line break.
If I then put the same document inside a command substitution:
x=$(cat <<'EOT'
abc ` def
ghi \
jkl
EOT
)
then I get an error "Lexing error: unbalanced parentheses". Removing or escaping
the backtick removes the error, but being inside a command substitution should
not change the parsing of here-documents (Per §2.6.3, "Any valid shell script
can be used for command"). The given word is then "$(cat <<'EOT' abc \` def
ghi jkl EOT )", which is missing all of the newlines - I'm not sure whether
that's presentational or another difference.
*
, ?
and character ranges are always parsed as literals
finnish the migration of colis.git/software/shparser
This would require a better encapsulation of the lexing buffer.
with the --debug option, and when using --from-stdin, morbig should also tell the pathname of each
script it opens. That would help in case of an error on a large corpus of scripts.
README.md says >= 4.02.1, opam says >= 4.02.3. Which one is right?
Compiling an other project (in this case, shstats
) that also has an Options module leads to an error:
Error: The files /path/to/libmorbig/libmorbig.cmi
and options.cmi make inconsistent assumptions over interface Options
This has not always been the case. I bisected this bug and found out that the last known good commit was 245208c and that the first bad commit is 0ca98a7. Sadly, there are two commits in between that do not compile, and 0ca98a7 is a very large commit, making it hard to find what went wrong.
I did not understand quite well what is causing the issue in shstats, and I didn't manage to produce a minimal example (although I'm still working on it). But it probably should not happen in any way in my opinion.
I will try to find more information about that and I keep you posted.
Just a TODO, not a bug: we should review what precisely is exported in the library. For instance, we probably do not want to export all of CSTHelpers.
We should integrate OSX as a target of our travis-based continuous integration.
Hi,
using the docker image and defining the shell function, I encounter a Fatal error: exception Sys_error("script.sjson: Is a directory")
error. The file mentioned is created, but empty.
Is there a workaround?
I haven't tried compiling morbig myself yet, so I don't know if the error also happens in that case.
When any part of the delimiter of a here document is quoted, the backslash in the here document shall behave as inside doubel quotes, that is it has a special meaning only when followed by $ ` " \ .
For example:
cat << "EOF1"
\a$\
EOF1
cat << EOF2
\a$\
EOF2
In the first case, the contents of the here documt shall be
\a$\
while in the second case it shall be
\a$\
Looking at the module hereDocument, it turns out that the flags which are pushed on the stack expanded are in fact never used (or poped).
% cd examples/c/
% make
ocamlfind: Package `libmorbig' not found
ocamlfind: Package `libmorbig' not found
cc -o dump -I /usr/lib/ocaml -I \
dump.c -lm -ldl /libmorbigc.a
cc: error: /libmorbigc.a: No such file or directory
Makefile:7: recipe for target 'all' failed
make: *** [all] Error 1
after changing in examples/c/Makefile
all libmorbig to morbig:
% make
cc -o dump -I /usr/lib/ocaml -I /usr/local/lib/ocaml/4.02.3/morbig \
dump.c -lm -ldl /usr/local/lib/ocaml/4.02.3/morbig/morbigc.a
cc: error: /usr/local/lib/ocaml/4.02.3/morbig/morbigc.a: No such file or directory
Makefile:7: recipe for target 'all' failed
Issue #15 is fixed but that was only for syntax errors. The same should be done for lexical errors, for instance with the test cases from tests/bad/2.3_token-recognition:
% morbig unterminated-backquote.sh
Fatal error: exception Prelexer.LexingError("Unterminated nesting!")
-Ralf.
Is there a reason why Morbig's parse_file
function returns a list of CST.complete_command
and not CST.complete_command located
? Wouldn't we want that information?
When running morbig --as simple
on the file at the end of this issue, I get
Failure: lexing: empty token.
as an output from Morbig. The Shell script seems to be accepted by other shells. I'll try to narrow this script down to the place where the "bug" occurs.
if [ "${0#/usr/share/initramfs-tools/hooks/}" != "$0" ] ||
[ "${0#/etc/initramfs-tools/hooks/}" != "$0" ]; then
# called from an initramfs-tools hook script
TABFILE="$DESTDIR/cryptroot/crypttab"
elif [ "${0#/scripts/}" != "$0" ]; then
# called at initramfs stage from a boot script
TABFILE="/cryptroot/crypttab"
else
TABFILE="${TABFILE-/etc/crypttab}"
fi
export DM_DEFAULT_NAME_MANGLING_MODE=hex # for dmsetup(8)
# Logging helpers. Send the argument list to plymouth(1), or fold it
# and print it to the standard error.
cryptsetup_message() {
local IFS=' '
if [ "${0#/scripts/}" != "$0" ] && [ -x /bin/plymouth ] && plymouth --ping; then
plymouth message --text="cryptsetup: $*"
elif [ ${#*} -lt 70 ]; then
echo "cryptsetup: $*" >&2
else
# use busybox's fold(1) and sed(1) at initramfs stage
echo "cryptsetup: $*" | fold -s | sed '1! s/^/ /' >&2
fi
return 0
}
# crypttab_parse_options([--export], [--quiet])
# Parse $_CRYPTTAB_OPTIONS, a comma-separated option string from the
# crypttab(5) 4th column, and sets corresponding variables
# CRYPTTAB_OPTION_<option>=<value> (which are added to the environment
# if --export is set).
# For error and warning messages, CRYPTTAB_NAME, (resp. CRYPTTAB_KEY)
# should be set to the (unmangled) mapped device name (resp. key
# file).
# Return 1 on parsing error, 0 otherwise (incl. if unknown options
# were encountered).
crypttab_parse_options() {
local quiet="n" export="n"
while [ $# -gt 0 ]; do
case "$1" in
--quiet) quiet="y";;
--export) export="y";;
*) cryptsetup_message "WARNING: crypttab_parse_options(): unknown option $1"
esac
shift
done
local IFS=',' x OPTION VALUE
unset -v CRYPTTAB_OPTION_cipher \
CRYPTTAB_OPTION_size \
CRYPTTAB_OPTION_hash \
CRYPTTAB_OPTION_offset \
CRYPTTAB_OPTION_skip \
CRYPTTAB_OPTION_verify \
CRYPTTAB_OPTION_readonly \
CRYPTTAB_OPTION_discard \
CRYPTTAB_OPTION_plain \
CRYPTTAB_OPTION_luks \
CRYPTTAB_OPTION_tcrypt \
CRYPTTAB_OPTION_veracrypt \
CRYPTTAB_OPTION_swap \
CRYPTTAB_OPTION_tmp \
CRYPTTAB_OPTION_check \
CRYPTTAB_OPTION_checkargs \
CRYPTTAB_OPTION_tries \
CRYPTTAB_OPTION_initramfs \
CRYPTTAB_OPTION_noearly \
CRYPTTAB_OPTION_noauto \
CRYPTTAB_OPTION_loud \
CRYPTTAB_OPTION_quiet \
CRYPTTAB_OPTION_keyscript \
CRYPTTAB_OPTION_keyslot \
CRYPTTAB_OPTION_header \
CRYPTTAB_OPTION_tcrypthidden
# use $_CRYPTTAB_OPTIONS not $CRYPTTAB_OPTIONS as options values may
# contain '\054' which is decoded to ',' in the latter
for x in $_CRYPTTAB_OPTIONS; do
OPTION="${x%%=*}"
VALUE="${x#*=}"
if [ "$x" = "$OPTION" ]; then
unset -v VALUE
else
VALUE="$(printf '%b' "$VALUE")"
fi
if ! crypttab_validate_option; then
if [ "$quiet" = "n" ]; then
cryptsetup_message "ERROR: $CRYPTTAB_NAME: invalid value for '${x%%=*}' option, skipping"
fi
return 1
elif [ -z "${OPTION+x}" ]; then
continue
fi
if [ "$export" = "y" ]; then
export "CRYPTTAB_OPTION_$OPTION"="${VALUE-yes}"
else
eval "CRYPTTAB_OPTION_$OPTION"='${VALUE-yes}'
fi
done
IFS=" "
if [ "$quiet" = "n" ] &&
[ -z "${CRYPTTAB_OPTION_luks+x}" ] && [ -n "${CRYPTTAB_OPTION_header+x}" ]; then
cryptsetup_message "WARNING: Option 'luks' missing in crypttab for target $CRYPTTAB_NAME." \
"Headers are only supported for LUKS devices."
fi
if [ -z "${CRYPTTAB_OPTION_luks+x}" ] && [ -z "${CRYPTTAB_OPTION_tcrypt+x}" ]; then
# the compiled-in default for these are subject to change
options='cipher size'
if [ -n "${CRYPTTAB_OPTION_keyscript+x}" ] || [ "$CRYPTTAB_KEY" = "none" ]; then
options="$options hash" # --hash is being ignored in plain mode with keyfile specified
fi
for o in $options; do
if [ "$quiet" = "n" ] && eval [ -z "\${CRYPTTAB_OPTION_$o+x}" ]; then
cryptsetup_message "WARNING: Option '$o' missing in crypttab for plain dm-crypt" \
"mapping $CRYPTTAB_NAME. Please read /usr/share/doc/cryptsetup/README.initramfs and" \
"add the correct '$o' option to your crypttab(5)."
fi
done
fi
}
# crypttab_validate_option()
# Validate $OPTION=$VALUE (or flag $OPTION if VALUE is unset). return
# 1 on error, unsets OPTION for unknown or useless options.
crypttab_validate_option() {
# option aliases
case "$OPTION" in
read-only) OPTION="readonly";;
key-slot) OPTION="keyslot";;
tcrypt-hidden) OPTION="tcrypthidden";;
tcrypt-veracrypt) OPTION="veracrypt";;
esac
# sanitize the option name so CRYPTTAB_OPTION_$OPTION is a valid variable name
local o="$OPTION"
case "$o" in
keyfile-offset) OPTION="keyfile_offset";;
keyfile-size) OPTION="keyfile_size";;
esac
case "$o" in
# value must be a non-empty string
cipher|hash|header)
[ -n "${VALUE:+x}" ] || return 1
;;
# numeric options >0
size|keyfile-size)
if ! printf '%s' "${VALUE-}" | grep -Exq "0*[1-9][0-9]*"; then
return 1
fi
;;
# numeric options >=0
offset|skip|tries|keyslot|keyfile-offset)
if ! printf '%s' "${VALUE-}" | grep -Exq "[0-9]+"; then
return 1
fi
;;
tmp)
if [ -z "${VALUE+x}" ]; then
VALUE="ext4" # 'tmp flag'
elif [ -z "$VALUE" ]; then
return 1
fi
;;
check)
if [ -z "${VALUE+x}" ]; then
if [ -n "${CRYPTDISKS_CHECK-}" ]; then
VALUE="$CRYPTDISKS_CHECK"
else
unset -v OPTION
return 0
fi
fi
if [ -x "/lib/cryptsetup/checks/$VALUE" ] && [ -f "/lib/cryptsetup/checks/$VALUE" ]; then
VALUE="/lib/cryptsetup/checks/$VALUE"
elif [ ! -x "$VALUE" ] || [ ! -f "$VALUE" ]; then
return 1
fi
;;
checkargs)
[ -n "${VALUE+x}" ] || return 1 # must have a value (possibly empty)
;;
keyscript)
[ -n "${VALUE:+x}" ] || return 1 # must have a value
if [ "${VALUE#/}" = "$VALUE" ]; then
VALUE="/lib/cryptsetup/scripts/$VALUE"
fi
if [ ! -x "$VALUE" ] || [ ! -f "$VALUE" ]; then
return 1
fi
;;
# and now the flags
verify) ;;
loud) ;;
quiet) ;;
initramfs) ;;
noearly) ;;
noauto) ;;
readonly) ;;
discard) ;;
plain) ;;
luks) ;;
swap) ;;
tcrypt) ;;
veracrypt) ;;
tcrypthidden) ;;
*)
if [ "${quiet:-n}" = "n" ]; then
cryptsetup_message "WARNING: $CRYPTTAB_NAME: ignoring unknown option '$o'";
fi
unset -v OPTION
;;
esac
}
# crypttab_resolve_source()
# Resolve the CRYPTTAB_SOURCE variable, containing value of the second
# field of a crypttab(5)-like file.
# On error (non-existing source), CRYPTTAB_SOURCE is not changed and 1
# is returned.
crypttab_resolve_source() {
# return immediately if source is a regular file
[ ! -f "$CRYPTTAB_SOURCE" ] || return 0
# otherwise resolve the block device specification
local dev="$CRYPTTAB_SOURCE"
dev="$(resolve_device_spec "$dev")" && CRYPTTAB_SOURCE="$dev" || return 1
}
# run_keyscript($keyscriptarg, $tried_count)
# exec()'ute `$CRYPTTAB_OPTION_keyscript $keyscriptarg`.
# If $CRYPTTAB_OPTION_keyscript is unset or null and $keyscriptarg is
# "none" (meaning the passphrase is to be read interactively from the
# console), use `/lib/cryptsetup/askpass` as keyscript with a suitable
# prompt message.
# Since the shell process is replaced with the $CRYPTTAB_OPTION_keyscript
# program, run_keyscript() must be used on the left-hand side of a
# pipe, or similar.
run_keyscript() {
local keyscriptarg="$1" CRYPTTAB_TRIED="$2" keyscript;
export CRYPTTAB_NAME CRYPTTAB_SOURCE CRYPTTAB_OPTIONS
export CRYPTTAB_TRIED
if [ -n "${CRYPTTAB_OPTION_keyscript+x}" ] && \
[ "$CRYPTTAB_OPTION_keyscript" != "/lib/cryptsetup/askpass" ]; then
# 'keyscript' option is present: export its argument as
# $CRYPTTAB_KEY
export CRYPTTAB_KEY="$keyscriptarg"
keyscript="$CRYPTTAB_OPTION_keyscript"
elif [ "$keyscriptarg" = none ]; then
# don't export the prompt message as CRYPTTAB_KEY
keyscript="/lib/cryptsetup/askpass"
keyscriptarg="Please unlock disk $CRYPTTAB_NAME: "
fi
exec "$keyscript" "$keyscriptarg"
}
# get_crypt_type()
# Return the mapping's type, depending on its
# $CRYPTTAB_OPTION_<option> values
get_crypt_type() {
local type="plain" # assume plain dm-crypt device by default
if [ "${CRYPTTAB_OPTION_tcrypt-}" = "yes" ]; then
type="tcrypt"
elif [ "${CRYPTTAB_OPTION_plain-}" = "yes" ]; then
type="plain"
elif [ "${CRYPTTAB_OPTION_luks-}" = "yes" ] ||
/sbin/cryptsetup isLuks -- "${CRYPTTAB_OPTION_header-$CRYPTTAB_SOURCE}"; then
type="luks"
fi
echo "$type"
}
# unlock_mapping([$keyfile])
# Run cryptsetup(8) with suitable options and arguments to unlock
# $CRYPTTAB_SOURCE and setup dm-crypt managed device-mapper mapping
# $CRYPTTAB_NAME.
unlock_mapping() {
local keyfile="${1:--}"
if [ -n "${CRYPTTAB_OPTION_header+x}" ] && [ ! -f "$CRYPTTAB_OPTION_header" ]; then
cryptsetup_message "ERROR: $CRYPTTAB_NAME: LUKS header '$CRYPTTAB_OPTION_header' missing"
return 1
fi
local type="$(get_crypt_type)"
if [ "$type" = "luks" ] || [ "$type" = "tcrypt" ]; then
# ignored for LUKS and TCRYPT devices
unset -v CRYPTTAB_OPTION_cipher \
CRYPTTAB_OPTION_size \
CRYPTTAB_OPTION_hash \
CRYPTTAB_OPTION_offset \
CRYPTTAB_OPTION_skip
fi
if [ "$type" = "plain" ] || [ "$type" = "tcrypt" ]; then
unset -v CRYPTTAB_OPTION_keyfile_size
fi
if [ "$type" = "tcrypt" ]; then
# ignored for TCRYPT devices
unset -v CRYPTTAB_OPTION_keyfile_offset
else
# ignored for non-TCRYPT devices
unset -v CRYPTTAB_OPTION_veracrypt CRYPTTAB_OPTION_tcrypthidden
fi
if [ "$type" != "luks" ]; then
# ignored for non-LUKS devices
unset -v CRYPTTAB_OPTION_keyslot
fi
/sbin/cryptsetup -T1 \
${CRYPTTAB_OPTION_header:+--header="$CRYPTTAB_OPTION_header"} \
${CRYPTTAB_OPTION_cipher:+--cipher="$CRYPTTAB_OPTION_cipher"} \
${CRYPTTAB_OPTION_size:+--key-size="$CRYPTTAB_OPTION_size"} \
${CRYPTTAB_OPTION_hash:+--hash="$CRYPTTAB_OPTION_hash"} \
${CRYPTTAB_OPTION_offset:+--offset="$CRYPTTAB_OPTION_offset"} \
${CRYPTTAB_OPTION_skip:+--skip="$CRYPTTAB_OPTION_skip"} \
${CRYPTTAB_OPTION_verify:+--verify-passphrase} \
${CRYPTTAB_OPTION_readonly:+--readonly} \
${CRYPTTAB_OPTION_discard:+--allow-discards} \
${CRYPTTAB_OPTION_veracrypt:+--veracrypt} \
${CRYPTTAB_OPTION_keyslot:+--key-slot="$CRYPTTAB_OPTION_keyslot"} \
${CRYPTTAB_OPTION_tcrypthidden:+--tcrypt-hidden} \
${CRYPTTAB_OPTION_keyfile_size:+--keyfile-size="$CRYPTTAB_OPTION_keyfile_size"} \
${CRYPTTAB_OPTION_keyfile_offset:+--keyfile-offset="$CRYPTTAB_OPTION_keyfile_offset"} \
--type="$type" --key-file="$keyfile" \
open -- "$CRYPTTAB_SOURCE" "$CRYPTTAB_NAME"
}
# crypttab_key_check()
# Sanity checks for keyfile $CRYPTTAB_KEY. CRYPTTAB_NAME and
# CRYPTTAB_OPTION_<option> must be set appropriately.
crypttab_key_check() {
if [ ! -f "$CRYPTTAB_KEY" ] && [ ! -b "$CRYPTTAB_KEY" ] && [ ! -c "$CRYPTTAB_KEY" ] ; then
cryptsetup_message "WARNING: $CRYPTTAB_NAME: keyfile '$CRYPTTAB_KEY' not found"
return 0
fi
if [ "$CRYPTTAB_KEY" = "/dev/random" ] || [ "$CRYPTTAB_KEY" = "/dev/urandom" ]; then
if [ -n "${CRYPTTAB_OPTION_luks+x}" ] || [ -n "${CRYPTTAB_OPTION_tcrypt+x}" ]; then
cryptsetup_message "WARNING: $CRYPTTAB_NAME: has random data as key"
return 1
else
return 0
fi
fi
local mode="$(stat -L -c"%04a" -- "$CRYPTTAB_KEY")"
if [ $(stat -L -c"%u" -- "$CRYPTTAB_KEY") -ne 0 ] || [ "${mode%00}" = "$mode" ]; then
cryptsetup_message "WARNING: $CRYPTTAB_NAME: key file $CRYPTTAB_KEY has" \
"insecure ownership, see /usr/share/doc/cryptsetup/README.Debian."
fi
}
# resolve_device_spec($spec)
# Resolve LABEL=<label>, UUID=<uuid>, PARTUUID=<partuuid> and
# PARTLABEL=<partlabel> to a block special device. If $spec is
# already a (link to a block special device) then it is echoed as is.
# Return 1 if $spec doesn't correspond to a block special device.
resolve_device_spec() {
local spec="$1"
case "$spec" in
UUID=*|LABEL=*|PARTUUID=*|PARTLABEL=*)
# don't use /dev/disk/by-label/... to avoid gessing udev mangling
spec="$(blkid -l -t "$spec" -o device)" || spec=
;;
esac
[ -b "$spec" ] && printf '%s\n' "$spec" || return 1
}
# dm_blkdevname($name)
# Print the mapped device name, or return 1 if the the device doesn't exist.
dm_blkdevname() {
local name="$1" dev
# /dev/mapper/$name isn't reliable due to udev mangling
if dev="$(dmsetup info -c --noheadings -o blkdevname -- "$name" 2>/dev/null)" &&
[ -n "$dev" ] && [ -b "/dev/$dev" ]; then
echo "/dev/$dev"
return 0
else
return 1
fi
}
# crypttab_find_entry([--quiet], $target)
# Search in the crypttab(5) for the given $target, and sets the
# variables CRYPTTAB_NAME, CRYPTTAB_SOURCE, CRYPTTAB_KEY and
# CRYPTTAB_OPTIONS accordingly. (In addition _CRYPTTAB_NAME,
# _CRYPTTAB_SOURCE, _CRYPTTAB_KEY and _CRYPTTAB_OPTIONS are set to the
# unmangled values before decoding the escape sequence.) If there are
# duplicates then only the first match is considered.
# Return 0 if a match is found, and 1 otherwise.
crypttab_find_entry() {
local target="$1" quiet="n" IFS
if [ "$target" = "--quiet" ] && [ $# -eq 2 ]; then
quiet="y"
target="$2"
fi
if [ -f "$TABFILE" ]; then
while IFS=" " read -r _CRYPTTAB_NAME _CRYPTTAB_SOURCE _CRYPTTAB_KEY _CRYPTTAB_OPTIONS; do
if [ "${_CRYPTTAB_NAME#\#}" != "$_CRYPTTAB_NAME" ] || [ -z "$_CRYPTTAB_NAME" ]; then
# ignore comments and empty lines
continue
fi
# unmangle names
CRYPTTAB_NAME="$(printf '%b' "$_CRYPTTAB_NAME")"
if [ -z "$_CRYPTTAB_SOURCE" ] || [ -z "$_CRYPTTAB_KEY" ] || [ -z "$_CRYPTTAB_OPTIONS" ]; then
cryptsetup_message "WARNING: '$CRYPTTAB_NAME' is missing some arguments, see crypttab(5)"
continue
elif [ "$CRYPTTAB_NAME" = "$target" ]; then
CRYPTTAB_SOURCE="$( printf '%b' "$_CRYPTTAB_SOURCE" )"
CRYPTTAB_KEY="$( printf '%b' "$_CRYPTTAB_KEY" )"
CRYPTTAB_OPTIONS="$(printf '%b' "$_CRYPTTAB_OPTIONS")"
return 0
fi
done <"$TABFILE"
fi
if [ "$quiet" = "n" ]; then
cryptsetup_message "WARNING: target '$target' not found in $TABFILE"
fi
return 1
}
# crypttab_foreach_entry($callback)
# Iterate through the crypttab(5) and run the given $callback for each
# entry found. Variables CRYPTTAB_NAME, CRYPTTAB_SOURCE, CRYPTTAB_KEY
# and CRYPTTAB_OPTIONS are set accordingly and available to the
# $callback. (In addition _CRYPTTAB_NAME, _CRYPTTAB_SOURCE,
# _CRYPTTAB_KEY and _CRYPTTAB_OPTIONS are set to the unmangled values
# before decoding the escape sequence.)
# Return 0 if a match is found, and 1 otherwise.
crypttab_foreach_entry() {
local callback="$1" IFS
local _CRYPTTAB_NAME _CRYPTTAB_SOURCE _CRYPTTAB_KEY _CRYPTTAB_OPTIONS \
CRYPTTAB_NAME CRYPTTAB_SOURCE CRYPTTAB_KEY CRYPTTAB_OPTIONS
[ -f "$TABFILE" ] || return
while IFS=" " read -r _CRYPTTAB_NAME _CRYPTTAB_SOURCE _CRYPTTAB_KEY _CRYPTTAB_OPTIONS <&9; do
if [ "${_CRYPTTAB_NAME#\#}" != "$_CRYPTTAB_NAME" ] || [ -z "$_CRYPTTAB_NAME" ]; then
# ignore comments and empty lines
continue
fi
# unmangle names
CRYPTTAB_NAME="$( printf '%b' "$_CRYPTTAB_NAME" )"
CRYPTTAB_SOURCE="$( printf '%b' "$_CRYPTTAB_SOURCE" )"
CRYPTTAB_KEY="$( printf '%b' "$_CRYPTTAB_KEY" )"
CRYPTTAB_OPTIONS="$(printf '%b' "$_CRYPTTAB_OPTIONS")"
if [ -z "$CRYPTTAB_SOURCE" ] || [ -z "$CRYPTTAB_KEY" ] || [ -z "$CRYPTTAB_OPTIONS" ]; then
cryptsetup_message "WARNING: '$CRYPTTAB_NAME' is missing some arguments, see crypttab(5)"
continue
fi
"$callback" 9<&-
done 9<"$TABFILE"
}
# vim: set filetype=sh :
add a target to the makefile to uninstall
on the corpus sid.amd64.main+contrib+non-free.2018-05-23:
$ examples/scripts/debian-maintainer-scripts corpus
Cleaning up ... done.
Start parsing 31302 files.
Fatal error: exception File "prelexerState.ml", line 287, characters 31-37: Assertion failed
Command exited with non-zero status 2
I am trying to find out on which file this happens, but this is surprisingly not so easy.
The Posix standard says:
... resulting word that is identified to be the command name word of a simple command shall be examined to determine whether it is an unquoted, valid alias name ...
In my understanding (which may be wrong) this means that only a WORD which is obtained
through this sub-derivation
simple_command -> cmd_name -> WORD
or maybe as the first WORD in
simple_command -> cmd_name cmd_suffix -> WORD cmd_suffix
can be candidate for alias substitution. How these can be found before applying any grammar rules, as the standard requires, escapes me but that is a different matter.
For instance, the following shell script
alias show=echo
show show
should print "show", and "not "echo" . Dash does this right, but when you look at the json produced by morbig you see that it has in fact replaced both occurrences of "show".
The same error occurs on toplevel-unalias.sh in the tests/good directory: If you look at the json produced by morbig you see that the unalias has not been executed. The reason for this is
that the first alias has also replaced the argument of the succeeding unalias, making it ineffective.
I don't really see the use case. In particular because it depends on where morbig has been called.
Is it really useful to have that in the marshalized value?
When parsing the following file
alias x='true '
x
I obtain (only the simgle command corresponding to the second line shown)
"SimpleCommand_CmdName",
[
"CmdName_Word",
[ "Word", "'true '", [ [ "WordLiteral", "x" ] ] ]
]
The x in the list of word components should have been replaced.
When parsing here-documents, the current version of Morbig includes in the word describing their contents the end marker.
Here is an example in for_loop_here_document.sh
where the contents of the here-document should be foo\n
and the separator should be DATA
. Morbig indeed says that the marker is DATA
. But it also says that the contents is foo\nDATA
, which is wrong:
write a C API
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.