progrium/bashstyle

Let's do Bash right!

Hello, I'm Jeff! 👋

I'm a creator and entrepreneur known for my work and ideas in open source developer tooling.

I try to push the boundaries of computing as a magical tool and compiler for the imagination.

Since you found me on GitHub, you can join my Discord.

bashstyle's Issues

Only capitalize exported/environment variable names

This is one of the stylistic rules I'm using for Plushu, and it's working out pretty well: only use ALL_CAPS_SNAKE_CASE when you're going to be getting a variable from the environment, or when you're going to export it to the environment. Names for script-local variables that will only appear in subshells should be small_caps_snake_case.

support for differences when considering zsh/sh compatibility?

e.g. using [ vs. [[

Mention shellcheck

http://www.shellcheck.net/ is a linter for Bash that covers a lot of this (and more, too).

repo at https://github.com/koalaman/shellcheck

maintain in bpkg ?

Heyo - would you be into maintaining this in bpkg ? It'd be great if we could spread these same ideas about bash in bpkg as part of its core philosophy. I could always just reference : ]

Using STDOUT, STDERR, and the other file descriptors

STDOUT should be easily parseable. Output from STDOUT should avoid using locale-dependent human language wherever possible. (The closer you can match the syntax used by an existing, older utility your output would be piped to, the better.)

STDERR should be used for any warnings or errors (since they won't go in a pipe and will normally go to the terminal). See this Stack Overflow question.

The other file descriptors should maybe have a convention, taking into account how they get used and what bugs there are around them (isn't there something about fd5 not being copied for subcommands or something)? For example, fd6 should be used as the holding descriptor when swapping two other file descriptors.

I know I used the convention in Plushu that a command's programmatic output was to be output on STDOUT (so it could be captured with command substitution), and the command's terminal output (the output from its calls' STDOUT) was output on fd3.

Use `command -v` instead of `which` &c

command -v is standard and portable, which isn't.

Shell linter

Probably makes sense to write a tool that analyzes shell scripts to tell you if you did it right.

Coverage

Please take a look at LCOV.SH full BASH implementation of coverage… no need for additional interpreters like RUBY or BINARY executable in your machine. Check coverage of BASH just with BASH https://github.com/javanile/lcov.sh

Fully-qualified paths

Even relative paths should be qualified with ./: ideally, all paths should be fully-qualified down to the root, so that they'll be equally valid when passed to another command with a different PWD.

Never use paths with an implicit containing directory (like rm *): if one of these filenames start with a hyphen, it's entirely likely to be interpreted as an option rather than a filename (hope that rm * wasn't run in a directory with a file named -r)!

Example of "This can be queried/extracted with a simple function using reflection."

In the section talking about declare desc="description", it talks about using a function to extract this. What would this look like?

Would it be worth putting an example in the bottom section?

Use [[ -n "$whatever" ]] rather than [[ "$whatever" ]]

Okay, so, in Bash, with double-brace tests, this will never actually be misinterpreted.

However:

• Using -n makes your test have an operator, just like every other test will, making it perfectly clear that this wasn't, say, somebody forgetting to add an operator/operand to the test.
• In older shell constructs like the single-brace test (which your tests could be converted to if somebody copies a snippet of it to convert to zsh or something), [ "$whatever" ... is how you specify a variable test operator (yikes!), and thus this opens up a potential channeling attack (see #18).

Google's Bash styleguide link is broken

Hi, the new link is: https://google.github.io/styleguide/shell.xml

Calling conventions: short-opts, long-opts, equals, keywords, etc

I think a section on (or link to) how to parse, and design, script arguments, and specifically options (both long and short), would be useful.

Shellcheck utility reference

I've used: ShellCheck in its independently-runnable form for a while and tie its output into a Jenkins job that shows warnings in a visible way. It might need some love, but it'd be a great to connect these "rules" with a tool that helped you understand where you're making mistakes.

Bash Unit Testing

There's more than just shunit2 out there for bash unit testing. I've used:

• roundup
• plus a bunch of additions such as test reporting via junit (internal, which I'd like to get out to the community), namely better integration with:

Mocking:

• bock

I've also been wanting to try out:

• bats

Bash Profiler

This project is quite handy when you are trying to improve the performance of your scripts:
https://github.com/F-Hauri/bashProfiler

set -u

I would add set -u into your advice for set -eo pipefail, i.e. set -euo pipefail

An enhancement to that is to fail early in a script with useful feedback.

: ${FOO:?You have not set the FOO variable}

Why to always use "if [[ ... ]]; then; fi" instead of a conditional statement

Statements joined with && don't end a script even when it's running with set -e, but if they're the last statement in your file, their non-zero exit code will be your script's exit code (think of it like an implicit return), and if that script was getting called by something with set -e, the whole house of cards will come tumbling down.

In other words, while it's an easily-avoided ledge, it still needs a railing, because a single careless bump will send you way the hell crash tumbling down.

I used to remember this pitfall, but then I went away for a few months, and when I came back I couldn't remember why I didn't do this, and I did it all over my code, and immediately got bitten by this. In general, if you're using set -e, only use tests to test things.

posix shell instead of bash?

Why not encourage posix shell instead of bash? It's still possible to stick with /bin/sh shell and make things looks as neat as bash.

.bash_profile tips

Hey @progrium , loved the idea to put this repo together.
I was wondering what would be a good advice when having to export and set alias on .bash_profile.

Is there any good convention to keep those organised like:

aliases() {
    alias lsa="ls -la"
    …
}

exports() {
    export PATH="/usr/local/bin"
}

cheers

Ownership and modes

(This is a best practice that sort of runs into the "protips" area, one that applies more in Plushu or Dokku than it does to most Bash scripts.)

Ownership and sudo

Whenever you're creating something in a script, you should make sure that the files will be owned by the correct user.

For example, if you have a script that creates a directory of files, and this script is meant to be run as root, after creating it, you should do something like this:

if [[ "$EUID" == 0 && -n "$SUDO_USER" ]]; then
  chown -R "$SUDO_USER:" "$created_dir"
fi

Take care to note the colon after the username in the chown command. This tells chown to change not only the owning user on the files, but also the owning group on those files to the specified user's group (the same ownership it would have had had that user created the files themselves).

Permission bits

If you want to create a file with certain modes unset, you can run the command that creates the file in a subshell, prefixed by a umask command which will unset permission bits for any file created in that subshell:

(umask 0226; printf '%s\n' \
  "$PLUSHU_USER ALL=(ALL)NOPASSWD:`command -v nginx` -s reload" \
  >/etc/sudoers.d/plushu-reload-nginx)

Note that the umask is an inverted octal bitmask to restrict the permissions that files will be created with. If the script will normally create files with permission bits 0666 (-rw-rw-rw-), a umask of 0226 will create files with permissions of 0440 (-r--r-----).

This is specifically useful for creating a sudoers file (either the main /etc/sudoers or a file included from it), as sudo will refuse to run when a file in sudo's configuration does not have the proper permission bits. This can also be useful when working with files in a user's .ssh directory.

Tabs are not evil!

I using tabs, because of the great application in these heredocs:

#!/usr/bin/env bash

main() {
    cat > "message.txt" <<-EOF
        Hello, ${USER}!

        This has no leading whitespace.
        It is also nested in a more readable way.
        All indentation here is with tabs.
    EOF
}

[[ "$0" == "${BASH_SOURCE}" ]] && main "$@"

Locale independence

Setting LC_ALL=C

See http://unix.stackexchange.com/questions/87745/what-does-lc-all-c-do

If you don't set LC_ALL=C for, say, sorting a list of numbers from a container/network source, it's going to be dependent on the system locale. That's a problem, if, say, that system's locale is designed for using commas as the decimal separator rather than dots.

Even if you're managing the environment in a container, make sure that the default locale in that container's environment is LC_ALL=C, rather than something like en_US.

Article reference link appears broken.

This link: http://www.kfirlavi.com/blog/2012/11/14/defensive-bash-programming/ is now broken. Not sure where I would look to PR a replacement :/

Add license information

Hello,

can you please add license information.

Regards
steigr

Throwing `set -a` and `set +a` around a variable pair list

set -a makes it so that every assignment has an implicit export in front of it. set +a turns this off.

This makes it really easy to keep configuration in a file like this (which is compatible with Docker's --env option and EnvironmentFile in systemd):

SECRET_BASE=qweasd4321
SPECIAL_KEY=ghfdwqu98

and then export its variables (eg. setting up an environment for a script):

set -a; . config_file; set +a

Tracing and Debugging

One nice thing to put in a script's boilerplate is an environment variable to read that will enable xtrace and verbose, something like:

if [[ -n "$PLUSHU_TRACE" ]]; then set -vx; fi

This makes it possible to debug a script without having to mess with its option parsing or shebang.

Use `printf '%s\n'` rather than `echo` to echo strings

If the string you're echoing is -n, passing that string to echo will output nothing, potentially causing tons of stuff to break. This is why you should use printf '%s\n' for all line-echoes, except maybe when you're echoing fixed messages to STDERR with echo >&2.

When, and how, to use xargs

Since this is apparently a somewhat obscure tool for a very necessary task (converting strings to argument lists while preserving quoting/grouping), I think it's called for to have a note on using xargs -x (and xargs -xa when you can't clobber STDIN) to deserialize arguments from strings/streams, involving how to interpolate multiple inputs to output your string as a stream (use a sequence of commands in a subshell).

Google's Shell Styleguide

Another good reference to pull ideas from or link to is http://google-styleguide.googlecode.com/svn/trunk/shell.xml

Use curly braces for all variable references

How about standardizing on braces?

# Yep
echo "${var}"
echo "$1"

# Nope
echo "$var"

Notes on randomness, /tmp files, cleanup, and signal traps

• When you need to save a stream, use a file created by mktemp. Note that mktemp will use some (crypto-strong? I don't even think so) randomness to give your file an unpredictable name that's not going to collide with anything like another concurrent running instance of your script.
  • This is the most stable and secure solution I could find: I'd be glad if someone had a better solution here, as this does kind of give me that "this is totally a hole" feeling.
• When you're making temporary files, make sure they're deleted as soon as possible.
• If you've set -eo pipefail, your script could exit at any point, so make sure you've got your cleanup run as part of a trap for the exit signal.

Serializing with "$(printf %q "$VARIABLE")"

This is one of those holes that can open up if you're not being conscious of how you're outputting your strings. If it could have spaces, quotes, or dollar signs (and you should always treat strings as potentially having these things), and you're putting it somewhere where it will be treated as part of a string representing an argument list (to be deserialized using xargs -x, see #17), you should output it with printf %q rather than just naively echoing it (which should also be done with printf, see #18).

Guidance on documenting functions

What's the preferred style for documenting functions?

a() { # a does foo
    ...
}

# b does foo
b() {
    ...
}

c() {
    # c does foo
    ...
}

Namespacing / environmental consciousness; and, when to use args and when to use vars

There should maybe be a note about how general to make the environment variables you choose: remember that an ENV variable like DIR can be the only usage of DIR throughout the entire call stack leading up to your command, and if you use too general a name for too specific a thing, you're going to force authors to wrap your command in something that sets the environment up for your command anyway. (Besides, for this specific example, you should be using PWD for the one "dir" you're going to work in - that's why it's called the working directory.)

Similarly, maybe there should be a note on when you should use environment variables (for long-running deep truths about the environment in which the command and all lower commands are being executed) and when you should use arguments (when it's a parameter for this specific operation). Maybe even some thoughts on when to use positional opts vs. long opts vs. short opts vs. subcommands?

Using defaults and "${VAR:?}"

If you don't assert the existence of certain important variables, your script could inadvertently wipe the user's home directory:

ValveSoftware/steam-for-linux#3671

https://news.ycombinator.com/item?id=8896186

[discuss] Suffix all bash scripts with .sh

Use .sh or .bash extension if file is meant to be included/sourced. Never on executable script.

I'm curious about the philosophy of NOT including .sh as a suffix for scripts.

The main reason I always suffix shell scripts with .sh is to make shellcheck linting simpler. A makefile rule with find . -type f -name '*.sh' | xargs shellcheck is the simplest way I know to ensure quality control for all scripts in a repo.

`if expr` vs `if [[ expr ]]`?

In what cases does it make sense to use if expr?

I recall there is some edge case scenario where it's more straight forward to do that, but I can't remember when.