bats-core/bats-support

Supporting library for Bats test helpers

bats-support

bats-support is a supporting library providing common functions to test helper libraries written for Bats.

Features:

• error reporting
• output formatting
• language tools

See the shared documentation to learn how to install and load this library.

If you want to use this library in your own helpers or just want to learn about its internals see the developer documentation in the source files.

Error reporting

fail

Display an error message and fail. This function provides a convenient way to report failure in arbitrary situations. You can use it to implement your own helpers when the ones available do not meet your needs. Other functions use it internally as well.

@test 'fail()' {
  fail 'this test always fails'
}

The message can also be specified on the standard input.

@test 'fail() with pipe' {
  echo 'this test always fails' | fail
}

This function always fails and simply outputs the given message.

this test always fails

Output formatting

Many test helpers need to produce human readable output. This library provides a simple way to format simple messages and key value pairs, and display them on the standard error.

Simple message

Simple messages without structure, e.g. one-line error messages, are simply wrapped in a header and a footer to help them stand out.

-- ERROR: assert_output --
`--partial' and `--regexp' are mutually exclusive
--

Key-Value pairs

Some helpers, e.g. assertions, structure output as key-value pairs. This library provides two ways to format them.

When the value is one line long, a pair can be displayed in a columnar fashion called two-column format.

-- output differs --
expected : want
actual   : have
--

When the value is longer than one line, the key and value must be displayed on separate lines. First, the key is displayed along with the number of lines in the value. Then, the value, indented by two spaces for added readability, starting on the next line. This is called multi-line format.

-- command failed --
status : 1
output (2 lines):
  Error! Something went terribly wrong!
  Our engineers are panicing... \`>`;/
--

Sometimes, for clarity, it is a good idea to display related values also in this format, even if they are just one line long.

-- output differs --
expected (1 lines):
  want
actual (3 lines):
  have 1
  have 2
  have 3
--

Language and Execution

Restricting invocation to specific locations

Sometimes a helper may work properly only when called from a certain location. Because it depends on variables to be set or some other side effect.

A good example is cleaning up temporary files only if the test has succeeded. The outcome of a test is only available in teardown. Thus, to avoid programming mistakes, it makes sense to restrict such a clean-up helper to that function.

batslib_is_caller checks the call stack and returns 0 if the caller was invoked from a given function, and 1 otherwise. This function becomes really useful with the --indirect option, which allows calls through intermediate functions, e.g. the calling function may be called from a function that was called from the given function.

Staying with the example above, the following code snippet implements a helper that is restricted to teardown or any function called indirectly from it.

clean_up() {
  # Check caller.
  if batslib_is_caller --indirect 'teardown'; then
    echo "Must be called from \`teardown'" \
      | batslib_decorate 'ERROR: clean_up' \
      | fail
    return $?
  fi

  # Body goes here...
}

In some cases a helper may be called from multiple locations. For example, a logging function that uses the test name, description or number, information only available in setup, @test or teardown, to distinguish entries. The following snippet implements this restriction.

log_test() {
  # Check caller.
  if ! ( batslib_is_caller --indirect 'setup' \
      || batslib_is_caller --indirect "$BATS_TEST_NAME" \
      || batslib_is_caller --indirect 'teardown' )
  then
    echo "Must be called from \`setup', \`@test' or \`teardown'" \
      | batslib_decorate 'ERROR: log_test' \
      | fail
    return $?
  fi

  # Body goes here...
}

bats-support's Issues

How Should `batslib_count_lines` Handle Trailing Newlines?

Currently, batslib_count_lines is written to try and ignore the last newline to be equivalent to it not being there.

This leads '' and '\n' as both being "0 line". Similarly, 'a' and 'a\n' will both be considered as "1 line". This is explicitly checked in tests.

Issue

However, when using run --keep-empty-lines, assertions will check trailing newlines. However, this leads batslib_is_single_line to still return 0, and the error messages will be printed by batslib_print_kv_single instead of batslib_print_kv_multi (which shows the line numbers, a helpful indicator for debugging).
Example (using bats-assert):

@test "trailing empty line" {
  run --keep-empty-lines printf 'a\n'
  assert_output "$(printf 'a')"
}

prints:

-- output differs --
expected : a
actual   : a

--

The new line is there, but it's hard to really notice. More egregiously, a multi-line output will be very misleading.
Example:

@test "trailing empty line on multiline" {
  run --keep-empty-lines printf 'a\nb\n'
  assert_output "$(printf 'a\nb')"
}

prints:

-- output differs --
expected (2 lines):
a
b
actual (2 lines):
a
b
--  

This both hides the extra new line and states that the line counts are the same.

Discussion

I'm writing out this GH Issue (and not just sending some PR(s)) because there's a few ways that this could possibly be addressed. I wanted to chat this over and see which way you'd think would be best. I'm happy to put together a PR for wherever we land.

Here's some options that I've considered, feel free to suggest more:

1. Update the impl for batslib_count_lines.

This could be done by changing

batslib_count_lines() {
  local -i n_lines=0
  local line
  while IFS='' read -r line || [[ -n $line ]]; do
    (( ++n_lines ))
  done < <(printf '%s' "$1")
  echo "$n_lines"
}

to

batslib_count_lines() {
  local -r input="$1"
  local -i n_lines=0
  local line
  if [ -n "$input" ]; then
    while IFS='' read -r line; do
      (( ++n_lines ))
    done < <(printf '%s' "$input")
    # Increment for the EOF (not captured by the loop above).
    (( ++n_lines ))
  fi
  echo "$n_lines"
}

There are some related extra changes that are necessary with this approach. batslib_prefix needs to be updated in a similar way (else batslib_print_kv_multi in batslib_print_kv_single_or_multi will print the incorrect number of lines).
Potential impl change from

batslib_prefix() {
  local -r prefix="${1:-  }"
  local line
  while IFS='' read -r line || [[ -n $line ]]; do
    printf '%s%s\n' "$prefix" "$line"
  done
}

to

batslib_prefix() {
  local -r prefix="${1:-  }"
  local line=''
  local -i lines_read=0
  while IFS='' read -r line; do
    printf '%s%s\n' "$prefix" "$line"
    (( ++lines_read ))
  done
  # Print for the line with EOF (not captured by the loop above).
  # Do not print if stdin was completely empty.
  if (( lines_read > 0 )) || [[ -n "$line" ]]; then
    printf '%s%s\n' "$prefix" "$line"
  fi
}

batslib_mark would likely also need to be updated in a similar way (these 3 functions all use while IFS='' read -r line || [[ -n $line ]]; do).

While I think this is pretty straightforward (and my personal preference), I understand that this may have backwards compatibility issues and may have implications for other functionality built upon this function (which may really care about ignoring trailing newlines).

I can see that bats-assert makes use of these 3 functions in refute_line. The suggested changes to batslib_prefix and batslib_mark wouldn't really change anything. Though the indirect change to batslib_is_single_line (via the change to batslib_count_lines) might lead to some more calls being made to batslib_print_kv_multi instead of batslib_print_kv_single. I'd suspect that wouldn't affect most cases, as the trailing newline is removed unless explicitly forced to be kept (which is likely desirable).

2. Add --keep-empty-lines (or similar) to batslib_count_lines.

This option keeps the old functionality and allows the caller to change into tracking all newlines. This could either be done as opt-in with --keep-empty-lines or --include-trailing-newline; or it could be opt-out, where the default is to include it and --ignore-trailing-newline is added.

This option (specifically the opt-in version) is more backwards compatible with existing logic. However, it does require adding/piping the flag through various functions, which can be a bit verbose.
Concerns about differentiating --keep... from intended input can be worked-around via there should always be an even number of positional arguments to these specific functions.

3. Create alternative batslib_count_lines function.

In this option, a new function (e.g. batslib_count_all_lines) can be added to use the new logic.
This is very backwards compatible, as it does not touch the existing batslib_count_lines function. However, it would require doing the same for other functions like batslib_is_single_line, batslib_print_kv_multi, batslib_print_kv_single_or_multi, etc.

---

Those are the suggestions that I have thus far. As I mentioned, I think option 1 is the best option. Let me know what you think.

Thanks!

P.S. There's a psudo-related TODO(ztombol) on batslib_count_lines about fixing inconsistencies vs ${#lines[@]}. This isn't really the same issue, but it's somewhat related. Thought I'd mention it.

CC0 no longer allowed in Fedora. Move to 0BSD?

The use of CC0 for code is no longer allowed in Fedora (see https://lwn.net/Articles/902410/.) This kind of changes tend to extend to other distributions such as Debian and SUSE after a while.

Would it be possible to relicense bats-support to another, more widely accepted, public-domain-equivalent license such as 0BSD¹?

Footnotes

1. https://opensource.org/licenses/0BSD ↩

Missing version numbers?

This repo has no version number tags.

The README implies there should be that it got the versions number from bats-core when it split out? It's confusing. I'm guessing there were multiple moves and renames.

Anyway, ztombol/bats-support is version 0.3.0 so I assume there should be version numbers here as well.