yihui/testit

A simple package for testing R packages

testit

This package provides two simple functions (30 lines of code in total):

• assert(fact, ...): think of it as message(fact) + stopifnot(...)

• test_pkg(package): runs tests with all objects (exported or non-exported) in the package namespace directly available, so no need to use the triple-colon package:::name for non-exported objects

Why?

Because it is tedious to type these commands repeatedly in tests:

message('checking if these numbers are equal...')
stopifnot(all.equal(1, 1+1e-10), 10*.1 == 1)

message('checking if a non-exported function works...')
stopifnot(is.character(package:::utility_foo(x = 'abcd', y = 1:100)))

With the two simple functions above, we type six letters (assert) instead of sixteen (message + stopifnot), and assert is also a more intuitive function name for testing purposes (you assert a fact followed by evidence):

assert('These numbers are equal', {

  (all.equal(1, 1 + 1e-10))

  (10 * .1 == 1)

})

assert('A non-exported function works', {
  res = utility_foo(x = 'abcd', y = 1:100)
  (is.character(res))
})

assert('T is TRUE and F is FALSE by default, but can be changed', {
  (T == TRUE )
  (F == FALSE)

  T = FALSE
  (T == FALSE)
})

R CMD check

Put the tests under the directory pkg_name/tests/testit/ (where pkg_name is the root directory of your package), and write a test-all.R under pkg_name/tests/:

library(testit)
test_pkg('pkg_name')

That is all for R CMD check. For package development, it is recommended to use devtools. In particular, Ctrl/Cmd + Shift + L in RStudio makes all objects in a package visible to you, and you can run tests interactively.

Installation

Stable version on CRAN:

install.packages('testit')

Development version:

remotes::install_github('yihui/testit')

More

How about testthat? Well, this package is far less sophisticated than testthat. There is nothing fancy in this package. Please do consider testthat if your tests require more granularity. I myself do not use testthat because I'm too lazy to learn the new vocabulary (testthat::expect_xxx). For testit, I do not need to think if I should use expect_equal, expect_equivalent, or expect_identical; I just write test conditions in parentheses that are expected to return TRUE. That is the only single rule to remember.

There is no plan to add new features or reinvent anything in this package. It is an intentionally tiny package.

Although he did not really mean it, Xunzi said something that happens to apply well to unit testing:

不积跬步，无以至千里；不积小流，无以成江海。

This package is free and open source software, licensed under GPL-3.

testit's Issues

Fix coverage report in CI

Just to not forget it currently fails
https://github.com/yihui/testit/runs/1445996248#step:8:10

Something related to posting to coveralls.

I noticed other repo used codecov . We could take the opportunity to switch to be consistent instead of fix.

User cannot suppress error message in has_error()

The function argument silent is not exposed to the user in the has_error() function. It depends entirely on whether or not the function is running interactively because of

As a feature request, it would be nice as a user to override the default behavior when needed. Like this:

has_error(stop('An intentional error'), silent=TRUE);
#> [1] TRUE

has_error(1 + 'a', silent=FALSE)
#> Error in 1 + "a" : non-numeric argument to binary operator
#> [1] TRUE

should the test files be placed under ./tests/testit or ./inst/tests/testit

I'm confused because clearly test_pkg()'s source code suggests ./inst/tests/testit but testit itself place the tests under ./tests/testit.

testit/R/testit.R

Line 172 in 65768f8

path = available_dir(c(dir, system.file('tests', 'testit', package = package)))

So what's the correct way to do?

Thanks.

Which GPL

With regards to a819fb7

Could you clarify which GPL license applies. Ideally create a LICENSE file in the repository which can be included in distro packaging, such as RPM files.

rcmdcheck fail

Hi! I tried to run tests in rcmdcheck but with no success.

Log:

* checking tests ...
  Running ‘test-all.R’
 ERROR
Running the tests in ‘tests/test-all.R’ failed.
Complete output:
  > library(testit)
  > test_pkg('rgugik')
  Linking to GEOS 3.8.1, GDAL 3.1.1, PROJ 6.3.1
  Error from parse_and_simplify(txt = txt, simplifyVector = simplifyVector,  ... 
  Error in parse_con(txt, bigint_as_char) : 
    lexical error: invalid char in json text.
                                         <html><head><title>Request Reje
                       (right here) ------^
  Calls: test_pkg ... <Anonymous> -> parse_and_simplify -> parseJSON -> parse_con
  Execution halted

Using Test Package (Ctrl + Shift + T) works fine:

==> Sourcing R files in 'tests' directory

Linking to GEOS 3.8.0, GDAL 3.0.4, PROJ 6.3.1
trying URL 'https://opendata.geoportal.gov.pl/ortofotomapa/41/41_3756_N-33-130-D-b-2-3.tif'
Content type 'application/octet-stream' length 2137288 bytes (2.0 MB)
==================================================
downloaded 2.0 MB

Tests complete

covr does not correctly interact with testit multiple line check

Hi, not sure if this is really an error, but it seems to me like testit would intend to support this common setup:

• testthat for unit tests
• testit for runtime assertions in functions
• covr for code coverage

The setup works well unless I use testit::assert on several lines, like in the testit README file

Example:
file R/myfuncs.R

failFunc <- function() {
    assert("Foo", {
        (FALSE == TRUE)
        (TRUE == TRUE)
    })
}

test file tests/testthat/test.myfuncs.R

test_that("failFunc fails", {
     expect_error(failFunc())
})

when running devtools:test() the test passes as expected. However, with the command covr::package_coverage() the test does not pass, apparently because the first assertion (FALSE == TRUE) is not evaluated. Changing the function fixes this, but is not in the spirit of testit:

failFunc <- function() {
    assert("Foo", {
        (FALSE == TRUE) &&
        (TRUE == TRUE)
    })
}

Not sure if this is related to the way covr works or is part of the testit internals.

Will this work with covr?

I currently use testthat and covr (for code coverage) but I find your README compelling. Is this pkg compatible with covr?