README for xpectr

Ludvig Renbo Olsen

2020-02-09

Abstract

This vignette is an introduction to the xpectr package. xpectr provides a set of utilities and RStudio addins for generating tests for testthat unit testing.  
 
Contact author at r-pkgs@ludvigolsen.dk  
 

When developing R packages, it’s good practice to build a good set of unit tests that will notify you when something breaks in the future. For this, the testthat package is commonly used. Often though, we end up writing a similar set of tests again and again, which can be both tedious and time-consuming.

xpectr helps you by generating common testthat tests. Its goal is not to replace the active thinking process of finding meaningful tests for your functions, but to help systematize and ease that process.

One such example is gxs_function(), which generates tests for a set of argument value combinations. This allows you to focus on the selection of argument values to test. Once the tests have been generated, you can go through each of them to ensure your function is working as intended and to add relevant tests.

Note: If you comment out the call to gxs_function(), it is easy to later regenerate the tests. By using a diff tool, you can check that only the intended changes have been made to the file.

Installation

You can install the development version with:

# install.packages("devtools")  
devtools::install_github("ludvigolsen/xpectr")

You can install the latest CRAN release of the package with:

install.packages("xpectr")

Main functions

Generator functions

These functions are used for generating expectations (gxs).

Functions for use in tests

Helper functions

Addins

Using in packages

Suggestion: Add xpectr in the Suggests field in the DESCRIPTION file.

Examples

library(xpectr)
library(testthat)
library(dplyr)

# Set a seed
# When R > 3.6.0, it sets sampling.kind to "Rounding" to make
# tests compatible with previous versions of R
set_test_seed(42)
# Some data
num_vec <- 1:10
long_vec <- c(LETTERS, letters)
a_factor <- factor(c("a","b","c"))

df <- data.frame(
  'a' = c(1, 2, 3),
  'b' = c('t', 'y', 'u'),
  "c" = a_factor,
  stringsAsFactors = FALSE
) %>% 
  dplyr::group_by(a)

# A function with side effects
fn <- function(raise = FALSE){
  message("Hi! I'm Kevin, your favorite message!")
  warning("G'Day Mam! I'm a warning to the world!")
  message("Kevin is ma name! Yesss!")
  warning("Hopefully the whole world will see me :o")
  if (isTRUE(raise)){
    stop("Lord Evil Error has arrived! Yeehaaa")
  }
  "the output"
}

gxs_selection

Note: gxs_selection() can be used with the Insert Expectations addin. See ?insertExpectationsAddin for instructions on how to set up a key command.

Selection is a vector

Selection is a function call with side effects

When the selected code generates an error, warning or message, we can test those as well. An error is tested with expect_error(), while messages and warnings are tested as character vectors, to make sure we catch additional warnings/messages.

When running testthat unit tests on different systems, they sometimes vary in the use of punctuation and newlines (\n). The strip() and strip_msg() functions are wrapped around the side effects to remove non-alphanumeric symbols from the tested strings. This helps the expect_* functions to ignore those differences. In cases where such differences are important to catch, you can set strip = FALSE in gxs_selection().

We assign the output of the function to an output_12345 variable, so we don’t have to run it more than once. The number is randomly generated and is not guaranteed to be unique. suppress_mw() suppresses the messages and warnings.

# Generate expectations
gxs_selection("fn()")

# Inserts the following tests:

## Testing 'fn()'                                                           ####
## Initially generated by xpectr
xpectr::set_test_seed(42)
# Testing side effects
# Assigning side effects
side_effects_19148 <- xpectr::capture_side_effects(fn())
expect_equal(
  xpectr::strip(side_effects_19148[['warnings']]),
  xpectr::strip(c("G'Day Mam! I'm a warning to the world!", 
                  "Hopefully the whole world will see me :o")),
  fixed = TRUE)
expect_equal(
  xpectr::strip(side_effects_19148[['messages']]),
  xpectr::strip(c("Hi! I'm Kevin, your favorite message!\n", 
                  "Kevin is ma name! Yesss!\n")),
  fixed = TRUE)
# Assigning output
output_19148 <- xpectr::suppress_mw(fn())
# Testing class
expect_equal(
  class(output_19148),
  "character",
  fixed = TRUE)
# Testing type
expect_type(
  output_19148,
  type = "character")
# Testing values
expect_equal(
  output_19148,
  "the output",
  fixed = TRUE)
# Testing names
expect_equal(
  names(output_19148),
  NULL,
  fixed = TRUE)
# Testing length
expect_equal(
  length(output_19148),
  1L)
# Testing sum of element lengths
expect_equal(
  sum(xpectr::element_lengths(output_19148)),
  1L)
## Finished testing 'fn()'                                                  ####

# In case of errors, the warnings and messages aren't tested

gxs_selection("fn(raise = TRUE)")

# Inserts the following tests:

## Testing 'fn(raise = TRUE)'                                               ####
## Initially generated by xpectr
xpectr::set_test_seed(42)
# Testing side effects
expect_error(
  xpectr::strip_msg(fn(raise = TRUE)),
  xpectr::strip("Lord Evil Error has arrived! Yeehaaa"),
  fixed = TRUE)
## Finished testing 'fn(raise = TRUE)'                                      ####

gxs_function

When testing the inputs to a function, gxs_function() allows us to quickly specify values to check and generates tests for each of them.

The first value supplied for an argument is considered the valid baseline value. For each argument, we create tests for each of the supplied values, where the other arguments have their baseline value.

By default, each argument is tested with the NULL object as well, why we only need to specify it when the baseline value should be NULL.

It is important that we manually read through the generated tests to make sure that our function is behaving as intended, and to check if any important tests are missing.

# Define a function with arguments
fn <- function(x, y, z) {
  if (x > 3) stop("'x' > 3")
  if (y < 0) warning("'y'<0")
  if (z == 10) message("'z' was 10!")
  x + y + z
}

# Create tests for the function
# Note: We currently need to specify the list of arguments
# in the function call
gxs_function(fn = fn,
             args_values = list(
               "x" = list(2, 4, NA),
               "y" = list(0,-1),
               "z" = list(5, 10)
             ))

# Inserts the following tests:

## Testing 'fn'                                                             ####
## Initially generated by xpectr
# Testing different combinations of argument values

# Testing fn(x = 2, y = 0, z = 5)
xpectr::set_test_seed(42)
# Assigning output
output_19148 <- fn(x = 2, y = 0, z = 5)
# Testing class
expect_equal(
  class(output_19148),
  "numeric",
  fixed = TRUE)
# Testing type
expect_type(
  output_19148,
  type = "double")
# Testing values
expect_equal(
  output_19148,
  7,
  tolerance = 1e-4)
# Testing names
expect_equal(
  names(output_19148),
  NULL,
  fixed = TRUE)
# Testing length
expect_equal(
  length(output_19148),
  1L)
# Testing sum of element lengths
expect_equal(
  sum(xpectr::element_lengths(output_19148)),
  1L)

# Testing fn(x = 4, y = 0, z = 5)
# Changed from baseline: x
xpectr::set_test_seed(42)
# Testing side effects
expect_error(
  xpectr::strip_msg(fn(x = 4, y = 0, z = 5)),
  xpectr::strip("'x' > 3"),
  fixed = TRUE)

# Testing fn(x = NA, y = 0, z = 5)
# Changed from baseline: x
xpectr::set_test_seed(42)
# Testing side effects
expect_error(
  xpectr::strip_msg(fn(x = NA, y = 0, z = 5)),
  xpectr::strip("missing value where TRUE/FALSE needed"),
  fixed = TRUE)

# Testing fn(x = NULL, y = 0, z = 5)
# Changed from baseline: x
xpectr::set_test_seed(42)
# Testing side effects
expect_error(
  xpectr::strip_msg(fn(x = NULL, y = 0, z = 5)),
  xpectr::strip("argument is of length zero"),
  fixed = TRUE)

# Testing fn(x = 2, y = -1, z = 5)
# Changed from baseline: y
xpectr::set_test_seed(42)
# Testing side effects
# Assigning side effects
side_effects_16417 <- xpectr::capture_side_effects(fn(x = 2, y = -1, z = 5))
expect_equal(
  xpectr::strip(side_effects_16417[['warnings']]),
  xpectr::strip("'y'<0"),
  fixed = TRUE)
expect_equal(
  xpectr::strip(side_effects_16417[['messages']]),
  xpectr::strip(character(0)),
  fixed = TRUE)
# Assigning output
output_16417 <- xpectr::suppress_mw(fn(x = 2, y = -1, z = 5))
# Testing class
expect_equal(
  class(output_16417),
  "numeric",
  fixed = TRUE)
# Testing type
expect_type(
  output_16417,
  type = "double")
# Testing values
expect_equal(
  output_16417,
  6,
  tolerance = 1e-4)
# Testing names
expect_equal(
  names(output_16417),
  NULL,
  fixed = TRUE)
# Testing length
expect_equal(
  length(output_16417),
  1L)
# Testing sum of element lengths
expect_equal(
  sum(xpectr::element_lengths(output_16417)),
  1L)

# Testing fn(x = 2, y = NULL, z = 5)
# Changed from baseline: y
xpectr::set_test_seed(42)
# Testing side effects
expect_error(
  xpectr::strip_msg(fn(x = 2, y = NULL, z = 5)),
  xpectr::strip("argument is of length zero"),
  fixed = TRUE)

# Testing fn(x = 2, y = 0, z = 10)
# Changed from baseline: z
xpectr::set_test_seed(42)
# Testing side effects
# Assigning side effects
side_effects_17365 <- xpectr::capture_side_effects(fn(x = 2, y = 0, z = 10))
expect_equal(
  xpectr::strip(side_effects_17365[['warnings']]),
  xpectr::strip(character(0)),
  fixed = TRUE)
expect_equal(
  xpectr::strip(side_effects_17365[['messages']]),
  xpectr::strip("'z' was 10!\n"),
  fixed = TRUE)
# Assigning output
output_17365 <- xpectr::suppress_mw(fn(x = 2, y = 0, z = 10))
# Testing class
expect_equal(
  class(output_17365),
  "numeric",
  fixed = TRUE)
# Testing type
expect_type(
  output_17365,
  type = "double")
# Testing values
expect_equal(
  output_17365,
  12,
  tolerance = 1e-4)
# Testing names
expect_equal(
  names(output_17365),
  NULL,
  fixed = TRUE)
# Testing length
expect_equal(
  length(output_17365),
  1L)
# Testing sum of element lengths
expect_equal(
  sum(xpectr::element_lengths(output_17365)),
  1L)

# Testing fn(x = 2, y = 0, z = NULL)
# Changed from baseline: z
xpectr::set_test_seed(42)
# Testing side effects
expect_error(
  xpectr::strip_msg(fn(x = 2, y = 0, z = NULL)),
  xpectr::strip("argument is of length zero"),
  fixed = TRUE)

## Finished testing 'fn'                                                    ####

wrapStringAddin

The wrapStringAddin splits long strings and wraps them with paste0().