14  Test suite

Published

2024-09-19


testthat Workflow:

  • use_testthat(): setup testing infrastructure in your app-package

    • Include edition (i.e., use_testthat(3))

  • use_test(): creates new test files (with test- prefix)

    • Each file under R/ should a corresponding test- file

  • test_active_file(): runs tests in the current open test file

  • test_coverage_active_file(): test coverage for the current open test file

Behavior-driven development functions:

  • describe(): provides context (user specification or feature) for tests and test code

  • it(): used to test functional requirement (i.e., expectation functions).

Testing Shiny applications poses some unique challenges. Shiny functions are written in the context of its reactive model,1 so some standard testing techniques and methods for regular R packages don’t directly apply. This chapter covers setting up the testthat’s infrastructure, keyboard shortcuts for commonly used functions, and running tests in RStudio vs. Positron .

I’ve created the shinypak R package In an effort to make each section accessible and easy to follow:

Install shinypak using pak (or remotes):

# install.packages('pak')
pak::pak('mjfrigaard/shinypak')

Review the chapters in each section:

library(shinypak)
list_apps(regex = 'test')
## # A tibble: 7 × 2
##   branch               last_updated       
##   <chr>                <dttm>             
## 1 10_test-suite        2024-09-23 12:05:04
## 2 11_tests-specs       2024-09-03 22:33:22
## 3 12.1_tests-fixtures  2024-09-14 07:13:41
## 4 12.2_tests-helpers   2024-09-03 22:39:27
## 5 12.3_tests-snapshots 2024-11-06 08:09:57
## 6 13_tests-modules     2024-09-03 23:11:59
## 7 14_tests-system      2024-09-03 23:23:17

Launch an app:

launch(app = "11_tests-specs")

14.1 testthat framework

testthat is the standard package for testing in R packages and one of the most widely used and supported packages on CRAN. Its widespread adoption is due to its ability to simplify the setup, creation, and execution of unit tests.

In our app-package, we’ll use testthat unit tests to ensure the underlying logic (i.e., non-reactive utility functions) behaves correctly. We can extend testthat’s framework for integration tests with Shiny’s testServer() function and system tests with the shinytest2 package. Together, these tools provide a comprehensive testing suite for an app-package.

14.2 Setting up testthat tests

The testthat package has been around for over a decade and thus has undergone various changes that require us to specify the edition we intend to use (currently, it’s the third):2

usethis::use_testthat(3)

Setting up your testing infrastructure with use_testthat() does the following:

  1. In the DESCRIPTION file, testthat (>= 3.0.0) is listed under Suggests

  2. Config/testthat/edition: 3 is also listed in the DESCRIPTION to specify the testthat edition

  3. A new tests/ folder is created, with a testthat/ subfolder

  4. The tests/testthat/testthat.R file is created

We now have a tests/ folder to store our testthat tests.

tests/
  ├── testthat/
  └── testthat.R

2 directories, 1 file
1
Referred to as the ‘test runner,’ because it runs all our tests (do not edit this file).

14.3 Creating unit tests

Launch app with the shinypak package:

launch('10_test-suite')

The standard workflow for writing testthat unit tests consists of the following:

New tests are created with usethis::use_test():

usethis::use_test("scatter_plot")
  • testthat recommends having a corresponding test file in tests/testthat/ (with the test- prefix) for the files in R/.

14.3.1 test- files

Test files: the IDE will automatically create and open the new test file:

✔ Writing 'tests/testthat/test-scatter_plot.R'
• Modify 'tests/testthat/test-scatter_plot.R'

14.3.2 test_that() tests

Each new test file contains a boilerplate test_that() test:

test_that(desc = "multiplication works", code = {
 
})
1
desc is the test context (supplied in "quotes"), and code is the test code (supplied in {curly brackets}).

14.3.3 expect_ations

The expectations typically have two arguments: observed and expected.

expect_equal(
  object = 2 * 2,
  expected = 4
  )
1
A testthat expectation function
2
The output or behavior being tested
3
A predefined output or behavior

The observed object is an artifact of some code we’ve written, and it’s being compared against an expected result.

14.3.4 BDD test functions

At the time of this writing, the 2024.09.0-1 pre-release of Positron was available for testing.

testthat also has two behavior-driven development (BDD) functions for performing tests: describe() and it().

“Use describe() to verify that you implement the right things and use [it()] to ensure you do the things right.” - testthat documentation

describe("Description of feature or specification",
      code = { 
    it("Functionality under test",
        code = {
            expect_equal( 
            object = 2 * 2, 
            expected = 4 
            )
        })
    })
1
describe() the feature or specification
2
Capture it() in a test
3
Write expectations

We’ll cover BDD more in the next chapter, but for now just know that each call to it() behaves like test_that().

14.4 Running tests

Another devtools habit to adopt is regularly writing and running tests. Below we’ll cover writing and running tests in RStudio and Positron .

14.4.1 Keyboard shortcuts

R Packages, 2ed also suggests binding test_active_file() and test_coverage_active_file() to keyboard shortcuts. I highly recommend using a shortcut while developing tests because it will improve your ability to iterate quickly.

devtools function

 

Keyboard shortcut

test()

 

Ctrl/Cmd + Shift + T

test_active_file()

 

Ctrl/Cmd + T

test_coverage_active_file()

 

Ctrl/Cmd + Shift + R


Follow these instructions to create a new keyboard shortcut in RStudio . Positron already includes the devtools::test() shortcut, but the other two (test_active_file() and test_coverage_active_file()) will have to be added manually (see instructions here).

14.4.2 Tests in RStudio

In RStudio , test_active_file() (or Ctrl/Cmd + T) will test the current test file:

test_active_file() or Ctrl/Cmd + T in RStudio

test_active_file() or Ctrl/Cmd + T in RStudio

The output will provide feedback on whether the test passes or fails (and occasionally some encouragement).

When we’ve written multiple test files, we can run all the tests in our app-package using the Build pane or the keyboard shortcut (Ctrl/Cmd + Shift + T)

Run all tests

Run all tests

devtools::test()

devtools::test()

RStudio can be configured to include additional columns, which can be helpful during test development. Below is an example workflow with the R/scatter_plot.R file, it’s accompanying test file, the Build pane, and the Console.

Test workflow in RStudio

Test workflow in RStudio

14.4.3 Tests in Positron

If we click on the Testing sidebar menu item with our test file open, Positron will display the hierarchy of the describe() and it() functions:

Testing icon in Positron sidebar menu

Testing icon in Positron sidebar menu

BDD testing functions in Positron

BDD testing functions in Positron

Positron gives us multiple options for running tests. For running individual tests, we can use the Run Test icons:

Run Test in Testing sidebar menu item

Run Test in Testing sidebar menu item

Run Test in test file

Run Test in test file

When testing the active file in Positron , the results are displayed in the Console:

test_active_file() in Positron

test_active_file() in Positron

To run all the tests in our app-package, we can use the keyboard shortcut or the Run Tests icon:

Run Tests in Testing sidebar menu item

Run Tests in Testing sidebar menu item

Run Tests in test file

Run Tests in test file

The Ctrl/Cmd + Shift + T shortcut will call devtools::test() and display the results in a new R Terminal task:

Recap

RECAP  


testthat setup

  • use_testthat(): sets up testing infrastructure in your app-package

Test files

  • use_test(): creates new test files (with test- prefix). The test file names should generally match the file names be belowR/.

BDD test functions

  • describe(): Feature descriptions and any relevant background information

  • it(): Scenarios and test code with expectations (Then statement = functional requirement).

Running tests

  • test_active_file(): runs tests in the current open test file

  • test_coverage_active_file(): test coverage for the current open test file

Please open an issue on GitHub

  1. The ‘Reactivity - An overview’ article gives an excellent description (and mental module) of reactive programming.↩︎

  2. Read more about changes to the third edition to testthat in R Packages, 2ed↩︎