# to get the mstsap package used in this post:
::load_all()
devtoolslibrary(mstsap)
This is the third post in a series on testing shiny applications. I’ll cover testing shiny module server functions using the testhat
package and shiny’s testServer()
function.
Testing shiny modules
Shiny functions pose a couple of unique challenges for testing. First, we can’t execute shiny server
functions in the console. Second, as Shiny apps become more complex, it’s highly recommended to break up the code base into modules. Modules have additional challenges due to their reactivity being split between interconnected UI and server functions.
The shiny
package doesn’t provide a direct, built-in way to test modules, but the testServer()
function addresses these challenges by testing “reactive interactions” in module server functions. testServer()
also works with testthat
, which means we can structure these ‘reactive interaction’ tests just like other unit tests (for non-application functions).
A Shiny App-Package
testthat
is designed to work within an R package, and the mstsap
branch of sapkgs
has a Mastering Shiny testServer app-package to demonstrate writing tests with testServer()
.
The functions, modules, and applications in mstsap
come from the Shiny Modules chapter of Mastering Shiny.1 If you haven’t read this chapter–start there.
Modules
In a previous post, I used the following definition for unit tests,
“A unit test is a piece of code that invokes a unit of work and checks one specific end result of that unit of work.” - The Art of Unit Testing, 2nd edition
Shiny modules can also be broken into discrete ‘units of work’ with expected ‘end results.’ Modules are ‘a pair of UI and server functions’ designed to compartmentalize input and output IDs into distinct namespaces.2
However, the ‘unit of work’ for a Shiny module might be accomplished with a combination of a module UI and server functions, and a helper/utility function.
mstsap
contains three modules: dataset
, selectVar
, and selectDataVar
. If you’re like more information on these modules, click on the links below.
1) Dataset input module
datasetInput
/datasetServer
: loads and returns data object from the datasets
package (filtered by data frames or matrices)
The objects from datasets
are filtered in the UI module function with a filter
argument that can be used to “limit the options to built-in datasets that are either data frames (filter = is.data.frame
) or matrices (filter = is.matrix
)”. The names
are passed to the choices
in the selectInput()
:
show/hide choices in datasetInput()
<- ls("package:datasets")
names if (!is.null(filter)) {
<- lapply(names, get, "package:datasets")
data <- names[vapply(data, filter, logical(1))]
names }
The datasets
object is returned with get()
(wrapped in reactive()
). See below:
show/hide returned data from datasetServer()
::reactive(
shinyget(input$dataset, "package:datasets")
)
2) selectVar module
selectVarInput
/selectVarServer
: displays a selectInput()
that “allows the user to select variables of specified type from a given reactive dataset.”
The data
argument in selectVarServer()
is the returned value from datasetServer()
. The data()
is used with the filter
argument in the find_vars()
function:
show/hide find_vars()
<- function(data, filter) {
find_vars # I've included the updated version with the 'stopifnot()' checks!
stopifnot(is.data.frame(data))
stopifnot(is.function(filter))
names(data)[vapply(data, filter, logical(1))]
}
The filter argument can be used to return variables by class/type (using is.*
functions like is.numeric()
or is.character()
).
When data()
changes, the output from find_vars()
updates the choices in the variable selectInput()
(i.e., input$var
). See below:
selectVarServer()
also returns the selected variable (input$var
) as a reactive value (var()
)
3) selectDataVar module
selectDataVarUI
/selectDataVarServer
: The selectDataVar
module is from the section titled, “Modules inside of modules”, so here we see the dataset
and selectVar
modules placed inside the selectDataVar
module (each with a new namespace (NS()
)).
Standalone App Functions
mstsap
contains three standalone functions for running each set of module functions.
I’ve made a small change to each standalone app function–each app has a call to reactiveValuesToList()
that displays in the UI.
print reactive values
::verbatimTextOutput("vals")
shiny
$vals <- shiny::renderPrint({
output<- shiny::reactiveValuesToList(input,
x all.names = TRUE)
print(x)
})
datasetApp
datasetApp()
contains a call to the dataset
module, and includes a tableOutput()
to render the selected data object:
When datasetApp()
is run, the app displays the dataset object in the tableOutput()
, and the verbatimTextOutput()
renders the reactive values as a text:
The output above shows what NS()
does in the dataset
module–it appends the module id
argument to the inputId
(which is why we see dataset-dataset
).
dataset-
: the module iddataset-dataset
theinputId
from theselectInput()
selectVarApp
selectVarApp()
includes both dataset
and selectVar
modules, but instead of rendering the output in a table, the UI renders the variable output in a verbatimTextOutput()
.
Note that selectVarApp()
contains namespaces for two modules:
"data"
: the namespace for thedatasetnput()
anddatasetServer()
modules, inheriting thefilter
argument and creating the data object"var"
: theselectVar
modules are linked with the"var"
id.selectVarServer()
uses thedata
object created bydatasetServer()
(and also inherits thefilter
argument).
These namespaced IDs are rendered below with reactiveValuesToList()
:
There’s a lot happening in selectVarApp()
, so I’ve created the figure below to display the code for the modules with their displayed outputs:
As we can see, the data
output from the dataset
module is used to generate the vars()
reactive for the verbatimTextOutput()
in selectVarApp()
. Note that both dataset
and selectVar
modules don’t contain any output functions–these have been provided in the UI for both datasetApp()
and selectVarApp()
.
selectDataVarApp
The final app in mstsap
is selectDataVarApp()
. Here the inputs from dataset
and selectVar
have been moved into the sidebarPanel()
, and the output is rendered in the mainPanel()
.
The reactive values here show how the ‘Modules inside of modules’ work–by adding the additional call to NS()
in the datasetInput()
and selectVarInput()
functions within selectDataVarUI()
and selectDataVarServer()
, an additional namespace is appended to the reactive values (input$dataset
and input$var
):
Below is a figure that displays the contents of the selectDataVar
modules (I’ve removed the tagList()
and moduleServer()
for simplicity), the selectDataVarApp()
, and the rendered outputs:
selectDataVarApp
schema
dataset
and selectVar
modules inside selectDataVar
module with rendered outputs
testServer()
Module server functions can be tested the same way as a traditional shiny server
function, as long as you provide the inputs and verify the correct outputs. Below I’ll cover some general advice on module server tests (and the arguments in testServer()
).
What should I test?
The best bit of advice I’ve found helpful when writing tests comes from R Packages,
“focus your time on code that you’re not sure about, is fragile, or has complicated interdependencies”
The quote isn’t in reference to testing modules or Shiny application functions, but I’ve found it’s easy to fall into the trap of trying to test everything when a targeted approach is more efficient (and equally valid).
The items below have been compiled from Mastering Shiny, R Packages, and Engineering Production-Grade Shiny Apps:
- Do the inputs/outputs behave as expected?
- These tests verify the module server function
inputId
s andoutputId
s are properly namespaced and accessible
- These tests verify the module server function
- Does the module contain the expected reactive values/objects?
- Tests should verify it’s reactivity–module server functions will automatically recompute the outputs when it’s inputs change, so tests should verify changes to inputs produce the expected behaviors and outputs. This includes any returned values from the module (and any additional function arguments).
- Are the calculations correct?
- If the module server function performs calculations or data manipulations, the tests should verify the module produces the correct result (ideally for a variety of inputs and edge cases).
- How are errors handled in the module?
- What errors are displayed from the module? Tests should simulate scenarios that can test if the module: 1) returns errors that are informative, 2) fails silently (when appropriate), or 3) falls back to the correct default behavior.
The first test I’ll perform is for datasetServer()
, the module used to return a data object from the datasets
package.
Arguments
testServer()
has the following arguments:
app
can be a module server function (i.e.,datasetServer
), or anyshiny.appobj
expr
is where I’ll add thetestthat
expectations and other test codeargs
is alist()
I can use to include any module server function arguments
Testing inputs
I’ll start by testing if the initial input value (input$dataset
) in datasetServer()
is set to NULL
. The module server function is the first argument in testServer()
:
testServer(app = datasetServer, expr = {
expect_equal(input$dataset, NULL)
cat("\ndatasetServer: dataset$input is NULL", "\n")
})
Add custom messages with
cat()
and theinputId
we’re testing. Then load, document, and install the packagedevtools::load_all() ℹ Loading mstsap devtools::document() ℹ Updating mstsap documentation ℹ Loading mstsap Restarting R session... library(mstsap)
- And run the test with
testthat::test_file()
:
test_file("tests/testthat/test-datasetServer.R")
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 1 ] datasetServer: dataset$input is NULL
- And run the test with
Setting test inputs
testServer()
allows us to mimic changing application (or module) inputId
s with session$setInputs()
like so:
$setInputs(inputId = "value") session
I’ll demonstrate with a test for input$dataset
in datasetServer()
:
$setInputs(dataset = "faithful")
sessionexpect_equal(
object = input$dataset,
expected = "faithful")
test_cmt("datasetServer", "dataset$input")
Returned values
Any returned values from module server functions can be accessed in testServer()
with session$returned()
. I’ll verify input$dataset
returns an object from datasetServer()
by testing the class of session$returned()
:
show/hide test with session$returned()
$setInputs(dataset = "airquality")
sessionexpect_equal(
object = class(session$returned()),
expected = "data.frame")
test_cmt("datasetServer", "class(session$returned())")
$setInputs(dataset = "WorldPhones")
sessionexpect_true(
object = is.matrix(session$returned()))
test_cmt("datasetServer", "is.matrix(session$returned())")
Note that both methods above can be used to check the class of the returned object.
I can also use the
typeof(datasets::mtcars)
for a direct comparison:show/hide test with session$returned()
$setInputs(dataset = "mtcars") sessionexpect_equal( # app value... object = typeof(session$returned()), # ...compared to actual output expected = typeof(datasets::mtcars)) test_cmt("datasetServer", "typeof(session$returned())")
Server function arguments
If the module server function has additional arguments beyond id
, then it has additional functionality to verify with unit tests. To test additional module server arguments, pass these to testServer(args = list())
. The args
list should include named arguments from the module server function, i.e., list(param1 = "value1", param2 = "value2")
.
For example, selectVarServer()
has data
and filter
arguments:
data
is the returned reactive object fromdatasetServer()
filter
is the function passed to thefind_vars()
utility function
dataset()
-> selectVar()
datasetServer()
and passed to selectVarServer()
Below is a test for selectVarServer()
using args
to verify the reactive data()
is datasets::mtcars
:
testServer(selectVarServer,
args = list(data = mtcars,
filter = is.numeric), expr = {
expect_true(
object = is.reactive(data))
test_cmt("selectVarServer", "is.reactive(data())")
})
But this fails with the following error:
test_file("tests/testthat/test-selectVarServer.R")
[ FAIL 1 | WARN 0 | SKIP 0 | PASS 0 ]
── Error (test-selectVarServer.R:1:1): (code run outside of `test_that()`) ─── Error in `(function (id, data, filter = is.numeric)
What happened?
I’ve included this example because it’s not in the testServer()
documentation, and it’s common to pass values between modules (see here in Engineering Production-Grade Shiny Apps and here in Mastering Shiny)
Testing module communication
The error message above tells me the issue is originating from the stopifnot()
calls in selectVarServer()
.
I’ll stop a moment here to address what’s happening in each module:
The
datasetServer()
returns the results ofinput$dataset
as a reactive (data()
)data()
entersselectVarServer()
in thedata
argumentInside
selectVarServer()
, twostopifnot()
functions evaluate the reactivity ofdata
andfilter
withshiny::is.reactive()
In datasetServer()
, the return object is wrapped in the reactive()
function, so the items args = list()
also need to be wrapped in reactive()
.
I’ll re-write the test above to a more basic test using is.reactive()
:
testServer(selectVarServer,
args = list(data = reactive(mtcars),
filter = is.numeric), expr = {
expect_true(
object = is.reactive(data()))
test_cmt("selectVarServer", "is.reactive(data())")
})
test_file("tests/testthat/test-selectVarServer.R")
[ FAIL 1 | WARN 0 | SKIP 0 | PASS 0 ]
selectVarServer: is.reactive(data())
── Failure (test-selectVarServer.R:1:1): (code run outside of `test_that()`) ───
is.reactive(data()) is not TRUE
`actual`: FALSE `expected`: TRUE
Another failure???
The results of this test might seem confusing given my advice to wrap the args
list in reactive()
, but some reading of the x
argument in is.reactive()
will clear up the error:
For
is.reactive()
, an object to test. Forreactive()
, an expression.
Removing the parentheses from data()
will result in the proper test results:
testServer(selectVarServer,
args = list(data = reactive(mtcars),
filter = is.numeric), expr = {
expect_true(
object = is.reactive(data))
test_cmt("selectVarServer", "is.reactive(data())")
})
test_file("tests/testthat/test-selectVarServer.R")
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 1 ] selectVarServer: is.reactive(data())
Now that I have a reactive data()
input, I can explore how this value is used inside selectVarServer()
. To update input$var
, the data()
input is passed to find_vars()
(a function that uses a filter
argument “used to select which variables to list”). See the example below:
find_vars(
data = chickwts,
filter = is.factor)
#> [1] "feed"
I’ll write an expectation that captures the behavior of find_vars()
in selectVarServer()
:
testServer(selectVarServer,
args = list(data = reactive(chickwts),
filter = is.numeric), expr = {
expect_equal(
object = find_vars(data(), is.factor),
expected = "feed")
test_cmt("selectVarServer", "find_vars()")
})
The results are below:
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 2 ] selectVarServer: find_vars()
To verify that the returned object from selectVarServer()
is the selected column, I’ll need to simulate the application behavior in the tests:
Create a reactive
data()
input inselectVarServer()
:setting args = list()
testServer(selectVarServer, args = list(data = reactive(chickwts), filter = is.numeric), expr = { # include expectations below... })
Set the
input$var
and verify theinput$var
:verify input$var
$setInputs(var = "weight") sessionexpect_equal(object = input$var, expected = "weight") test_cmt("selectVarServer", "input$var")
Set the
input$var
and verify thesession$returned()
verify session$returned()
$setInputs(var = "feed") sessionexpect_equal(object = session$returned(), expected = datasets::chickwts[["feed"]]) test_cmt("selectVarServer", "session$returned()")
The results from these tests are below:
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 3 ]
selectVarServer: input$var
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 4 ] selectVarServer: session$returned()
Module outputs
Rendered outputs can be accessed in testServer()
just like inputs (i.e., with output$outputId
). But the modules in mstsap
don’t have outputs–these are included in the standalone app functions (datasetApp()
, selectVarApp()
, and selectDaraVarApp()
).
Fortunately, app functions can also be passed to the app
argument of testServer()
. I’ll use datasetApp()
to demonstrate.
<- datasetApp()
ds_app testServer(ds_app,
expr = {
})
Testing a standalone app function is similar to testing a module server function, but with a few minor differences. First, the output from the standalone app function is assigned to an object (ds_app
), then placed in the app
argument.
To use session$setInputs()
need to include the namespace for the inputId
. The output from reactiveValuesToList()
in datasetApp()
shows me how to access the inputId
in the datasetServer()
module (i.e., input$`dataset-dataset`
):
<- datasetApp()
ds_app testServer(ds_app,
expr = {
$setInputs(
session`dataset-dataset` = "chickwts")
})
Output testing strategy
Testing outputs with testServer()
is different than testing outputs in regular unit tests, because Shiny outputs are executed in the server, but then rendered as HTML in the UI. testServer()
outlines a testing strategy for complex outputs:
*The goal for your tests should be to ask “is the code that I wrote producing the plot I want?” There are two components to that question:
- Does the plot generate without producing an error?
- Is the plot visually correct?
testServer
is great for assessing the first component here. By merely referencingoutput$plot
in your test, you’ll confirm that the plot was generated without an error.
If we replace plot with table in the advice above, the tests for datasetApp()
should confirm output$data
is generated without producing an error.
Instead of writing an expectation, we’ll use cat()
to display the contents of output$data
after setting the `dataset-dataset`
input:
<- datasetApp()
ds_app testServer(ds_app, expr = {
$setInputs(`dataset-dataset` = "chickwts")
sessioncat("\n\toutput$data:\n", output$data, "\n")
})
The results from the test is below:
test_file("tests/testthat/test-datasetApp.R")
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 0 ] output$data:
<table class = 'table shiny-table table- spacing-s' style = 'width:auto;'>
<thead>
<tr>
<th style='text-align: right;'> weight </th>
<th style='text-align: left;'> feed </th>
</tr>
</thead>
<tbody>
<tr> <td align="right"> 179.00 </td> <td> horsebean </td> </tr>
<tr> <td align="right"> 160.00 </td> <td> horsebean </td> </tr>
<tr> <td align="right"> 136.00 </td> <td> horsebean </td> </tr>
<tr> <td align="right"> 227.00 </td> <td> horsebean </td> </tr>
<tr> <td align="right"> 217.00 </td> <td> horsebean </td> </tr>
<tr> <td align="right"> 168.00 </td> <td> horsebean </td> </tr>
</tbody>
</table>
The output is the HTML used to render the table in the UI. This doesn’t add a passing test, but it confirms that the table is being generated from the data()
reactive.
The tests for datasetApp()
will confirm the inputId
, and verify the class
and names
of the data()
reactive (which will be passed to the renderTable()
function):
expect_equal(
object = input$`dataset-dataset`,
expected = "chickwts")
test_cmt("datasetApp", "input$`dataset-dataset`")
expect_true(
object = is.data.frame(data()))
test_cmt("datasetApp", "is.data.frame(data())")
expect_equal(
object = names(data()),
expected = names(datasets::chickwts))
test_cmt("datasetApp", "names(data())")
I can include a test for the class
of output$data
, but note that this is a character output:
expect_equal(
object = class(output$data),
expected = "character")
test_cmt("datasetApp", "class(output$data)")
The results from test_file()
are below:
test_file("tests/testthat/test-datasetApp.R")
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 1 ]
datasetApp: input$`dataset-dataset`
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 2 ]
datasetApp: is.data.frame(data())
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 3 ]
datasetApp: names(data())
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 4 ] datasetApp: class(output$data)
The same method can be used to test the selectVarApp()
, but note this app requires passing both inputId
s to session$setInputs()
:
show/hide selectVarApp() tests
<- selectVarApp()
sv_app testServer(app = sv_app, expr = {
$setInputs(`var-var` = "Ozone",
session`data-dataset` = "airquality")
# confirm contents of output$out
cat("\n\toutput$out:\n", output$out, "\n")
# confirm var is reactive
expect_true(object = is.reactive(var))
# confirm var input
expect_equal(
object = input$`var-var`,
expected = "Ozone")
# confirm data is reactive
expect_true(object = is.reactive(data))
# confirm data() is a data.frame
expect_true(
object = is.data.frame(data()))
# confirm 'data' can be subsetted with 'var'
expect_equal(
object = data()[[input$`var-var`]],
expected = airquality[["Ozone"]])
})
Testing nested modules
I highly recommend viewing the output of reactiveValuesToList()
if your application has nested modules. It’s easy to lose track of ids if they span multiple layers.
We know selectDataVarApp()
contains ‘modules inside other modules’, and these layers are reflected in the namespaces:
To access the inputId
s in the nested modules, we need to pass the full ‘appended’ namespace:
<- selectDataVarApp()
dv_app testServer(app = dv_app, expr = {
$setInputs(`var-var-var` = "Ozone",
session`var-data-dataset` = "airquality")
})
After setting the inputs, I can confirm the contents of output$out
<- selectDataVarApp()
dv_app testServer(app = dv_app, expr = {
$setInputs(`var-var-var` = "Ozone",
session`var-data-dataset` = "airquality")
cat("\n\toutput$out:\n", output$out, "\n")
})
test_file("tests/testthat/test-selectDataVarApp.R")
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 0 ]
output$out:
[1] 41 36 12 18 NA 28 23 19 8 NA 7 16 11 14 18
[16] 14 34 6 30 11 1 11 4 32 NA NA NA 23 45 115
[31] 37 NA NA NA NA NA NA 29 NA 71 39 NA NA 23 NA
[46] NA 21 37 20 12 13 NA NA NA NA NA NA NA NA NA
[61] NA 135 49 32 NA 64 40 77 97 97 85 NA 10 27 NA
[76] 7 48 35 61 79 63 16 NA NA 80 108 20 52 82 50
[91] 64 59 39 9 16 78 35 66 122 89 110 NA NA 44 28
[106] 65 NA 22 59 23 31 44 21 9 NA 45 168 73 NA 76
[121] 118 84 85 96 78 73 91 47 32 20 23 21 24 44 21
[136] 28 9 13 46 18 13 24 16 13 23 36 7 14 30 NA [151] 14 18 20
After confirming output$out
, I’ll test the inputs:
expect_equal(
object = input$`var-var-var`,
expected = "Ozone")
test_cmt("selectDataVarApp", "input$`var-var-var`")
expect_equal(
object = input$`var-data-dataset`,
expected = "airquality")
test_cmt("selectDataVarApp", "input$`var-data-dataset`")
test_file("tests/testthat/test-selectDataVarApp.R")
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 1 ]
selectDataVarApp: input$`var-var-var`
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 2 ] selectDataVarApp: input$`var-data-dataset`
I can also verify the contents of the reactive var()
inside the test:
expect_true(object = is.reactive(var))
test_cmt("selectDataVarApp", "is.reactive(var)")
cat("\n\tvar:\n", var(), "\n")
test_file("tests/testthat/test-selectDataVarApp.R")
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 3 ]
selectDataVarApp: is.reactive(var)
var:
41 36 12 18 NA 28 23 19 8 NA 7 16 11 14 18 14 34 6 30 11 1 11 4 32 NA NA NA 23
45 115 37 NA NA NA NA NA NA 29 NA 71 39 NA NA 23 NA NA 21 37 20 12 13 NA NA NA
NA NA NA NA NA NA NA 135 49 32 NA 64 40 77 97 97 85 NA 10 27 NA 7 48 35 61 79
63 16 NA NA 80 108 20 52 82 50 64 59 39 9 16 78 35 66 122 89 110 NA NA 44 28
65 NA 22 59 23 31 44 21 9 NA 45 168 73 NA 76 118 84 85 96 78 73 91 47 32 20 23 21 24 44 21 28 9 13 46 18 13 24 16 13 23 36 7 14 30 NA 14 18 20
Recap
This post has shown how shiny’s testServer()
function allows you to isolate and test module server functions, which makes it easier to ensure that your server
function behaves as expected (and locate and fix bugs).
I hope you have a better understanding of how you can use testServer()
to test a modules inputs/outputs, reactivity, calculations, and errors.
In the next post I’ll cover performing integration tests with shinytest2
!