An overview of testing in R

Db8efd836c9a09b71e3d8e1c60d6ea84?s=47 Colin Fay
September 09, 2020

An overview of testing in R

Talk at the AI Testing online meetup

Db8efd836c9a09b71e3d8e1c60d6ea84?s=128

Colin Fay

September 09, 2020
Tweet

Transcript

  1. An overview of testing in R rdev::secure() 2020-09-09 Colin Fay

    - ThinkR Colin FAY (@_ColinFay) - https://rtask.thinkr.fr 1 / 41
  2. $ whoami Colin FAY Data Scientist & R-Hacker at ThinkR,

    a french company focused on Data Science & R. Hyperactive open source developer, lead developer of the {golem} project. https://thinkr.fr https://rtask.thinkr.fr https://twitter.com/_colinfay https://github.com/colinfay https://colinfay.me Colin FAY (@_ColinFay) - https://rtask.thinkr.fr 2 / 41
  3. ThinkR Colin FAY (@_ColinFay) - https://rtask.thinkr.fr 3 / 41

  4. Data Science engineering, focused on R.  Training  Software

    Engineering  R in production  Consulting ThinkR Colin FAY (@_ColinFay) - https://rtask.thinkr.fr 4 / 41
  5. An overview of testing in R Securing youR development Colin

    FAY (@_ColinFay) - https://rtask.thinkr.fr 5 / 41
  6. Today's menu A (non-comprehensive) overview of testing technologies in the

    R world Program Testing in R, why? Testing R packages Benchmark and loadtest Disclaimer I know nothing about AI Colin FAY (@_ColinFay) - https://rtask.thinkr.fr 6 / 41
  7. An overview of testing in R Part 0, The Current

    State of Unit Test Colin FAY (@_ColinFay) - https://rtask.thinkr.fr 7 / 41
  8. The two states of unit test Colin FAY (@_ColinFay) -

    https://rtask.thinkr.fr 8 / 41
  9. The two states of unit test Colin FAY (@_ColinFay) -

    https://rtask.thinkr.fr 9 / 41
  10. An overview of testing in R Part 1, Why Colin

    FAY (@_ColinFay) - https://rtask.thinkr.fr 10 / 41
  11. Everything That's Not Tested Will Eventually Break Colin FAY (@_ColinFay)

    - https://rtask.thinkr.fr 11 / 41
  12. Don't let your users be your unit test Colin FAY

    (@_ColinFay) - https://rtask.thinkr.fr 12 / 41
  13. Control your software load See also: Don't DoS your own

    server Colin FAY (@_ColinFay) - https://rtask.thinkr.fr 13 / 41
  14. Welcome new collaborators Colin FAY (@_ColinFay) - https://rtask.thinkr.fr 14 /

    41
  15. Make your code reproducible Colin FAY (@_ColinFay) - https://rtask.thinkr.fr 15

    / 41
  16. An overview of testing in R Part 2, Testing R

    packages Colin FAY (@_ColinFay) - https://rtask.thinkr.fr 16 / 41
  17. R CMD check devtools::check() rcmdcheck::rcmdcheck() Command line interface & can

    be used on CI/CD Testing R packages Package infrastructure comes with native testing Colin FAY (@_ColinFay) - https://rtask.thinkr.fr 17 / 41
  18. {testthat} test_that("The meaning of life is 42", { expect_equal( meaning_of_life(),

    42 ) }) Testing R packages Colin FAY (@_ColinFay) - https://rtask.thinkr.fr 18 / 41
  19. Testing R packages {testthat} Comes with a lot of native

    expectations library(testthat) ls("package:testthat", pattern = "^expect_") [1] "expect_condition" "expect_cpp_tests_pass" "expect_equal" [4] "expect_equal_to_reference" "expect_equivalent" "expect_error" [7] "expect_failure" "expect_false" "expect_gt" [10] "expect_gte" "expect_identical" "expect_invisible" [13] "expect_is" "expect_known_failure" "expect_known_hash" [16] "expect_known_output" "expect_known_value" "expect_length" [19] "expect_less_than" "expect_lt" "expect_lte" [22] "expect_mapequal" "expect_match" "expect_message" [25] "expect_more_than" "expect_named" "expect_null" [28] "expect_output" "expect_output_file" "expect_reference" [31] "expect_s3_class" "expect_s4_class" "expect_setequal" [34] "expect_silent" "expect_success" "expect_that" [37] "expect_true" "expect_type" "expect_vector" [40] "expect_visible" "expect_warning" Colin FAY (@_ColinFay) - https://rtask.thinkr.fr 19 / 41
  20. Testing R packages {testthat} Custom expectations can be defined expect_file_exists

    <- function(file) { act <- quasi_label(rlang::enquo(file), arg = "fls") act$val <- file.exists(file) expect( isTRUE(act$val), sprintf("File %s doesn't exist.", fls) ) invisible(act$val) } Colin FAY (@_ColinFay) - https://rtask.thinkr.fr 20 / 41
  21. Testing R packages Other tools to define tests for functions

    {testit}, which comes with two functions (assert() and test_pkg()) {testthis}, to enhance unit testing (use_testdata(), test_coverage()) {tinytest}, a lightweight and dependency free unit testing framework Complementary tools {webmockr} and {vcr}, to test for HTTP requests {httptest}: webtesting through {httr} Document {covr} & {covrpage}, to document test coverage {testdown}: {testthat} results as a bookdown Colin FAY (@_ColinFay) - https://rtask.thinkr.fr 21 / 41
  22. CI/CD Github Actions https://github.com/r-lib/actions GitHub Actions for the R language

    - uses: r-lib/actions/setup-r@v1 - uses: r-lib/actions/setup-pandoc@v1 - uses: r-lib/actions/setup-tinytex@master - name: Test run: Rscript -e 'rcmdcheck::rcmdcheck()' Colin FAY (@_ColinFay) - https://rtask.thinkr.fr 22 / 41
  23. CI/CD Travis https://docs.travis-ci.com/user/languages/r/ Set up R specific dependencies, and test

    the package language: r before_install: - tlmgr install index install: - R -e 'devtools::install_deps(dep = T)' script: - R CMD build . - R CMD check *tar.gz Colin FAY (@_ColinFay) - https://rtask.thinkr.fr 23 / 41
  24. CI/CD GitLab Some docker images can be used as a

    base: image: colinfay/r-ci-tidyverse:3.6.0 test: script: - R -e 'remotes::install_local()' - R -e 'devtools::check()' Colin FAY (@_ColinFay) - https://rtask.thinkr.fr 24 / 41
  25. {shiny} specific {shinytests} Test visual regression of your application puppeteer

    Command line tool to mock a web session, in NodeJS {crrry} R tool to drive a {shiny} session Colin FAY (@_ColinFay) - https://rtask.thinkr.fr 25 / 41
  26. Everything is a package Colin FAY (@_ColinFay) - https://rtask.thinkr.fr 26

    / 41
  27. Everything is a package Analysis as a package Applications as

    a package Tools as a package Colin FAY (@_ColinFay) - https://rtask.thinkr.fr 27 / 41
  28. An overview of testing in R Part 3, benchmark and

    load Colin FAY (@_ColinFay) - https://rtask.thinkr.fr 28 / 41
  29. library(profvis) top <- function(){ pause(0.1) lapply(1:10, function(x){ x * 10

    }) middle() } middle <- function(){ pause(0.2); 1e4 * 9 } profvis({ top() }) Benchmark and load Profiling R code with {profvis} Colin FAY (@_ColinFay) - https://rtask.thinkr.fr 29 / 41
  30. Benchmark and load Profiling memory with {profmem} library(profmem) p <-

    profmem({ x <- raw(1000) A <- matrix(rnorm(100), ncol = 10) }) p2 <- subset(p, bytes > 1000) print(p2) Rprofmem memory profiling of: { x <- raw(1000) A <- matrix(rnorm(100), ncol = 10) } Memory allocations: what bytes calls 1 alloc 1048 raw() Colin FAY (@_ColinFay) - https://rtask.thinkr.fr 30 / 41
  31. Benchmark and load Timing with {bench} res <- bench::mark( plus

    = 1+1, sum = sum(1, 1), iterations = 10000 ) res # A tibble: 2 x 6 expression min median `itr/sec` mem_alloc `gc/sec` <bch:expr> <bch:tm> <bch:tm> <dbl> <bch:byt> <dbl> 1 plus 71ns 101ns 4561272. 0B 0 2 sum 222ns 266ns 1733893. 0B 0 Colin FAY (@_ColinFay) - https://rtask.thinkr.fr 31 / 41
  32. Benchmark and load ggplot2::autoplot(res) Colin FAY (@_ColinFay) - https://rtask.thinkr.fr 32

    / 41
  33. Benchmark and load in {shiny} {shinyloadtest} : native R package

    + Cli to record and replay load tests {dockerstats} : get Docker stats inside R {crrry} + {dockerstats} : replay session and watch the Docker stats Colin FAY (@_ColinFay) - https://rtask.thinkr.fr 33 / 41
  34. Wrapper around docker stats Turns the stats from Docker into

    an R dataframe Can be called recursively Focus - {dockerstats} Colin FAY (@_ColinFay) - https://rtask.thinkr.fr 34 / 41
  35. How - {dockerstats} # Launch the container from the terminal

    docker run --name hexmake --rm -p 2811:80 colinfay/hexmake # Call dockerstats, only selecting specific columns dockerstats::dockerstats("hexmake")[, c("Name", "MemUsage", "CPUPerc", "MemPerc", "record_time")] Name MemUsage CPUPerc MemPerc record_time 1 hexmake 137MiB 0.08 1.72 2020-09-09 21:08:48 Colin FAY (@_ColinFay) - https://rtask.thinkr.fr 35 / 41
  36. {dockerstats} Colin FAY (@_ColinFay) - https://rtask.thinkr.fr 36 / 41

  37. An overview of testing in R Conclusion Colin FAY (@_ColinFay)

    - https://rtask.thinkr.fr 37 / 41
  38. To conclude Test often, test soon Everything can, and should

    be scripted Testing is not just about functions, it's also about load and speed R Packages make our life easier Colin FAY (@_ColinFay) - https://rtask.thinkr.fr 38 / 41
  39. devtools.r-lib.org github.com/r-lib/rcmdcheck testthat.r-lib.org github.com/yihui/testit github.com/s-fleck/testthis github.com/markvanderloo/tinytest github.com/ropensci/webmockr github.com/ropensci/vcr covr.r-lib.org/ github.com/yonicd/covrpage

    github.com/ThinkR-open/testdown github.com/nealrichardson/httptest rstudio.github.io/shinytest pptr.dev github.com/ColinFay/crrry rstudio.github.io/profvis github.com/HenrikBengtsson/profmem bench.r-lib.org rstudio.github.io/shinyloadtest github.com/ColinFay/dockerstats Colin FAY (@_ColinFay) - https://rtask.thinkr.fr 39 / 41
  40. engineering-shiny.org Colin FAY (@_ColinFay) - https://rtask.thinkr.fr 40 / 41

  41. Thx! Questions? Colin Fay Online colin@thinkr.fr http://twitter.com/_colinfay http://twitter.com/thinkr_fr https://github.com/ColinFay https://thinkr.fr/

    https://rtask.thinkr.fr/ https://colinfay.me/ Colin FAY (@_ColinFay) - https://rtask.thinkr.fr 41 / 41