Building and maintaining OpenIntro using the R ecosystem (useR 2021)

Building and maintaining OpenIntro using the R ecosystem

OpenIntro's (openintro.org) mission is to make educational products that are free and transparent and that lower barriers to education. The products include textbooks (in print and online), supporting resources for instructors as well as for students. From day one, OpenIntro materials have been built using tools within the R ecosystem. In this talk we will discuss how the OpenIntro project has shaped and grown over the years, our process for developing and publishing open-source textbooks at the high school and college level, and our computing resources such as interactive R tutorials and R packages as well as labs in various languages. We will highlight recent workflows we have developed and lessons learned for converting books from LaTeX to bookdown and give an overview of our project organization and tooling for authoring, collaboration, and maintenance, much of which is built with R, R Markdown, Git, and GitHub. Finally, we will discuss opportunities for getting involved for educators and students contributing to the development of open-source educational resources under the OpenIntro umbrella and beyond.

Transcript

1. building and maintaining
   using the R ecosystem
   mine-cetinkaya-rundel
   cetinkaya.mine@gmail.com
   @minebocek
   🔗 bit.ly/openintro-useR2021
   mine çetinkaya-runde
   l
   
   duke university & rstudio
   

   View Slide

2. OpenIntro’s mission is to make educational
   products that are free, transparent, and
   lower barriers to education.
   mission
   🔗 bit.ly/openintro-useR2021
   

   View Slide

3. 🔗 bit.ly/openintro-useR2021
   openintro.org
   

   View Slide

4. 🔗 bit.ly/openintro-useR2021
   

   View Slide

5. team
   🔗 bit.ly/openintro-useR2021
   

   View Slide

6. books labs tutorials other
   packages
   🔗 bit.ly/openintro-useR2021
   

   View Slide

7. Preliminary Edition
   how it
   
   
   started
   how it’s going
   🔗 bit.ly/openintro-useR2021
   

   View Slide

8. Preliminary Edition First Edition Second Edition
   First Edition
   First Edition
   🔗 bit.ly/openintro-useR2021
   

   View Slide

9. ๏Applications as motivation
   
   
   ๏Real, recent, relatable datasets
   
   
   ๏Emphasis on data exploration, multivariable
   relationships, and statistical reasoning
   
   
   ๏Guided practices and worked examples
   interspersed in text
   
   
   ๏Lots of end of chapter exercises, with solutions
   to odd numbered questions in the back
   pedagogy
   🔗 bit.ly/openintro-useR2021
   

   View Slide

10. license
    🔗 bit.ly/openintro-useR2021
    

    View Slide

11. ๏HTML textbook freely available
    
    
    ๏PDF also freely available —
    distributed through LeanPub with a
    suggested donation (min $0)
    
    
    ๏Paperback available at low cost
    — cost of printing + minimal
    royalty that goes back to
    OpenIntro (US-based nonpro
    fi
    t)
    “business” model
    🔗 bit.ly/openintro-useR2021
    

    View Slide

12. how it started
    ๏Each chapter its own LaTeX
    fi
    le
    
    
    ๏Each
    fi
    gure in a folder with
    associated R code that
    generates it
    +
    how it’s made
    🔗 bit.ly/openintro-useR2021
    

    View Slide

13. how it’s going
    ๏Each chapter its own Rmd
    fi
    le
    
    
    ๏Output to HTML and PDF with the same source code
    (and lots of customization!)
    how it’s made
    🔗 bit.ly/openintro-useR2021
    

    View Slide

14. ๏Challenge: Convert from LaTeX to
    R Markdown
    
    
    ๏Step 1: Pandoc magic!
    conversion
    # Load packages ----------------------------------------------------------------
    
    
    library(fs)
    
    
    library(tidyverse)
    
    
    # Convert from tex to md -------------------------------------------------------
    
    
    tex_to_md <- function(tex_file_name){
    
    
    
    
    # convert
    
    
    md_file_name <- str_replace(tex_file_name, "\\.tex$", ".md")
    
    
    system_call <- paste("pandoc -s" , tex_file_name, "-o", md_file_name)
    
    
    system(system_call)
    
    
    
    
    # delete tex
    
    
    file_delete(tex_file_name)
    
    
    }
    
    
    # Convert from md to rmd -------------------------------------------------------
    
    
    md_to_rmd <- function(md_file_name){
    
    
    
    
    rmd_file_name <- md_file_name %>%
    
    
    str_replace("\\.md$", ".Rmd")
    
    
    
    
    file_move(
    
    
    path = md_file_name,
    
    
    new_path = rmd_file_name
    
    
    )
    
    
    
    
    }
    
    
    # Get tex file names -----------------------------------------------------------
    
    
    tex_file_names <- dir_info(recurse = 2, glob = "*.tex") %>%
    
    
    pull(path)
    
    
    # Convert from .tex to .md -----------------------------------------------------
    
    
    walk(tex_file_names, tex_to_md)
    md_file_name <- str_replace(tex_file_name, "\\.tex$", ".md")
    
    
    system_call <- paste("pandoc -s" , tex_file_name, "-o", md_file_name)
    
    
    system(system_call)
    tex_file_names <- dir_info(recurse = 2, glob = "*.tex") %>%
    
    
    pull(path)
    
    
    walk(tex_file_names, tex_to_md)
    🔗 bit.ly/openintro-useR2021
    

    View Slide

15. conversion
    ๏ stringr and regular expressions for cleaning custom LaTeX! And when in doubt, add more //s!
    tex_file_clean <- tex_file %>%
    
    
    str_replace("\\\\qt\\{(.*?)(\n)?(\\s)*(.*?)\\\\label\\{.*?\\}\\}", "\\\\textbf{\\1 \\4}") %>%
    
    
    str_replace_all("begin\\{parts\\}", "begin{enumerate}\n\\\\def\\\\labelenumii{\\\\alph{enumii}.}") %>%
    
    
    str_replace_all("end\\{parts\\}", "end{enumerate}") %>%
    
    
    str_replace_all("\\\\footfullcite", " \\\\citep") %>%
    
    
    str_replace_all("\\.pdf", ".png")
    ๏ magick::image_convert() to convert PDFs to PNGs
    ๏ fs::file_delete() to remove
    fi
    les no longer needed
    ๏Challenge: Convert from LaTeX to
    R Markdown
    
    
    ๏Step 2: Post-processing
    🔗 bit.ly/openintro-useR2021
    

    View Slide

16. custom blocks
    ๏Challenge: Achieve similar looking custom blocks in HTML and PDF output
    
    
    ๏Solution: Fenced div blocks + Custom blocks chapter of the R Markdown Cookbook
    🔗 bit.ly/openintro-useR2021
    ::: {.guidedpractice data-latex=""}
    
    
    :::
    Rmd
    .guidedpractice {
    
    
    padding: 1em 1em 1em 4em;
    
    
    margin-top: 30px;
    
    
    margin-bottom: 30px;
    
    
    background: #f8f8f8 5px center/3em no-repeat;
    
    
    border-top: 1px solid #569BBD; /* openintro::COL[1,1] */
    
    
    border-bottom: 1px solid #569BBD; /* openintro::COL[1,1] */
    
    
    background-image: url("../images/_icons/guided-practice.png");
    
    
    background-position: 0.5em 1em;
    
    
    }
    
    
    .guidedpractice:before {
    
    
    content: "Guided practice";
    
    
    clear: right;
    
    
    display: block;
    
    
    font-weight: bold;
    
    
    font-variant: small-caps;
    
    
    color: #569BBD;
    
    
    }
    CSS
    \newenvironment{guidedpractice}{
    
    
    \vspace{4mm}
    
    
    \begin{mdframedwithfootGPWE}
    
    
    \begin{minipage}[t]{0.10\textwidth}
    
    
    {$\:$ \\ \setkeys{Gin}{width=2.5em,keepaspectratio}
    \includegraphics{images/_icons/guided-practice.png}}
    
    
    \end{minipage}
    
    
    \hfill
    
    
    \begin{minipage}[t]{0.90\textwidth}
    
    
    \vspace{-2mm}
    
    
    \setlength{\parskip}{1em}
    
    
    \noindent\textbf{\color{oiB}\small\fontfamily{phv}
    \selectfont{\MakeUppercase{Guided Practice}}} $\:$ \\ \\
    
    
    }
    
    
    {\end{minipage}
    
    
    \end{mdframedwithfootGPWE}
    
    
    \vspace{4mm}
    
    
    }
    LaTeX
    

    View Slide

17. plots
    ๏Challenge: A consistent and branded look for plots
    
    
    ๏Solution: De
    fi
    ne theme_openintro() and rede
    fi
    ne ggplot2 defaults
    🔗 bit.ly/openintro-useR2021
    ggplot2::update_geom_defaults("point", list(color = openintro::IMSCOL["blue","full"],
    
    
    fill = openintro::IMSCOL["blue","full"]))
    
    
    ggplot2::update_geom_defaults("bar", list(fill = openintro::IMSCOL["blue","full"],
    
    
    color = "#FFFFFF"))
    
    
    ggplot2::update_geom_defaults("col", list(fill = openintro::IMSCOL["blue","full"],
    
    
    color = "#FFFFFF"))
    
    
    ggplot2::update_geom_defaults("boxplot", list(color = openintro::IMSCOL["blue","full"]))
    
    
    ggplot2::update_geom_defaults("density", list(color = openintro::IMSCOL["blue","full"]))
    
    
    ggplot2::update_geom_defaults("line", list(color = openintro::IMSCOL["gray", "full"]))
    
    
    ggplot2::update_geom_defaults("smooth", list(color = openintro::IMSCOL["gray", "full"]))
    
    
    ggplot2::update_geom_defaults("dotplot", list(color = openintro::IMSCOL["blue","full"],
    
    
    fill = openintro::IMSCOL["blue","full"]))
    

    View Slide

18. ๏Challenge:
    
    
    ๏ Source code for what’s in the printed book should be publicly available
    
    
    ๏ Source code for full solutions should not be publicly available
    
    
    ๏ Question, full solution, and short answer should all live in the same repo
    
    
    ๏Solution:
    
    
    ๏ Separate repo for exercises: 1 folder per exercise, containing 1 Rmd / question, short answer, full solution
    
    
    ๏ Programmatically generate a single Rmd for exercises for each chapter
    map(question_paths, read_lines) %>%
    
    
    map_chr(~ paste(.x, collapse = "\n")) %>%
    
    
    paste(collapse = "\n\n") %>%
    
    
    # fix figure path
    
    
    str_replace_all("images/", glue("exercises/images/")) %>%
    
    
    # write out
    
    
    write_lines(out_file_path)
    questions_odd <- tibble(
    
    
    question_no = 1:length(questions),
    
    
    question_label = questions
    
    
    ) %>%
    
    
    mutate(odd_even = if_else((question_no %% 2) != 0, "odd", "even")) %>%
    
    
    filter(odd_even == "odd") %>%
    
    
    pull(question_label)
    ๏ Programmatically generate a single Rmd for short solutions of odd numbered exercises for each chapter
    exercises
    🔗 bit.ly/openintro-useR2021
    

    View Slide

19. books labs tutorials other
    packages
    🔗 bit.ly/openintro-useR2021
    

    View Slide

20. 🔗 bit.ly/openintro-useR2021
    

    View Slide

21. on: push
    
    
    name: Render Rmd
    
    
    jobs:
    
    
    render:
    
    
    name: Render Rmd
    
    
    runs-on: macOS-latest
    
    
    steps:
    
    
    - uses: actions/checkout@v2
    
    
    - uses: r-lib/actions/setup-r@v1
    
    
    - uses: r-lib/actions/setup-pandoc@v1
    
    
    - uses: Install rmarkdown, remotes, and the local package
    
    
    run: |
    
    
    install.packages("remotes")
    
    
    remotes::install_cran(c("rmarkdown", "tidyverse", "openintro", "infer", "statsr", "GGally", "skimr"))
    
    
    shell: Rscript {0}
    
    
    - name: Render Lab 01
    
    
    run: Rscript -e 'rmarkdown::render("01_intro_to_r/intro_to_r.Rmd")'
    
    
    …
    
    
    - name: Render Lab 09
    
    
    run: Rscript -e 'rmarkdown::render("09_multiple_regression/multiple_regression.Rmd")'
    
    
    - name: Commit results
    
    
    run: |
    
    
    git config --local user.email "actions@github.com"
    
    
    git config --local user.name "GitHub Actions"
    
    
    git commit -a -m 'Re-build Rmd' || echo "No changes to commit"
    
    
    git push origin || echo "No changes to commit"
    ๏Challenge: Maintenance
    
    
    ๏Solution: Use GitHub Actions (borrowed from r-lib/actions) to re-render Rmds with each
    push and PR merge
    🔗 bit.ly/openintro-useR2021
    

    View Slide

22. books labs tutorials other
    packages
    🔗 bit.ly/openintro-useR2021
    

    View Slide

23. ๏Challenge: Keep main text software agnostic, but teach modern statistics!
    
    
    ๏Solution: Supplement with interactive R tutorials developed with learnr
    🔗 bit.ly/openintro-useR2021
    

    View Slide

24. books labs tutorials other
    packages
    🔗 bit.ly/openintro-useR2021
    

    View Slide

25. 🔗 bit.ly/openintro-useR2021
    

    View Slide

26. ๏Challenge: Datasets blow up package size
    
    
    ๏Solution: Break into smaller packages, and bring them
    along automatically
    Depends:
    
    
    R (>= 2.10),
    
    
    airports,
    
    
    cherryblossom,
    
    
    usdata
    +
    🔗 bit.ly/openintro-useR2021
    

    View Slide

27. contributing
    🔗 bit.ly/openintro-useR2021
    

    View Slide

28. ๏Open an issue
    
    
    ๏Make a pull request
    github.com/openintrostat
    🔗 bit.ly/openintro-useR2021
    

    View Slide

29. openintro.org
    ๏Send feedback
    
    
    ๏Report a typo
    🔗 bit.ly/openintro-useR2021
    

    View Slide

30. openintro.org/teachers/get_involved
    ๏Datasets: Data hunter
    
    
    ๏Exams: Exam contributor
    
    
    ๏Exercises:
    
    
    ๏ Exercise contributor
    
    
    ๏ MyOpenMath exercise conversion
    
    
    ๏Labs:
    
    
    ๏ Lab solution developer
    
    
    ๏ New software owner
    
    
    ๏ New lab developer
    
    
    ๏Modules:
    
    
    ๏ Module developer
    
    
    ๏ Module editor
    
    
    ๏Translation
    
    
    ๏Your ideas?
    🔗 bit.ly/openintro-useR2021
    

    View Slide

31. thank you!
    mine-cetinkaya-rundel
    cetinkaya.mine@gmail.com
    @minebocek
    mine çetinkaya-runde
    l
    
    duke university & rstudio
    bit.ly/openintro-useR2021
    

    View Slide