Save 37% off PRO during our Black Friday Sale! »

Continuous Integration and TeX with Org-Mode

Continuous Integration and TeX with Org-Mode

Presented at the 42nd TEX Users Group Meeting:

- Associated site post ::


Rohit Goswami

August 07, 2021


  1. Continuous Integration and TEX with Org-mode TEX in the cloud

    by Rohit Goswami,MInstP August 6, 2021
  2. Introduction 2/28 Find me here: Who? Rohit Goswami MInstP

    ▶ Doctoral Researcher in the Jonsson Group, University of Iceland, Faculty of Physical Sciences and Science Institute
  3. Big Picture 3/28 TeX is the lingua franca of academic

    communities Collaborations with TeX revolve around proprietary systems Overleaf Or collaborators require some expertise with TeX
  4. Mitigation Mechanisms 4/28 Everyone wants TeX output without writing TeX

    pandoc, orgmode promise TeX without the pain Cloud build machines are cheap to deploy now Goals A nonexpert TeX workflow which requires no proprietary tools Transparent git and CI setup Expert friendly in terms of templating
  5. Writing TEX 5/28 1 \documentclass{article} 2 \author{John Doe} 3 \title{Astounding

    Discoveries} 4 \begin{document} 5 \maketitle 6 \section{TeX} 7 Hello World 8 \end{document} Not bad Fairly comprehensive Quickly gets out of hand Trivial for all examples which fit on slides
  6. Splitting Code 6/28 .cls files Loaded with \documentclass and \usepackage

    .sty files Style files or packages (including beamer themes) .rc files Control files for build systems (.latexmkrc or Makefile) What CTAN handles typically Popularly managed by texlive distributions Abstracts TeX and LaTeX (styling) away from document writing Great for collaboration
  7. Straying Away 7/28 Orgmode 1 #+author: John Doe 2 #+title:

    Astounding Discoveries 3 * TeX 4 Hello World 1 (org-BACKEND-export-to-FRONT) 2 (org-latex-export-to-latex) Pandoc Markdown 1 # TeX 2 Hello World 1 pandoc -s -o thing.tex -)metadata title=”Astounding Discoveries” author=”John Doe” ↪ ↪ ↪ Appears more readable and easier to write however…
  8. Polluted Outputs 8/28 1 wc -l {base,orgOne,pandocOne}.tex 8 base.tex 15

    orgOne.tex 63 pandocOne.tex 86 total Generated files involve template substitution
  9. Pandoc Substitution 9/28 Top down approach Fixed locations in a

    template (e.g. zenYoda) Variables expanded into TeX YAML metadata 1 $for(header-includes)$ 2 $header-includes$ 3 $endfor$ 1 header-includes: 2 - \numberwithin{figure}{section} 3 - \numberwithin{equation}{section}
  10. Orgmode Substitution 10/28 Bottom up approach tangle to an output

    Structure defined per-file #+TITLE: Continuous Integration and TeX with Org-mode #+SUBTITLE: TeX in the cloud #+LATEX_COMPILER: xelatex #+LaTeX_CLASS: beamer #+LaTeX_CLASS_OPTIONS: [unknownkeysallowed,aspectratio=169] #+LATEX_HEADER: \usepackage{biblatex} #+ATTR_LaTeX: :width 0.4\linewidth Not strictly true (preset variables)
  11. Conceptual Differences 11/28 org exporter options assume only one output

    Allows arbitrary emacs-lisp evaluations Sharing configurations can be clunky pandoc shares configuration system for multiple outputs Sane defaults, good templating options Easy to share templates
  12. Continuous Integration 12/28 No one likes switching computers to test

    MacOS, Windows (WSL often), Many Linux distributions There are far too many options nowadays Wercker, Travis CI, Shippable, GitLab CI, Github Actions Mostly transient docker or nix based systems Setup can be annoying without nix TEX Gains Single reproducible source of truth for TeX The CI machine configuration
  13. Teaching CI about TEX 13/28 Relying on build machine OS

    texlive is fragile texliveonfly can get packages “on the fly” Basic TeXLive Profile 1 selected_scheme scheme-basic 2 TEXDIR /tmp/texlive 3 TEXMFCONFIG ~/.texlive/texmf-config 4 TEXMFHOME ~/texmf 5 TEXMFLOCAL /tmp/texlive/texmf-local 6 TEXMFSYSCONFIG /tmp/texlive/texmf-config 7 TEXMFSYSVAR /tmp/texlive/texmf-var 8 TEXMFVAR ~/.texlive/texmf-var 9 option_doc 0 10 option_src 0
  14. TexLive CI Script 14/28 1 export PATH=/tmp/texlive/bin/x86_64-linux:$PATH 2 if !

    command -v texlua > /dev/null; then 3 wget http:)) 4 tar -xzf install-tl-unx.tar.gz 5 cd install-tl-20* 6 ./install-tl -)profile=$1 7 cd .) 8 fi 9 tlmgr install luatex scheme-small \ 10 biber \ 11 beamer \ 12 xetex \ 13 pdflatex \ 14 latexmk \ 15 etoolbox \ 16 minted \ 17 texliveonfly 18 tlmgr option -) autobackup 0 19 tlmgr update -)self -)all -)no-auto-install
  15. GitHub Actions TeXLive 15/28 1 jobs: 2 deploy: 3 runs-on:

    ubuntu-latest 4 steps: 5 - uses: actions/checkout@v2.3.4 6 - name: Install package 7 run: | 8 sudo apt-get install -y python-pygments emacs 9 - name: Setup LaTeX 10 run: | 11 export PATH=/tmp/texlive/bin/x86_64-linux:$PATH 12 export PATH=$HOME/texmf/bin:$PATH 13 scripts/ $(pwd)/scripts/texlive.profile
  16. Minimal Lisp for TEX 16/28 Running functions Setting variables 1

    (require 'ox-extra) ;) :ignoreheading:ignore: 2 (ox-extras-activate '(ignore-headlines)) 3 (org-babel-tangle) 4 (setq org-latex-pdf-process (list ”latexmk -shell-escape -f -pdfxe %f”)) 5 (setq org-latex-listings 'minted) 6 (setq org-latex-minted-options 7 '((”bgcolor” ”white”) (”breaklines” ”true”) (”linenos” ”true”) (”style” ”tango”))) ↪ 8 (add-hook 'after-save-hook '(lambda () (org-beamer-export-to-latex) t)) ;) Export
  17. Org Syntax for TEX 17/28 Source blocks #+begin_src <lang> :exports

    <code/none/results> :eval <never> +#end_src Direct TeX export #+begin_export <lang> #+end_export
  18. Org and Packages 18/28 Effectively generates .cls and .sty files

  19. Org and Headers 19/28 In body TeX can be directly

    written in export blocks #+LATEX_HEADER: can be used to add to document headers
  20. Generating Classes 20/28 #+LATEX_CLASS: myclass is populated from org-latex-classes So

    we need to add to it before use Or use it as part of the single file setup 1 (append-to-list 2 'org-latex-classes 3 '((”tufte-book” 4 ”\)documentclass[a4paper, sfsidenotes, openany, justified]{tufte-book}” 5 (”\)part{%s}” . ”\)part*{%s}”) 6 (”\)chapter{%s}” . ”\)chapter*{%s}”) 7 (”\)section{%s}” . ”\)section*{%s}”) 8 (”utf8” . ”utf8x”) 9 (”\)subsection{%s}” . ”\)subsection*{%s}”))))
  21. Replacing Jupyter 21/28 #+PROPERTY: header-args:python :python /home/haozeke/.pyenv/shims/python :session OnePy :results

    output :exports both :tangle pyCode.py3
  22. PDF Output 22/28 Much nicer (and more native) than Jupyter

  23. Teaching CI Org-TEX 23/28 1 (require 'package) 2 (setq package-check-signature

    nil) 3 (add-to-list 'package-archives '(”melpa” . ”https:))”) t) 4 (package-initialize) 5 (unless package-archive-contents (package-refresh-contents)) 6 (package-install 'use-package) 7 (package-install 'org) 8 (dolist (package '(use-package)) 9 (unless (package-installed-p package) 10 (package-install package))) 11 (use-package org-ref 12 :ensure t) 13 (require 'ox-latex) 14 ;) Define an interactive function for easy testing 15 (defun org-beamer-export-to-pdf-directory (files) 16 ”Export all FILES to latex.” 17 (interactive ”Export org files to tex”) 18 ;) Export all org files given on the command line 19 (org-beamer-export-to-pdf-directory argv)
  24. GH Actions and Org-TEX 24/28 More completely, see this script

    With this action 1 - name: Generate TeX 2 run: emacs -q -nl -)script scripts/org2tex.el src/ 3 - name: Build pdf 4 run: | 1 export PATH=/tmp/texlive/bin/x86_64-linux:$PATH 2 export PATH=$HOME/texmf/bin:$PATH 3 cd src/ 4 texliveonfly -c latexmk -a ”-pdfxe -shell-escape -f” wgtqc.tex
  25. Omitted Topics 25/28 Caching CI rebuilds can be sped up

    with caching mechanisms Emacs-Lisp Too much and too irrelevant for TeX in general Advanced Concepts CI configurations and custom emacs setups; a lot more detail here Jupyter and Org Orgmode can be used as a fully fledged multi-language plain text Jupyter replacement for data science
  26. Advanced Concepts 26/28 Going beyond single files with :noweb yes

    Uses named blocks for clarity #+NAME: orgConf ▶ Named blocks are not tangled 1 (eval-after-load 'ox '(require 'ox-koma-letter)) 2 (with-eval-after-load 'ox-latex 3 <)tex_process>) 4 <)common_pkgs>) 5 <)tufte_book>) 6 <)koma_art>) 7 )
  27. Conclusions 27/28 orgmode provides a viable alternative syntax for writing

    TeX Can be used on public clouds without knowing emacs TeX is here to stay Abstracting complexity away from users is good Public cloud usage spares installation issues ▶ Enables git workflows Alternative syntaxes provide more natural usage for novices orgmode facilitates native execution
  28. End 28/28 Thank you