Continuous Integration and TeX with Org-Mode

Presented at the 42nd TEX Users Group Meeting: https://tug.org/tug2021/sched.html

- Associated site post :: https://rgoswami.me/posts/tug-2021-meta

Transcript

1. Continuous Integration and TEX with Org-mode TEX in the cloud by Rohit Goswami,MInstP August 6, 2021
2. Introduction 2/28 Find me here: https://rgoswami.me Who? Rohit Goswami MInstP ▶ Doctoral Researcher in the Jonsson Group, University of Iceland, Faculty of Physical Sciences and Science Institute
3. Big Picture 3/28 TeX is the lingua franca of academic communities Collaborations with TeX revolve around proprietary systems Overleaf Or collaborators require some expertise with TeX
4. Mitigation Mechanisms 4/28 Everyone wants TeX output without writing TeX pandoc, orgmode promise TeX without the pain Cloud build machines are cheap to deploy now Goals A nonexpert TeX workflow which requires no proprietary tools Transparent git and CI setup Expert friendly in terms of templating
5. Writing TEX 5/28 1 \documentclass{article} 2 \author{John Doe} 3 \title{Astounding Discoveries} 4 \begin{document} 5 \maketitle 6 \section{TeX} 7 Hello World 8 \end{document} Not bad Fairly comprehensive Quickly gets out of hand Trivial for all examples which fit on slides
6. Splitting Code 6/28 .cls files Loaded with \documentclass and \usepackage .sty files Style files or packages (including beamer themes) .rc files Control files for build systems (.latexmkrc or Makefile) What CTAN handles typically Popularly managed by texlive distributions Abstracts TeX and LaTeX (styling) away from document writing Great for collaboration
7. Straying Away 7/28 Orgmode 1 #+author: John Doe 2 #+title: Astounding Discoveries 3 * TeX 4 Hello World 1 (org-BACKEND-export-to-FRONT) 2 (org-latex-export-to-latex) Pandoc Markdown 1 # TeX 2 Hello World 1 pandoc -s thing.md -o thing.tex -)metadata title=”Astounding Discoveries” author=”John Doe” ↪ ↪ ↪ Appears more readable and easier to write however…
8. Polluted Outputs 8/28 1 wc -l {base,orgOne,pandocOne}.tex 8 base.tex 15 orgOne.tex 63 pandocOne.tex 86 total Generated files involve template substitution
9. Pandoc Substitution 9/28 Top down approach Fixed locations in a template (e.g. zenYoda) Variables expanded into TeX YAML metadata 1 $for(header-includes)$ 2 $header-includes$ 3 $endfor$ 1 header-includes: 2 - \numberwithin{figure}{section} 3 - \numberwithin{equation}{section}
10. Orgmode Substitution 10/28 Bottom up approach tangle to an output Structure defined per-file #+TITLE: Continuous Integration and TeX with Org-mode #+SUBTITLE: TeX in the cloud #+LATEX_COMPILER: xelatex #+LaTeX_CLASS: beamer #+LaTeX_CLASS_OPTIONS: [unknownkeysallowed,aspectratio=169] #+LATEX_HEADER: \usepackage{biblatex} #+ATTR_LaTeX: :width 0.4\linewidth Not strictly true (preset variables)
11. Conceptual Differences 11/28 org exporter options assume only one output Allows arbitrary emacs-lisp evaluations Sharing configurations can be clunky pandoc shares configuration system for multiple outputs Sane defaults, good templating options Easy to share templates
12. Continuous Integration 12/28 No one likes switching computers to test MacOS, Windows (WSL often), Many Linux distributions There are far too many options nowadays Wercker, Travis CI, Shippable, GitLab CI, Github Actions Mostly transient docker or nix based systems Setup can be annoying without nix TEX Gains Single reproducible source of truth for TeX The CI machine configuration
13. Teaching CI about TEX 13/28 Relying on build machine OS texlive is fragile texliveonfly can get packages “on the fly” Basic TeXLive Profile 1 selected_scheme scheme-basic 2 TEXDIR /tmp/texlive 3 TEXMFCONFIG ~/.texlive/texmf-config 4 TEXMFHOME ~/texmf 5 TEXMFLOCAL /tmp/texlive/texmf-local 6 TEXMFSYSCONFIG /tmp/texlive/texmf-config 7 TEXMFSYSVAR /tmp/texlive/texmf-var 8 TEXMFVAR ~/.texlive/texmf-var 9 option_doc 0 10 option_src 0
14. TexLive CI Script 14/28 1 export PATH=/tmp/texlive/bin/x86_64-linux:$PATH 2 if ! command -v texlua > /dev/null; then 3 wget http:))mirror.ctan.org/systems/texlive/tlnet/install-tl-unx.tar.gz 4 tar -xzf install-tl-unx.tar.gz 5 cd install-tl-20* 6 ./install-tl -)profile=$1 7 cd .) 8 fi 9 tlmgr install luatex scheme-small \ 10 biber \ 11 beamer \ 12 xetex \ 13 pdflatex \ 14 latexmk \ 15 etoolbox \ 16 minted \ 17 texliveonfly 18 tlmgr option -) autobackup 0 19 tlmgr update -)self -)all -)no-auto-install
15. GitHub Actions TeXLive 15/28 1 jobs: 2 deploy: 3 runs-on: ubuntu-latest 4 steps: 5 - uses: actions/checkout@v2.3.4 6 - name: Install package 7 run: | 8 sudo apt-get install -y python-pygments emacs 9 - name: Setup LaTeX 10 run: | 11 export PATH=/tmp/texlive/bin/x86_64-linux:$PATH 12 export PATH=$HOME/texmf/bin:$PATH 13 scripts/getTexLive.sh $(pwd)/scripts/texlive.profile
16. Minimal Lisp for TEX 16/28 Running functions Setting variables 1 (require 'ox-extra) ;) :ignoreheading:ignore: 2 (ox-extras-activate '(ignore-headlines)) 3 (org-babel-tangle) 4 (setq org-latex-pdf-process (list ”latexmk -shell-escape -f -pdfxe %f”)) 5 (setq org-latex-listings 'minted) 6 (setq org-latex-minted-options 7 '((”bgcolor” ”white”) (”breaklines” ”true”) (”linenos” ”true”) (”style” ”tango”))) ↪ 8 (add-hook 'after-save-hook '(lambda () (org-beamer-export-to-latex) t)) ;) Export
17. Org Syntax for TEX 17/28 Source blocks #+begin_src <lang> :exports <code/none/results> :eval <never> +#end_src Direct TeX export #+begin_export <lang> #+end_export
18. Org and Packages 18/28 Effectively generates .cls and .sty files
19. Org and Headers 19/28 In body TeX can be directly written in export blocks #+LATEX_HEADER: can be used to add to document headers
20. Generating Classes 20/28 #+LATEX_CLASS: myclass is populated from org-latex-classes So we need to add to it before use Or use it as part of the single file setup 1 (append-to-list 2 'org-latex-classes 3 '((”tufte-book” 4 ”\)documentclass[a4paper, sfsidenotes, openany, justified]{tufte-book}” 5 (”\)part{%s}” . ”\)part*{%s}”) 6 (”\)chapter{%s}” . ”\)chapter*{%s}”) 7 (”\)section{%s}” . ”\)section*{%s}”) 8 (”utf8” . ”utf8x”) 9 (”\)subsection{%s}” . ”\)subsection*{%s}”))))
21. Replacing Jupyter 21/28 #+PROPERTY: header-args:python :python /home/haozeke/.pyenv/shims/python :session OnePy :results output :exports both :tangle pyCode.py3
22. PDF Output 22/28 Much nicer (and more native) than Jupyter
23. Teaching CI Org-TEX 23/28 1 (require 'package) 2 (setq package-check-signature nil) 3 (add-to-list 'package-archives '(”melpa” . ”https:))melpa.org/packages/”) t) 4 (package-initialize) 5 (unless package-archive-contents (package-refresh-contents)) 6 (package-install 'use-package) 7 (package-install 'org) 8 (dolist (package '(use-package)) 9 (unless (package-installed-p package) 10 (package-install package))) 11 (use-package org-ref 12 :ensure t) 13 (require 'ox-latex) 14 ;) Define an interactive function for easy testing 15 (defun org-beamer-export-to-pdf-directory (files) 16 ”Export all FILES to latex.” 17 (interactive ”Export org files to tex”) 18 ;) Export all org files given on the command line 19 (org-beamer-export-to-pdf-directory argv)
24. GH Actions and Org-TEX 24/28 More completely, see this script With this action 1 - name: Generate TeX 2 run: emacs -q -nl -)script scripts/org2tex.el src/filename.org 3 - name: Build pdf 4 run: | 1 export PATH=/tmp/texlive/bin/x86_64-linux:$PATH 2 export PATH=$HOME/texmf/bin:$PATH 3 cd src/ 4 texliveonfly -c latexmk -a ”-pdfxe -shell-escape -f” wgtqc.tex
25. Omitted Topics 25/28 Caching CI rebuilds can be sped up with caching mechanisms Emacs-Lisp Too much and too irrelevant for TeX in general Advanced Concepts CI configurations and custom emacs setups; a lot more detail here Jupyter and Org Orgmode can be used as a fully fledged multi-language plain text Jupyter replacement for data science
26. Advanced Concepts 26/28 Going beyond single files with :noweb yes Uses named blocks for clarity #+NAME: orgConf ▶ Named blocks are not tangled 1 (eval-after-load 'ox '(require 'ox-koma-letter)) 2 (with-eval-after-load 'ox-latex 3 <)tex_process>) 4 <)common_pkgs>) 5 <)tufte_book>) 6 <)koma_art>) 7 )
27. Conclusions 27/28 orgmode provides a viable alternative syntax for writing TeX Can be used on public clouds without knowing emacs TeX is here to stay Abstracting complexity away from users is good Public cloud usage spares installation issues ▶ Enables git workflows Alternative syntaxes provide more natural usage for novices orgmode facilitates native execution
28. End 28/28 Thank you