Jbuilder presentation at Docker Cambridge

Presentation on jbuilder given at Docker offices in Cambridge.

Jbuilder is a build system for OCaml developed at Jane Street. It solves many common problems faced by OCaml developers in general and makes the whole experience of developing and maintaining OCaml projects smoother.

Transcript

1. Jbuilder A build system for OCaml projects
2. OCaml development Different approaches
3. At Jane Street • Monolithic repository • Continuous release process • Almost no versioning story • Mostly in-house tooling (jenga, iron, hydra, …)
4. Open Source World • Many small repositories maintained by various entities • Stable releases / development repositories • Complex version constraints • oasis, ocamlfind, ocamlbuild, make, opam…
5. Open Source at Jane Street
6. History • Releasing packages for about 5 years • from 1 package to ~100 • Becoming more and more important
7. The public release process (before jbuilder)
8. Step 1: split the repo JANE Core Async Sexplib
9. Step 2: setup repos • LICENSE.txt • _oasis, setup.ml • myocamlbuild.ml, _tags • META • install.ml • opam
10. Complex process • Requires lots of manual adjustments • Code doesn’t always build with ocamlbuild • Hard to debug errors • Just too much tooling and layers for something that is now simple
11. Jenga and Jane Street Rules
12. Jenga • Full featured build system • Scale to huge repositories • Support polling builds • Knows how to talk to Emacs
13. Jane Street setup • Build system admins maintain the Jenga rules (in OCaml) • The rules read jbuild files • Developers write jbuild files
14. jbuild example (library ((name foo) (libraries (core async))))
15. Public release? • Jenga: No • Jbuild: Yes
16. Jbuilder = Jenga Rules - Jenga (almost)
17. Alternatives • Generating Makefiles • Bootstrapping jenga…
18. Jbuilder specific features
19. Automatic discovery • No need to specify where libraries/executables are • Easy to move things around • Multi-project development (imported from jenga)
20. Build contexts • Can build simultaneously against several opam switches/roots • Artefacts in _build/<context>/ • Specific to the rules, not part of the core
21. Cross-compilation • Rules can cross build contexts • Can easily depend on the host version of an executable
22. And all the rest • Generates .install files • Supports repositories defining multiple packages • Generates META files • Supports parallel builds • Portable • Tries to report usable error messages
23. What it doesn’t support • polling builds • communication with Emacs
24. Jbuilder internals
25. Keep rules simple • static set of targets • no need for clever rules when you have List.iter • -> immediately get the whole DAG
26. Build arrow • type rule = (unit, Action.t) Build.t • reusable code • allow to analyse the graph without running commands
27. Example rule (dep_graph >>> Build.dyn_paths (arr (String_map.find name)) >>> Build.run (Dep ctx.ocamlc) [ Dep src; A “-c”; Target dst ])
28. Parallel builds • Mini Lwt/Async like monad • Scheduler only knows how to multiplex processes
29. Future plans
30. Release 1.0.0 • Documentation generation • Versioning story for jbuild files
31. Bridge with Jenga • Write a Jenga plugin that reuses Jbuilder rules
32. OCaml plugins • So that complex rules can be written outside of Jbuilder • Automatic loading of plugins • (x:ctypes …) • Allow plugins inside the workspace
33. The end