Keynote at the OCaml Workshop 2020 (part of the ACM ICFP 2020 series). The talk describes the advances in the OCaml community around tooling and the processes for the construction of the OCaml Platform.
State of the OCaml Platform 2020 With many contributions from the OCaml community and maintainers. OCaml Workshop @ ICFP August 2020 Anil Madhavapeddy, University of Cambridge Computer Laboratory OCaml Labs
The OCaml Platform makes users productive with the OCaml language This talk covers: Integrated Development Environments Next Steps for the OCaml Platform Plans for 2020-2021
Integrated Development Environments • Most of the OCaml developer experience is spent within an IDE. • Traditionally, OCaml has had good support for Emacs, and "ok" support for Vim. • Editor support is complex due to the number of tools and latency requirements.
Integrated Development Environments Emacs Vim Atom Language Server Protocol (LSP)
Server New OCaml project: github.com/ocaml/ocaml-lsp Integrates pieces of merlin, odoc, and omd into a single binary, so works out of the box. Builds on previous efforts, but in pure OCaml.
• LSP provides a sustainable basis for expanding IDE support for OCaml. • Single binary means that complexities of tooling can be hidden behind it. • We also developed a complete Visual Studio Code plugin: • VSCode is popular, extensible and cross-platform.
• Install the OCaml Platform extension and off you go.
• Supports all the features newcomers expect from a modern IDE. Integrated Development Environments
• Try it out now by installing the "OCaml Platform" plugin in the VSCode Marketplace.
• 100% free and open-source! Integrated Development Environments The plugin will transition to the "ocaml" organisation on GitHub. Next: how do we accept and maintain such packages?
The OCaml Platform ocamlfind packaging 1.8.1 MIT sustain opam-publish packaging 2.0.2 LGPLv2 active opam packaging 2.0.7 LGPLv2 active dune-release packaging 1.4.0 ISC active odoc documenting 1.5.1 ISC incubate ocamlbuild building 0.14.0 LGPLv2 sustain dune building 2.7.0 MIT active merlin editing 3.3.7 MIT active utop editing 2.6.0 BSD3 active ocp-indent editing 1.8.1 LGPLv2 sustain ocamlformat editing 0.15.0 MIT incubate lsp-server editing 1.0.0 ISC incubate mdx testing 1.7.0 ISC incubate bun testing 0.3.3 MIT incubate omp ast 2.0.0 LGPLv2 sustain ppxlib ast 0.16.0 MIT active
• How do we decide what is in scope for the Platform? • Initially, just tools. Libraries will be supported as "transitive dependencies". (e.g. even ppx compiles into a binary driver)
• Principles for acceptance: • Sharing: does this assist with the reuse and publication of OCaml code? • Development: does this improve the OCaml development process, ranging from onboarding new developers to maintaining a legacy codebase? • Evolution: is there a strategy for incremental change and backwards compatibility over several years? • Openness: is the source code liberally licensed, and are there any operational restrictions (such as online services) that the tool depends on? • Structural requirements: • Maintainer resources, establish a community need, demonstrate some adoption. The OCaml Platform: Scope
The OCaml Platform: lifecycle Incubate Active Sustain Deprecate Cornerstone projects in the ecosystem: opam: source-based package management dune: composable build system for OCaml merlin: IDE and type-checker integration
utop: interactive REPL via a CLI
ppxlib: AST manipulation infrastructure
opam-publish: release OCaml packages Usage Guidelines: Are the recommendation for new projects.
Advanced users can continue to use toolchain directly.
The OCaml Platform: lifecycle Incubate Active Sustain Deprecate Filling a gap in the tooling ecosystem: odoc: cross-referenced documentation ocamlformat: mechanical code formatting lsp-server: OCaml backend for IDEs
mdx: automated testing of code fragments
bun: automated fuzz testing in CI
dune-release: easier releases to opam How to get promoted? Establish a migration path and need. odoc: replace all major uses of ocamldoc
ocamlformat: support partial file indentation (ocp-indent)
lsp-server: a release cycle for time and testing
dune-release: integrate with opam-publish, avoid duplication
The OCaml Platform: lifecycle Incubate Active Sustain Deprecate Stable and being maintained best-effort: ocamlfind: compilation unit manager
ocamlbuild: build system (formerly included with compiler) ocp-indent: as-you-type code formatting Sustain projects are best-effort If your projects use them, then it is safe to continue, but there may be better alternatives. ocamlfind: generally no need to use directly
ocamlbuild: recommend a migration to dune, but older projects will still build fine. Newer OCaml options may not be exposed.
ocp-indent: works fine and stable, but steadily supplanted by ocamlformat.
The OCaml Platform: lifecycle Incubate Active Sustain Deprecate Deprecated projects: camlp4: no longer included in compiler or used as a dependency in tools
oasis: no longer supported for modern versions of OCaml Deprecated projects are only deprecated for the Platform In all cases, any community user can step up and take over maintainership. However, the guarantees about being released in sync with the OCaml compiler no longer hold.
camlp4: now maintained by ygrek for the community (thanks!)
oasis: still builds old code (e.g. Mirage libraries) fine, not used in new packages.
The OCaml Platform: innovation Incubate Active Sustain Deprecate Balance between progress and stability: A thought experiment: if an OCaml Platform in 2013 had required ocamlbuild, would dune exist now? So incubated projects with new points of view are vital, but they must progress to maturity before replacing active ones.
Improving syntax and metadata is important. Dev meeting notes regularly recorded where practical: https://github.com/ocaml/opam/wiki
https://github.com/ocaml/dune/wiki
https://github.com/ocaml/odoc/wiki Maintainer guidelines: Active projects need to remain open to contributors.
No active OCaml Platform project shall be restricted due to employer, and shall retain a liberal license.
The OCaml Platform: maintainers odoc documenting Jon Ludlam, Leo White incubate ocamlformat editing Josh Berdine, Guillaume Petiot incubate lsp-server editing Rudi Grinberg incubate mdx testing Guillaume Petiot, Nathan Rebours incubate bun testing Mindy Preston incubate dune-release packaging Nathan Rebours, Thomas Gazagnaire incubate opam packaging Raja Boujbel, Louis Gesbert, David Allsopp active opam-publish packaging Louis Gesbert active dune building Jeremie Dimino, Rudi Grinberg active merlin editing Fred Bour, Thomas Refis active utop editing ZAN DoYe, Jeremie Dimino active ppxlib ast Jeremie Dimino, Nathan Rebours active ocamlfind packaging Gerd Stolpmann sustain ocamlbuild building Gabriel Scherer sustain ocp-indent editing Louis Gesbert sustain omp ast Fred Bour, Jeremie Dimino sustain camlp4 ast ygrek deprecate These maintainers work for a diverse range of organisations (Tarides, Jane Street, OCamlPro, OCaml Labs, Inria, Facebook, OCaml Software Foundation) or act as individual contributors.
ocaml.org services docs.ocaml.org central repository of API docs incubate git.ocaml.org private mirror of repositories incubate inbox.ocaml.org email list archives active ci.ocaml.org continuous integration for opam active discuss.ocaml.org forum for threaded discussion active discord.ocaml.org server for Discord chat bot active www.ocaml.org main website active lists.ocaml.org email list management sustain opam.ocaml.org opam package lists sustain forge.ocaml.org source code hosting deprecate Some services will be deprecated over the next 12 months: lists.ocaml.org is quite a maintenance burden
forge.ocaml.org will finally have DNS entries removed. Some services will improve over next 12 months: ci.ocaml.org will be replaced with an ocurrent.org-based CI, backed by a multiarch computer cluster (see Thomas Leonard's talk)
docs.ocaml.org: odoc is seeing major improvements to support this (see Jon Ludlam's talk) Thanks to Packet, Scaleway, Amazon and Cambridge University for sponsorship
The OCaml Platform Incubate Active Sustain Deprecate What are the next steps? • merge opam.ocaml.org with the main OCaml website.
• add a newsfeed to the OCaml website for platform announcements (from discuss.ocaml.org).
• regular newsletter about Platform developments (like the "multicore monthlies").
• add workflow guides to ocaml.org There are still some major missing development gaps, however: Windows support, and better turnkey installers. Onto the development plans for the next 12 months...
- generate dune-compatible monorepos as opam lock files
- (the tool formerly known as duniverse, now integrated with opam) These are all under development, and examples of how we can experiment with new workflows from within the familiar opam framework
opam 2.2, aka the "Windows release" As soon as opam 2.1.0 is released, the maintainers have decided that we need to focus together on end-to-end Windows support. This will integrate fdopen's superb Cygwin fork. Requires some key client features, which also help other platforms:
• Native shell integration • Do not assume the existence of a unix environment
• Package parameters • Support variable tracking through opam files
• Helps rationalise compiler variants as well (500+ now).
• Layered switches • allow binaries to be made available across switches.
• Build environments • generalise sandboxing to support more toolchains
• could potentially use this to build directly in containers too More details: https://github.com/ocaml/opam/wiki/opam-2.2-slides.pdf
avsm
norimuraz
x_ttyszk
yumayabe
microcms
tarappo
oracle4engineer
lefred
iselegant
hayaosuzuki
takahana
shimap_sampo
yoshidashingo
sonatard
lemiorhan
carmenintech
chriscoyier
roundedbygravity
nonsquared
jcasabona
smashingmag
brettharned
productmarketing
shpigford
paulrobertlloyd