State of the
 OCaml Platform 2020 With many contributions from the OCaml community and maintainers. OCaml Workshop @ ICFP August 2020 Anil Madhavapeddy, University of Cambridge Computer Laboratory
 OCaml Labs 

The OCaml Platform makes users productive with the OCaml language This talk covers: Integrated Development Environments Next Steps for the OCaml Platform Plans for 2020-2021 

Integrated Development Environments • Most of the OCaml developer experience is spent within an IDE. • Traditionally, OCaml has had good support for Emacs, and "ok" support for Vim. • Editor support is complex due to the number of tools and latency requirements. 

Integrated Development Environments Emacs Vim Atom Tuareg Vimrc Plugin Merlin OCamlformat Odoc 

Integrated Development Environments Emacs Vim Atom Merlin OCamlformat Odoc Language
 Server Protocol (LSP)
 
 Server 

Integrated Development Environments Emacs Vim Atom Language
 Server Protocol (LSP)
 
 Server New OCaml project: github.com/ocaml/ocaml-lsp Integrates pieces of merlin, odoc, and omd into a single binary, so works out of the box. Builds on previous efforts, but in pure OCaml. 

• LSP provides a sustainable basis for expanding IDE support for OCaml. • Single binary means that complexities of tooling can be hidden behind it. • We also developed a complete Visual Studio Code plugin: • VSCode is popular, extensible and cross-platform. • Install the OCaml Platform extension and off you go. • Supports all the features newcomers expect from a modern IDE. Integrated Development Environments 

Types on hover 

Autocomplete 

Destructing pattern matches 

Project sandboxing 

• Try it out now by installing the "OCaml Platform" plugin in the VSCode Marketplace. • 100% free and open-source! Integrated Development Environments The plugin will transition to the "ocaml" organisation on GitHub. Next: how do we accept and maintain such packages? 

• For the first time, OCaml 4.11.0 was released simultaneously with support for popular tools. • Combined efforts of Florian Angeletti (OCaml release), Kate Deplaix (opam) and David Allsopp (OCaml core), and all the developers of the tools. • See ocaml/opam-repository#6539 • Requires an extended release cycle for OCaml, with a new alpha phase before the more public beta. The OCaml Platform: Synchronous Release 

The OCaml Platform ocamlfind packaging 1.8.1 MIT sustain opam-publish packaging 2.0.2 LGPLv2 active opam packaging 2.0.7 LGPLv2 active dune-release packaging 1.4.0 ISC active odoc documenting 1.5.1 ISC incubate ocamlbuild building 0.14.0 LGPLv2 sustain dune building 2.7.0 MIT active merlin editing 3.3.7 MIT active utop editing 2.6.0 BSD3 active ocp-indent editing 1.8.1 LGPLv2 sustain ocamlformat editing 0.15.0 MIT incubate lsp-server editing 1.0.0 ISC incubate mdx testing 1.7.0 ISC incubate bun testing 0.3.3 MIT incubate omp ast 2.0.0 LGPLv2 sustain ppxlib ast 0.16.0 MIT active 

• Incubate • Projects that fill a gap in the ecosystem, or perform an existing function differently. • Not yet ready for mainstream release and have unreliable backwards compatibility. • Active • The day-to-day workhorse projects, with strong backwards compatibility. • May be adding major features, but not radically changing their workflows. • Sustain • No major new features added, but updated for OCaml releases. • Continue to use these, but be aware that there may be active alternatives. • Can be an extremely important part of the ecosystem and in this mode indefinitely. • Deprecate • Being pushed out of the OCaml distribution and platform. • There will be recommended alternatives, and projects should migrate. The OCaml Platform: Classification 

The OCaml Platform odoc documenting 1.5.1 ISC incubate ocamlformat editing 0.15.0 MIT incubate lsp-server editing 1.0.0 ISC incubate mdx testing 1.7.0 ISC incubate bun testing 0.3.3 MIT incubate dune-release packaging 1.4.0 ISC incubate opam packaging 2.0.7 LGPLv2 active opam-publish packaging 2.0.2 LGPLv2 active dune building 2.7.0 MIT active merlin editing 3.3.7 MIT active utop editing 2.6.0 BSD3 active ppxlib ast 0.16.0 MIT active ocamlfind packaging 1.8.1 MIT sustain ocamlbuild building 0.14.0 LGPLv2 sustain ocp-indent editing 1.8.1 LGPLv2 sustain omp ast 2.0.0 LGPLv2 sustain camlp4 ast 4.11.0 LGPLv2 deprecate oasis build 0.4.11 LGPLv2 deprecate 

• How do we decide what is in scope for the Platform? • Initially, just tools. Libraries will be supported as "transitive dependencies".
 (e.g. even ppx compiles into a binary driver) • Principles for acceptance: • Sharing: does this assist with the reuse and publication of OCaml code? • Development: does this improve the OCaml development process, ranging from onboarding new developers to maintaining a legacy codebase? • Evolution: is there a strategy for incremental change and backwards compatibility over several years? • Openness: is the source code liberally licensed, and are there any operational restrictions (such as online services) that the tool depends on? • Structural requirements: • Maintainer resources, establish a community need, demonstrate some adoption. The OCaml Platform: Scope 

The OCaml Platform: lifecycle Incubate Active Sustain Deprecate 

The OCaml Platform: lifecycle Incubate Active Sustain Deprecate Cornerstone projects in the ecosystem: opam: source-based package management dune: composable build system for OCaml merlin: IDE and type-checker integration utop: interactive REPL via a CLI ppxlib: AST manipulation infrastructure opam-publish: release OCaml packages Usage Guidelines: Are the recommendation for new projects. Advanced users can continue to use toolchain directly. Avoid duplication of functionality. Metadata files are versioned reliably. 

The OCaml Platform: lifecycle Incubate Active Sustain Deprecate Filling a gap in the tooling ecosystem: odoc: cross-referenced documentation ocamlformat: mechanical code formatting lsp-server: OCaml backend for IDEs mdx: automated testing of code fragments bun: automated fuzz testing in CI dune-release: easier releases to opam How to get promoted? Establish a migration path and need.
 odoc: replace all major uses of ocamldoc ocamlformat: support partial file indentation (ocp-indent) lsp-server: a release cycle for time and testing dune-release: integrate with opam-publish, avoid duplication 

The OCaml Platform: lifecycle Incubate Active Sustain Deprecate Stable and being maintained best-effort: ocamlfind: compilation unit manager ocamlbuild: build system (formerly included with compiler) ocp-indent: as-you-type code formatting Sustain projects are best-effort If your projects use them, then it is safe to continue, but there may be better alternatives.
 ocamlfind: generally no need to use directly ocamlbuild: recommend a migration to dune, but older projects will still build fine. Newer OCaml options may not be exposed. ocp-indent: works fine and stable, but steadily supplanted by ocamlformat. 

The OCaml Platform: lifecycle Incubate Active Sustain Deprecate Deprecated projects: camlp4: no longer included in compiler
 or used as a dependency in tools
 
 oasis: no longer supported for modern
 versions of OCaml Deprecated projects are only deprecated for the Platform In all cases, any community user can step up and take over maintainership. However, the guarantees about being released in sync with the OCaml compiler no longer hold. camlp4: now maintained by ygrek for the community (thanks!) oasis: still builds old code (e.g. Mirage libraries) fine, not used in new packages. 

The OCaml Platform: innovation Incubate Active Sustain Deprecate Balance between progress and stability: A thought experiment: if an OCaml Platform
 in 2013 had required ocamlbuild, would dune exist now?
 So incubated projects with new points of view are vital, but they must progress to maturity before replacing active ones. Improving syntax and metadata is important. Dev meeting notes regularly recorded where practical: https://github.com/ocaml/opam/wiki https://github.com/ocaml/dune/wiki https://github.com/ocaml/odoc/wiki Maintainer guidelines: Active projects need to remain open to contributors. No active OCaml Platform project shall be restricted due to employer, and shall retain a liberal license. 

The OCaml Platform: maintainers odoc documenting Jon Ludlam, Leo White incubate ocamlformat editing Josh Berdine, Guillaume Petiot incubate lsp-server editing Rudi Grinberg incubate mdx testing Guillaume Petiot, Nathan Rebours incubate bun testing Mindy Preston incubate dune-release packaging Nathan Rebours, Thomas Gazagnaire incubate opam packaging Raja Boujbel, Louis Gesbert, David Allsopp active opam-publish packaging Louis Gesbert active dune building Jeremie Dimino, Rudi Grinberg active merlin editing Fred Bour, Thomas Refis active utop editing ZAN DoYe, Jeremie Dimino active ppxlib ast Jeremie Dimino, Nathan Rebours active ocamlfind packaging Gerd Stolpmann sustain ocamlbuild building Gabriel Scherer sustain ocp-indent editing Louis Gesbert sustain omp ast Fred Bour, Jeremie Dimino sustain camlp4 ast ygrek deprecate These maintainers work for a diverse range of organisations (Tarides, Jane Street, OCamlPro, OCaml Labs, Inria, Facebook, OCaml Software Foundation) or act as individual contributors. 

ocaml.org services docs.ocaml.org central repository of API docs incubate git.ocaml.org private mirror of repositories incubate inbox.ocaml.org email list archives active ci.ocaml.org continuous integration for opam active discuss.ocaml.org forum for threaded discussion active discord.ocaml.org server for Discord chat bot active www.ocaml.org main website active lists.ocaml.org email list management sustain opam.ocaml.org opam package lists sustain forge.ocaml.org source code hosting deprecate Some services will be deprecated over the next 12 months: lists.ocaml.org is quite a maintenance burden forge.ocaml.org will finally have DNS entries removed. Some services will improve over next 12 months: ci.ocaml.org will be replaced with an ocurrent.org-based CI, backed by a multiarch computer cluster (see Thomas Leonard's talk) docs.ocaml.org: odoc is seeing major improvements to support this (see Jon Ludlam's talk) Thanks to Packet, Scaleway, Amazon and Cambridge University for sponsorship 

The OCaml Platform Incubate Active Sustain Deprecate What are the next steps? • merge opam.ocaml.org with the main OCaml website. • add a newsfeed to the OCaml website for platform announcements (from discuss.ocaml.org). • regular newsletter about Platform developments (like the "multicore monthlies"). • add workflow guides to ocaml.org There are still some major missing development gaps, however: Windows support, and better turnkey installers. Onto the development plans for the next 12 months... 

OCaml Platform
 Development Directions for 2020-2021 

OCaml Community Metrics total number of opam packages 

OCaml Community Metrics number of unique opam packages 

OCaml Community Metrics number of package contributors 

several opam 2 betas to gather feedback and improve workflows Feb 2017: opam 2~beta1 Mar 2017: opam2~beta2 May 2017: opam2~beta3 June 2017: opam 1.2.0 deprecated Aug 2017: opam 2~beta4 Nov 2017: opam 2~beta5 Jan 2018: opam 2~rc1 May 2018: critical build issue in camlp5 May 2018: opam 2~rc2 (sandboxing) July 2018: opam 2~rc3 Aug 2018: opam 2~rc4 Sep 2018: opam 2.0.0 stable released! Sep 2018: opam-repo using 2.0 format. opam 2.0.0 released with v1.2-v2.0 migration and secure sandboxing Jan 2019: opam 2.0.3 Apr 2019: opam 2.0.4 (fix pin-depends) Jul 2019: opam 2.0.5 (improve linting) Jan 2020: opam 2.0.6 (dune caching) Apr 2020: opam 2.0.7 refining and bugfixes 

several opam 2 betas to gather feedback and improve workflows Feb 2017: opam 2~beta1 Mar 2017: opam2~beta2 May 2017: opam2~beta3 June 2017: opam 1.2.0 deprecated Aug 2017: opam 2~beta4 Nov 2017: opam 2~beta5 Jan 2018: opam 2~rc1 May 2018: critical build issue in camlp5 May 2018: opam 2~rc2 (sandboxing) July 2018: opam 2~rc3 Aug 2018: opam 2~rc4 Sep 2018: opam 2.0.0 stable released! Sep 2018: opam-repo using 2.0 format. opam 2.0.0 released with v1.2-v2.0 migration and secure sandboxing Jan 2019: opam 2.0.3 Apr 2019: opam 2.0.4 (fix pin-depends) Jul 2019: opam 2.0.5 (improve linting) Jan 2020: opam 2.0.6 (dune caching) Apr 2020: opam 2.0.7 refining and bugfixes opam 2.1.0~beta1! 

opam 2.1.0 beta1 opam 2.1.0 contains hundreds of bug fixes, and these major new features: - integrated external dependency handling - opam install will intelligently check for apt/rpm/brew depends. - no more need for an external opam depext - opam lock integration - generate fixed version files via "opam lock" - create reproducible switches from these lock files. - opam cli versioning - in scripts, just set OPAM_CLI=2.1 to guarantee 2.1 behaviour, even in future clients. - switch invariants - richer version constraints to specify base packages in a switch, such as upgrading automatically on OCaml patch releases. - significant performance improvements: - faster repository loading - interleaving downloading and installation - more efficient copying during package installation - new "0install solver" optimised for finding from-scratch solutions 

opam 2.1.0 beta1 Also incubating new plugins to support more workflows: - opam compiler - https://github.com/ocaml-opam/opam-compiler - manage OCaml compiler installations much more easily - opam tools - https://github.com/avsm/opam-tools - install Platform tools within a local switch automatically - opam monorepo - https://github.com/ocamllabs/duniverse - generate dune-compatible monorepos as opam lock files - (the tool formerly known as duniverse, now integrated with opam) These are all under development, and examples of how we can experiment with new workflows from within the familiar opam framework 

opam 2.2, aka the "Windows release" As soon as opam 2.1.0 is released, the maintainers have decided that we need to focus together on end-to-end Windows support. This will integrate fdopen's superb Cygwin fork. Requires some key client features, which also help other platforms: • Native shell integration • Do not assume the existence of a unix environment • Package parameters • Support variable tracking through opam files • Helps rationalise compiler variants as well (500+ now). • Layered switches • allow binaries to be made available across switches. • Build environments • generalise sandboxing to support more toolchains • could potentially use this to build directly in containers too More details:
 https://github.com/ocaml/opam/wiki/opam-2.2-slides.pdf 

Le Fin See you on discuss.ocaml.org and clowdr!