Data Structure Adventures for Fun, Profit, and Performance

Data Structure Adventures for Fun, Profit, and Performance

Erlang comes with numerous data structures ranging from built-in lists, tuples, records, and (recently) maps to standard lib offerings such as orddict, dict, gbtrees, sets, orddsets, queue, and array. Likewise, use cases that need a concurrent data structure can use ETS in various ways as a concurrent hash map, concurrent ordered tree, or even a set of concurrent counters.

While these built-in offerings are often adequate, anyone who has tried to build high performance Erlang system often runs into performance challenges with these existing offerings. Often times, these systems move to using external systems or custom native code (via NIFs) to meet the performance demands.

This talk presents on-going work to build a new set of high performance data structures for Erlang, including both single process data structures as well as various concurrent data structures.

The primary goal of this work is to provide data structures that have good performance, are memory-efficient, and play well with the Erlang scheduler and therefore enable more code to be written in Erlang rather than resorting to native code or external systems/libraries.


Joseph Blomstedt

March 27, 2015