Data Structure Adventures for Fun, Profit, and Performance

Erlang comes with numerous data structures ranging from built-in lists, tuples, records, and (recently) maps to standard lib offerings such as orddict, dict, gbtrees, sets, orddsets, queue, and array. Likewise, use cases that need a concurrent data structure can use ETS in various ways as a concurrent hash map, concurrent ordered tree, or even a set of concurrent counters.

While these built-in offerings are often adequate, anyone who has tried to build high performance Erlang system often runs into performance challenges with these existing offerings. Often times, these systems move to using external systems or custom native code (via NIFs) to meet the performance demands.

This talk presents on-going work to build a new set of high performance data structures for Erlang, including both single process data structures as well as various concurrent data structures.

The primary goal of this work is to provide data structures that have good performance, are memory-efficient, and play well with the Erlang scheduler and therefore enable more code to be written in Erlang rather than resorting to native code or external systems/libraries.

Transcript

1. Erlang Factory 2015 Joe Blomstedt Basho Technologies Data Structure Adventures Monday, March 30, 15
2. 2 Erlang is highly productive language Monday, March 30, 15
3. 3 Scalable Monday, March 30, 15
4. 4 Distributed Monday, March 30, 15
5. 5 Fault Tolerant Monday, March 30, 15
6. 6 Performance envy Monday, March 30, 15
7. 7 Solution Write a NIF! Monday, March 30, 15
8. 8 Monday, March 30, 15
9. 9 Data structures Dispatch/protection Statistics Monday, March 30, 15
10. Data Structures 10 Monday, March 30, 15
11. 11 orddict dict gb_trees Monday, March 30, 15
12. 12 1000 10k 100k 1mm 10mm orddict 25 2770 -- -- -- dict 2 21 315 16485 -- gb_trees 3 44 577 8095 -- Monday, March 30, 15
13. 13 1000 10k 100k 1mm 10mm bt 4 59 708 8894 -- dict 2 21 315 16485 -- gb_trees 3 44 577 8095 -- Monday, March 30, 15
14. 14 Immutable Shared structure Monday, March 30, 15
15. 15 D1 = dict:new(), D2 = dict:store(1, 10, D1), D3 = dict:store(1, 15, D2), 10 = dict:fetch(1, D2), 15 = dict:fetch(1, D3). Monday, March 30, 15
16. 16 1000 10k 100k 1mm 10mm ets 8 8 49 497 5296 dict 2 21 315 16485 -- gb_trees 3 44 577 8095 -- Monday, March 30, 15
17. 17 Concurrent Fast Off Heap Monday, March 30, 15
18. 18 Immutable Shared Structure Concurrent Fast Off Heap Monday, March 30, 15
19. CoW B-Tree 19 Monday, March 30, 15
20. 20 (1,10),(5,50),(9,90) root Monday, March 30, 15
21. 21 (1,10),(2,20) (1,A),(5,B) (5,50),(9,90) root A B Monday, March 30, 15
22. 22 (1,10),(2,20),(3,30) (1,A),(5,B) (5,50),(9,90) root A B Monday, March 30, 15
23. 23 (1,10),(2,20),(3,30) (1,A),(5,B) (5,50),(6,60),(9,90) root A B Monday, March 30, 15
24. 24 (1,10),(2,20) (1,A),(3,C),(5,B) (3,30),(4,40) (5,50),(6,60),(9,90) root A C B Monday, March 30, 15
25. 25 (1,10),(2,20) (1,A),(3,C) (3,30),(4,40) (5,50),(6,60) (7,70),(9,90) (1,E),(5,F) (5,B),(7,D) root E F A C B D Monday, March 30, 15
26. 26 [{1, [{1,[{1,10},{2,20}]}, {3,[{3,30},{4,40}]}]}, {5, [{5,[{5,50},{6,60}]}, {7,[{7,70},{9,90}]}]}] Monday, March 30, 15
27. 27 find(Key, #tree{height=Height, root=Root}) -> find(1, Height, Key, Root). find(Depth, Height, Key, Node) when Depth == Height -> %% leaf node orddict:find(Key, Node); find(Depth, Height, Key, Node) -> %% inner node {_, Child} = search(Node, Key), find(Depth + 1, Height, Key, Child). Monday, March 30, 15
28. 28 1000 10k 100k 1mm 10mm bt 4 59 708 8894 -- dict 2 21 315 16485 -- gb_trees 3 44 577 8095 -- Monday, March 30, 15
29. 29 Immutable Shared Structure Concurrent Fast Off Heap Monday, March 30, 15
30. 30 Rewrite as a NIF Monday, March 30, 15
31. 31 Allocation Snapshots Reclamation (SMR) Atomics / Ordering Monday, March 30, 15
32. 32 Epoch Reclamation Monday, March 30, 15
33. 33 Grace Period Detection Monday, March 30, 15
34. 34 read T0 T1 T2 logical delete synchronize delete read Monday, March 30, 15
35. 35 0 T0 T1 T2 logical delete synchronize delete 1 2 3 0 1 2 3 1 1 2 3 Monday, March 30, 15
36. 36 std::atomic<Root*> root; void reader() { while(true) { epoch_begin(); Root *r = root; do_something(r); epoch_end(); } } void writer() { while(true) { Root *old_root = root; Root *new_root = update(root); root = new_root; epoch_synchronize(); delete root; } } Monday, March 30, 15
37. 37 std::atomic<Root*> root; uint64_t epoch; void reader() { while(true) { epoch_begin(); Root *r = root; snapshot(r.epoch); epoch_end(); do_something(r); } } void writer() { while(true) { epoch++; Root *old_root = root; Root *new_root = update(root); new_root->birth = epoch; old_root->death = epoch; root = new_root; garbage.push_back(old_root); collect(); } } Monday, March 30, 15
38. 38 void collect() { epoch_synchronize(); for(auto *item : garbage) { bool live = false; for(auto epoch : snapshots) { if((epoch >= item->birth) && (epoch < item->death)) { live = true; break; } } if(live) keep.push_back(item); else delete item; } garbage.swap(keep); } Monday, March 30, 15
39. 39 Flat Combining Monday, March 30, 15
40. 40 (request) ready? (request) ready? (request) ready? locked? requests[] Monday, March 30, 15
41. 41 (request) ready? (request) ready? (request) ready? locked? requests[] Monday, March 30, 15
42. 42 (request) ready? (request) ready? (request) ready? locked? requests[] Monday, March 30, 15
43. 43 (request) ready? (request) ready? (request) ready? locked? requests[] Monday, March 30, 15
44. 44 (request) ready? (request) ready? (request) ready? locked? requests[] Monday, March 30, 15
45. 45 (request) ready? (request) ready? (request) ready? locked? requests[] Monday, March 30, 15
46. 46 (request) ready? (request) ready? (request) ready? locked? requests[] Monday, March 30, 15
47. 47 B1 = btn:new(), B2 = btn:store(1, 10, B1), B3 = btn:store(1, 15, B2), 10 = btn:fetch(1, B1), 15 = btn:fetch(1, B2). Monday, March 30, 15
48. 48 1000 10k 100k 1mm 10mm bt 4 59 708 8894 -- ets 8 8 49 497 5296 bt_nif 11 23 267 2393 20908 Monday, March 30, 15
49. 49 btn:m_new(test), btn:m_store(test, 1, 10). btn:m_store(test, 1, 15), spawn(fun() -> 15 = btn:m_fetch(1) end). Monday, March 30, 15
50. 50 1000 10k 100k 1mm 10mm bt_nif2 3 6 68 744 8513 ets 8 8 49 497 5296 bt_nif 11 23 267 2393 20908 Monday, March 30, 15
51. Worker Dispatch 51 Monday, March 30, 15
52. 52 Monday, March 30, 15
53. 53 Load balancing Overload protection Monday, March 30, 15
54. 54 sidejob Monday, March 30, 15
55. 55 1 5 3 0 8 8 message counters (ETS) Monday, March 30, 15
56. 56 1 5 3 0 8 8 message counters (ETS) Monday, March 30, 15
57. 57 1 5 3 0 8 8 message counters (ETS) Monday, March 30, 15
58. 58 1 6 3 0 8 8 message counters (ETS) Monday, March 30, 15
59. 59 1 6 3 0 8 8 message counters (ETS) Monday, March 30, 15
60. 60 1 6 3 0 8 8 message counters (ETS) Monday, March 30, 15
61. 61 1 6 3 0 8 8 message counters (ETS) Monday, March 30, 15
62. 62 1 6 3 0 8 8 message counters (ETS) Monday, March 30, 15
63. 63 2 6 3 0 8 8 message counters (ETS) Monday, March 30, 15
64. 64 2 6 3 0 8 8 message counters (ETS) Monday, March 30, 15
65. 65 dispatch NIF Monday, March 30, 15
66. dispatch:new(test). sender() -> Pid = dispatch:find(test), Pid ! Msg, ok. worker() -> Name = dispatch:listen(test, self()), worker(Name). worker(Name) -> receive Msg -> do_something(Msg), dispatch:ack(test, Name), worker(Name) end. Monday, March 30, 15
67. 67 Faster under contention (About 1.5x) Monday, March 30, 15
68. Statistics 68 Monday, March 30, 15
69. 69 folsom exometer (others?) Monday, March 30, 15
70. 70 Challenge min/max mean latency percentiles metrics++ Monday, March 30, 15
71. 71 Riak issue for years Monday, March 30, 15
72. 72 Optimized counters in Riak 1.3 Monday, March 30, 15
73. 73 “Scheduler” Partitioned counters in ETS Monday, March 30, 15
74. 74 Histograms still a challenge Monday, March 30, 15
75. 75 Reservoir sampling (ETS) Monday, March 30, 15
76. 76 Let’s write a NIF! Monday, March 30, 15
77. 77 min/max via atomics Monday, March 30, 15
78. 78 template<class TA, class T> void atomic_max(TA &atomic, T val) { T current = atomic.load(std::memory_order_relaxed); while(val > current) { current = val; val = atomic.exchange(val); } } Monday, March 30, 15
79. 79 Normal reservoir sampling with atomic increments Monday, March 30, 15
80. 80 200 metrics 10 million events each 8-16 workers 30-40s runtime Monday, March 30, 15
81. 81 50 million events/s Monday, March 30, 15
82. 82 Future Work Partitioned Reservoir sampling Monday, March 30, 15
83. 83 http://gregable.com/2007/10/ reservoir-sampling.html Monday, March 30, 15
84. Conclusion 84 Monday, March 30, 15
85. 85 NIFs can help Monday, March 30, 15
86. 86 Optimized reusable components Monday, March 30, 15
87. 87 NIFs are hard Monday, March 30, 15
88. 88 Garbage collection/SMR Allocation Copying Monday, March 30, 15
89. 89 Erlang + NIFs are hard Monday, March 30, 15
90. 90 But, think it’s work it Monday, March 30, 15
91. 91 Much more work to do Monday, March 30, 15
92. 92 github.com/jtuple/ef2015 Monday, March 30, 15
93. Questions? 93 Monday, March 30, 15