The Implementation and Use of a Generic Dataflow Behavior for Erlang

Erlang Workshop 2015
Christopher Meiklejohn, Peter Van Roy

Transcript

1. The Implementation and Use of a Generic Dataflow Behavior for Erlang CHRISTOPHER MEIKLEJOHN* AND PETER VAN ROY Basho Technologies, Inc. and Université catholique de Louvain We propose a new “generic” abstraction for Erlang/OTP that aids in the implementation of dataflow programming languages and models on the Erlang VM. This abstraction simplifies the implementation of “processing elements” in dataflow languages by providing a simple callback interface in the style of the gen_server and gen_fsm abstractions. We motivate the use of this new abstraction by examining the implementation of a distributed dataflow programming variant called Lasp. Categories and Subject Descriptors: D.1.3 [Programming Techniques]: Concurrent Programming; E.1 [Data Structures]: Distributed data structures Keywords: Dataflow Programming, Erlang, Concurrent Programming * work performed when the author was employed by Basho Technologies, Inc.
2. Motivation
3. • Provide a dataflow abstraction for Erlang in the style of the “generic” abstractions • Focuses on data and control flow • Support the development of dataflow based programming models* “Derflow”, Bravo et al., Erlang Workshop 2014 
 “DerflowL”, Meiklejohn et al., LADIS Workshop 2014 
 “Lasp”, Meiklejohn et al., PPDP 2015
4. {3,4,5} Intersection {1,2,3} {3}
5. {3,4,5} Intersection {3,4} {1,2,3,4}
6. Behaviour
7. • Module:init/1
 Initialize any necessary state • Module:read/1
 Provide a list of functions that are used to retrieve the value for each argument to processing • Module:process/2
 Process computation given the most recently received arguments from each “source”
8. -module(gen_flow_example). -behaviour(gen_flow). -export([start_link/1]). -export([init/1, read/1, process/2]). -record(state, {pid}). start_link(Args) -> gen_flow:start_link(?MODULE, Args). init([Pid]) -> {ok, #state{pid=Pid}}. read(State) -> ReadFuns = [ fun(_) -> sets:from_list([1,2,3]) end, fun(_) -> sets:from_list([3,4,5]) end ], {ok, ReadFuns, State}. process(Args, #state{pid=Pid}=State) -> case Args of [undefined, _] -> ok; [_, undefined] -> ok; [X, Y] -> Set = sets:intersection(X, Y), Pid ! {ok, sets:to_list(Set)}, ok end, {ok, State}.
9. Execution of gen_flow
10. 1. Launch read functions
 Launch a linked process to attempt to read all source variables 2. Cache responses
 As responses are received, cache the most recent value 3. Module:process/2
 Reprocess the function with the most recent values from the cache 4. Propagate result
 The implementer can decide how to propagate values forward with an arbitrary function; via a TCP socket, Erlang message passing, etc.
11. gen_server gen_server gen_flow gen_server
12. gen_server gen_server gen_flow gen_server 1.
13. gen_server gen_server gen_flow gen_server 1. 2.
14. gen_server gen_server gen_flow gen_server 1. 2. 3.
15. gen_server gen_server gen_flow gen_server 1. 2. 3. 4.
16. • If any of the read functions fail; collapse and restart the entire tree. • Results are cached for subsequent executions, given processing may require all.
17. gen_server gen_server gen_flow gen_server 1.
18. gen_server gen_server gen_flow gen_server 1.
19. gen_server gen_server gen_flow gen_server
20. gen_server gen_server gen_flow gen_server 1.
21. Lasp
22. • Distributed, deterministic dataflow
 Distributed, deterministic dataflow programming model for “eventually consistent” computations • Prototypical implementation
 Implementation in Erlang using gen_flow • Convergent modules
 Primary data abstraction is the CRDT
23. RA RB
24. RA RB 1 set(1)
25. RA RB 1 set(1)
26. RA RB 1 set(1) 3 set(3)
27. RA RB 1 set(1) 3 set(3) 2 set(2)
28. RA RB 1 set(1) 3 set(3) 2 set(2)
29. RA RB 1 set(1) 3 set(3) 2 set(2) ? ?
30. • Concurrency
 How do we resolve concurrent operations? • Semantic resolution
 Resolved by implementer; techniques can be error prone; arbitration results in “anomalies”
31. RA RB 1 set(1)
32. RA RB 1 set(1)
33. RA RB 1 set(1) 3 set(3)
34. RA RB 1 set(1) 3 set(3) 2 set(2)
35. RA RB 1 set(1) 3 set(3) 2 set(2)
36. RA RB 1 set(1) 3 set(3) 2 set(2) 3 3 max(2,3) max(2,3)
37. • Deterministic resolution
 Define a deterministic resolution for objects that closely resembles sequential data structures • Strong Eventual Consistency (SEC)
 Deterministic convergence guarantee once all messages have been delivered to all “replicas” • Non-monotonic operations
 How do we handle operations that are nondeterministic?
38. RA RB RC
39. RA RB RC {1} (1, {a}, {}) add(1)
40. RA RB RC {1} (1, {a}, {}) add(1) {1} (1, {b}, {}) add(1)
41. RA RB RC {1} (1, {a}, {}) add(1) {1} (1, {b}, {}) add(1) {} (1, {b}, {b}) remove(1)
42. RA RB RC {1} (1, {a}, {}) add(1) {1} (1, {b}, {}) add(1) {} (1, {b}, {b}) remove(1) {1} {1} {1} (1, {a, b}, {b}) (1, {a, b}, {b}) (1, {a, b}, {b})
43. • External non-monotonicity
 Implemented through the use of monotonic metadata • Metadata reduction
 Various ways to perform metadata reduction* Bieniusa et al., INRIA RR-8083, 2012 
 Brown et al., PaPEC 2014
44. Lasp API
45. • declare(T)
 Declare a variable of type T • bind(X, V)
 Bind a value to a variable; computes the least- upper-bound between the new and current value • update(X, Op, Actor)
 Update value of a variable with an identifier and a unique actor identifier
46. • read(X, V)
 Read variable at a logical time equal to or greater than a previously observed value • strict_read(X, V)
 Strict inflation version of read(X, V)
47. • map(X, F, Y)
 Apply function over X into Y • filter(X, P, Y)
 Filter X into Y using predicate P • fold(X, Op, Y)
 Fold values from X into Y using operation Op
48. • product(X, Y, Z)
 Compute product of X and Y into Z • union(X, Y, Z)
 Compute union of X and Y into Z • intersection(X, Y, Z)
 Compute intersection of X and Y into Z
49. %% Create initial set. {ok, S1} = lasp:declare(riak_dt_orset), %% Add elements to initial set and update. {ok, _} = lasp:update(S1, {add_all, [1,2,3]}, a), %% Create second set. {ok, S2} = lasp:declare(riak_dt_orset), %% Apply map operation between S1 and S2. {ok, _} = lasp:map(S1, fun(X) -> X * 2 end, S2).
50. {1,2,3} Map {2,4,6}
51. Map {2,4,6} {} {1,2,3} Map Map {} {2,4,6} {1,2,3}
52. Map {2,4,6} {} {1,2,3} Map Map {} {2,4,6} {1,2,3}
53. • Read operations
 Reads should not appear to “go back in time” • Causality
 Functions applied to CRDTs must preserve causal metadata • Quorum operations
 Take the “merge” of a quorum of replicas
54. Lasp Processes
55. • Processes
 Responsible for the read-then-write cycle of Lasp operations • Strict inflations
 Lasp operations read forward; do not return the result of a read operation until “time has advanced” logically
56. -module(lasp_process). -behaviour(gen_flow). -export([start_link/1]). -export([init/1, read/1, process/2]). -record(state, {read_funs, function}). start_link(Args) -> gen_flow:start_link(?MODULE, Args). %% @doc Initialize state. init([ReadFuns, Function]) -> {ok, #state{read_funs=ReadFuns, function=Function}}.
57. %% @doc Return list of read functions. read(#state{read_funs=ReadFuns0}=State) -> ReadFuns = [gen_read_fun(Id, ReadFun) || {Id, ReadFun} <- ReadFuns0], {ok, ReadFuns, State}. %% @doc Generate ReadFun. gen_read_fun(Id, ReadFun) -> fun(Value0) -> Value = case Value0 of undefined -> undefined; {_, _, V} -> V end, {ok, Value1} = ReadFun(Id, {strict, Value}), Value1 end.
58. %% @doc Computation to execute when inputs change. process(Args, #state{function=Function}=State) -> case lists:any(fun(X) -> X =:= undefined end, Args) of true -> ok; false -> erlang:apply(Function, Args) end, {ok, State}.
59. Evaluation
60. • Repeated pattern
 Appeared in previous work on Derflow, DerflowL , and Lasp • Extracted from Lasp
 gen_flow was extracted from the Lasp runtime • Code reduction
 Assisted in ease of implementation of the Lasp language as we added new “operations” • Macros as read functions
 Replacement of read functions at compile time based on EQC, eunit, or actual execution
61. Recent Work
62. • Spawn via proc_lib
 Spawn via the proc_lib facility for proper supervision • System messages
 Support system messages to replace or return state • Debugging
 Support Erlang debugging for tracking received messages • Elixir
 Elixir is looking to bring ideas from gen_flow into their generic abstraction gen_router
63. Related Work
64. • Kahn Process Networks (KPNs)
 Inspiration for gen_flow • Flowpools (EPFL 2013)
 Concurrent, lock-free dataflow abstraction over collections • Javelin (Clojure)
 Cell-based, generic dataflow programming library using Clojure’s protocols • Riak Pipe (Erlang Workshop 2012)
 Distributed dataflow abstraction; fixed topology using Erlang messaging
65. Future Work
66. • Reduce copying between processes
 Reduce the amount of data copied between processes when reading values. • Visualization
 Identify a way to visualize the flow of information between processes.
67. Questions?