Tweet
Tweet

Slide 1

Slide 1 text

Effective Programming in OCaml “KC” Sivaramakrishnan

Slide 2

Slide 2 text

• Adds native support for concurrency and parallelism to OCaml Multicore OCaml

Slide 3

Slide 3 text

• Adds native support for concurrency and parallelism to OCaml Multicore OCaml Overlapped execution A B A C B Time

Slide 4

Slide 4 text

• Adds native support for concurrency and parallelism to OCaml Multicore OCaml Overlapped execution A B A C B Time Simultaneous execution A B C Time

Slide 5

Slide 5 text

• Adds native support for concurrency and parallelism to OCaml Multicore OCaml Overlapped execution A B A C B Time Simultaneous execution A B C Time Effect Handlers

Slide 6

Slide 6 text

• Adds native support for concurrency and parallelism to OCaml Multicore OCaml Overlapped execution A B A C B Time Simultaneous execution A B C Time Effect Handlers Domains

Slide 7

Slide 7 text

• Adds native support for concurrency and parallelism to OCaml Multicore OCaml Overlapped execution A B A C B Time Simultaneous

Slide 8

Slide 8 text

Concurrency is not parallelism Parallelism is a performance hack whereas concurrency is a program structuring mechanism

Slide 9

Slide 9 text

Concurrency is not parallelism • OS threads give you parallelism and concurrenc y ✦ Too heavy weight for concurrent programmin g ✦ Http server with 1 OS thread per request is a terrible idea Parallelism is a performance hack whereas concurrency is a program structuring mechanism

Slide 10

Slide 10 text

Concurrency is not parallelism • OS threads give you parallelism and concurrenc y ✦ Too heavy weight for concurrent programmin g ✦ Http server with 1 OS thread per request is a terrible idea • Programming languages provide concurrent programming mechanisms as primitives ✦ async/await, generators, coroutines, etc. Parallelism is a performance hack whereas concurrency is a program structuring mechanism

Slide 11

Slide 11 text

Concurrency is not parallelism • OS threads give you parallelism and concurrenc y ✦ Too heavy weight for concurrent programmin g ✦ Http server with 1 OS thread per request is a terrible idea • Programming languages provide concurrent programming mechanisms as primitives ✦ async/await, generators, coroutines, etc. • Often include different primitives for concurrent programmin g ✦ JavaScript has async/await, generators, promises, and callbacks!! Parallelism is a performance hack whereas concurrency is a program structuring mechanism

Slide 12

Slide 12 text

Concurrent Programming in OCaml • OCaml does not have primitive support for concurrent programming

Slide 13

Slide 13 text

Concurrent Programming in OCaml • OCaml does not have primitive support for concurrent programming • Lwt and Async - concurrent programming libraries in OCam l ✦ Callback-oriented programming with monadic syntax >>= ✦ But do not satisfy monad laws

Slide 14

Slide 14 text

Concurrent Programming in OCaml • OCaml does not have primitive support for concurrent programming • Lwt and Async - concurrent programming libraries in OCam l ✦ Callback-oriented programming with monadic syntax >>= ✦ But do not satisfy monad laws • Suffers many pitfalls of callback-oriented programming ✦ No backtraces, No exception, monadic syntax, too many closures

Slide 15

Slide 15 text

Concurrent Programming in OCaml • OCaml does not have primitive support for concurrent programming • Lwt and Async - concurrent programming libraries in OCam l ✦ Callback-oriented programming with monadic syntax >>= ✦ But do not satisfy monad laws • Suffers many pitfalls of callback-oriented programming ✦ No backtraces, No exception, monadic syntax, too many closures • Go (goroutines) and GHC Haskell (threads) have better abstractions — lightweight threads

Slide 16

Slide 16 text

Concurrent Programming in OCaml • OCaml does not have primitive support for concurrent programming • Lwt and Async - concurrent programming libraries in OCam l ✦ Callback-oriented programming with monadic syntax >>= ✦ But do not satisfy monad laws • Suffers many pitfalls of callback-oriented programming ✦ No backtraces, No exception, monadic syntax, too many closures • Go (goroutines) and GHC Haskell (threads) have better abstractions — lightweight threads Should we add lightweight threads to OCaml?

Slide 17

Slide 17 text

Effect Handlers • A mechanism for programming with user-de fi ned effects

Slide 18

Slide 18 text

Effect Handlers • A mechanism for programming with user-de fi ned effects • Modular basis of non-local control- fl ow mechanism s ✦ Exceptions, generators, lightweight threads, promises, asynchronous IO, coroutines

Slide 19

Slide 19 text

Effect Handlers • A mechanism for programming with user-de fi ned effects • Modular basis of non-local control- fl ow mechanism s ✦ Exceptions, generators, lightweight threads, promises, asynchronous IO, coroutines • Effect handlers ~= fi rst-class, restartable exception s ✦ Similar to exceptions, performing an effect separate from handling it

Slide 20

Slide 20 text

An example effect E : string let comp () = print_string "0 "; print_string (perform E); print_string "3 " let main () = try comp () with effect E k -> print_string "1 "; continue k "2 "; print_string “4 "

Slide 21

Slide 21 text

An example effect E : string let comp () = print_string "0 "; print_string (perform E); print_string "3 " let main () = try comp () with effect E k -> print_string "1 "; continue k "2 "; print_string “4 " effect declaration

Slide 22

Slide 22 text

An example effect E : string let comp () = print_string "0 "; print_string (perform E); print_string "3 " let main () = try comp () with effect E k -> print_string "1 "; continue k "2 "; print_string “4 " computation effect declaration

Slide 23

Slide 23 text

An example effect E : string let comp () = print_string "0 "; print_string (perform E); print_string "3 " let main () = try comp () with effect E k -> print_string "1 "; continue k "2 "; print_string “4 " computation handler effect declaration

Slide 24

Slide 24 text

An example effect E : string let comp () = print_string "0 "; print_string (perform E); print_string "3 " let main () = try comp () with effect E k -> print_string "1 "; continue k "2 "; print_string “4 " computation handler suspends current computation effect declaration

Slide 25

Slide 25 text

An example effect E : string let comp () = print_string "0 "; print_string (perform E); print_string "3 " let main () = try comp () with effect E k -> print_string "1 "; continue k "2 "; print_string “4 " computation handler delimited continuation suspends current computation effect declaration

Slide 26

Slide 26 text

An example effect E : string let comp () = print_string "0 "; print_string (perform E); print_string "3 " let main () = try comp () with effect E k -> print_string "1 "; continue k "2 "; print_string “4 " computation handler delimited continuation suspends current computation resume suspended computation effect declaration

Slide 27

Slide 27 text

Stepping through the example effect E : string let comp () = print_string "0 "; print_string (perform E); print_string "3 " let main () = try comp () with effect E k -> print_string "1 "; continue k "2 "; print_string “4 " pc main sp

Slide 28

Slide 28 text

Stepping through the example effect E : string let comp () = print_string "0 "; print_string (perform E); print_string "3 " let main () = try comp () with effect E k -> print_string "1 "; continue k "2 "; print_string “4 " pc main sp

Slide 29

Slide 29 text

comp Stepping through the example effect E : string let comp () = print_string "0 "; print_string (perform E); print_string "3 " let main () = try comp () with effect E k -> print_string "1 "; continue k "2 "; print_string “4 " pc main sp parent Fiber: A piece of stack + effect handler

Slide 30

Slide 30 text

comp comp Stepping through the example effect E : string let comp () = print_string "0 "; print_string (perform E); print_string "3 " let main () = try comp () with effect E k -> print_string "1 "; continue k "2 "; print_string “4 " pc main sp parent 0

Slide 31

Slide 31 text

comp comp Stepping through the example effect E : string let comp () = print_string "0 "; print_string (perform E); print_string "3 " let main () = try comp () with effect E k -> print_string "1 "; continue k "2 "; print_string “4 " pc main sp k 0

Slide 32

Slide 32 text

comp comp Stepping through the example effect E : string let comp () = print_string "0 "; print_string (perform E); print_string "3 " let main () = try comp () with effect E k -> print_string "1 "; continue k "2 "; print_string “4 " pc main sp k 0

Slide 33

Slide 33 text

comp comp Stepping through the example effect E : string let comp () = print_string "0 "; print_string (perform E); print_string "3 " let main () = try comp () with effect E k -> print_string "1 "; continue k "2 "; print_string “4 " pc main sp k 0

Slide 34

Slide 34 text

comp comp Stepping through the example effect E : string let comp () = print_string "0 "; print_string (perform E); print_string "3 " let main () = try comp () with effect E k -> print_string "1 "; continue k "2 "; print_string “4 " pc main sp k 0 1

Slide 35

Slide 35 text

comp comp Stepping through the example effect E : string let comp () = print_string "0 "; print_string (perform E); print_string "3 " let main () = try comp () with effect E k -> print_string "1 "; continue k "2 "; print_string “4 " pc main sp k 0 1

Slide 36

Slide 36 text

comp comp Stepping through the example effect E : string let comp () = print_string "0 "; print_string (perform E); print_string "3 " let main () = try comp () with effect E k -> print_string "1 "; continue k "2 "; print_string “4 " pc main sp k parent 0 1

Slide 37

Slide 37 text

comp comp Stepping through the example effect E : string let comp () = print_string "0 "; print_string (perform E); print_string "3 " let main () = try comp () with effect E k -> print_string "1 "; continue k "2 "; print_string “4 " pc main sp k parent 0 1 2

Slide 38

Slide 38 text

Stepping through the example effect E : string let comp () = print_string "0 "; print_string (perform E); print_string "3 " let main () = try comp () with effect E k -> print_string "1 "; continue k "2 "; print_string “4 " pc main sp k 0 1 2 3

Slide 39

Slide 39 text

Stepping through the example effect E : string let comp () = print_string "0 "; print_string (perform E); print_string "3 " let main () = try comp () with effect E k -> print_string "1 "; continue k "2 "; print_string “4 " pc main sp k 0 1 2 3 4

Slide 40

Slide 40 text

effect A : unit effect B : unit let baz () = perform A let bar () = try baz () with effect B k -> continue k () let foo () = try bar () with effect A k -> continue k () Handlers can be nested foo bar baz sp parent parent pc

Slide 41

Slide 41 text

effect A : unit effect B : unit let baz () = perform A let bar () = try baz () with effect B k -> continue k () let foo () = try bar () with effect A k -> continue k () Handlers can be nested foo bar baz sp parent parent pc

Slide 42

Slide 42 text

effect A : unit effect B : unit let baz () = perform A let bar () = try baz () with effect B k -> continue k () let foo () = try bar () with effect A k -> continue k () Handlers can be nested foo bar baz sp parent pc k

Slide 43

Slide 43 text

effect A : unit effect B : unit let baz () = perform A let bar () = try baz () with effect B k -> continue k () let foo () = try bar () with effect A k -> continue k () Handlers can be nested foo bar baz sp parent pc k • Linear search through handler s • Handler stacks shallow in practice

Slide 44

Slide 44 text

Lightweight Threading effect Fork : (unit -> unit) -> unit effect Yield : unit

Slide 45

Slide 45 text

Lightweight Threading effect Fork : (unit -> unit) -> unit effect Yield : unit let run main = ... (* assume queue of continuations *) let run_next () = match dequeue () with | Some k -> continue k () | None -> () in let rec spawn f = match f () with | () -> run_next () (* value case *) | effect Yield k -> enqueue k; run_next () | effect (Fork f) k -> enqueue k; spawn f in spawn main

Slide 46

Slide 46 text

Lightweight Threading effect Fork : (unit -> unit) -> unit effect Yield : unit let run main = ... (* assume queue of continuations *) let run_next () = match dequeue () with | Some k -> continue k () | None -> () in let rec spawn f = match f () with | () -> run_next () (* value case *) | effect Yield k -> enqueue k; run_next () | effect (Fork f) k -> enqueue k; spawn f in spawn main let fork f = perform (Fork f) let yield () = perform Yield

Slide 47

Slide 47 text

Lightweight threading let main () = fork (fun _ -> print_endline "1.a"; yield (); print_endline "1.b"); fork (fun _ -> print_endline "2.a"; yield (); print_endline “2.b") ;; run main

Slide 48

Slide 48 text

Lightweight threading let main () = fork (fun _ -> print_endline "1.a"; yield (); print_endline "1.b"); fork (fun _ -> print_endline "2.a"; yield (); print_endline “2.b") ;; run main 1.a 2.a 1.b 2.b

Slide 49

Slide 49 text

Lightweight threading let main () = fork (fun _ -> print_endline "1.a"; yield (); print_endline "1.b"); fork (fun _ -> print_endline "2.a"; yield (); print_endline “2.b") ;; run main 1.a 2.a 1.b 2.b • Direct-style (no monads) • User-code need not be aware of effects

Slide 50

Slide 50 text

Generators

Slide 51

Slide 51 text

Generators • Generators — non-continuous traversal of data structure by yielding value s ✦ Primitives in JavaScript and Python

Slide 52

Slide 52 text

Generators • Generators — non-continuous traversal of data structure by yielding value s ✦ Primitives in JavaScript and Python function* generator(i) { yield i; yield i + 10; } const gen = generator(10); console.log(gen.next().value); // expected output: 10 console.log(gen.next().value); // expected output: 20

Slide 53

Slide 53 text

Generators • Generators — non-continuous traversal of data structure by yielding value s ✦ Primitives in JavaScript and Python • Can be derived automatically from any iterator using effect handlers function* generator(i) { yield i; yield i + 10; } const gen = generator(10); console.log(gen.next().value); // expected output: 10 console.log(gen.next().value); // expected output: 20

Slide 54

Slide 54 text

Generators: effect handlers module MkGen (S :sig type 'a t val iter : ('a -> unit) -> 'a t -> unit end) : sig val gen : 'a S.t -> (unit -> 'a option) end = struct

Slide 55

Slide 55 text

Generators: effect handlers module MkGen (S :sig type 'a t val iter : ('a -> unit) -> 'a t -> unit end) : sig val gen : 'a S.t -> (unit -> 'a option) end = struct let gen : type a. a S.t -> (unit -> a option) = fun l -> let module M = struct effect Yield : a -> unit end in let open M in let rec step = ref (fun () -> match S.iter (fun v -> perform (Yield v)) l with | () -> None | effect (Yield v) k -> step := (fun () -> continue k ()); Some v) in fun () -> !step () end

Slide 56

Slide 56 text

Generators: List module L = MkGen (struct type 'a t = 'a list let iter = List.iter end)

Slide 57

Slide 57 text

Generators: List module L = MkGen (struct type 'a t = 'a list let iter = List.iter end) let next = L.gen [1;2;3] next() (* Some 1 *) next() (* Some 2 *) next() (* Some 3 *) next() (* None *)

Slide 58

Slide 58 text

Generators: Tree type 'a tree = | Leaf | Node of 'a tree * 'a * 'a tree let rec iter f = function | Leaf -> () | Node (l, x, r) -> iter f l; f x; iter f r module T = MkGen(struct type 'a t = 'a tree let iter = iter end)

Slide 59

Slide 59 text

Generators: Tree type 'a tree = | Leaf | Node of 'a tree * 'a * 'a tree let rec iter f = function | Leaf -> () | Node (l, x, r) -> iter f l; f x; iter f r module T = MkGen(struct type 'a t = 'a tree let iter = iter end) (* Make a complete binary tree of depth [n] using [O(n)] space *) let rec make = function | 0 -> Leaf | n -> let t = make (n-1) in Node (t,n,t)

Slide 60

Slide 60 text

Generators: Tree type 'a tree = | Leaf | Node of 'a tree * 'a * 'a tree let rec iter f = function | Leaf -> () | Node (l, x, r) -> iter f l; f x; iter f r module T = MkGen(struct type 'a t = 'a tree let iter = iter end) let t = make 2 2 1 1 (* Make a complete binary tree of depth [n] using [O(n)] space *) let rec make = function | 0 -> Leaf | n -> let t = make (n-1) in Node (t,n,t)

Slide 61

Slide 61 text

Generators: Tree type 'a tree = | Leaf | Node of 'a tree * 'a * 'a tree let rec iter f = function | Leaf -> () | Node (l, x, r) -> iter f l; f x; iter f r module T = MkGen(struct type 'a t = 'a tree let iter = iter end) let t = make 2 2 1 1 (* Make a complete binary tree of depth [n] using [O(n)] space *) let rec make = function | 0 -> Leaf | n -> let t = make (n-1) in Node (t,n,t) let next = T.gen t next() (* Some 1 *) next() (* Some 2 *) next() (* Some 1 *) next() (* None *)

Slide 62

Slide 62 text

Static Semantics

Slide 63

Slide 63 text

Static Semantics • No effect safet y ✦ No static guarantee that all the effects performed are handled (c.f. exceptions ) ✦ perform E at the top-level raises Unhandled exception

Slide 64

Slide 64 text

Static Semantics • No effect safet y ✦ No static guarantee that all the effects performed are handled (c.f. exceptions ) ✦ perform E at the top-level raises Unhandled exception • Effect system in the work s ✦ See also Eff, Koka, Links, Helium

Slide 65

Slide 65 text

Static Semantics • No effect safet y ✦ No static guarantee that all the effects performed are handled (c.f. exceptions ) ✦ perform E at the top-level raises Unhandled exception • Effect system in the work s ✦ See also Eff, Koka, Links, Helium • Effective OCam l ✦ Track both user-de fi ned and built-in (ref, io, exceptions) effect s ✦ OCaml becomes a pure language (in the Haskell sense — divergence allowed)

Slide 66

Slide 66 text

Static Semantics • No effect safet y ✦ No static guarantee that all the effects performed are handled (c.f. exceptions ) ✦ perform E at the top-level raises Unhandled exception • Effect system in the work s ✦ See also Eff, Koka, Links, Helium • Effective OCam l ✦ Track both user-de fi ned and built-in (ref, io, exceptions) effect s ✦ OCaml becomes a pure language (in the Haskell sense — divergence allowed) let foo () = print_string "hello, world" val foo : unit -[ io ]-> unit Syntax is still in the works

Slide 67

Slide 67 text

Retro fi tting Challenges

Slide 68

Slide 68 text

Retro fi tting Challenges • Millions of lines of legacy cod e ✦ Written without non-local control- fl ow in min d ✦ Cost of refactoring sequential code itself is prohibitive

Slide 69

Slide 69 text

Retro fi tting Challenges • Millions of lines of legacy cod e ✦ Written without non-local control- fl ow in min d ✦ Cost of refactoring sequential code itself is prohibitive Backwards compatibility before fancy new features

Slide 70

Slide 70 text

Systems Programming • OCaml is a systems programming languag e ✦ Manipulates resources such as fi les, sockets, buffers, etc.

Slide 71

Slide 71 text

Systems Programming • OCaml is a systems programming languag e ✦ Manipulates resources such as fi les, sockets, buffers, etc. • OCaml code is written in defensive style to guard against exceptional behaviour and clear up resources

Slide 72

Slide 72 text

Systems Programming • OCaml is a systems programming languag e ✦ Manipulates resources such as fi les, sockets, buffers, etc. • OCaml code is written in defensive style to guard against exceptional behaviour and clear up resources let copy ic oc = let rec loop () = let l = input_line ic in output_string oc (l ^ "\n"); loop () in try loop () with | End_of_file -> close_in ic; close_out oc | e -> close_in ic; close_out oc; raise e

Slide 73

Slide 73 text

Systems Programming • OCaml is a systems programming languag e ✦ Manipulates resources such as fi les, sockets, buffers, etc. • OCaml code is written in defensive style to guard against exceptional behaviour and clear up resources let copy ic oc = let rec loop () = let l = input_line ic in output_string oc (l ^ "\n"); loop () in try loop () with | End_of_file -> close_in ic; close_out oc | e -> close_in ic; close_out oc; raise e raises End_of_file at the end

Slide 74

Slide 74 text

Systems Programming • OCaml is a systems programming languag e ✦ Manipulates resources such as fi les, sockets, buffers, etc. • OCaml code is written in defensive style to guard against exceptional behaviour and clear up resources let copy ic oc = let rec loop () = let l = input_line ic in output_string oc (l ^ "\n"); loop () in try loop () with | End_of_file -> close_in ic; close_out oc | e -> close_in ic; close_out oc; raise e raise Sys_error when channel is closed raises End_of_file at the end

Slide 75

Slide 75 text

Systems Programming • OCaml is a systems programming languag e ✦ Manipulates resources such as fi les, sockets, buffers, etc. • OCaml code is written in defensive style to guard against exceptional behaviour and clear up resources let copy ic oc = let rec loop () = let l = input_line ic in output_string oc (l ^ "\n"); loop () in try loop () with | End_of_file -> close_in ic; close_out oc | e -> close_in ic; close_out oc; raise e We would like to make this code transparently asynchronous raise Sys_error when channel is closed raises End_of_file at the end

Slide 76

Slide 76 text

Asynchronous IO effect In_line : in_channel -> string effect Out_str : out_channel * string -> unit

Slide 77

Slide 77 text

Asynchronous IO effect In_line : in_channel -> string effect Out_str : out_channel * string -> unit let input_line ic = perform (In_line ic) let output_string oc s = perform (Out_str (oc,s))

Slide 78

Slide 78 text

Asynchronous IO let run_aio f = match f () with | v -> v | effect (In_line chan) k -> register_async_input_line chan k; run_next () | effect (Out_str (chan, s)) k -> register_async_output_string chan s k; run_next () effect In_line : in_channel -> string effect Out_str : out_channel * string -> unit let input_line ic = perform (In_line ic) let output_string oc s = perform (Out_str (oc,s))

Slide 79

Slide 79 text

Asynchronous IO let run_aio f = match f () with | v -> v | effect (In_line chan) k -> register_async_input_line chan k; run_next () | effect (Out_str (chan, s)) k -> register_async_output_string chan s k; run_next () • Continue with appropriate value when the asynchronous IO call returns effect In_line : in_channel -> string effect Out_str : out_channel * string -> unit let input_line ic = perform (In_line ic) let output_string oc s = perform (Out_str (oc,s))

Slide 80

Slide 80 text

Asynchronous IO let run_aio f = match f () with | v -> v | effect (In_line chan) k -> register_async_input_line chan k; run_next () | effect (Out_str (chan, s)) k -> register_async_output_string chan s k; run_next () • Continue with appropriate value when the asynchronous IO call returns • But what about termination? — End_of_file and Sys_error exceptional cases. effect In_line : in_channel -> string effect Out_str : out_channel * string -> unit let input_line ic = perform (In_line ic) let output_string oc s = perform (Out_str (oc,s))

Slide 81

Slide 81 text

Discontinue • We add a discontinue primitive to resume a continuation by raising an exceptio n • On End_of_file and Sys_error, the asynchronous IO scheduler uses discontinue to raise the appropriate exception discontinue k End_of_file

Slide 82

Slide 82 text

Linearity • Resources such as sockets, fi le descriptors, channels and buffers are linear resource s ✦ Created and destroyed exactly once

Slide 83

Slide 83 text

Linearity • Resources such as sockets, fi le descriptors, channels and buffers are linear resource s ✦ Created and destroyed exactly once • OCaml functions return exactly once with value or exception ✦ Defensive programming already guards against exceptional return cases

Slide 84

Slide 84 text

Linearity • Resources such as sockets, fi le descriptors, channels and buffers are linear resource s ✦ Created and destroyed exactly once • OCaml functions return exactly once with value or exception ✦ Defensive programming already guards against exceptional return cases • With effect handlers, functions may return at-most once if continuation not resumed ✦ This breaks resource-safe legacy code

Slide 85

Slide 85 text

Linearity effect E : unit let foo () = perform E

Slide 86

Slide 86 text

Linearity effect E : unit let foo () = perform E let bar () = let ic = open_in "input.txt" in match foo () with | v -> close_in ic | exception e -> close_in ic; raise e

Slide 87

Slide 87 text

Linearity effect E : unit let foo () = perform E let bar () = let ic = open_in "input.txt" in match foo () with | v -> close_in ic | exception e -> close_in ic; raise e let baz () = try bar () with | effect E _ -> () (* leaks ic *)

Slide 88

Slide 88 text

Linearity effect E : unit let foo () = perform E We assume that captured continuations are resumed exactly once either using continue or discontinue let bar () = let ic = open_in "input.txt" in match foo () with | v -> close_in ic | exception e -> close_in ic; raise e let baz () = try bar () with | effect E _ -> () (* leaks ic *)

Slide 89

Slide 89 text

Backtraces • OCaml has excellent compatibility with debugging and pro fi ling tools — gdb, lldb, perf, libunwind, etc . ✦ DWARF stack unwinding support

Slide 90

Slide 90 text

Backtraces • OCaml has excellent compatibility with debugging and pro fi ling tools — gdb, lldb, perf, libunwind, etc . ✦ DWARF stack unwinding support • Multicore OCaml supports DWARF stack unwinding across fi bers

Slide 91

Slide 91 text

Backtraces effect E : unit let foo () = perform E let bar () = let ic = open_in "input.txt" in match foo () with | v -> close_in ic | exception e -> close_in ic; raise e let baz () = try bar () with | effect E _ -> () (* leak *) • OCaml has excellent compatibility with debugging and pro fi ling tools — gdb, lldb, perf, libunwind, etc . ✦ DWARF stack unwinding support • Multicore OCaml supports DWARF stack unwinding across fi bers

Slide 92

Slide 92 text

Backtraces effect E : unit let foo () = perform E let bar () = let ic = open_in "input.txt" in match foo () with | v -> close_in ic | exception e -> close_in ic; raise e let baz () = try bar () with | effect E _ -> () (* leak *) • OCaml has excellent compatibility with debugging and pro fi ling tools — gdb, lldb, perf, libunwind, etc . ✦ DWARF stack unwinding support • Multicore OCaml supports DWARF stack unwinding across fi bers foo baz bar Stack grows down Fiber 1 Fiber 2

Slide 93

Slide 93 text

Backtraces effect E : unit let foo () = perform E let bar () = let ic = open_in "input.txt" in match foo () with | v -> close_in ic | exception e -> close_in ic; raise e let baz () = try bar () with | effect E _ -> () (* leak *) • OCaml has excellent compatibility with debugging and pro fi ling tools — gdb, lldb, perf, libunwind, etc . ✦ DWARF stack unwinding support • Multicore OCaml supports DWARF stack unwinding across fi bers foo baz bar Stack grows down Fiber 1 Fiber 2 Bespoke DWARF bytecode for unwinding across fi bers

Slide 94

Slide 94 text

Backtraces effect E : unit let foo () = perform E let bar () = let ic = open_in "input.txt" in match foo () with | v -> close_in ic | exception e -> close_in ic; raise e let baz () = try bar () with | effect E _ -> () (* leak *) (lldb) bt * thread #1, name = 'a.out', stop reason = … * #0: 0x58b208 caml_perform #1: 0x56aa5d camlTest__foo_83 at test.ml:4 #2: 0x56aae2 camlTest__bar_85 at test.ml:9 #3: 0x56a9fc camlTest__fun_199 at test.ml:14 #4: 0x58b322 caml_runstack + 70 #5: 0x56ab99 camlTest__baz_91 at test.ml:14 #6: 0x56ace6 camlTest__entry at test.ml:21 #7: 0x56a41c caml_program + 60 #8: 0x58b0b7 caml_start_program + 135 #9: …

Slide 95

Slide 95 text

Performance let foo () = (* a *) try (* b *) perform E (* d *) with effect E k -> (* c *) continue k () (* e *)

Slide 96

Slide 96 text

Performance let foo () = (* a *) try (* b *) perform E (* d *) with effect E k -> (* c *) continue k () (* e *) Instruction Sequence a to b b to c c to d d to e Signi fi cance Create a new stack & run the computation Performing & handling an e ff ect Resuming a continuation Returning from a computation & free the stack • Each of the instruction sequences involves a stack switch

Slide 97

Slide 97 text

Performance let foo () = (* a *) try (* b *) perform E (* d *) with effect E k -> (* c *) continue k () (* e *) Instruction Sequence a to b b to c c to d d to e Signi fi cance Create a new stack & run the computation Performing & handling an e ff ect Resuming a continuation Returning from a computation & free the stack • Each of the instruction sequences involves a stack switch • Intel(R) Xeon(R) Gold 5120 CPU @ 2.20GH z ★ For calibration, memory read latency is 90 ns (local NUMA node) and 145 ns (remote NUMA node)

Slide 98

Slide 98 text

Performance let foo () = (* a *) try (* b *) perform E (* d *) with effect E k -> (* c *) continue k () (* e *) Instruction Sequence a to b b to c c to d d to e Signi fi cance Create a new stack & run the computation Performing & handling an e ff ect Resuming a continuation Returning from a computation & free the stack Time (ns) 23 5 11 7 • Each of the instruction sequences involves a stack switch • Intel(R) Xeon(R) Gold 5120 CPU @ 2.20GH z ★ For calibration, memory read latency is 90 ns (local NUMA node) and 145 ns (remote NUMA node)

Slide 99

Slide 99 text

Performance: Generators • Traverse a complete binary-tree of depth 2 5 ✦ 226 stack switches

Slide 100

Slide 100 text

Performance: Generators • Traverse a complete binary-tree of depth 2 5 ✦ 226 stack switches • Iterator — idiomatic recursive traversal

Slide 101

Slide 101 text

Performance: Generators • Traverse a complete binary-tree of depth 2 5 ✦ 226 stack switches • Iterator — idiomatic recursive traversal • Generato r ✦ Hand-written generator (hw-generator ) ✤ CPS translation + defunctionalization to remove intermediate closure allocatio n ✦ Generator using effect handlers (eh-generator)

Slide 102

Slide 102 text

Performance: Generators Variant Time (milliseconds) Iterator (baseline) 202 hw-generator 837 (3.76x) eh-generator 1879 (9.30x) Multicore OCaml

Slide 103

Slide 103 text

Performance: Generators Variant Time (milliseconds) Iterator (baseline) 202 hw-generator 837 (3.76x) eh-generator 1879 (9.30x) Multicore OCaml Variant Time (milliseconds) Iterator (baseline) 492 generator 43842 (89.1x) nodejs 14.07

Slide 104

Slide 104 text

Performance: WebServer • Effect handlers for asynchronous I/O in direct-styl e ✦ https://github.com/kayceesrk/ocaml-aeio/ • Variant s ✦ Go + net/http (GOMAXPROCS=1 ) ✦ OCaml + http/af + Lwt (explicit callbacks ) ✦ OCaml + http/af + Effect handlers (MC ) • Performance measured using wrk2

Slide 105

Slide 105 text

Performance: WebServer • Effect handlers for asynchronous I/O in direct-styl e ✦ https://github.com/kayceesrk/ocaml-aeio/ • Variant s ✦ Go + net/http (GOMAXPROCS=1 ) ✦ OCaml + http/af + Lwt (explicit callbacks ) ✦ OCaml + http/af + Effect handlers (MC ) • Performance measured using wrk2

Slide 106

Slide 106 text

Performance: WebServer • Effect handlers for asynchronous I/O in direct-styl e ✦ https://github.com/kayceesrk/ocaml-aeio/ • Variant s ✦ Go + net/http (GOMAXPROCS=1 ) ✦ OCaml + http/af + Lwt (explicit callbacks ) ✦ OCaml + http/af + Effect handlers (MC ) • Performance measured using wrk2 • Direct style (no monadic syntax)

Slide 107

Slide 107 text

Thanks! • Multicore OCaml — https://github.com/ocaml-multicore/ocaml- multicore • Effects Examples — https://github.com/ocaml-multicore/effects- examples • Sivaramakrishnan et al, “Retro fi tting Parallelism onto OCaml", ICFP 2020 • Dolan et al, “Concurrent System Programming with Effect Handlers”, TFP 2017 $ opam switch create 4.10.0+multicore \ --packages=ocaml-variants.4.10.0+multicore \ --repositories=multicore=git+https://github.com/ocaml-multicore/multicore-opam.git,default Install Multicore OCaml

Slide 108

Slide 108 text

Bonus Slides

Slide 109

Slide 109 text

No effects performance

Slide 110

Slide 110 text

No effects performance coq irmin menhir alt-ergo

Slide 111

Slide 111 text

No effects performance coq irmin menhir alt-ergo • ~1% faster than stock (geomean of normalised running times ) ✦ Difference under measurement noise mostl y ✦ Outliers due to difference in allocators