Parallel Implementations The problematic Sequential algorithm Parallel algorithms Results Todo 1 The problematic: How to prove properties in reasonable time? 2 Sequential algorithm 3 Parallel algorithms 4 Results 5 Todo An Efficient and Parallel Abstract Interpreter in Scala — 3×3 Parallel Implementations 2 / 22
Parallel Implementations The problematic Sequential algorithm Parallel algorithms Results Todo From undecidable problem to fast decidable technique We want prove properties about programs. But it is an undecidable problem. ↓ Abstract interpretation is a static analysis technique that evaluates abstraction of programs. Decidable but high time complexity. ↓ A possible way to make that faster is to parallelize. . . An Efficient and Parallel Abstract Interpreter in Scala — 3×3 Parallel Implementations 3 / 22
Parallel Implementations The problematic Sequential algorithm Parallel algorithms Results Todo Parallelization of Scala-AM Scala-AM “(Abstract) Abstract Machine Experiments using Scala” (forked around September 14, 2017) ↓ Restriction to the simplest abstract machine (AAM), without optimization Restriction to analyze only Scheme programs. ↓ 3×3 parallel implementations (ParAAM∗) with actors (Akka) ↓ Scala-Par-AM https://bitbucket.org/OPiMedia/scala-par-am/ An Efficient and Parallel Abstract Interpreter in Scala — 3×3 Parallel Implementations 4 / 22
Parallel Implementations The problematic Sequential algorithm Parallel algorithms Results Todo 1 The problematic: How to prove properties in reasonable time? 2 Sequential algorithm 3 Parallel algorithms 4 Results 5 Todo An Efficient and Parallel Abstract Interpreter in Scala — 3×3 Parallel Implementations 5 / 22
Parallel Implementations The problematic Sequential algorithm Parallel algorithms Results Todo Worklist and step function, walk over the state graph (letrec ((abs (lambda (x) (if (>= x 0) x (- x))))) (abs -42)) initial state (abs -42) step function (if (>= x 0) x (- x)) (>= x 0) ko(Bool) x (- x) ko(Int) ( Ò ( × x ) ( (>= x 0) x (− x ) ) ) ( × −42) # state: 8 # final value: 1 (Int) # already visited: 1 Worklist max size: 2 Sizes of successors: # size 1: 6 # size 2: 1 Essential parts of abstract interpretation for this presentation: main loop on a worklist of states the step function: takes a state and returns a set of states Sequential algorithm (Scala code of SeqAAM) w o r k l i s t = empty l i s t v i s i t e d = empty s e t w o r k l i s t = w o r k l i s t ∪ { i n i t i a l s t a t e } Ö Ô Ø s t a t e = p i c k one from w o r k l i s t s t a t e not i n v i s i t e d Ø Ò v i s i t e d = v i s i t e d ∪ { s t a t e } e v a l u a t i o n of s t a t e (essentially s u c c e s s o r s = ste p ( s t a t e )) w o r k l i s t = w o r k l i s t ∪ s u c c e s s o r s other update (final values. . . ) ÙÒØ Ð w o r k l i s t i s empty The completed state graph does not depend of the order of the evaluation. Crucial for parallelization. An Efficient and Parallel Abstract Interpreter in Scala — 3×3 Parallel Implementations 6 / 22
Parallel Implementations The problematic Sequential algorithm Parallel algorithms Results Todo Very simple Scheme example, bigger state graph (letrec ((fibonacci (lambda (n) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2))))))) (fibonacci 10)) (fibonacci 10) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) (<= n 1) ko(Bool) n (+ (fibonacci (- n 1)) (fibonacci (- n 2))) ko(Int) (fibonacci (- n 1)) (- n 1) ko(Int) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) (<= n 1) ko(Bool) n (+ (fibonacci (- n 1)) (fibonacci (- n 2))) n (+ (fibonacci (- n 1)) (fibonacci (- n 2))) ko(Int) (fibonacci (- n 1)) ko(Int) (fibonacci (- n 1)) (- n 1) (fibonacci (- n 2)) (- n 1) ko(Int) (- n 2) ko(Int) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) ko(Int) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) (<= n 1) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) (<= n 1) ko(Bool) n (+ (fibonacci (- n 1)) (fibonacci (- n 2))) n (+ (fibonacci (- n 1)) (fibonacci (- n 2))) ko(Bool) n (+ (fibonacci (- n 1)) (fibonacci (- n 2))) n n (+ (fibonacci (- n 1)) (fibonacci (- n 2))) (+ (fibonacci (- n 1)) (fibonacci (- n 2))) ko(Int) ko(Int) (fibonacci (- n 2)) (fibonacci (- n 2)) ko(Int) (fibonacci (- n 1)) ko(Int) ko(Int) (fibonacci (- n 1)) (fibonacci (- n 1)) (- n 1) (fibonacci (- n 2)) (- n 1) (- n 1) (- n 2) (- n 2) ko(Int) ko(Int) ko(Int) ko(Int) (- n 2) ko(Int) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) ko(Int) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) (<= n 1) (<= n 1) (<= n 1) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) ko(Bool) ko(Bool) ko(Bool) n (+ (fibonacci (- n 1)) (fibonacci (- n 2))) n (+ (fibonacci (- n 1)) (fibonacci (- n 2))) n (+ (fibonacci (- n 1)) (fibonacci (- n 2))) n (+ (fibonacci (- n 1)) (fibonacci (- n 2))) (+ (fibonacci (- n 1)) (fibonacci (- n 2))) (+ (fibonacci (- n 1)) (fibonacci (- n 2))) n n (+ (fibonacci (- n 1)) (fibonacci (- n 2))) n (+ (fibonacci (- n 1)) (fibonacci (- n 2))) (+ (fibonacci (- n 1)) (fibonacci (- n 2))) n n ko(Int) (fibonacci (- n 1)) ko(Int) ko(Int) ko(Int) (fibonacci (- n 1)) (fibonacci (- n 1)) ko(Int) ko(Int) ko(Int) ko(Int) ko(Int) (- n 1) (fibonacci (- n 2)) (fibonacci (- n 2)) (- n 1) (- n 1) (fibonacci (- n 2)) (fibonacci (- n 2)) (fibonacci (- n 2)) (- n 2) (- n 2) ko(Int) (- n 2) (- n 2) ko(Int) ko(Int) (- n 2) ko(Int) ko(Int) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) ko(Int) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) ko(Int) ko(Int) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) (<= n 1) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) (<= n 1) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) (<= n 1) ko(Bool) ko(Bool) (<= n 1) ko(Bool) (+ (fibonacci (- n 1)) (fibonacci (- n 2))) (+ (fibonacci (- n 1)) (fibonacci (- n 2))) n (+ (fibonacci (- n 1)) (fibonacci (- n 2))) n n n n (+ (fibonacci (- n 1)) (fibonacci (- n 2))) (+ (fibonacci (- n 1)) (fibonacci (- n 2))) n (+ (fibonacci (- n 1)) (fibonacci (- n 2))) ko(Bool) n (+ (fibonacci (- n 1)) (fibonacci (- n 2))) n (+ (fibonacci (- n 1)) (fibonacci (- n 2))) n (+ (fibonacci (- n 1)) (fibonacci (- n 2))) ko(Int) ko(Int) ko(Int) ko(Int) ko(Int) ko(Int) n (+ (fibonacci (- n 1)) (fibonacci (- n 2))) n (+ (fibonacci (- n 1)) (fibonacci (- n 2))) (+ (fibonacci (- n 1)) (fibonacci (- n 2))) n ko(Int) (fibonacci (- n 1)) ko(Int) (fibonacci (- n 1)) ko(Int) (fibonacci (- n 2)) (fibonacci (- n 2)) (fibonacci (- n 2)) (fibonacci (- n 2)) (fibonacci (- n 2)) ko(Int) (fibonacci (- n 1)) ko(Int) (fibonacci (- n 1)) ko(Int) (- n 1) (- n 1) (- n 2) (- n 2) (- n 2) (- n 2) (- n 2) (- n 1) (- n 1) ko(Int) ko(Int) ko(Int) ko(Int) ko(Int) ko(Int) ko(Int) ko(Int) ko(Int) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) (<= n 1) (<= n 1) (<= n 1) ko(Bool) ko(Bool) ko(Bool) n n (+ (fibonacci (- n 1)) (fibonacci (- n 2))) n (+ (fibonacci (- n 1)) (fibonacci (- n 2))) (+ (fibonacci (- n 1)) (fibonacci (- n 2))) (+ (fibonacci (- n 1)) (fibonacci (- n 2))) (+ (fibonacci (- n 1)) (fibonacci (- n 2))) n n (+ (fibonacci (- n 1)) (fibonacci (- n 2))) n n (+ (fibonacci (- n 1)) (fibonacci (- n 2))) (+ (fibonacci (- n 1)) (fibonacci (- n 2))) (+ (fibonacci (- n 1)) (fibonacci (- n 2))) n n ko(Int) (fibonacci (- n 1)) ko(Int) ko(Int) ko(Int) ko(Int) ko(Int) ko(Int) ko(Int) ko(Int) (fibonacci (- n 2)) (- n 1) (- n 2) ko(Int) ko(Int) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) (<= n 1) ko(Bool) (+ (fibonacci (- n 1)) (fibonacci (- n 2))) n n (+ (fibonacci (- n 1)) (fibonacci (- n 2))) n (+ (fibonacci (- n 1)) (fibonacci (- n 2))) (fibonacci (- n 1)) ko(Int) ko(Int) ko(Int) (- n 1) ko(Int) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) ( Ò ( f i b o n a c c i n ) ( (<= n 1) n (+ ( f i b o n a c c i (− n 1)) ( f i b o n a c c i (− n 2 ) ) ) ) ) ( f i b o n a c c i 10) # state: 276 # final value: 1 (Int) # already visited: 71 Worklist max size: 24 Sizes of successors: # size 1: 231 # size 2: 13 # size 3: 3 # size 4: 2 # size 6: 12 An Efficient and Parallel Abstract Interpreter in Scala — 3×3 Parallel Implementations 7 / 22
Parallel Implementations The problematic Sequential algorithm Parallel algorithms Results Todo Worklist implementations Possible (immutable) data structures to implement the worklist (tested on Sergey/jfp/primtest.scm) • list of states 2.304s (Scala code of SeqAAM) s s s s s s • set of states 2.466s (Scala code of SeqAAMS) s s s s s s • list of sets of states 2.315s (Scala code of SeqAAMLS) s s s s s s avoid useless concatenations Each set corresponds to a result of the step function. Selected for parallel implementations and SeqAAMLS becomes the reference for comparisons. These time results and following are averages on 10 repetitions (after 3 repetitions skipped), made on Intel Xeon Gold 6148 2.40GHz, 20 cores available (VUB Hydra cluster), with Scala 2.12.7 – Akka 2.5.21 – Java 1.8.0 – GraalVM 1.0.0-rc14. An Efficient and Parallel Abstract Interpreter in Scala — 3×3 Parallel Implementations 8 / 22
Parallel Implementations The problematic Sequential algorithm Parallel algorithms Results Todo 1 The problematic: How to prove properties in reasonable time? 2 Sequential algorithm 3 Parallel algorithms 4 Results 5 Todo An Efficient and Parallel Abstract Interpreter in Scala — 3×3 Parallel Implementations 9 / 22
Parallel Implementations The problematic Sequential algorithm Parallel algorithms Results Todo First parallel implementation: ParAAM-L-SA-state main thread SenderAggregator N = p − 1 Eval actors w o r k l i s t = empty l i s t v i s i t e d = empty s e t shared data w o r k l i s t = w o r k l i s t ∪{ i n i t i a l s t a t e } Ö Ô Ø send c u r r e n t w o r k l i s t wait r e c e i v e new w o r k l i s t ÙÒØ Ð w o r k l i s t i s empty ÓÖ s t a t e from c u r r e n t w o r k l i s t send s t a t e update N r e c e i v e d send f i n i s h e d e v a l u a t e s t a t e send r e s u l t s to Eval n◦ i%N I name this implementation ParAAM-L-SA-state, for ParAAM – Loop – SenderAggregator – state. An Efficient and Parallel Abstract Interpreter in Scala — 3×3 Parallel Implementations 10 / 22
Parallel Implementations The problematic Sequential algorithm Parallel algorithms Results Todo First alternative: ParAAM-L-SA-set main thread SenderAggregator N = p − 1 Eval actors w o r k l i s t = empty l i s t v i s i t e d = empty s e t shared data w o r k l i s t = w o r k l i s t ∪{ i n i t i a l s t a t e } Ö Ô Ø send c u r r e n t w o r k l i s t wait r e c e i v e new w o r k l i s t ÙÒØ Ð w o r k l i s t i s empty f o r e a c h s e t from c u r r e n t w o r k l i s t send s e t update N r e c e i v e d send f i n i s h e d f o r e a c h s t a t e from s e t e v a l u a t e s t a t e send r e s u l t s to Eval n◦ i%N First improvement: send more than one state at a time. Set a set of states. ParAAM-L-SA-set, for ParAAM – Loop – SenderAggregator – set. An Efficient and Parallel Abstract Interpreter in Scala — 3×3 Parallel Implementations 11 / 22
Parallel Implementations The problematic Sequential algorithm Parallel algorithms Results Todo Second alternative: ParAAM-L-SA-part main thread SenderAggregator N = p − 1 Eval actors w o r k l i s t = empty l i s t v i s i t e d = empty s e t shared data w o r k l i s t = w o r k l i s t ∪{ i n i t i a l s t a t e } Ö Ô Ø send c u r r e n t w o r k l i s t wait r e c e i v e new w o r k l i s t ÙÒØ Ð w o r k l i s t i s empty f o r e a c h e qual part from c u r r e n t w o r k l i s t send part update N r e c e i v e d send f i n i s h e d f o r e a c h s t a t e from part e v a l u a t e s t a t e send r e s u l t s to Eval n◦ i Other possibility: send “equals” part of the worklist. ParAAM-L-SA-part, for ParAAM – Loop – SenderAggregator – part. An Efficient and Parallel Abstract Interpreter in Scala — 3×3 Parallel Implementations 12 / 22
Parallel Implementations The problematic Sequential algorithm Parallel algorithms Results Todo 3×3 parallel implementations ParAAM-L-SA-state ParAAM-C-S-state ParAAM-C-state state SeqAAMLS ParAAM-L-SA-set ParAAM-C-S-set ParAAM-C-set set ParAAM-L-SA-part ParAAM-C-S-part ParAAM-C-part part L-SA C-S C Loop – SenderAggregator Concurrent – Sender Concurrent 1 special actor p-1 eval actors main loop (with a barrier) SA send worklist each eval actor send results to SA 1 special actor p-1 eval actors no barrier S send worklist each eval actor update 0 special actor p eval actors no barrier each eval actor send worklist each eval actor update An Efficient and Parallel Abstract Interpreter in Scala — 3×3 Parallel Implementations 13 / 22
Parallel Implementations The problematic Sequential algorithm Parallel algorithms Results Todo Other strategy: ParAAM-C-S-state main thread Sender N = p − 1 Eval actors w o r k l i s t = empty l i s t v i s i t e d = empty s e t shared data send s t a r t wait send i n i t i a l s t a t e ÓÖ ( state , a c t o r ) from c u r r e n t w o r k l i s t and a v a i l a b l e Eval a c t o r s send s t a t e to a c t o r w o r k l i s t i s empty and a l l Eval a c t o r s a v a i l a b l e send f i n i s h e d e v a l u a t e s t a t e update (with synchronization) send a v a i l a b l e to Eval n◦ 0 Eval n◦ i available I name this implementation ParAAM-C-S-state, for ParAAM – Concurrent – Sender – state. And these similar improvements: ParAAM-C-S-set, for ParAAM – Concurrent – Sender – set and ParAAM-C-S-part, for ParAAM – Concurrent – Sender – part. An Efficient and Parallel Abstract Interpreter in Scala — 3×3 Parallel Implementations 14 / 22
Parallel Implementations The problematic Sequential algorithm Parallel algorithms Results Todo Last strategy: ParAAM-C-state main thread N = p Eval actors v i s i t e d = empty s e t a c t o r s = empty l i s t shared data send s t a r t wait send i n i t i a l s t a t e e v a l u a t e s t a t e update (with synchronization) ÓÖ ( state , a c t o r ) from c u r r e n t w o r k l i s t and a v a i l a b l e Eval a c t o r s send s t a t e to a c t o r w o r k l i s t i s empty and a l l Eval a c t o r s a v a i l a b l e send f i n i s h e d I name this implementation ParAAM-C-state, for ParAAM – Concurrent – state. And these similar improvements: ParAAM-C-set, for ParAAM – Concurrent – set and ParAAM-C-part, for ParAAM – Concurrent – part. An Efficient and Parallel Abstract Interpreter in Scala — 3×3 Parallel Implementations 15 / 22
Parallel Implementations The problematic Sequential algorithm Parallel algorithms Results Todo 1 The problematic: How to prove properties in reasonable time? 2 Sequential algorithm 3 Parallel algorithms 4 Results 5 Todo An Efficient and Parallel Abstract Interpreter in Scala — 3×3 Parallel Implementations 16 / 22
Parallel Implementations The problematic Sequential algorithm Parallel algorithms Results Todo 1 The problematic: How to prove properties in reasonable time? 2 Sequential algorithm 3 Parallel algorithms 4 Results 5 Todo An Efficient and Parallel Abstract Interpreter in Scala — 3×3 Parallel Implementations 20 / 22
Parallel Implementations The problematic Sequential algorithm Parallel algorithms Results Todo Done / Todo Implementation is (almost) done (Need little checkings, and maybe change the update of visited set for the worst implementations.) Todo Select which Scheme examples to use to produce final benchmarks and estimate computation times Run computations and collect data (benchmarks and some statistics) Sort data and finalize analyzes Write and write again (some parts of theoretical background can be taken from preparatory work) An Efficient and Parallel Abstract Interpreter in Scala — 3×3 Parallel Implementations 21 / 22
Parallel Implementations The problematic Sequential algorithm Parallel algorithms Results Todo Thank you! Questions time. . . Implementation: Scala-Par-AM – https:// Ø Ù ØºÓÖ »ÇÈ Å /scala-par-am/ This presentation on https://×Ô Ö º ÓÑ»ÓÔ Ñ / efficient-parallel-abstract-interpreter-in-scala-3x3-implementations Document, L A TEX sources, other references and previous presentations on https://bitbucket.org/OPiMedia/efficient-parallel-abstract-interpreter-in-scala An Efficient and Parallel Abstract Interpreter in Scala — 3×3 Parallel Implementations 22 / 22
opimedia
terensarnic
sharkira
seansaito
sansandsoc
ryoozawa
shogo_osawa
taka88
kamakiri1225
marktimemedia
jnunemaker
roundedbygravity
samanthasiow
hursman
jacobian
revolveconf
erikaheidi
matthewcrist
akmur
speakerdeck
sugarenia
