Efficient Parallel Abstract Interpreter in Scala — 3x3 Parallel Implementations

Universit´
e Libre de Bruxelles
Computer Science Department
MEMO-F524 Masters thesis
An Efficient and Parallel
Abstract Interpreter in Scala
— 3×3 Parallel Implementations —
Olivier Pirson — [email protected]
orcid.org/0000-0001-6296-9659
March 22, 2019
(Last modiﬁcations: June 18, 2020)
https://speakerdeck.com/opimedia/
efficient-parallel-abstract-interpreter-in-scala-3x3-implementations
Vrije Universiteit Brussel
Promotors Coen De Roover
Wolfgang De Meuter
Advisor Quentin Sti´
evenart

View Slide

An Eﬃcient and
Parallel Abstract
Interpreter
in Scala
—
3×3 Parallel
Implementations
The problematic
Sequential
algorithm
Parallel
algorithms
Results
Todo
1 The problematic: How to prove properties in reasonable time?
2 Sequential algorithm
3 Parallel algorithms
4 Results
5 Todo
An Eﬃcient and Parallel Abstract Interpreter in Scala — 3×3 Parallel Implementations 2 / 22

View Slide

An Eﬃcient and
Parallel Abstract
Interpreter
in Scala
—
3×3 Parallel
Implementations
The problematic
Sequential
algorithm
Parallel
algorithms
Results
Todo
From undecidable problem to fast decidable technique
We want prove properties about programs.
But it is an undecidable problem.
↓
Abstract interpretation is a static analysis technique that
evaluates abstraction of programs.
Decidable but high time complexity.
↓
A possible way to make that faster is to parallelize. . .

View Slide

An Eﬃcient and
Parallel Abstract
Interpreter
in Scala
—
3×3 Parallel
Implementations
The problematic
Sequential
algorithm
Parallel
algorithms
Results
Todo
Parallelization of Scala-AM
Scala-AM
“(Abstract) Abstract Machine Experiments using Scala”
(forked around September 14, 2017)
↓
Restriction to the simplest abstract machine (AAM),
without optimization
Restriction to analyze only Scheme programs.
↓
3×3 parallel implementations (ParAAM∗) with actors (Akka)
↓
Scala-Par-AM
https://bitbucket.org/OPiMedia/scala-par-am/

View Slide

An Eﬃcient and
Parallel Abstract
Interpreter
in Scala
—
3×3 Parallel
Implementations
The problematic
Sequential
algorithm
Parallel
algorithms
Results
Todo
4 Results
5 Todo

View Slide

An Efficient and
Parallel Abstract
Interpreter
in Scala
—
3×3 Parallel
Implementations
The problematic
Sequential
algorithm
Parallel
algorithms
Results
Todo
Worklist and step function, walk over the state graph
(letrec ((abs (lambda (x) (if (>= x 0) x (- x))))) (abs -42))
initial state
(abs -42)
step function
(if (>= x 0) x (- x))
(>= x 0)
ko(Bool)
x (- x)
ko(Int)
( Ò ( × x )
( (>= x 0)
x
(− x ) ) )
( × −42)
# state: 8
# final value: 1 (Int)
# already visited: 1
Worklist max size: 2
Sizes of successors:
# size 1: 6
# size 2: 1
Essential parts of abstract interpretation for this presentation:
main loop on a worklist of states
the step function:
takes a state
and returns a set of states
Sequential algorithm (Scala code of SeqAAM)
w o r k l i s t = empty l i s t
v i s i t e d = empty s e t
w o r k l i s t = w o r k l i s t ∪ { i n i t i a l s t a t e }
Ö Ô Ø
s t a t e = p i c k one from w o r k l i s t
s t a t e not i n v i s i t e d Ø Ò
v i s i t e d = v i s i t e d ∪ { s t a t e }
e v a l u a t i o n of s t a t e
(essentially s u c c e s s o r s = ste p ( s t a t e ))
w o r k l i s t = w o r k l i s t ∪ s u c c e s s o r s
other update (final values. . . )
ÙÒØ Ð w o r k l i s t i s empty
The completed state graph
does not depend of the order of the evaluation.
Crucial for parallelization.

View Slide

An Eﬃcient and
Parallel Abstract
Interpreter
in Scala
—
3×3 Parallel
Implementations
The problematic
Sequential
algorithm
Parallel
algorithms
Results
Todo
Very simple Scheme example, bigger state graph
(letrec ((fibonacci (lambda (n) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2))))))) (fibonacci 10))
(fibonacci 10)
(if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2))))
(<= n 1)
ko(Bool)
n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))
ko(Int) (fibonacci (- n 1))
(- n 1)
ko(Int)
(<= n 1)
ko(Bool)
(- n 1)
(fibonacci (- n 2)) (- n 1)
ko(Int)
(- n 2) ko(Int)
ko(Int) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2))))
(<= n 1)
(<= n 1)
ko(Bool)
n
(+ (fibonacci (- n 1)) (fibonacci (- n 2)))
ko(Bool)
n
n n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))
ko(Int) ko(Int)
(fibonacci (- n 2)) (fibonacci (- n 2))
ko(Int)
(fibonacci (- n 1))
ko(Int) ko(Int) (fibonacci (- n 1))
(fibonacci (- n 1))
(- n 1)
(fibonacci (- n 2)) (- n 1)
(- n 1)
(- n 2) (- n 2)
ko(Int) ko(Int)
ko(Int) ko(Int)
(- n 2) ko(Int)
ko(Int) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2))))
(<= n 1)
(<= n 1)
(<= n 1)
ko(Bool)
ko(Bool)
ko(Bool)
n
(+ (fibonacci (- n 1)) (fibonacci (- n 2))) n
n
(+ (fibonacci (- n 1)) (fibonacci (- n 2))) (+ (fibonacci (- n 1)) (fibonacci (- n 2)))
n
n
n
n n
ko(Int)
(fibonacci (- n 1))
ko(Int) ko(Int)
ko(Int)
ko(Int)
ko(Int)
ko(Int)
ko(Int) ko(Int)
(- n 1) (fibonacci (- n 2))
(fibonacci (- n 2))
(- n 1) (- n 1)
(fibonacci (- n 2))
(- n 2) (- n 2) ko(Int) (- n 2) (- n 2)
ko(Int) ko(Int)
(- n 2)
ko(Int) ko(Int)
ko(Int)
ko(Int) ko(Int)
(if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) (<= n 1) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2))))
(<= n 1)
(<= n 1)
ko(Bool)
ko(Bool)
(<= n 1)
ko(Bool) (+ (fibonacci (- n 1)) (fibonacci (- n 2))) (+ (fibonacci (- n 1)) (fibonacci (- n 2)))
n
n
n
n
(+ (fibonacci (- n 1)) (fibonacci (- n 2))) (+ (fibonacci (- n 1)) (fibonacci (- n 2))) n
ko(Bool)
n
n
ko(Int)
ko(Int)
ko(Int)
ko(Int) ko(Int)
ko(Int)
n
n (+ (fibonacci (- n 1)) (fibonacci (- n 2))) (+ (fibonacci (- n 1)) (fibonacci (- n 2))) n
ko(Int)
(fibonacci (- n 1))
ko(Int)
(fibonacci (- n 2))
(fibonacci (- n 2))
(fibonacci (- n 2))
ko(Int)
(fibonacci (- n 1))
ko(Int)
(fibonacci (- n 1))
ko(Int)
(- n 1)
(- n 1)
(- n 2) (- n 2)
(- n 2)
(- n 2)
(- n 2)
(- n 1)
(- n 1)
ko(Int)
ko(Int)
ko(Int)
ko(Int) ko(Int) ko(Int)
ko(Int)
ko(Int)
ko(Int)
(if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2))))
(if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2)))) (if (<= n 1) n (+ (fibonacci (- n 1)) (fibonacci (- n 2))))
(<= n 1) (<= n 1) (<= n 1)
ko(Bool) ko(Bool)
ko(Bool)
n n
n
(+ (fibonacci (- n 1)) (fibonacci (- n 2))) (+ (fibonacci (- n 1)) (fibonacci (- n 2))) (+ (fibonacci (- n 1)) (fibonacci (- n 2)))
n n
n n
n
n ko(Int)
(fibonacci (- n 1))
ko(Int)
ko(Int)
ko(Int) ko(Int)
ko(Int)
ko(Int) ko(Int)
ko(Int)
(fibonacci (- n 2))
(- n 1)
(- n 2)
ko(Int)
ko(Int)
(<= n 1)
ko(Bool)
n n
(fibonacci (- n 1))
ko(Int) ko(Int)
ko(Int)
(- n 1)
ko(Int)
( Ò ( f i b o n a c c i n )
( (<= n 1)
n
(+ ( f i b o n a c c i (− n 1))
( f i b o n a c c i (− n 2 ) ) ) ) )
( f i b o n a c c i 10)
# state: 276
# ﬁnal value: 1 (Int)
Sizes of successors:
# size 1: 231
# size 2: 13
# size 3: 3
# size 4: 2
# size 6: 12

View Slide

An Eﬃcient and
Parallel Abstract
Interpreter
in Scala
—
3×3 Parallel
Implementations
The problematic
Sequential
algorithm
Parallel
algorithms
Results
Todo
Worklist implementations
Possible (immutable) data structures to implement the worklist
(tested on Sergey/jfp/primtest.scm)
• list of states 2.304s (Scala code of SeqAAM)
s s s s s s
• set of states 2.466s (Scala code of SeqAAMS)
s s
s
s
s
s
• list of sets of states 2.315s (Scala code of SeqAAMLS)
s s s s s
s avoid useless concatenations
Each set corresponds to a result of the step function.
Selected for parallel implementations
and SeqAAMLS becomes the reference for comparisons.
These time results and following are averages on 10 repetitions (after 3
repetitions skipped), made on Intel Xeon Gold 6148 2.40GHz, 20 cores
available (VUB Hydra cluster), with Scala 2.12.7 – Akka 2.5.21 – Java
1.8.0 – GraalVM 1.0.0-rc14.

View Slide

An Eﬃcient and
Parallel Abstract
Interpreter
in Scala
—
3×3 Parallel
Implementations
The problematic
Sequential
algorithm
Parallel
algorithms
Results
Todo
4 Results
5 Todo

View Slide

An Eﬃcient and
Parallel Abstract
Interpreter
in Scala
—
3×3 Parallel
Implementations
The problematic
Sequential
algorithm
Parallel
algorithms
Results
Todo
First parallel implementation: ParAAM-L-SA-state
main thread SenderAggregator N = p − 1 Eval actors
shared data
w o r k l i s t =
w o r k l i s t ∪{ i n i t i a l s t a t e }
Ö Ô Ø
send c u r r e n t w o r k l i s t
wait
r e c e i v e new w o r k l i s t
ÓÖ s t a t e
from c u r r e n t w o r k l i s t
send s t a t e
update
N r e c e i v e d
send f i n i s h e d
e v a l u a t e s t a t e
send r e s u l t s
to Eval n◦ i%N
I name this implementation
ParAAM-L-SA-state, for ParAAM – Loop – SenderAggregator – state.

View Slide

An Eﬃcient and
Parallel Abstract
Interpreter
in Scala
—
3×3 Parallel
Implementations
The problematic
Sequential
algorithm
Parallel
algorithms
Results
Todo
First alternative: ParAAM-L-SA-set
shared data
w o r k l i s t =
Ö Ô Ø
wait
f o r e a c h s e t
send s e t
update
N r e c e i v e d
f o r e a c h s t a t e from s e t
send r e s u l t s
to Eval n◦ i%N
First improvement: send more than one state at a time. Set a set of states.
ParAAM-L-SA-set, for ParAAM – Loop – SenderAggregator – set.

View Slide

An Eﬃcient and
Parallel Abstract
Interpreter
in Scala
—
3×3 Parallel
Implementations
The problematic
Sequential
algorithm
Parallel
algorithms
Results
Todo
Second alternative: ParAAM-L-SA-part
shared data
w o r k l i s t =
Ö Ô Ø
wait
f o r e a c h e qual part
send part
update
N r e c e i v e d
f o r e a c h s t a t e from part
send r e s u l t s
to Eval n◦ i
Other possibility: send “equals” part of the worklist.
ParAAM-L-SA-part, for ParAAM – Loop – SenderAggregator – part.

View Slide

An Eﬃcient and
Parallel Abstract
Interpreter
in Scala
—
3×3 Parallel
Implementations
The problematic
Sequential
algorithm
Parallel
algorithms
Results
Todo
3×3 parallel implementations
ParAAM-L-SA-state ParAAM-C-S-state ParAAM-C-state state
SeqAAMLS
ParAAM-L-SA-set ParAAM-C-S-set ParAAM-C-set set
ParAAM-L-SA-part ParAAM-C-S-part ParAAM-C-part part
L-SA C-S C
Loop – SenderAggregator Concurrent – Sender Concurrent
1 special actor
p-1 eval actors
main loop (with a barrier)
SA send worklist
each eval actor send results to SA
1 special actor
p-1 eval actors
no barrier
S send worklist
each eval actor update
0 special actor
p eval actors
no barrier
each eval actor send worklist
each eval actor update

View Slide

An Eﬃcient and
Parallel Abstract
Interpreter
in Scala
—
3×3 Parallel
Implementations
The problematic
Sequential
algorithm
Parallel
algorithms
Results
Todo
Other strategy: ParAAM-C-S-state
main thread Sender N = p − 1 Eval actors
shared data
send s t a r t
wait send i n i t i a l s t a t e
ÓÖ ( state , a c t o r )
and a v a i l a b l e Eval a c t o r s
send s t a t e to a c t o r
w o r k l i s t i s empty
and a l l Eval a c t o r s a v a i l a b l e
update (with synchronization)
send a v a i l a b l e
to Eval n◦ 0
Eval n◦ i available
ParAAM-C-S-state, for ParAAM – Concurrent – Sender – state.
And these similar improvements:
ParAAM-C-S-set, for ParAAM – Concurrent – Sender – set and
ParAAM-C-S-part, for ParAAM – Concurrent – Sender – part.

View Slide

An Eﬃcient and
Parallel Abstract
Interpreter
in Scala
—
3×3 Parallel
Implementations
The problematic
Sequential
algorithm
Parallel
algorithms
Results
Todo
Last strategy: ParAAM-C-state
main thread N = p Eval actors
a c t o r s = empty l i s t
shared data
send s t a r t
wait send i n i t i a l s t a t e
update (with synchronization)
ÓÖ ( state , a c t o r )
and a v a i l a b l e Eval a c t o r s
send s t a t e to a c t o r
w o r k l i s t i s empty
and a l l Eval a c t o r s a v a i l a b l e
ParAAM-C-state, for ParAAM – Concurrent – state.
And these similar improvements:
ParAAM-C-set, for ParAAM – Concurrent – set and
ParAAM-C-part, for ParAAM – Concurrent – part.

View Slide

An Eﬃcient and
Parallel Abstract
Interpreter
in Scala
—
3×3 Parallel
Implementations
The problematic
Sequential
algorithm
Parallel
algorithms
Results
Todo
4 Results
5 Todo

View Slide

An Eﬃcient and
Parallel Abstract
Interpreter
in Scala
—
3×3 Parallel
Implementations
The problematic
Sequential
algorithm
Parallel
algorithms
Results
Todo
Computation time of all implementations
0
2
4
6
8
10
12
0 2 4 6 8 10 12 14 16 18 20
time (seconds)
p
SeqAAMLS
ParAAM-L-SA-state
ParAAM-L-SA-set
ParAAM-L-SA-part
ParAAM-C-S-state
ParAAM-C-S-set
ParAAM-C-S-part
ParAAM-C-state
ParAAM-C-set
ParAAM-C-part
Results computed on Sergey/jfp/primtest.scm
Statistic computed with SeqAAMLS
# state: 243472
# ﬁnal: 1 (Int)
# error: 1
Max sizes of successors: 6

View Slide

An Eﬃcient and
Parallel Abstract
Interpreter
in Scala
—
3×3 Parallel
Implementations
The problematic
Sequential
algorithm
Parallel
algorithms
Results
Todo
Speedup
0
2
4
6
8
10
0 2 4 6 8 10 12 14 16 18 20
speedup
p
0
0.5
1
1.5
2
2.5
3
0 2 4 6 8 10 12 14 16 18 20
time (seconds)
p
SeqAAMLS
ParAAM-L-SA-state
ParAAM-L-SA-set
ParAAM-L-SA-part
ParAAM-C-S-state
ParAAM-C-S-set
ParAAM-C-S-part
ParAAM-C-state
ParAAM-C-set
ParAAM-C-part
speedup = time of SeqAAMLS
time
(Super-linear speedup is unexplained for now!)

View Slide

An Efficient and
Parallel Abstract
Interpreter
in Scala
—
3×3 Parallel
Implementations
The problematic
Sequential
algorithm
Parallel
algorithms
Results
Todo
Efficiency
0
0.2
2 ¡
2 ¢
2 £
1
¤ ¥
¤ ¡
0 2 4 6 8 10 12 14 16 18 20
effiency
p
0
2
4
6
8
10
0 2 4 6 8 10 12 14 16 18 20
speedup
p
SeqAAMLS
ParAAM-L-SA-state
ParAAM-L-SA-set
ParAAM-L-SA-part
ParAAM-C-S-state
ParAAM-C-S-set
ParAAM-C-S-part
ParAAM-C-state
ParAAM-C-set
ParAAM-C-part
efficiency = speedup
p

View Slide

An Eﬃcient and
Parallel Abstract
Interpreter
in Scala
—
3×3 Parallel
Implementations
The problematic
Sequential
algorithm
Parallel
algorithms
Results
Todo
4 Results
5 Todo

View Slide

An Efficient and
Parallel Abstract
Interpreter
in Scala
—
3×3 Parallel
Implementations
The problematic
Sequential
algorithm
Parallel
algorithms
Results
Todo
Done / Todo
Implementation is (almost) done
(Need little checkings,
and maybe change the update of visited set for the worst implementations.)
Todo
Select which Scheme examples to use to produce final benchmarks
and estimate computation times
Run computations and collect data (benchmarks and some statistics)
Sort data and finalize analyzes
Write and write again
(some parts of theoretical background
can be taken from preparatory work)

View Slide

An Eﬃcient and
Parallel Abstract
Interpreter
in Scala
—
3×3 Parallel
Implementations
The problematic
Sequential
algorithm
Parallel
algorithms
Results
Todo
Thank you!
Questions time. . .
Implementation:
Scala-Par-AM – https:// Ø Ù ØºÓÖ »ÇÈ Å /scala-par-am/
This presentation on
https://×Ô Ö º ÓÑ»ÓÔ Ñ /
efficient-parallel-abstract-interpreter-in-scala-3x3-implementations
Document, L
A
TEX sources, other references and previous presentations on
https://bitbucket.org/OPiMedia/efficient-parallel-abstract-interpreter-in-scala

View Slide

Efficient Parallel Abstract Interpreter in Scala — 3x3 Parallel Implementations

Efficient Parallel Abstract Interpreter in Scala — 3x3 Parallel Implementations

🌳 Olivier Pirson — OPi 🇧🇪🇫🇷🇬🇧 🐧 👨‍💻 👨‍🔬

More Decks by 🌳 Olivier Pirson — OPi 🇧🇪🇫🇷🇬🇧 🐧 👨‍💻 👨‍🔬

Other Decks in Science

Featured

Transcript