PyPy JIT Under the Hood

PyPy JIT under the hood
Antonio Cuni
PyCon UK 2012
September 28, 2012
antocuni (PyCon UK 2012) PyPy JIT under the hood September 28, 2012 1 / 29

View Slide

About me
PyPy core dev
PyPy py3k tech leader
pdb++, fancycompleter, ...
Consultant, trainer
You can hire me :-)
http://antocuni.eu

View Slide

About this talk
What is PyPy? (in 30 seconds)
Overview of tracing JITs
The PyPy JIT generator

View Slide

Part 0: What is PyPy?
RPython toolchain
subset of Python
ideal for writing VMs
JIT & GC for free
Python interpreter
written in RPython
Whatever (dynamic) language you want
smalltalk, prolog, javascript, ...

View Slide

Part 1
Overview of tracing JITs

View Slide

Compilers
When?
Batch or Ahead Of Time
Just In Time
How?
Static
Dynamic or Adaptive
What?
Method-based compiler
Tracing compiler
PyPy: JIT, Dynamic, Tracing

View Slide

Assumptions
Pareto Principle (80-20 rule)
the 20% of the program accounts for the 80% of the
runtime
hot-spots
Fast Path principle
optimize only what is necessary
fall back for uncommon cases
Most of runtime spent in loops
Always the same code paths (likely)

View Slide

Tracing JIT
Interpret the program as usual
Detect hot loops
Tracing phase
linear trace
Compiling
Execute
guards to ensure correctness
Proﬁt :-)

View Slide

Tracing JIT phases
Interpretation

View Slide

Tracing JIT phases
Interpretation
Tracing
hot loop detected

View Slide

Tracing JIT phases
Interpretation
Tracing
hot loop detected
Compilation

View Slide

Tracing JIT phases
Interpretation
Tracing
hot loop detected
Compilation
Running

View Slide

Tracing JIT phases
Interpretation
Tracing
hot loop detected
Compilation
Running
cold guard failed

View Slide

Tracing JIT phases
Interpretation
Tracing
hot loop detected
Compilation
Running
cold guard failed
entering compiled loop

View Slide

Tracing JIT phases
Interpretation
Tracing
hot loop detected
Compilation
Running
cold guard failed
guard failure → hot

View Slide

Tracing JIT phases
Interpretation
Tracing
hot loop detected
Compilation
Running
cold guard failed
guard failure → hot
hot guard failed

View Slide

Tracing Example (1)
java
interface Operation {
int DoSomething(int x);
}
class IncrOrDecr implements Operation {
public int DoSomething(int x) {
if (x < 0) return x-1;
else return x+1;
}
}
class tracing {
public static void main(String argv[]) {
int N = 100;
int i = 0;
Operation op = new IncrOrDecr();
while (i < N) {
i = op.DoSomething(i);
}
System.out.println(i);
}
}

View Slide

Tracing Example (2)
Java bytecode
class IncrOrDecr {
...
public DoSomething(I)I
ILOAD 1
IFGE LABEL_0
ILOAD 1
ICONST_1
ISUB
IRETURN
LABEL_0
ILOAD 1
ICONST_1
IADD
IRETURN
}
Java bytecode
class tracing {
...
public static main(
[Ljava/lang/String;)V
...
LABEL_0
ILOAD 2
ILOAD 1
IF_ICMPGE LABEL_1
ALOAD 3
ILOAD 2
INVOKEINTERFACE
Operation.DoSomething (I)I
ISTORE 2
GOTO LABEL_0
LABEL_1
...
}

View Slide

Tracing example (3)
INSTR: Instruction executed but not recorded
INSTR: Instruction added to the trace but not executed
Method Java code Trace Value

View Slide

Tracing example (3)
Main while (i < N) { ILOAD 2 3
ILOAD 1 100
IF ICMPGE LABEL 1 f alse
GUARD ICMPLT

View Slide

Tracing example (3)
ILOAD 1 100
GUARD ICMPLT
i = op.DoSomething(i); ALOAD 3 IncrOrDecr obj
ILOAD 2 3
INVOKEINTERFACE ...
GUARD CLASS(IncrOrDecr)

View Slide

Tracing example (3)
ILOAD 1 100
GUARD ICMPLT
ILOAD 2 3
INVOKEINTERFACE ...
DoSomething if (x < 0) ILOAD 1 3
IFGE LABEL 0 true
GUARD GE

View Slide

Tracing example (3)
ILOAD 1 100
GUARD ICMPLT
ILOAD 2 3
INVOKEINTERFACE ...
IFGE LABEL 0 true
GUARD GE
return x+1; ILOAD 1 3
ICONST 1 1
IADD 4
IRETURN

View Slide

Tracing example (3)
ILOAD 1 100
GUARD ICMPLT
ILOAD 2 3
INVOKEINTERFACE ...
IFGE LABEL 0 true
GUARD GE
return x+1; ILOAD 1 3
ICONST 1 1
IADD 4
IRETURN
Main ISTORE 2
i = op.DoSomething(i);
} GOTO LABEL 0
4

View Slide

Trace trees (1)
tracetree.java
public static void trace_trees() {
int a = 0;
int i = 0;
int N = 100;
while(i < N) {
if (i%2 == 0)
a++;
else
a*=2;
i++;
}
}

View Slide

Trace trees (2)
ILOAD 1
ILOAD 2
GUARD ICMPLT
ILOAD 1
ICONST 2
IREM
GUARD NE
ILOAD 0
ICONST 2
IMUL
ISTORE 0
IINC 1 1

View Slide

Trace trees (2)
ILOAD 1
ILOAD 2
GUARD ICMPLT
ILOAD 1
ICONST 2
IREM
GUARD NE
ILOAD 0
ICONST 2
IMUL
ISTORE 0
IINC 1 1
BLACKHOLE

View Slide

Trace trees (2)
ILOAD 1
ILOAD 2
GUARD ICMPLT
ILOAD 1
ICONST 2
IREM
GUARD NE
ILOAD 0
ICONST 2
IMUL
ISTORE 0
IINC 1 1
BLACKHOLE
INTERPRETER

View Slide

Trace trees (2)
ILOAD 1
ILOAD 2
GUARD ICMPLT
ILOAD 1
ICONST 2
IREM
GUARD NE
ILOAD 0
ICONST 2
IMUL
ISTORE 0
IINC 1 1

View Slide

Trace trees (2)
ILOAD 1
ILOAD 2
GUARD ICMPLT
ILOAD 1
ICONST 2
IREM
GUARD NE
ILOAD 0
ICONST 2
IMUL
ISTORE 0
IINC 1 1
IINC 0 1
IINC 1 1

View Slide

Part 2
The PyPy JIT generator

View Slide

General architecture
def LOAD_GLOBAL(self):
...
def STORE_FAST(self):
...
def BINARY_ADD(self):
...
RPYTHON

View Slide

...
...
...
RPYTHON
CODEWRITER

View Slide

...
...
...
RPYTHON
CODEWRITER
...
p0 = getfield_gc(p0, 'func_globals')
p2 = getfield_gc(p1, 'strval')
call(dict_lookup, p0, p2)
....
...
p0 = getfield_gc(p0, 'locals_w')
setarrayitem_gc(p0, i0, p1)
....
...
promote_class(p0)
i0 = getfield_gc(p0, 'intval')
promote_class(p1)
i2 = int_add(i0, i1)
if (overflowed) goto ...
p2 = new_with_vtable('W_IntObject')
setfield_gc(p2, i2, 'intval')
....
JITCODE

View Slide

...
...
...
RPYTHON
CODEWRITER
...
....
...
....
...
promote_class(p0)
promote_class(p1)
....
JITCODE
compile-time
runtime

View Slide

...
...
...
RPYTHON
CODEWRITER
...
....
...
....
...
promote_class(p0)
promote_class(p1)
....
JITCODE
compile-time
runtime
META-TRACER

View Slide

...
...
...
RPYTHON
CODEWRITER
...
....
...
....
...
promote_class(p0)
promote_class(p1)
....
JITCODE
compile-time
runtime
META-TRACER
OPTIMIZER

View Slide

...
...
...
RPYTHON
CODEWRITER
...
....
...
....
...
promote_class(p0)
promote_class(p1)
....
JITCODE
compile-time
runtime
META-TRACER
OPTIMIZER
BACKEND

View Slide

...
...
...
RPYTHON
CODEWRITER
...
....
...
....
...
promote_class(p0)
promote_class(p1)
....
JITCODE
compile-time
runtime
META-TRACER
OPTIMIZER
BACKEND
ASSEMBLER

View Slide

PyPy trace example
def fn():
c = a+b
...

View Slide

PyPy trace example
def fn():
c = a+b
...
LOAD_GLOBAL A
LOAD_GLOBAL B
BINARY_ADD
STORE_FAST C

View Slide

PyPy trace example
def fn():
c = a+b
...
LOAD_GLOBAL A
LOAD_GLOBAL B
BINARY_ADD
STORE_FAST C
...
...

View Slide

PyPy trace example
def fn():
c = a+b
...
LOAD_GLOBAL A
LOAD_GLOBAL B
BINARY_ADD
STORE_FAST C
...
...
...
...

View Slide

PyPy trace example
def fn():
c = a+b
...
LOAD_GLOBAL A
LOAD_GLOBAL B
BINARY_ADD
STORE_FAST C
...
...
...
...
...
guard_class(p0, W_IntObject)
i2 = int_add(00, i1)
guard_not_overflow()
...

View Slide

PyPy trace example
def fn():
c = a+b
...
LOAD_GLOBAL A
LOAD_GLOBAL B
BINARY_ADD
STORE_FAST C
...
...
...
...
...
i2 = int_add(00, i1)
guard_not_overflow()
...
...
....

View Slide

PyPy optimizer
intbounds
constant folding / pure operations
virtuals
string optimizations
heap (multiple get/setﬁeld, etc)
fﬁ
unroll

View Slide

Intbound optimization (1)
intbound.py
def fn():
i = 0
while i < 5000:
i += 2
return i

View Slide

Intbound optimization (2)
unoptimized
...
i17 = int_lt(i15, 5000)
guard_true(i17)
i19 = int_add_ovf(i15, 2)
guard_no_overflow()
...
optimized
...
i17 = int_lt(i15, 5000)
guard_true(i17)
i19 = int_add(i15, 2)
...
It works often
array bound checking
intbound info propagates all over the trace

View Slide

Virtuals (1)
virtuals.py
def fn():
i = 0
while i < 5000:
i += 2
return i

View Slide

Virtuals (2)
unoptimized
...
i1 = getfield_pure(p0, ’intval’)
i2 = int_add(i1, 2)
p3 = new(W_IntObject)
setfield_gc(p3, i2, ’intval’)
...
optimized
...
i2 = int_add(i1, 2)
...
The most important optimization (TM)
It works both inside the trace and across the loop
It works for tons of cases
e.g. function frames

View Slide

Constant folding (1)
constfold.py
def fn():
i = 0
while i < 5000:
i += 2
return i

View Slide

Constant folding (2)
unoptimized
...
i2 = getfield_pure(,
’intval’)
...
optimized
...
i3 = int_add(i1, 2)
...
It “ﬁnishes the job”
Works well together with other optimizations (e.g.
virtuals)
It also does “normal, boring, static” constant-folding

View Slide

Out of line guards (1)
outoﬂineguards.py
N = 2
def fn():
i = 0
while i < 5000:
i += N
return i

View Slide

Out of line guards (2)
unoptimized
...
quasiimmut_field(, ’val’)
guard_not_invalidated()
p0 = getfield_gc(, ’val’)
...
optimized
...
guard_not_invalidated()
...
i3 = int_add(i1, 2)
...
Python is too dynamic, but we don’t care :-)
No overhead in assembler code
Used a bit “everywhere”
Credits to Mark Shannon
for the name :-)

View Slide

Guards
guard_true
guard_false
guard_class
guard_no_overﬂow
guard_value

View Slide

Promotion
guard_value
specialize code
make sure not to overspecialize
example: type of objects
example: function code objects, ...

View Slide

Conclusion
PyPy is cool :-)
Any question?

View Slide

PyPy JIT Under the Hood

PyPy JIT Under the Hood

Antonio Cuni

More Decks by Antonio Cuni

Other Decks in Programming

Featured

Transcript