PyPy Overview • PyPy is Python implemented in Python Interpreter (ANSI C) Python Program Interpreter (Python) Python Program CPython PyPy • Take the C version of the interpreter and rewrite it as a Python program.
rpython • PyPy is actually implemented in "rpython" • rpython is not an "interpreter", but a restricted subset of the Python language Python rpython • It can run as valid Python code, but that's about the only similarity
rpython • rpython is a completely different language • Python syntax, yes. • Must be compiled (like C, C++, etc.) • Static typing via type inference • Very different than anything you're used to
R is for Restricted • rpython allows no dynamic typing def add(x,y): return x+y def main(argv): r1 = add(2,3) # Ok r2 = add("Hello","World") # Error return 0 • Functions can only have one type signature
R is for Restricted • Containers can only have a single type numbers = [1,2,3,4,5] # Ok items = [1, "Hello", 3.5] # Error names = { # Ok 'dabeaz' : 'David Beazley', 'gaynor' : 'Alex Gaynor', } record = { # Error 'name' : 'ACME', 'shares' : 100 } • Think C, not Python.
R is for Restricted • Attributes can only be a single type class Pair(object): def __init__(self,x,y): self.x = x self.y = y a = Pair(2,3) # OK (first use) b = Pair("Hello","World") # Error • Again, think C
Looking at the C Code • rpython generates C code and places it into a temporary directory ... [translation:info] usession directory: /var/folders/M7/ M7Q2OurGGbezUFSLGEgQZ++++TI/-Tmp-/usession-unknown-0 [translation:info] created: /Users/beazley/Desktop/PyPyResearch/fib-c [Timer] Timings: [Timer] annotate --- 2.2 s [Timer] rtype_lltype --- 1.8 s [Timer] backendopt_lltype --- 1.2 s [Timer] stackcheckinsertion_lltype --- 0.0 s [Timer] database_c --- 16.7 s [Timer] source_c --- 2.8 s [Timer] compile_c --- 2.2 s [Timer] ========================================= [Timer] Total: --- 26.8 s bash %
Looking at the C Code • Go look for the "testing_1" directory bash % cd /var/folders/M7/M7Q2OurGGbezUFSLGEgQZ++++TI/-Tmp-/usession-unknown-0 bash % cd testing_1 bash % ls *.c data_objspace_flow_specialcase.c data_rlib_rdtoa.c data_rlib_rposix.c data_rlib_rstack.c data_rlib_rstack_1.c data_rpython_lltypesystem_rffi.c data_rpython_lltypesystem_rlist.c data_rpython_memory_gc_env.c data_rpython_memory_gc_minimark.c data_rpython_memory_gc_minimark_1.c data_rpython_memory_gctransform_framework.c debug_print.c implement.c nonfuncnodes.c objspace_flow_specialcase.c profiling.c ...
Essential Files • Here's where most of the generated code from your program gets placed • implement.c (functions) • nonfuncnodes.c (globals) • structdef.h (data structures) • Look at them if you dare... yes.
Experimental Coding • You can try different things in rpython and go look at the output C code • A bit of a challenge • But interesting to study what happens
Accessing C Code • If everything in PyPy is written in rpython, how does it access low-level C libraries? • os modules • time functions • math functions • General question: How would you access C code from any Python program?
rpython rffi • Foreign Function Interface from pypy.rpython.lltypesystem import rffi sin = rffi.llexternal("sin", [rffi.DOUBLE], rffi.DOUBLE) cos = rffi.llexternal("cos", [rffi.DOUBLE], rffi.DOUBLE) ... • Declares external C functions with types • Can use in your rpython program y = sin(x) + cos(x) ... • Instructive to look at low-level C code
rffi Commentary • The rpython rffi is highly developed • Most C primitive datatypes • Arrays • Structures • Pointers • Memory management • (More advanced example shortly)
Configuration System • There is a C compilation/configuration system from pypy.translator.tool.cbuild import ExternalCompilationInfo from pypy.rpython.tool import rffi_platform as platform class CConfig: _compilation_info_ = ExternalCompilationInfo( includes = ['math.h'], libraries = ['m'], ) M_PI = platform.DefinedConstantDouble('M_PI') M_E = platform.DefinedConstantDouble('M_E') config = platform.configure(CConfig) M_PI = config['M_PI'] M_E = config['M_E']
Configuration System • There is a C compilation/configuration system from pypy.translator.tool.cbuild import ExternalCompilationInfo from pypy.rpython.tool import rffi_platform as platform class CConfig: _compilation_info_ = ExternalCompilationInfo( includes = ['math.h'], libraries = ['m'], ) M_PI = platform.DefinedConstantDouble('M_PI') M_E = platform.DefinedConstantDouble('M_E') config = platform.configure(CConfig) M_PI = config['M_PI'] M_E = config['M_E'] C compiler specification (includes, libraries, paths, etc.)
Configuration System • There is a C compilation/configuration system from pypy.translator.tool.cbuild import ExternalCompilationInfo from pypy.rpython.tool import rffi_platform as platform class CConfig: _compilation_info_ = ExternalCompilationInfo( includes = ['math.h'], libraries = ['m'], ) M_PI = platform.DefinedConstantDouble('M_PI') M_E = platform.DefinedConstantDouble('M_E') config = platform.configure(CConfig) M_PI = config['M_PI'] M_E = config['M_E'] Some "queries" for things you want to know from C
Configuration System • There is a C compilation/configuration system from pypy.translator.tool.cbuild import ExternalCompilationInfo from pypy.rpython.tool import rffi_platform as platform class CConfig: _compilation_info_ = ExternalCompilationInfo( includes = ['math.h'], libraries = ['m'], ) M_PI = platform.DefinedConstantDouble('M_PI') M_E = platform.DefinedConstantDouble('M_E') config = platform.configure(CConfig) M_PI = config['M_PI'] M_E = config['M_E'] Run the C compiler and get results back
Configuration Comments • There is no centralized "configuration" • Individual program modules simply request information from the C compilation environment whenever they need it • Somehow (magically), the system will invoke the C compiler as needed. • It hurts my head...
Now What?!? • We haven't even talked about PyPy yet! • .... or the JIT • Implemented in rpython • So, all of this is just a starting point • More talks? (maybe)