Gccpy – Python Front-end to GCC Philip Herron - @redzor [email protected] - http://redbrain.co.uk

Introduction ● Fully ahead of time compiler for Pure Python code (no Rpython) ● Currently targeting Python 2.4 syntax (simple exception handling) ● Implemented on-top of GCC for a feature rich Middle and backend. ● Not using LLVM as it wasn't as mature back in 2009 when I was developing the idea ● I didn't like LLVM's code base at the time.

What is a Front End?

Why develop this ● GOAL: Is Jit compilation better than Ahead of time with current technologies and languages? – No one has really tested this fully but ends up in flame wars. ● GCCPY vs PyPy Basically. ● To learn and gain a deep understanding in Compilers and code- generation - Python argued to be slow ● Inspired by PHC a php compiler Paul Biggar's PHD optimizing dynamic languages ● Surely making python code native is faster right...? – This isn't necessarily true

History ● Always wanted a deep full project to get into and got accepted to Gsoc 2010 ● PHC - http://www.phpcompiler.org/ Php compiler: Paul Biggar – PHD thesis: http://paulbiggar.com/research/#phd-dissertation ● Wanted to learn how compilers and languages really worked ● Python is awesome, Paul also wanted to do this exact project from his google talk on his phd its on youtube search his name. ● In the end PHC Failed for many reasons I believe mostly because it was a home-brew compiler! – If you are to reuse GCC with all its lovely middle/back end optimizers we could have something special – Also why I can't stand compiler's which output C because its LAZY

Hello World $ gccpy -g -O2 -fpy-gen-main helloworld.py -o helloworld $ ./helloworld

Compiling Python – DOT IL ● Developed DOT_IL (Python with braces ;) ) – It is a dynamic typed language with some optimizations – constant folding at a highlevel – Sits on-top of GCC GENERIC which is very C like no control structure almost 3-address-code – Lowered to Gimple and into GCC middle-end optimizers and finally thr backend with target specific optimizations.

The whole Pipeline together!

Dynamic Typing ● What something points to! ● Basically looks similar to Cpython gpy_object_t, PYObject are very similar structures in principle. ● The runtime is more optimized to Ahead of time languages more runtime behaviour is compiled in rather than dynamically figured out at runtime. ● Objects are allocated at runtime into gpy_object_t

Memory Addressing

Issues ● Empty class declarations such as: class myClass: pass ● Won't be supported by gccpy

Latency and Speed ● Programmers have forgot how powerful C is! ● Programmers have forgot how powerful old hardware is. ● Add more resources argument is broken.

Demo 1 Hello World!..........

Your gonna make this face next

Structure of the Gccpy Front-End

Gccpy VS other's ● Nuitka is pretty much a clone of ShedSkin.... – They generate optimized, embedded python/C++ depending on libpython.so. ● Cython is awesome for mixing pure python + C – Dialect of python supports C types – Can be used to 'compile' python but its not really ● PyPy Jit'd python boasting lots of runtime optimization. ● Numba and parakeet extremely DSL specific

Compile Time vs Runtime ● EVERYTHING is already Jit'ed before running your program – Reduces the size of the software stack no intermediate Virtual Machine. ● Runtime optimizations on the language very hard to do. – Jit optimization mostly revolves around target cpu funky'ness ● GCC is already aware of this -mtune – Languages and Architectures don't lend themselves to current state of dynamic languages. –

Good and the Bad ● Developing this is a huge task! – Takes long time everything from scratch ● Generating sensible code – Jit's in principle are better idea for the future of dynamic languages but I don't agree with this in our current landscape – PyPy has a budget! – Ahead of time binaries are always going to be around – Embedded systems will always benefit from this type of project. – Native applications without need to change language

Gccpy Optimizations ● Global Dictionary Lookup offset's all calculated at compile time – No runtime lookups! ● Jump tables calculated at compile time with labels and goto's in the compiled code! – No runtime jump tables ● No middle-ish crap to manage like parsing ● No need to pseudo compile .pyc or manage the jit.

Project Status ● First milestone (lucy) accomplished – All basic core principles in place – List/dict/integers/string all types need their api filled out more community has been helping with that – Stdlib needs done (huge task) going to implement enough to bootstrap the rest of Cpython stdlib. ● __builtin__ sys etc...

Major Projects ● Module compilation with __init__.py (should modules be .so or .a or both?) ● Threading and multi-core no GIL (only should affect the gvs (gccpy global virtual stack)) ● Need a garbage Collector – bohem-gc used in gcj (gnu-java and objc/objc++) is meant to be slow but will do in the mean time I think)

Thanks! Twitter @redzor [email protected] http://redbrain.co.uk https://github.com/redbrain/gccpy http://gcc.gnu.org/wiki/PythonFrontEnd Questions?!