Short bio Short bio PyPy core dev since 2006 pdb++, CFFI, vmprof, capnpy, ... @antocuni https://github.com/antocuni (https://github.com/antocuni) https://bitbucket.org/antocuni (https://bitbucket.org/antocuni)
obj = json.loads(string) for item in obj['data']: id = item['id'] name = item['name'] ingredients = [] for ingr in item["ingredients"]: ingredients.append(ingr['name'])
Example of temporary objects Example of temporary objects Bound methods Bound methods In [ ]: class A(object): def foo(self): return 42 a = A() bound_foo = a.foo %timeit a.foo() %timeit bound_foo()
Ideally Ideally Think of concepts, not implementation details Think of concepts, not implementation details Real world Real world Details leak to the user Details leak to the user
2. Work around in the language specs 2. Work around in the language specs range vs xrange dict.keys vs .iterkeys int vs long array.array vs list Easier to implement Harder to use Clutter the language unnecessarily More complex to understand Not really Pythonic
3. Stay in C as much as possible 3. Stay in C as much as possible In [29]: In [31]: numbers = range(1000) % timeit [x*2 for x in numbers] import numpy as np numbers = np.arange(1000) % timeit numbers*2 10000 loops, best of 3: 47.1 µs per loop The slowest run took 17.59 times longer than the fastest. This could mean that an intermediate result is being cached. 1000000 loops, best of 3: 1.48 µs per loop
Python in the HPC world Python in the HPC world Python as a glue-only language Python as a glue-only language Tradeo between speed and code quality Tradeo between speed and code quality
How fast is PyPy? How fast is PyPy? Wrong question Wrong question Up to 80x faster in extreme cases 10x faster in good cases 2x faster on "random" code sometime it's just slower
Example: Sobel lter Example: Sobel lter Extendend version "The Joy of PyPy: Abstractions for Free", EP 2017 https://speakerdeck.com/antocuni/the-joy-of-pypy-jit-abstractions-for-free (https://speakerdeck.com/antocuni/the-joy-of-pypy-jit-abstractions-for-free) https://www.youtube.com/watch?v=NQfpHQII2cU (https://www.youtube.com/watch?v=NQfpHQII2cU)
cpyext cpyext PyPy version of Python.h Compatibility layer Most C extensions just work: numpy, scipy, pandas, etc. Slow :( Use CFFI whenever it's possible
We are working on it We are working on it Future status (hopefully) Future status (hopefully) All C extensions will just work C code as fast as today, Python code super-fast The best of both worlds PyPy as the default choice for HPC My personal estimate: 6 months of work and we have a fast cpyext (let's talk about money :))