Cython
➤ Cython is a source-to-source compiler (a.k.a. transcompiler).
➤ Cython is a superset of Python.
➤ Provides optional static type declarations.
➤ Makes writing C extensions for Python easier.
➤ Makes Python faster by pre-compiling and static type.
➤ Sometimes faster by orders of magnitude.
➤ The pandas uses Cython!
2
Slide 3
Slide 3 text
.so
C compiler
.c
.py import
Python
.pyx
Cython
Slide 4
Slide 4 text
Mosky
➤ Python Charmer at Pinkoi.
➤ Has spoken at: PyCons in
TW, MY, KR, JP
, SG, HK,
COSCUPs, and TEDx, etc.
➤ Countless hours
on teaching Python.
➤ Own the Python packages like
ZIPCodeTW.
➤ http://mosky.tw/
4
Slide 5
Slide 5 text
Outline
1. Setup
2. Foundation
3. Practicing
4. Tips
5. Uncovered Topics
6. Other Solutions
5
Using C Lib
from libc.math cimport sin
# or
cdef extern from "math.h":
double sin(double x)
21
Slide 22
Slide 22 text
Function Visibility
22
Same
File
Other
.pyx
.py
Funcs in .h/.c
Visible
directly
Visible
via cdef extern
Invisible
cdef
Visible
via .pxd &
cimport
cpdef
Visible
via import
def
Slide 23
Slide 23 text
Binding During
23
Same
File
Other
.pyx
.py
Funcs in .h/.c
compile-time
compile-time x
cdef
cpdef
run-time
def
Slide 24
Slide 24 text
Type Conversions
24
C From Py To Py
short, int, long int int
float, double int/float float
char* str/bytes bytes
array iterable list
struct dict
Slide 25
Slide 25 text
Practicing
Slide 26
Slide 26 text
The Suggestion
➤ LIB_NAME.pyx
➤ Has functions.
➤ test_LIB_NAME.py
➤ Call the functions.
26
Slide 27
Slide 27 text
The Overflow
➤ Static types may also overflow in Cython silently.
➤ Try to make an overflow!
➤ In C:
➤ http://j.mp/test_overflow_in_c_c
➤ In Cython:
➤ http://j.mp/overflow_in_pyx_pyx
27
Slide 28
Slide 28 text
Using C Functions
➤ Try to use the functions in C.
➤ Playing with fork, the system call, may be fun.
➤ In C:
➤ http://j.mp/test_fork_c
➤ In Cython:
➤ http://j.mp/fork_in_pyx_pyx
28
Slide 29
Slide 29 text
The Functions
➤ Write three functions
defined in
def, cdef, and cpdef.
➤ Try to call them in:
➤ The same file,
➤ Another .pyx file,
➤ And a .py file.
➤ Refer to the table,
“Function Visibility”.
➤ The Cython code:
➤ http://j.mp/
lib_in_pyx_pyx
➤ http://j.mp/
use_lib_in_pyx_pyx
➤ http://j.mp/
test_lib_in_pyx_py
29
Slide 30
Slide 30 text
Tips
Slide 31
Slide 31 text
cython -a
➤ cython -a NAME.pyx
➤ cython -a -3 NAME.pyx
➤ open NAME.html
31
Uncovered Topics
➤ Differences between C and Cython expressions
➤ http://docs.cython.org/src/userguide/
language_basics.html#differences-between-c-and-cython-
expressions
➤ Propagating Exceptions in cdef
➤ http://docs.cython.org/src/userguide/
language_basics.html#error-return-values
35
Slide 36
Slide 36 text
➤ Extension Type — cdef class
➤ http://docs.cython.org/src/userguide/
extension_types.html
➤ Generic programming using Cython's Template
➤ http://docs.cython.org/src/userguide/fusedtypes.html
➤ Conditional Compilation
➤ http://docs.cython.org/src/userguide/
language_basics.html#conditional-compilation
36
Slide 37
Slide 37 text
➤ Profiling
➤ http://docs.cython.org/src/tutorial/profiling_tutorial.html
➤ Parallelism (No GIL + OpenMP)
➤ http://docs.cython.org/src/userguide/parallelism.html
➤ Using C++ in Cython
➤ http://docs.cython.org/src/userguide/
wrapping_CPlusPlus.html
37
Slide 38
Slide 38 text
Other Solutions
Slide 39
Slide 39 text
No content
Slide 40
Slide 40 text
Other Solutions
➤ Numba: translates Python to optimized machine code.
➤ @jit
➤ CFFI: calls C code in Python dynamically.
➤ Cython versus CFFI – fuzzy notepad
➤ pybind11: makes C++ code available in Python.
➤ Cython, pybind11, cffi – which tool should you choose? –
Setfans Welt
➤ Dask: provides advanced parallelism for analytics.
➤ Comparison to Spark – Dask
➤ panda and Enhancing performance – pandas.
40
Slide 41
Slide 41 text
Cool Down
➤ Did you use NumPy and SciPy efficiently?
➤ Profile your program.
➤ Consider the portability — you are writing C programs!
➤ Algorithm still does matter.
➤ Pick the most suitable solution.
41
Slide 42
Slide 42 text
Recap
➤ static types by cdef and in function definition.
➤ cdef function & cpdef function.
➤ .pxd exposes cdef functions.
➤ cdef extern for C functions.
42