Cython vs SWIG, Fight! by Mark Kohler

Slide 1

Slide 1 text

Cython vs. SWIG, Fight! Mark Kohler 2013-03-16 Wrapping C libraries for Python

Slide 2

Slide 2 text

Pre-fight ● import statement ● libadder ● passing ints ● passing structs ● C strings ● memory management ● generalizations

Slide 3

Slide 3 text

Code first and ask questions later 1.C code 2.SWIG code 3.SWIG demonstration 4.Cython code 5.Cython demonstration

Slide 4

Slide 4 text

We are Here ● import statement ● libadder ● passing ints ● passing structs ● C strings ● memory management ● generalizations

Slide 5

Slide 5 text

import this >>> import socket >>> import datetime >>> import time ● What is socket.__file__? ● What is datetime.__file__? ● What is time.__file__?

Slide 6

Slide 6 text

import socket >>> socket.__file__ '/usr/lib/python2.7/socket.pyc' >>>

Slide 7

Slide 7 text

import datetime >>> datetime.__file__ '/usr/lib/python2.7/lib- dynload/datetime.so' >>> $ file /usr/lib/python2.7/lib- dynload/datetime.so ELF 64-bit LSB shared object, x86-64, dynamically linked $

Slide 8

Slide 8 text

import time >>> time.__file__ Traceback (most recent call last): File "", line 1, in AttributeError: 'module' object has no attribute '__file__' >>>

Slide 9

Slide 9 text

We are Here ● import statement ● libadder ● passing ints ● passing structs ● C strings ● memory management ● generalizations

Slide 10

Slide 10 text

adder.c: add() int add(int x, int y) { return x + y; }

Slide 11

Slide 11 text

adder.h: add() int add(int x, int y);

Slide 12

Slide 12 text

Building libadder.so 1.Compile adder.c + adder.h --> adder.o 2.Link adder.o --> libadder.so

Slide 13

Slide 13 text

We are Here ● import statement ● libadder ● passing ints ● passing structs ● C strings ● memory management ● generalizations

Slide 14

Slide 14 text

adder.i (SWIG interface file) %module adder %{ #include "adder.h" %} int add(int, int);

Slide 15

Slide 15 text

SWIG build diagram 1.Start with: adder.h, libadder.so 2.SWIG adder.h + adder.i --> adder_wrap.c + adder.py 3.Compile adder.h + adder-wrap.c --> adder_wrap.o 4.Link libadder.so + adder_wrap.o --> _adder.so

Slide 16

Slide 16 text

demo of SWIG's add() >>> import adder >>> adder.add(2, 3) 5 >>>

Slide 17

Slide 17 text

c_adder.pxd: Cython interface file cdef extern from "adder.h": int add(int x, int y)

Slide 18

Slide 18 text

cy_adder.pyx: Cython source file cimport c_adder def add(x, y): return c_adder.add(x, y)

Slide 19

Slide 19 text

Cython build diagram 1.Cython adder.h + c_adder.pxd + cy_adder.pyx --> cy_adder.c 2.Compile adder.h + cy_adder.c --> cy_adder.o 3.Link lib_adder.so + cy_adder.o --> cy_adder.so

Slide 20

Slide 20 text

demo of Cython's add() >>> import cy_adder >>> cy_adder.add(2, 3) 5 >>>

Slide 21

Slide 21 text

Cython build review 1.Given: adder.h, libadder.so 2.Cython adder.h + c_adder.pxd + cy_adder.pyx --> cy_adder.c 3.Compile adder.h + cy_adder.c --> cy_adder.o 4.Link libadder.so + cy_adder.o --> cy_adder.so

Slide 22

Slide 22 text

We are Here ● import statement ● libadder ● passing ints ● passing structs ● C strings ● memory management ● generalizations

Slide 23

Slide 23 text

adder.h: pair_add() typedef struct _PAIR { int x; int y; } PAIR; int pair_add(PAIR * ppair);

Slide 24

Slide 24 text

adder.c: pair_add() int pair_add(PAIR * ppair) { return ppair->x + ppair->y; }

Slide 25

Slide 25 text

adder.i: pair_add() typedef struct _PAIR { int x; int y; } PAIR; int pair_add(PAIR * ppair);

Slide 26

Slide 26 text

demo of SWIG's pair_add() >>> import adder >>> my_pair = adder.PAIR() >>> type(my_pair) >>> my_pair.x = 3 >>> my_pair.y = 4 >>> adder.pair_add(my_pair) 7 >>>

Slide 27

Slide 27 text

c_adder.pxd: pair_add() ctypedef struct PAIR: int x int y int pair_add(PAIR * ppair)

Slide 28

Slide 28 text

cy_adder.pyx: pair_add() def pair_add(x, y): cdef c_adder.PAIR my_pair my_pair.x = x my_pair.y = y return c_adder.pair_add(&my_pair)

Slide 29

Slide 29 text

demo of Cython's pair_add() >>> import cy_adder >>> cy_adder.pair_add(3, 4) 7 >>>

Slide 30

Slide 30 text

We are Here ● import statement ● libadder ● passing ints ● passing structs ● C strings ● memory management ● generalizations

Slide 31

Slide 31 text

adder.h: get_version() char * get_version(void);

Slide 32

Slide 32 text

adder.c: get_version() static char version[] = "v1.0"; char * get_version(void) { return version; }

Slide 33

Slide 33 text

adder.i: get_version() char * get_version(void);

Slide 34

Slide 34 text

demo of SWIG's get_version() >>> import adder >>> adder.get_version() 'v1.0' >>> adder.get_version().__class__ >>>

Slide 35

Slide 35 text

c_adder.pxd: get_version() char * get_version()

Slide 36

Slide 36 text

cy_adder.pyx: get_version() def get_version(): return c_adder.get_version()

Slide 37

Slide 37 text

demo of Cython's get_version() >>> import cy_adder >>> cy_adder.get_version() 'v1.0' >>> cy_adder.get_version().__class__ >>>

Slide 38

Slide 38 text

Cython and C Strings "C strings are slow and cumbersome" "...avoid using C strings where possible" "...more likely to introduce bugs"

Slide 39

Slide 39 text

SWIG and C Strings "The problems (and perils) of using char * are well-known. However, SWIG is not in the business of enforcing morality." SWIG documentation, Section 8.3 C String Handling

Slide 40

Slide 40 text

We are Here ● import statement ● libadder ● passing ints ● passing structs ● C strings ● memory management ● generalizations

Slide 41

Slide 41 text

adder.h: sgreeting() int sgreeting(char * name, char * outp, int buflen);

Slide 42

Slide 42 text

adder.c: sgreeting() static char hello[] = "Hello, "; int sgreeting(char * name, char * outp, int buflen) { if (outp && buflen) { if (buflen < (strlen(hello) + strlen(name) + 1)) { outp[0] = 0; return 0; } strcpy(outp, hello); strcat(outp, name); } return strlen(hello) + strlen(name); }

Slide 43

Slide 43 text

adder.i: sgreeting() %include "cstring.i" %cstring_output_maxsize(char * outp, int buflen); int sgreeting(char * name, char * outp, int buflen);

Slide 44

Slide 44 text

demo of SWIG's sgreeting() >>> import adder >>> adder.sgreeting("Monty", 100) [12, 'Hello, Monty'] >>>

Slide 45

Slide 45 text

c_adder.pxd: sgreeting() int sgreeting(char * name, char * output, int buflen)

Slide 46

Slide 46 text

cy_adder.pyx: sgreeting() def sgreeting(name): c_len = c_adder.sgreeting(name, 0, 0) py_str = 'x' * (c_len + 1) cdef char * c_str = py_str c_adder.sgreeting(name, c_str, len(py_str)) return c_str

Slide 47

Slide 47 text

demo of Cython's sgreeting() >>> import cy_adder >>> cy_adder.sgreeting("Monty") 'Hello, Monty' >>>

Slide 48

Slide 48 text

We are Here ● import statement ● libadder ● passing ints ● passing structs ● C strings ● memory management ● generalizations

Slide 49

Slide 49 text

SWIG Advantages ● Multi-language support ● More DRY than Cython

Slide 50

Slide 50 text

Cython Advantages ● It's Python and it's C ● explore performance trade-offs between C and Python

Slide 51

Slide 51 text

Alternatives to Cython and SWIG ● Python C/API http://docs.python.org/2/extending/ ● ctypes