Андрей Светлов (Python Core Developer) - Оптимизация производительности при помощи Cython

1
OPTIMIZATION
WITH CYTHON
WHY? WHEN? HOW?
Andrew Svetlov
http://asvetlov.blogspot.com
[email protected]
http://asvetlov.github.io/optimization-moscow-2016/

View Slide

2
BIO
Use Python for more than 16 years
Python Core Developer since 2012
asyncio committer
aiohttp maintainer
Author of a dozen libraries under aio-libs
umbrella

View Slide

3 . 1
WHY?
It's cool!!!
Learning new tech
Take out a time on non-business
task
...

View Slide

3 . 2
TO SPEEDUP YOUR
CODE

View Slide

4 . 1
OPTIMIZATION
TECHNIQUES
Improve
algorithms
Use python
tweaks
Rewrite with C
Use Cython

View Slide

4 . 2
PREREQUISITES
Python implementation
exists
100% test coverage
Bottleneck is found by
pro ler

View Slide

4 . 3
CYTHONIZE ALL
10% SPEEDUP
FAILS BY MANY REASONS:
CYTHON != PYTHON

View Slide

5 . 1
DISTRIBUTION

View Slide

5 . 2
SETUP.PY
f
r
o
m s
e
t
u
p
t
o
o
l
s i
m
p
o
r
t s
e
t
u
p
f
r
o
m C
y
t
h
o
n
.
B
u
i
l
d i
m
p
o
r
t c
y
t
h
o
n
i
z
e
s
e
t
u
p
(
e
x
t
_
m
o
d
u
l
e
s = c
y
t
h
o
n
i
z
e
(
"
h
e
l
l
o
w
o
r
l
d
.
p
y
x
"
)
)

View Slide

5 . 3
PYXIMPORT
i
m
p
o
r
t p
y
x
i
m
p
o
r
t
p
y
x
i
m
p
o
r
t
.
i
n
s
t
a
l
l
(
)
i
m
p
o
r
t h
e
l
l
o
w
o
r
l
d

View Slide

6 . 1
STEP BY STEP

View Slide

6 . 2
EXAMPLE: WEBSOCKET MASK APPLYING
n
a
t
i
v
e
_
b
y
t
e
o
r
d
e
r = s
y
s
.
b
y
t
e
o
r
d
e
r
d
e
f _
w
e
b
s
o
c
k
e
t
_
m
a
s
k
_
p
y
t
h
o
n
(
m
a
s
k
, d
a
t
a
)
:
a
s
s
e
r
t i
s
i
n
s
t
a
n
c
e
(
d
a
t
a
, b
y
t
e
a
r
r
a
y
)
, d
a
t
a
a
s
s
e
r
t l
e
n
(
m
a
s
k
) =
= 4
, m
a
s
k
d
a
t
a
l
e
n = l
e
n
(
d
a
t
a
)
i
f d
a
t
a
l
e
n =
= 0
:
r
e
t
u
r
n b
y
t
e
a
r
r
a
y
(
)
d
a
t
a = i
n
t
.
f
r
o
m
_
b
y
t
e
s
(
d
a
t
a
, n
a
t
i
v
e
_
b
y
t
e
o
r
d
e
r
)
m
a
s
k = i
n
t
.
f
r
o
m
_
b
y
t
e
s
(
m
a
s
k * (
d
a
t
a
l
e
n /
/ 4
) + m
a
s
k
[
: d
a
t
a
l
e
n % 4
]
,
n
a
t
i
v
e
_
b
y
t
e
o
r
d
e
r
)
r
e
t
u
r
n (
d
a
t
a ^ m
a
s
k
)
.
t
o
_
b
y
t
e
s
(
d
a
t
a
l
e
n
, n
a
t
i
v
e
_
b
y
t
e
o
r
d
e
r
)

View Slide

6 . 3
NAIVE CYTHONIZING: 9% BOOST
$ c
y
t
h
o
n
a m
o
d
u
l
e
.
p
y
x
$ x
d
g

o
p
e
n m
o
d
u
l
e
.
h
t
m
l

View Slide

6 . 4
ADD TYPES
f
r
o
m c
p
y
t
h
o
n c
i
m
p
o
r
t P
y
B
y
t
e
s
_
A
s
S
t
r
i
n
g
f
r
o
m l
i
b
c
.
s
t
d
i
n
t c
i
m
p
o
r
t u
i
n
t
3
2
_
t
, u
i
n
t
6
4
_
t
, u
i
n
t
m
a
x
_
t
c
d
e
f e
x
t
e
r
n f
r
o
m "
P
y
t
h
o
n
.
h
"
:
c
h
a
r
* P
y
B
y
t
e
A
r
r
a
y
_
A
s
S
t
r
i
n
g
(
b
y
t
e
a
r
r
a
y b
a
) e
x
c
e
p
t N
U
L
L
d
e
f _
w
e
b
s
o
c
k
e
t
_
m
a
s
k
_
c
y
t
h
o
n
(
b
y
t
e
s m
a
s
k
, b
y
t
e
a
r
r
a
y d
a
t
a
)
:
c
d
e
f
:
P
y
_
s
s
i
z
e
_
t d
a
t
a
_
l
e
n
, i
u
n
s
i
g
n
e
d c
h
a
r * i
n
_
b
u
f
c
o
n
s
t u
n
s
i
g
n
e
d c
h
a
r * m
a
s
k
_
b
u
f
u
i
n
t
3
2
_
t u
i
n
t
3
2
_
m
s
k
u
i
n
t
6
4
_
t u
i
n
t
6
4
_
m
s
k

View Slide

6 . 5
WORK WITH RAW DATA BUFFERS
a
s
s
e
r
t l
e
n
(
m
a
s
k
) =
= 4
d
a
t
a
_
l
e
n = l
e
n
(
d
a
t
a
)
i
n
_
b
u
f = <
u
n
s
i
g
n
e
d c
h
a
r
*
>
P
y
B
y
t
e
A
r
r
a
y
_
A
s
S
t
r
i
n
g
(
d
a
t
a
)
m
a
s
k
_
b
u
f = <
c
o
n
s
t u
n
s
i
g
n
e
d c
h
a
r
*
>
P
y
B
y
t
e
s
_
A
s
S
t
r
i
n
g
(
m
a
s
k
)
u
i
n
t
3
2
_
m
s
k = (
<
u
i
n
t
3
2
_
t
*
>
m
a
s
k
_
b
u
f
)
[
0
]

View Slide

6 . 6
CONVERT IN-PLACE
w
h
i
l
e d
a
t
a
_
l
e
n >
= 4
:
(
<
u
i
n
t
3
2
_
t
*
>
i
n
_
b
u
f
)
[
0
] ^
= u
i
n
t
3
2
_
m
s
k
i
n
_
b
u
f +
= 4
d
a
t
a
_
l
e
n
= 4
f
o
r i i
n r
a
n
g
e
(
0
, d
a
t
a
_
l
e
n
)
:
i
n
_
b
u
f
[
i
] ^
= m
a
s
k
_
b
u
f
[
i
]
r
e
t
u
r
n d
a
t
a

View Slide

6 . 7
64 BIT OPTIMIZATION
i
f s
i
z
e
o
f
(
s
i
z
e
_
t
) >
= 8
:
u
i
n
t
6
4
_
m
s
k = u
i
n
t
3
2
_
m
s
k
u
i
n
t
6
4
_
m
s
k = (
u
i
n
t
6
4
_
m
s
k <
< 3
2
) | u
i
n
t
3
2
_
m
s
k
w
h
i
l
e d
a
t
a
_
l
e
n >
= 8
:
(
<
u
i
n
t
6
4
_
t
*
>
i
n
_
b
u
f
)
[
0
] ^
= u
i
n
t
6
4
_
m
s
k
i
n
_
b
u
f +
= 8
d
a
t
a
_
l
e
n
= 8

View Slide

6 . 8
AWARE CYTHONIZING: 20X BOOST
$ c
y
t
h
o
n
a m
o
d
u
l
e
.
p
y
x
$ x
d
g

o
p
e
n m
o
d
u
l
e
.
h
t
m
l

View Slide

7 . 1
USAGE AND
TESTING

View Slide

7 . 2
IMPORT
d
e
f _
w
e
b
s
o
c
k
e
t
_
m
a
s
k
_
p
y
t
h
o
n
(
m
a
s
k
, d
a
t
a
)
:
.
.
.
i
f b
o
o
l
(
o
s
.
e
n
v
i
r
o
n
.
g
e
t
(
'
A
I
O
H
T
T
P
_
N
O
_
E
X
T
E
N
S
I
O
N
S
'
)
)
:
w
e
b
s
o
c
k
e
t
_
m
a
s
k = _
w
e
b
s
o
c
k
e
t
_
m
a
s
k
_
p
y
t
h
o
n
e
l
s
e
:
t
r
y
:
f
r
o
m .
_
w
e
b
s
o
c
k
e
t i
m
p
o
r
t _
w
e
b
s
o
c
k
e
t
_
m
a
s
k
_
c
y
t
h
o
n
w
e
b
s
o
c
k
e
t
_
m
a
s
k = _
w
e
b
s
o
c
k
e
t
_
m
a
s
k
_
c
y
t
h
o
n
e
x
c
e
p
t I
m
p
o
r
t
E
r
r
o
r
: # p
r
a
g
m
a
: n
o c
o
v
e
r
w
e
b
s
o
c
k
e
t
_
m
a
s
k = _
w
e
b
s
o
c
k
e
t
_
m
a
s
k
_
p
y
t
h
o
n

View Slide

7 . 3
TESTING
c
l
a
s
s W
S
T
e
s
t
M
i
x
i
n
:
m
a
s
k = N
o
n
e
d
e
f t
e
s
t
_
a
(
s
e
l
f
)
:
s
e
l
f
.
a
s
s
e
r
t
E
q
u
a
l
(
s
e
l
f
.
m
a
s
k
(
.
.
.
, .
.
.
)
, .
.
.
)
c
l
a
s
s T
e
s
t
C
y
t
h
o
n
(
W
S
T
e
s
t
M
i
x
i
n
, u
n
i
t
t
e
s
t
.
T
e
s
t
C
a
s
e
)
:
m
a
s
k = _
w
e
b
s
o
c
k
e
t
_
m
a
s
k
_
c
y
t
h
o
n
c
l
a
s
s T
e
s
t
C
y
t
h
o
n
(
W
S
T
e
s
t
M
i
x
i
n
, u
n
i
t
t
e
s
t
.
T
e
s
t
C
a
s
e
)
:
m
a
s
k = _
w
e
b
s
o
c
k
e
t
_
m
a
s
k
_
p
y
t
h
o
n

View Slide

8
CYTHON
PROFILING AND
COVERAGE

View Slide

9
QUESTIONS?
Andrew Svetlov
http://asvetlov.blogspot.com
[email protected]
http://asvetlov.github.io/optimization-moscow-2016/

View Slide

Андрей Светлов (Python Core Developer) - Оптимизация производительности при помощи Cython

Андрей Светлов (Python Core Developer) - Оптимизация производительности при помощи Cython

Moscow Python Meetup
PRO

More Decks by Moscow Python Meetup

Other Decks in Programming

Featured

Transcript

Андрей Светлов (Python Core Developer) - Оптимизация производительности при помощи Cython

Андрей Светлов (Python Core Developer) - Оптимизация производительности при помощи Cython

Moscow Python Meetup PRO

More Decks by Moscow Python Meetup

Other Decks in Programming

Featured

Transcript

Moscow Python Meetup
PRO