Tweet
Tweet

Slide 1

Slide 1 text

PyPy + Swift September 4th, 2013 Wednesday, September 4, 13

Slide 2

Slide 2 text

Me • Software engineer at Rackspace • PyPy, Django, OpenStack, and more hacker • PSF board member • PyCon 2014 program committee co-chair Wednesday, September 4, 13

Slide 3

Slide 3 text

What is PyPy? Wednesday, September 4, 13

Slide 4

Slide 4 text

Alexanders-MacBook-Pro:~ alex_gaynor$ python >>> print 42 42 >>> ^D Alexanders-MacBook-Pro:~ alex_gaynor$ pypy >>>> from __pypy__ import do_what_I_mean >>>> do_what_I_mean() 42 Wednesday, September 4, 13

Slide 5

Slide 5 text

Why do you care? Wednesday, September 4, 13

Slide 6

Slide 6 text

Faster (JIT) Wednesday, September 4, 13

Slide 7

Slide 7 text

Less memory (maybe) Wednesday, September 4, 13

Slide 8

Slide 8 text

CPU-bound parallelism (someday soon) Wednesday, September 4, 13

Slide 9

Slide 9 text

Why do you care? Wednesday, September 4, 13

Slide 10

Slide 10 text

Why doesn’t everyone use this? • C-extensions • Wacky Python code • It’s faster except when it’s not Wednesday, September 4, 13

Slide 11

Slide 11 text

Steps to making Swift+PyPy work Wednesday, September 4, 13

Slide 12

Slide 12 text

Close some files Wednesday, September 4, 13

Slide 13

Slide 13 text

--- x.py! 2013-09-03 21:24:48.000000000 -0700 +++ t.py! 2013-09-03 21:24:54.000000000 -0700 @@ -1 +1,2 @@ -open(path, "w").write("my bad data I don't care about") +with open(path, "w") as f: + f.write("my super great data") Wednesday, September 4, 13

Slide 14

Slide 14 text

Close your database cursors Wednesday, September 4, 13

Slide 15

Slide 15 text

--- x.py!2013-09-03 21:30:15.000000000 -0700 +++ t.py!2013-09-03 21:30:05.000000000 -0700 @@ -1 +1,3 @@ -conn.execute("SELECT bad from cpython") +from contextlib import closing +with closing(conn.cursor()) as cursor: + cursor.execute("SELECT awesome FROM pypy") Wednesday, September 4, 13

Slide 16

Slide 16 text

json.loads() is always unicode Wednesday, September 4, 13

Slide 17

Slide 17 text

Don’t monkeypatch __del__ Wednesday, September 4, 13

Slide 18

Slide 18 text

Fix a few bugs in PyPy and Eventlet [images not available] Wednesday, September 4, 13

Slide 19

Slide 19 text

Missing os.statvfs Wednesday, September 4, 13

Slide 20

Slide 20 text

Bug with file.seek() and file.tell() Wednesday, September 4, 13

Slide 21

Slide 21 text

GreenSockets not being closed correctly Wednesday, September 4, 13

Slide 22

Slide 22 text

Current stat of play: The monster is lives! Wednesday, September 4, 13

Slide 23

Slide 23 text

Lies, damned lies, and benchmarks Wednesday, September 4, 13

Slide 24

Slide 24 text

+ .client-venv/bin/swift-bench bench.conf swift-bench 2013-08-29 21:46:29,698 INFO Auth version: 1.0 swift-bench 2013-08-29 21:46:30,115 INFO Auth version: 1.0 swift-bench 2013-08-29 21:46:32,134 INFO 161 PUTS [0 failures], 79.9/s swift-bench 2013-08-29 21:46:41,377 INFO 1000 PUTS **FINAL** [0 failures], 88.8/s swift-bench 2013-08-29 21:46:41,377 INFO Auth version: 1.0 swift-bench 2013-08-29 21:46:43,380 INFO 672 GETS [0 failures], 335.9/s swift-bench 2013-08-29 21:46:58,385 INFO 6071 GETS [0 failures], 357.0/s swift-bench 2013-08-29 21:47:13,389 INFO 11407 GETS [0 failures], 356.4/s swift-bench 2013-08-29 21:47:28,389 INFO 16827 GETS [0 failures], 358.0/s swift-bench 2013-08-29 21:47:43,393 INFO 22119 GETS [0 failures], 356.7/s swift-bench 2013-08-29 21:47:58,394 INFO 27527 GETS [0 failures], 357.4/s swift-bench 2013-08-29 21:48:13,401 INFO 32925 GETS [0 failures], 357.8/s swift-bench 2013-08-29 21:48:28,401 INFO 38386 GETS [0 failures], 358.7/s swift-bench 2013-08-29 21:48:43,408 INFO 43747 GETS [0 failures], 358.5/s swift-bench 2013-08-29 21:48:58,409 INFO 49194 GETS [0 failures], 359.0/s swift-bench 2013-08-29 21:49:13,411 INFO 54573 GETS [0 failures], 359.0/s swift-bench 2013-08-29 21:49:28,413 INFO 59993 GETS [0 failures], 359.2/s swift-bench 2013-08-29 21:49:43,419 INFO 65342 GETS [0 failures], 358.9/s swift-bench 2013-08-29 21:49:58,431 INFO 70767 GETS [0 failures], 359.1/s swift-bench 2013-08-29 21:50:13,435 INFO 76149 GETS [0 failures], 359.1/s swift-bench 2013-08-29 21:50:28,436 INFO 81552 GETS [0 failures], 359.2/s swift-bench 2013-08-29 21:50:43,446 INFO 86947 GETS [0 failures], 359.2/s swift-bench 2013-08-29 21:50:58,454 INFO 92377 GETS [0 failures], 359.3/s swift-bench 2013-08-29 21:51:13,458 INFO 97742 GETS [0 failures], 359.2/s swift-bench 2013-08-29 21:51:19,617 INFO 100000 GETS **FINAL** [0 failures], 359.4/s swift-bench 2013-08-29 21:51:19,618 INFO Auth version: 1.0 swift-bench 2013-08-29 21:51:21,627 INFO 223 DEL [0 failures], 111.1/s swift-bench 2013-08-29 21:51:28,500 INFO 1000 DEL **FINAL** [0 failures], 112.6/s swift-bench 2013-08-29 21:51:28,500 INFO Auth version: 1.0 Wednesday, September 4, 13

Slide 25

Slide 25 text

+ .client-venv/bin/swift-bench bench.conf swift-bench 2013-08-30 19:48:08,159 INFO Auth version: 1.0 swift-bench 2013-08-30 19:48:09,096 INFO Auth version: 1.0 swift-bench 2013-08-30 19:48:11,104 INFO 66 PUTS [0 failures], 32.9/s swift-bench 2013-08-30 19:48:26,134 INFO 702 PUTS [0 failures], 41.2/s swift-bench 2013-08-30 19:48:31,961 INFO 1000 PUTS **FINAL** [0 failures], 43.7/s swift-bench 2013-08-30 19:48:31,961 INFO Auth version: 1.0 swift-bench 2013-08-30 19:48:33,968 INFO 359 GETS [0 failures], 179.2/s swift-bench 2013-08-30 19:48:48,979 INFO 4021 GETS [0 failures], 236.3/s swift-bench 2013-08-30 19:49:03,982 INFO 9010 GETS [0 failures], 281.4/s swift-bench 2013-08-30 19:49:18,983 INFO 14465 GETS [0 failures], 307.6/s swift-bench 2013-08-30 19:49:33,986 INFO 20732 GETS [0 failures], 334.3/s swift-bench 2013-08-30 19:49:48,986 INFO 27925 GETS [0 failures], 362.6/s swift-bench 2013-08-30 19:50:03,989 INFO 35434 GETS [0 failures], 385.1/s swift-bench 2013-08-30 19:50:18,994 INFO 42956 GETS [0 failures], 401.4/s swift-bench 2013-08-30 19:50:33,994 INFO 50736 GETS [0 failures], 415.8/s swift-bench 2013-08-30 19:50:48,995 INFO 58494 GETS [0 failures], 426.9/s swift-bench 2013-08-30 19:51:03,996 INFO 66256 GETS [0 failures], 435.8/s swift-bench 2013-08-30 19:51:18,998 INFO 73875 GETS [0 failures], 442.3/s swift-bench 2013-08-30 19:51:33,999 INFO 81612 GETS [0 failures], 448.3/s swift-bench 2013-08-30 19:51:49,000 INFO 89374 GETS [0 failures], 453.6/s swift-bench 2013-08-30 19:52:04,002 INFO 97165 GETS [0 failures], 458.2/s swift-bench 2013-08-30 19:52:09,564 INFO 100000 GETS **FINAL** [0 failures], 459.6/s swift-bench 2013-08-30 19:52:09,564 INFO Auth version: 1.0 swift-bench 2013-08-30 19:52:11,586 INFO 124 DEL [0 failures], 61.4/s swift-bench 2013-08-30 19:52:22,428 INFO 1000 DEL **FINAL** [0 failures], 77.8/s swift-bench 2013-08-30 19:52:22,428 INFO Auth version: 1.0 Wednesday, September 4, 13

Slide 26

Slide 26 text

~30% faster on GET unknown for PUT or DELETE Wednesday, September 4, 13

Slide 27

Slide 27 text

Warmup time suck Wednesday, September 4, 13

Slide 28

Slide 28 text

Lots of room for improvement (might help CPython as well) Wednesday, September 4, 13

Slide 29

Slide 29 text

Future directions Wednesday, September 4, 13

Slide 30

Slide 30 text

More faster. Wednesday, September 4, 13

Slide 31

Slide 31 text

Less syscalls Wednesday, September 4, 13

Slide 32

Slide 32 text

More careful attention to buffer allocation/ copying Wednesday, September 4, 13

Slide 33

Slide 33 text

What to do when PyPy and CPython diverge? Wednesday, September 4, 13

Slide 34

Slide 34 text

Get a PyPy release so this can get Wednesday, September 4, 13

Slide 35

Slide 35 text

Better benchmarking Wednesday, September 4, 13

Slide 36

Slide 36 text

Thanks! Questions? https://speakerdeck.com/alex Wednesday, September 4, 13