PyPy + Swift

As delivered at the Swift workshop in San Francisco.

Transcript

1. PyPy + Swift September 4th, 2013 Wednesday, September 4, 13
2. Me • Software engineer at Rackspace • PyPy, Django, OpenStack, and more hacker • PSF board member • PyCon 2014 program committee co-chair Wednesday, September 4, 13
3. What is PyPy? Wednesday, September 4, 13
4. Alexanders-MacBook-Pro:~ alex_gaynor$ python >>> print 42 42 >>> ^D Alexanders-MacBook-Pro:~ alex_gaynor$ pypy >>>> from __pypy__ import do_what_I_mean >>>> do_what_I_mean() 42 Wednesday, September 4, 13
5. Why do you care? Wednesday, September 4, 13
6. Faster (JIT) Wednesday, September 4, 13
7. Less memory (maybe) Wednesday, September 4, 13
8. CPU-bound parallelism (someday soon) Wednesday, September 4, 13
9. Why do you care? Wednesday, September 4, 13
10. Why doesn’t everyone use this? • C-extensions • Wacky Python code • It’s faster except when it’s not Wednesday, September 4, 13
11. Steps to making Swift+PyPy work Wednesday, September 4, 13
12. Close some files Wednesday, September 4, 13
13. --- x.py! 2013-09-03 21:24:48.000000000 -0700 +++ t.py! 2013-09-03 21:24:54.000000000 -0700 @@ -1 +1,2 @@ -open(path, "w").write("my bad data I don't care about") +with open(path, "w") as f: + f.write("my super great data") Wednesday, September 4, 13
14. Close your database cursors Wednesday, September 4, 13
15. --- x.py!2013-09-03 21:30:15.000000000 -0700 +++ t.py!2013-09-03 21:30:05.000000000 -0700 @@ -1 +1,3 @@ -conn.execute("SELECT bad from cpython") +from contextlib import closing +with closing(conn.cursor()) as cursor: + cursor.execute("SELECT awesome FROM pypy") Wednesday, September 4, 13
16. json.loads() is always unicode Wednesday, September 4, 13
17. Don’t monkeypatch __del__ Wednesday, September 4, 13
18. Fix a few bugs in PyPy and Eventlet [images not available] Wednesday, September 4, 13
19. Missing os.statvfs Wednesday, September 4, 13
20. Bug with file.seek() and file.tell() Wednesday, September 4, 13
21. GreenSockets not being closed correctly Wednesday, September 4, 13
22. Current stat of play: The monster is lives! Wednesday, September 4, 13
23. Lies, damned lies, and benchmarks Wednesday, September 4, 13
24. + .client-venv/bin/swift-bench bench.conf swift-bench 2013-08-29 21:46:29,698 INFO Auth version: 1.0 swift-bench 2013-08-29 21:46:30,115 INFO Auth version: 1.0 swift-bench 2013-08-29 21:46:32,134 INFO 161 PUTS [0 failures], 79.9/s swift-bench 2013-08-29 21:46:41,377 INFO 1000 PUTS **FINAL** [0 failures], 88.8/s swift-bench 2013-08-29 21:46:41,377 INFO Auth version: 1.0 swift-bench 2013-08-29 21:46:43,380 INFO 672 GETS [0 failures], 335.9/s swift-bench 2013-08-29 21:46:58,385 INFO 6071 GETS [0 failures], 357.0/s swift-bench 2013-08-29 21:47:13,389 INFO 11407 GETS [0 failures], 356.4/s swift-bench 2013-08-29 21:47:28,389 INFO 16827 GETS [0 failures], 358.0/s swift-bench 2013-08-29 21:47:43,393 INFO 22119 GETS [0 failures], 356.7/s swift-bench 2013-08-29 21:47:58,394 INFO 27527 GETS [0 failures], 357.4/s swift-bench 2013-08-29 21:48:13,401 INFO 32925 GETS [0 failures], 357.8/s swift-bench 2013-08-29 21:48:28,401 INFO 38386 GETS [0 failures], 358.7/s swift-bench 2013-08-29 21:48:43,408 INFO 43747 GETS [0 failures], 358.5/s swift-bench 2013-08-29 21:48:58,409 INFO 49194 GETS [0 failures], 359.0/s swift-bench 2013-08-29 21:49:13,411 INFO 54573 GETS [0 failures], 359.0/s swift-bench 2013-08-29 21:49:28,413 INFO 59993 GETS [0 failures], 359.2/s swift-bench 2013-08-29 21:49:43,419 INFO 65342 GETS [0 failures], 358.9/s swift-bench 2013-08-29 21:49:58,431 INFO 70767 GETS [0 failures], 359.1/s swift-bench 2013-08-29 21:50:13,435 INFO 76149 GETS [0 failures], 359.1/s swift-bench 2013-08-29 21:50:28,436 INFO 81552 GETS [0 failures], 359.2/s swift-bench 2013-08-29 21:50:43,446 INFO 86947 GETS [0 failures], 359.2/s swift-bench 2013-08-29 21:50:58,454 INFO 92377 GETS [0 failures], 359.3/s swift-bench 2013-08-29 21:51:13,458 INFO 97742 GETS [0 failures], 359.2/s swift-bench 2013-08-29 21:51:19,617 INFO 100000 GETS **FINAL** [0 failures], 359.4/s swift-bench 2013-08-29 21:51:19,618 INFO Auth version: 1.0 swift-bench 2013-08-29 21:51:21,627 INFO 223 DEL [0 failures], 111.1/s swift-bench 2013-08-29 21:51:28,500 INFO 1000 DEL **FINAL** [0 failures], 112.6/s swift-bench 2013-08-29 21:51:28,500 INFO Auth version: 1.0 Wednesday, September 4, 13
25. + .client-venv/bin/swift-bench bench.conf swift-bench 2013-08-30 19:48:08,159 INFO Auth version: 1.0 swift-bench 2013-08-30 19:48:09,096 INFO Auth version: 1.0 swift-bench 2013-08-30 19:48:11,104 INFO 66 PUTS [0 failures], 32.9/s swift-bench 2013-08-30 19:48:26,134 INFO 702 PUTS [0 failures], 41.2/s swift-bench 2013-08-30 19:48:31,961 INFO 1000 PUTS **FINAL** [0 failures], 43.7/s swift-bench 2013-08-30 19:48:31,961 INFO Auth version: 1.0 swift-bench 2013-08-30 19:48:33,968 INFO 359 GETS [0 failures], 179.2/s swift-bench 2013-08-30 19:48:48,979 INFO 4021 GETS [0 failures], 236.3/s swift-bench 2013-08-30 19:49:03,982 INFO 9010 GETS [0 failures], 281.4/s swift-bench 2013-08-30 19:49:18,983 INFO 14465 GETS [0 failures], 307.6/s swift-bench 2013-08-30 19:49:33,986 INFO 20732 GETS [0 failures], 334.3/s swift-bench 2013-08-30 19:49:48,986 INFO 27925 GETS [0 failures], 362.6/s swift-bench 2013-08-30 19:50:03,989 INFO 35434 GETS [0 failures], 385.1/s swift-bench 2013-08-30 19:50:18,994 INFO 42956 GETS [0 failures], 401.4/s swift-bench 2013-08-30 19:50:33,994 INFO 50736 GETS [0 failures], 415.8/s swift-bench 2013-08-30 19:50:48,995 INFO 58494 GETS [0 failures], 426.9/s swift-bench 2013-08-30 19:51:03,996 INFO 66256 GETS [0 failures], 435.8/s swift-bench 2013-08-30 19:51:18,998 INFO 73875 GETS [0 failures], 442.3/s swift-bench 2013-08-30 19:51:33,999 INFO 81612 GETS [0 failures], 448.3/s swift-bench 2013-08-30 19:51:49,000 INFO 89374 GETS [0 failures], 453.6/s swift-bench 2013-08-30 19:52:04,002 INFO 97165 GETS [0 failures], 458.2/s swift-bench 2013-08-30 19:52:09,564 INFO 100000 GETS **FINAL** [0 failures], 459.6/s swift-bench 2013-08-30 19:52:09,564 INFO Auth version: 1.0 swift-bench 2013-08-30 19:52:11,586 INFO 124 DEL [0 failures], 61.4/s swift-bench 2013-08-30 19:52:22,428 INFO 1000 DEL **FINAL** [0 failures], 77.8/s swift-bench 2013-08-30 19:52:22,428 INFO Auth version: 1.0 Wednesday, September 4, 13
26. ~30% faster on GET unknown for PUT or DELETE Wednesday, September 4, 13
27. Warmup time suck Wednesday, September 4, 13
28. Lots of room for improvement (might help CPython as well) Wednesday, September 4, 13
29. Future directions Wednesday, September 4, 13
30. More faster. Wednesday, September 4, 13
31. Less syscalls Wednesday, September 4, 13
32. More careful attention to buffer allocation/ copying Wednesday, September 4, 13
33. What to do when PyPy and CPython diverge? Wednesday, September 4, 13
34. Get a PyPy release so this can get Wednesday, September 4, 13
35. Better benchmarking Wednesday, September 4, 13
36. Thanks! Questions? https://speakerdeck.com/alex Wednesday, September 4, 13