PyCon Ireland 2013: On Concurrency

(via http://python.ie/pycon/2013/talks/on_concurrency/)
Speaker: Goran Peretin

This talk will start by giving a brief overview of what concurrency is, how it is different from parallelism and in which cases you could benefit from writing concurrent programs. We'll show how thread-based concurrency works and see that we can only get so far with it - we need a more flexible approach to dealing with lots of concurrent tasks. After that the talk will focus on different approaches available in Python to deal with that problem (mainly Greenlet-based and callback-based concurrency) and explain how they work. We'll point out pros and cons of each approach and show which libraries use which model. At the end, we'll take a look into the future and what PEP 3156 and Tulip will bring us.

The main goal of this talk will be to provide background for the audience so they can make better decisions on when to use some library/concept or know where to start doing further research on this topic.

There will be no details on inner workings of any of the mentioned libraries/frameworks.

I gave a similar talk at this years EuroPython, with a bit more focus on Greenlet-based concurrency. Video: http://www.youtube.com/watch?v=b9vTUZYmtiE Slides: https://speakerdeck.com/gperetin/greenlet-based-concurrency

This talk will be more general and will have more about PEP 3156 and Tulip.

Transcript

1. Greenlet-based concurrency Goran Peretin @gperetin
2. Who am I? ✤ Freelancer ✤ Interested in concurrent, parallel and distributed systems
3. What is this about? ✤ understand what <buzzword> is ✤ when should you use <buzzword> ✤ concurrency as execution model (as opposed to composition model)
4. There will be no... ✤ Turnkey solutions ✤ GIL ✤ Details
5. Buzzwords ahead!
6. ✤ concurrent vs parallel execution ✤ cooperative vs preemptive multitasking ✤ CPU bound vs IO bound task ✤ thread-based vs event-based concurrency
7. Mandatory definitions
8. Parallel execution ✤ Simultaneous execution of multiple tasks ✤ Must have multiple CPUs
9. Concurrent execution ✤ Executing multiple tasks in the same time frame ✤ ... but not necessarily at the same time ✤ Doesn’t require multiple CPU cores
10. Why do we want concurrent execution? ✤ We need it - more tasks than CPUs ✤ CPU is much faster than anything else
11. Thread-based concurrecy ✤ Executing multiple threads in the same time frame ✤ OS scheduler decides which thread runs when
12. How OS scheduler switches tasks? ✤ When current thread does IO operation ✤ When current thread used up it’s time slice
13. How OS scheduler switches tasks? ✤ When current thread does IO operation ✤ When current thread used up it’s time slice Preemptive multitasking
15. Mandatory GIL slide ✤ Global Interpreter Lock ✤ One Python interpreter can run just one thread at any point in time ✤ Only problem for CPU bound tasks
16. CPU bound vs IO bound ✤ CPU bound - time to complete a task is determined by CPU speed ✤ calculating Fibonacci sequence, video processing... ✤ IO bound - does a lot of IO, eg. reading from disk, network requests... ✤ URL crawler, most web applications...
17. Python anyone? ✤ import threading ✤ Python threads - real OS threads
18. Houston, we have a...
19. Problem? ✤ Lots of threads ✤ Thousands
20. Benchmarks!
21. Sample programs ✤ Prog 1: spawn some number of threads - each sleeps 200ms ✤ Prog 2: spawn some number of threads - each sleeps 90s
22. Prog 1 ✤ Sleep 200ms # of threads 100 1K 10K 100K Time 207 ms 327 ms 2.55 s 25.42 s
23. Prog 2 ✤ Sleep 90s # of threads 100 1K 10K 100K RAM ~4.9 GB ~11.8 GB ~82GB ? (256GB)
24. ... and more ✤ Number of threads is limited ✤ Preemptive multitasking
25. We need ✤ Fast to create ✤ Low memory footprint ✤ We decide when to switch
26. Green threads!
27. Green threads ✤ Not managed by OS ✤ 1:N with OS threads ✤ User threads, light-weight processes
28. Greenlets ✤ “...more primitive notion of micro- thread with no implicit scheduling; coroutines, in other words.” ✤ C extension
29. Greenlets ✤ Micro-thread ✤ No implicit scheduling ✤ Coroutines
30. Coroutine ✤ Function that can suspend it’s execution and then later resume ✤ Can also be implemented in pure Python (PEP 342) ✤ Coroutines decide when they want to switch
31. Coroutine ✤ Function that can suspend it’s execution and then later resume ✤ Can also be implemented in pure Python (PEP 342) ✤ Coroutines decide when they want to switch Cooperative multitasking
32. Cooperative multitasking ✤ Each task decides when to give others a chance to run ✤ Ideal for I/O bound tasks ✤ Not so good for CPU bound tasks
33. Using greenlets ✤ We need something that will know which greenlet should run next ✤ Our calls must not block ✤ We need something to notify us when our call is done
34. Using greenlets ✤ We need something that will know which greenlet should run next ✤ Our calls must not block ✤ We need something to notify us when our call is done Scheduler
35. Using greenlets ✤ We need something that will know which greenlet should run next ✤ Our calls must not block ✤ We need something to notify us when our call is done Scheduler Event loop
36. Event loop ✤ Listens for events from OS and notifies your app ✤ Asynchronous
38. ✤ Scheduler ✤ Event loop Greenlets + ...
39. Gevent
40. Gevent ✤ “...coroutine-based Python networking library that uses greenlet to provide a high-level synchronous API on top of the libevent event loop.”
42. Prog 1 ✤ Sleep 200ms # of threads 100 1K 10K 100K Time 207 ms 327 ms 2.55 s 25.42 s # of Greenlets 100 1K 10K 100K Time 204 ms 223 ms 421 ms 3.06 s
43. Prog 2 ✤ Sleep 90s # of threads 100 1K 10K 100K RAM 4.9 GB 11.8 GB 82GB ? (256GB) # of Greenlets 100 1K 10K 100K Time 33 MB 41 MB 114 MB 858 MB
44. Gevent ✤ Monkey-patching ✤ Event loop
45. Disadvantages ✤ Monkey-patching ✤ Doesn’t work with C extensions ✤ Greenlet implementation details ✤ Hard to debug
46. Alternatives ✤ Twisted ✤ Tornado ✤ Callback based
47. PEP 3156 & Tulip ✤ Attempt to standardize event loop API in Python ✤ Tulip is an implementation
48. Recap ✤ Concurrent execution helps with IO bound applications ✤ Use threads if it works for you ✤ Use async library if you have lots of connections
49. Thank you! ✤ Questions?
50. Resources ✤ http:/ /dabeaz.com/coroutines/Coroutines.pdf ✤ http:/ /www.gevent.org/ ✤ http:/ /greenlet.readthedocs.org/en/latest/