Concurrency 101

Walk through of concurrency models in Kotlin (JVM), Python, JavaScript (Node) & Golang

Supplementing code: https://github.com/sandeepraju/concurrency

Transcript

1. Concurrency 101 Sandeep Raju Prabhakar @sandeep sandeepraju
2. Before we get started ... • This is not a functional programming talk • Talk on concurrency models (implementation) which uses a few concurrency patterns (futures, async.. await, actors, callbacks). We are not going to discuss patterns in detail. • The concepts discussed are language and OS agnostic • Feel free to stop me at any time for questions
3. Overview • The need to understand concurrency • How it all began • Terminology: Concurrency, Parallelism, Workloads, Scheduling • Examples (Demo) • The modified Reactor Pattern (Event Loop) • Thread vs Coroutine vs Fiber vs Green Thread • Final thoughts
4. The unfortunate state of concurrency “Concurrency is generally someone else's problem ...
5. The unfortunate state of concurrency “Concurrency is generally someone else's problem ... ... unless that someone else is you.”
6. The need to understand concurrency Understanding how things work under the hood helps you: • Write performant code and scale your systems better. • Reason about why something works the way it does. • Picking a concurrency model when presented with options. • Generally being informed is better!
7. How it all began
8. Initially ... • There was only one control flow (no interleaving). • Programs (jobs) would be run one after another. • The execution order was “sequential / synchronous”. • Sequential / Synchronous: One job at a time. Execution of next job only after first one ends.
9. .. and then, a few years later • Still, there was only one control flow • But, programs (jobs) are “interleaved” (multi-tasking) • Interleaved: At any point in time, only a single job runs, but processor time is split among multiple jobs. • “Multiple jobs (tasks) are making progress overall” even though only one job runs at any point in time.
10. … few more years later (now) • Multiple processors (cores) == Run multiple jobs in “parallel” • Parallel: More than one control flow (one per processor) • Each control flow can be again interleaved. • Now, “multiple jobs are making progress (& running)”
11. Concurrency vs Parallelism
12. Concurrency vs Parallelism • Concurrency: Multiple jobs making progress without the need for all of them to be running at the same time. Eg. 4 threads on 1 processor == concurrent • Parallelism: Multiple jobs making progress and running at the same time. Eg: 4 threads on 4 processors == concurrent && parallel
13. Types of workloads
14. Types of workloads • I/O Intensive: ◦ Anything that waits on I/O devices (why?) ◦ Database queries, File read / write, Network requests • CPU Intensive: ◦ Anything that involves raw computations ◦ Image processing, Crypto algorithms, Math in general
15. Types of scheduling
16. Types of scheduling • Cooperative scheduling: ◦ Responsibility of the thread / task to “yield” ◦ Usually varying time-slices (a.k.a functional slices) ◦ Can be unfair if not programmed correctly • Preemptive scheduling: ◦ Usually Fixed time-slices (time-ticks). ◦ Forcefully context switch when time is up ◦ Maintains fairness
17. Examples (Demo)
18. Java Thread Pool
19. The ‘modified’ reactor pattern (a.k.a the event loop)
20. The ‘modified’ reactor pattern (event loop) • Should have the following components ◦ Resource: that can provide input or consume output ◦ Event Demuxer: Sends resource to dispatcher. Waits on select() or epoll() ◦ Dispatcher: Handlers registering and calling handlers ◦ Request Handler: Piece of code ◦ Handler Queue*: Holds the handlers that are ready
21. NodeJS Event Loop
22. NodeJS Event Loop
23. NodeJS Event Loop
24. Effect of GIL on Thread Pool in Python
25. Python Event Loop An event loop runs in a thread and executes all callbacks and tasks in the same thread. While a task is running in the event loop, no other task is running in the same thread. When the task uses yield, the task is suspended and the event loop executes the next task. More: https://cdn-images-1.medium.com/max/800/0*s1GH0YO9ZNdEEDxo.jpg
26. Goroutines
27. Threads Green threads Coroutines Fibers Generators Tasks
28. “Disclaimer”
29. Threads • Traditional OS (kernel-level) threads • Manifests as “pthread” implementations on *nix • A.k.a “lightweight” processes • Share the Heap & Global memory but NOT the stack • Instructions are scheduled by the Operating System “preemptively” • Java threads are “actual” threads
30. Green threads • User level threads, scheduled by the “runtime” or the “VM” • Simulate multi-threading on platforms that don’t support it • Preemptive multi-tasking • Green threads were the only threads in Java < 1.3 • Golang's "goroutines" are *kind of* like green threads • Python threads are NOT green threads (common misconception)
31. Coroutines • Generalization of sub-routines for non-preemptive multi-tasking (co-operative) • Multiple entry points for suspending and resuming execution at certain locations in a “routine” (function) • Well suited while working with event-loops • Python supports coroutines from 2.5 • Kotlin is getting it’s own coroutines very soon!
32. Fibers • Fibers are “light weight” threads (similar to green threads) • But, conceptually similar to coroutines (entry/exit points) • Fibers are considered generally as an implementation of the coroutine "concept" • Co-operative multi-tasking (mostly). One fiber yields to another fiber
33. Generators • Known as semi-coroutines (limited in functionality) • While co-routine can control where the execution can continue, generators continue just after the “yield” • Used to simplifying iterators, building infinite lists
34. Tasks • Blanket term for anything that is not an OS level thread or a process. • All the previous ones can be generalized as tasks • To add to the confusion, Python asyncio has something called “tasks”
35. As they say... “There are only two hard things in Computer Science: cache invalidation and naming things.”
36. Final thoughts
37. Final thoughts • Concurrency can he hard to wrap your head around conceptually due to time-slicing & scheduling • Non standard naming convention and various concurrency models don’t make it any easier • There is an initial learning curve and after that, it is just a mix and match of the core set of concepts • Learning the models of various languages helps understanding the concepts better
38. References
39. References • Nonblocking I/O, select() & epoll() • What the heck is the event loop anyway? • Concurrency is not parallelism • Concurrency from ground up (Python) • Understanding the Python GIL • Coroutines for the curious • Java (JDK) concurrency