Concurrency in Python

Mini PyCon MY 2014
March 2014, Kuala Lumpur

Transcript

1. Concurrency In Python
   Farzad Ghanei
   www.ghanei.net
   

   View Slide

2. != Parallel computing
   Concurrency
   == Handling tasks while others are still running
   

   View Slide

3. Blocked by IO
   Why?
   Processing takes long
   

   View Slide

4. Threads
   Solutions
   Processes
   Coroutines
   

   View Slide

5. Code Outline
   

   View Slide

6. Simple, sync IO API
   Threads
   Preemptive scheduling by OS
   Unpredictable execution ordering
   

   View Slide

7. IO bound threads
   

   View Slide

8. View Slide

9. CPU bound threads
   

   View Slide

10. CPU bound threads
    

    View Slide

11. View Slide

12. Threads release and acquire GIL
    Global Interpreter Lock
    The one thread that has GIL runs at a time
    

    View Slide

13. 1 CPU bound thread
    

    View Slide

14. 3 CPU bound threads
    

    View Slide

15. Not good for CPU bound
    Threads
    Good for user Interactivity, maybe IO
    Shared state might cause difficulties
    

    View Slide

16. Simple, sync IO API
    Processes
    Preemptive scheduling by OS
    Unpredictable execution ordering
    

    View Slide

17. IO bound processes
    

    View Slide

18. CPU bound processes
    

    View Slide

19. CPU bound processes
    

    View Slide

20. View Slide

21. Processes
    Good for parallel processing
    Use more resources than threads
    Shared state might cause difficulties
    

    View Slide

22. Predictable execution ordering
    Coroutines
    Cooperative concurrency
    

    View Slide

23. PEP 380 (yield from)
    Coroutines
    Generators (yield)
    Greenlets (green threads)
    

    View Slide

24. CPU bound greenlets
    

    View Slide

25. CPU bound greenlets
    

    View Slide

26. View Slide

27. Coroutines
    Have early results
    Controlling execution
    

    View Slide

28. Async IO
    No blocking calls
    Respond to IO Events
    

    View Slide

29. Event driven concurrency
    Callback based (Twisted)
    Coroutine based (Gevent)
    Both (asyncio)
    

    View Slide

30. Gevent
    Implicit coroutines (using greenlets)
    threading like API
    Monkey patching
    libev event loop
    

    View Slide

31. Gevent
    

    View Slide

32. View Slide

33. Twisted
    Explicit event loop
    Batteries included
    Deferreds
    No blocking calls
    

    View Slide

34. Twisted
    

    View Slide

35. View Slide

36. asyncio (tulip)
    PEP 3156 (asyncio)
    Callbacks and coroutines API
    Standard library
    Explicit event loop
    

    View Slide

37. Thanks
    

    View Slide