Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Concurrency in Python

Concurrency in Python

Mini PyCon MY 2014
March 2014, Kuala Lumpur

Farzad Ghanei

March 15, 2014
Tweet

More Decks by Farzad Ghanei

Other Decks in Programming

Transcript

  1. Concurrency In Python
    Farzad Ghanei
    www.ghanei.net

    View Slide

  2. != Parallel computing
    Concurrency
    == Handling tasks while others are still running

    View Slide

  3. Blocked by IO
    Why?
    Processing takes long

    View Slide

  4. Threads
    Solutions
    Processes
    Coroutines

    View Slide

  5. Code Outline

    View Slide

  6. Simple, sync IO API
    Threads
    Preemptive scheduling by OS
    Unpredictable execution ordering

    View Slide

  7. IO bound threads

    View Slide

  8. View Slide

  9. CPU bound threads

    View Slide

  10. CPU bound threads

    View Slide

  11. View Slide

  12. Threads release and acquire GIL
    Global Interpreter Lock
    The one thread that has GIL runs at a time

    View Slide

  13. 1 CPU bound thread

    View Slide

  14. 3 CPU bound threads

    View Slide

  15. Not good for CPU bound
    Threads
    Good for user Interactivity, maybe IO
    Shared state might cause difficulties

    View Slide

  16. Simple, sync IO API
    Processes
    Preemptive scheduling by OS
    Unpredictable execution ordering

    View Slide

  17. IO bound processes

    View Slide

  18. CPU bound processes

    View Slide

  19. CPU bound processes

    View Slide

  20. View Slide

  21. Processes
    Good for parallel processing
    Use more resources than threads
    Shared state might cause difficulties

    View Slide

  22. Predictable execution ordering
    Coroutines
    Cooperative concurrency

    View Slide

  23. PEP 380 (yield from)
    Coroutines
    Generators (yield)
    Greenlets (green threads)

    View Slide

  24. CPU bound greenlets

    View Slide

  25. CPU bound greenlets

    View Slide

  26. View Slide

  27. Coroutines
    Have early results
    Controlling execution

    View Slide

  28. Async IO
    No blocking calls
    Respond to IO Events

    View Slide

  29. Event driven concurrency
    Callback based (Twisted)
    Coroutine based (Gevent)
    Both (asyncio)

    View Slide

  30. Gevent
    Implicit coroutines (using greenlets)
    threading like API
    Monkey patching
    libev event loop

    View Slide

  31. Gevent

    View Slide

  32. View Slide

  33. Twisted
    Explicit event loop
    Batteries included
    Deferreds
    No blocking calls

    View Slide

  34. Twisted

    View Slide

  35. View Slide

  36. asyncio (tulip)
    PEP 3156 (asyncio)
    Callbacks and coroutines API
    Standard library
    Explicit event loop

    View Slide

  37. Thanks

    View Slide