Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Concurrency in Python

Concurrency in Python

Mini PyCon MY 2014
March 2014, Kuala Lumpur

Farzad Ghanei

March 15, 2014
Tweet

More Decks by Farzad Ghanei

Other Decks in Programming

Transcript

  1. Concurrency In Python
    Farzad Ghanei
    www.ghanei.net

    View full-size slide

  2. != Parallel computing
    Concurrency
    == Handling tasks while others are still running

    View full-size slide

  3. Blocked by IO
    Why?
    Processing takes long

    View full-size slide

  4. Threads
    Solutions
    Processes
    Coroutines

    View full-size slide

  5. Code Outline

    View full-size slide

  6. Simple, sync IO API
    Threads
    Preemptive scheduling by OS
    Unpredictable execution ordering

    View full-size slide

  7. IO bound threads

    View full-size slide

  8. CPU bound threads

    View full-size slide

  9. CPU bound threads

    View full-size slide

  10. Threads release and acquire GIL
    Global Interpreter Lock
    The one thread that has GIL runs at a time

    View full-size slide

  11. 1 CPU bound thread

    View full-size slide

  12. 3 CPU bound threads

    View full-size slide

  13. Not good for CPU bound
    Threads
    Good for user Interactivity, maybe IO
    Shared state might cause difficulties

    View full-size slide

  14. Simple, sync IO API
    Processes
    Preemptive scheduling by OS
    Unpredictable execution ordering

    View full-size slide

  15. IO bound processes

    View full-size slide

  16. CPU bound processes

    View full-size slide

  17. CPU bound processes

    View full-size slide

  18. Processes
    Good for parallel processing
    Use more resources than threads
    Shared state might cause difficulties

    View full-size slide

  19. Predictable execution ordering
    Coroutines
    Cooperative concurrency

    View full-size slide

  20. PEP 380 (yield from)
    Coroutines
    Generators (yield)
    Greenlets (green threads)

    View full-size slide

  21. CPU bound greenlets

    View full-size slide

  22. CPU bound greenlets

    View full-size slide

  23. Coroutines
    Have early results
    Controlling execution

    View full-size slide

  24. Async IO
    No blocking calls
    Respond to IO Events

    View full-size slide

  25. Event driven concurrency
    Callback based (Twisted)
    Coroutine based (Gevent)
    Both (asyncio)

    View full-size slide

  26. Gevent
    Implicit coroutines (using greenlets)
    threading like API
    Monkey patching
    libev event loop

    View full-size slide

  27. Twisted
    Explicit event loop
    Batteries included
    Deferreds
    No blocking calls

    View full-size slide

  28. asyncio (tulip)
    PEP 3156 (asyncio)
    Callbacks and coroutines API
    Standard library
    Explicit event loop

    View full-size slide