Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Thinking Outside the GIL

Thinking Outside the GIL

Have you ever written a small, elegant application that couldn't keep up with the growth of your data or user demand? Did your beautiful design end up buried in threads and locks? Did Python's very special Global Interpreter Lock make all of this an exercise in futility?

This talk is for you! With the combined powers of AsyncIO and multiprocessing, we'll redesign an old multithreaded application limited by the GIL into a modern solution that scales with the demand using only the standard library. No prior AsyncIO or multiprocessing experience required.

Presented at PyCon US 2018 in Cleveland: https://youtu.be/0kXaLh8Fz3k

John Reese

May 11, 2018
Tweet

More Decks by John Reese

Other Decks in Programming

Transcript

  1. View Slide

  2. Thinking Outside the GIL
    With AsyncIO and Multiprocessing
    John Reese
    Production Engineer, Facebook
    @n7cmdr

    github.com/jreese

    View Slide

  3. • Global Interpreter Lock
    • One VM thread at a time
    • No concurrent memory access
    • I/O wait releases lock
    What’s the GIL?

    View Slide

  4. • Gather ~100M data points
    • Process and aggregate anomalies
    • Easy to add new checks
    • Simple deployment
    • Few dependencies
    Stateful monitoring

    View Slide

  5. View Slide

  6. View Slide

  7. View Slide

  8. • One binary
    • Fetch the world
    • Process everything
    • Aggregate results
    • Thread pool for I/O
    #impact

    View Slide

  9. • Scales in time and memory
    • Runtime now too slow
    • Underutilizing hardware
    • Ultimately limited by the GIL
    Not aging well

    View Slide

  10. Give me options

    View Slide

  11. Switch to py3
    ~45% memory savings ~20% runtime reduction

    View Slide

  12. • Technically correct
    • Scales with number of workers
    • Complicated deployments
    • Communication overhead
    Sharding

    View Slide

  13. • Scales with CPU cores
    • Automatic IPC
    • Pool.map is really useful
    Multiprocessing

    View Slide

  14. View Slide

  15. View Slide

  16. • Scales with CPU cores
    • Automatic IPC
    • Pool.map is really useful
    • One task per process
    • Beware forking, pickling
    Multiprocessing

    View Slide

  17. • Based on futures
    • Faster than threads
    • Massive I/O concurrency
    AsyncIO

    View Slide

  18. View Slide

  19. View Slide

  20. View Slide

  21. View Slide

  22. • Based on futures
    • Faster than threads
    • Massive I/O concurrency
    • Processing still limited by GIL
    • Beware timeouts and queue length
    AsyncIO

    View Slide

  23. Why not both?

    View Slide

  24. • Use multiprocessing primitives
    • Event loop per process
    • Queues for work/results
    • Highly parallel workload
    • Need to do some plumbing
    Multiprocessing + AsyncIO

    View Slide

  25. But How?

    View Slide

  26. But How?

    View Slide

  27. But How?

    View Slide

  28. View Slide

  29. View Slide

  30. View Slide

  31. View Slide

  32. View Slide

  33. View Slide

  34. View Slide

  35. View Slide

  36. View Slide

  37. View Slide

  38. View Slide

  39. View Slide

  40. View Slide

  41. View Slide

  42. View Slide

  43. • Multiple work queues
    • Combine tasks into batches
    • Use spawned processes
    Optimizations

    View Slide

  44. • Minimize what you pickle
    • Prechunk work items
    • Aggregate results in the child
    • Use map/reduce
    Considerations

    View Slide

  45. Performance comparison
    Threads
    AsyncIO
    Multiprocessing
    Multi/Async Naive
    Multi/Async Tuned
    Multi/Async Map/Reduce
    Processed / Second

    View Slide

  46. I want it!

    View Slide

  47. $ pip install aiomultiprocess

    View Slide

  48. • Simple implementation
    • Emulates multiprocessing API
    • One shot or process pool
    • Supports map/reduce workloads
    aiomultiprocess
    github.com/jreese/aiomultiprocess

    View Slide

  49. View Slide

  50. Python is slow

    View Slide

  51. Python is slow powerful

    View Slide

  52. Great tools make
    complex tasks simple

    View Slide

  53. John Reese
    Production Engineer, Facebook
    @n7cmdr

    github.com/jreese

    View Slide

  54. View Slide