Thinking Outside the GIL

Thinking Outside the GIL

Have you ever written a small, elegant application that couldn't keep up with the growth of your data or user demand? Did your beautiful design end up buried in threads and locks? Did Python's very special Global Interpreter Lock make all of this an exercise in futility?

This talk is for you! With the combined powers of AsyncIO and multiprocessing, we'll redesign an old multithreaded application limited by the GIL into a modern solution that scales with the demand using only the standard library. No prior AsyncIO or multiprocessing experience required.

Presented at PyCon US 2018 in Cleveland: https://youtu.be/0kXaLh8Fz3k

9c98621c87f4f1333e0dcbb5282fa65b?s=128

John Reese

May 11, 2018
Tweet

Transcript

  1. None
  2. Thinking Outside the GIL With AsyncIO and Multiprocessing John Reese

    Production Engineer, Facebook @n7cmdr
 github.com/jreese
  3. • Global Interpreter Lock • One VM thread at a

    time • No concurrent memory access • I/O wait releases lock What’s the GIL?
  4. • Gather ~100M data points • Process and aggregate anomalies

    • Easy to add new checks • Simple deployment • Few dependencies Stateful monitoring
  5. None
  6. None
  7. None
  8. • One binary • Fetch the world • Process everything

    • Aggregate results • Thread pool for I/O #impact
  9. • Scales in time and memory • Runtime now too

    slow • Underutilizing hardware • Ultimately limited by the GIL Not aging well
  10. Give me options

  11. Switch to py3 ~45% memory savings ~20% runtime reduction

  12. • Technically correct • Scales with number of workers •

    Complicated deployments • Communication overhead Sharding
  13. • Scales with CPU cores • Automatic IPC • Pool.map

    is really useful Multiprocessing
  14. None
  15. None
  16. • Scales with CPU cores • Automatic IPC • Pool.map

    is really useful • One task per process • Beware forking, pickling Multiprocessing
  17. • Based on futures • Faster than threads • Massive

    I/O concurrency AsyncIO
  18. None
  19. None
  20. None
  21. None
  22. • Based on futures • Faster than threads • Massive

    I/O concurrency • Processing still limited by GIL • Beware timeouts and queue length AsyncIO
  23. Why not both?

  24. • Use multiprocessing primitives • Event loop per process •

    Queues for work/results • Highly parallel workload • Need to do some plumbing Multiprocessing + AsyncIO
  25. But How?

  26. But How?

  27. But How?

  28. None
  29. None
  30. None
  31. None
  32. None
  33. None
  34. None
  35. None
  36. None
  37. None
  38. None
  39. None
  40. None
  41. None
  42. None
  43. • Multiple work queues • Combine tasks into batches •

    Use spawned processes Optimizations
  44. • Minimize what you pickle • Prechunk work items •

    Aggregate results in the child • Use map/reduce Considerations
  45. Performance comparison Threads AsyncIO Multiprocessing Multi/Async Naive Multi/Async Tuned Multi/Async

    Map/Reduce Processed / Second
  46. I want it!

  47. $ pip install aiomultiprocess

  48. • Simple implementation • Emulates multiprocessing API • One shot

    or process pool • Supports map/reduce workloads aiomultiprocess github.com/jreese/aiomultiprocess
  49. None
  50. Python is slow

  51. Python is slow powerful

  52. Great tools make complex tasks simple

  53. John Reese Production Engineer, Facebook @n7cmdr
 github.com/jreese

  54. None