Thinking Outside the GIL

Have you ever written a small, elegant application that couldn't keep up with the growth of your data or user demand? Did your beautiful design end up buried in threads and locks? Did Python's very special Global Interpreter Lock make all of this an exercise in futility?

This talk is for you! With the combined powers of AsyncIO and multiprocessing, we'll redesign an old multithreaded application limited by the GIL into a modern solution that scales with the demand using only the standard library. No prior AsyncIO or multiprocessing experience required.

Presented at PyCon US 2018 in Cleveland: https://youtu.be/0kXaLh8Fz3k

Transcript

1. View Slide

2. Thinking Outside the GIL
   With AsyncIO and Multiprocessing
   John Reese
   Production Engineer, Facebook
   @n7cmdr

   github.com/jreese
   

   View Slide

3. • Global Interpreter Lock
   • One VM thread at a time
   • No concurrent memory access
   • I/O wait releases lock
   What’s the GIL?
   

   View Slide

4. • Gather ~100M data points
   • Process and aggregate anomalies
   • Easy to add new checks
   • Simple deployment
   • Few dependencies
   Stateful monitoring
   

   View Slide

5. View Slide

6. View Slide

7. View Slide

8. • One binary
   • Fetch the world
   • Process everything
   • Aggregate results
   • Thread pool for I/O
   #impact
   

   View Slide

9. • Scales in time and memory
   • Runtime now too slow
   • Underutilizing hardware
   • Ultimately limited by the GIL
   Not aging well
   

   View Slide

10. Give me options
    

    View Slide

11. Switch to py3
    ~45% memory savings ~20% runtime reduction
    

    View Slide

12. • Technically correct
    • Scales with number of workers
    • Complicated deployments
    • Communication overhead
    Sharding
    

    View Slide

13. • Scales with CPU cores
    • Automatic IPC
    • Pool.map is really useful
    Multiprocessing
    

    View Slide

14. View Slide

15. View Slide

16. • Scales with CPU cores
    • Automatic IPC
    • Pool.map is really useful
    • One task per process
    • Beware forking, pickling
    Multiprocessing
    

    View Slide

17. • Based on futures
    • Faster than threads
    • Massive I/O concurrency
    AsyncIO
    

    View Slide

18. View Slide

19. View Slide

20. View Slide

21. View Slide

22. • Based on futures
    • Faster than threads
    • Massive I/O concurrency
    • Processing still limited by GIL
    • Beware timeouts and queue length
    AsyncIO
    

    View Slide

23. Why not both?
    

    View Slide

24. • Use multiprocessing primitives
    • Event loop per process
    • Queues for work/results
    • Highly parallel workload
    • Need to do some plumbing
    Multiprocessing + AsyncIO
    

    View Slide

25. But How?
    

    View Slide

26. But How?
    

    View Slide

27. But How?
    

    View Slide

28. View Slide

29. View Slide

30. View Slide

31. View Slide

32. View Slide

33. View Slide

34. View Slide

35. View Slide

36. View Slide

37. View Slide

38. View Slide

39. View Slide

40. View Slide

41. View Slide

42. View Slide

43. • Multiple work queues
    • Combine tasks into batches
    • Use spawned processes
    Optimizations
    

    View Slide

44. • Minimize what you pickle
    • Prechunk work items
    • Aggregate results in the child
    • Use map/reduce
    Considerations
    

    View Slide

45. Performance comparison
    Threads
    AsyncIO
    Multiprocessing
    Multi/Async Naive
    Multi/Async Tuned
    Multi/Async Map/Reduce
    Processed / Second
    

    View Slide

46. I want it!
    

    View Slide

47. $ pip install aiomultiprocess
    

    View Slide

48. • Simple implementation
    • Emulates multiprocessing API
    • One shot or process pool
    • Supports map/reduce workloads
    aiomultiprocess
    github.com/jreese/aiomultiprocess
    

    View Slide

49. View Slide

50. Python is slow
    

    View Slide

51. Python is slow powerful
    

    View Slide

52. Great tools make
    complex tasks simple
    

    View Slide

53. John Reese
    Production Engineer, Facebook
    @n7cmdr

    github.com/jreese
    

    View Slide

54. View Slide