Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Concurrency Basics for Elixir

Concurrency Basics for Elixir

Slides from internal presentation at https://appunite.com

Maciej Kaszubowski

August 02, 2018
Tweet

More Decks by Maciej Kaszubowski

Other Decks in Programming

Transcript

  1. Concurrency basics
    For Elixir-based Systems

    View full-size slide

  2. So, what’s concurrency?

    View full-size slide

  3. Sequential Execution (3 functions, 1 thread)

    View full-size slide

  4. Sequential Execution (3 functions, 1 thread)
    Concurrent Execution (3 functions, 3 threads)

    View full-size slide

  5. Sequential Execution (3 functions, 1 thread)
    Concurrent Execution (3 functions, 3 threads)
    Preemptive scheduling

    View full-size slide

  6. Where’s the benefit?

    View full-size slide

  7. Req1 Req2 Req3 Resp
    Sequential
    Execution time Waiting time

    View full-size slide

  8. Req1 Req2 Req3 Resp
    Req1
    Resp
    Req2
    Req3
    Sequential
    Concurrent
    Execution time Waiting time

    View full-size slide

  9. CPU bound
    Re Re Re Res
    Re
    Res
    Re
    Re
    I/O bound

    View full-size slide

  10. Concurrent or Parallel
    What’s the difference?

    View full-size slide

  11. Concurrent Execution (3 functions, 3 threads)

    View full-size slide

  12. Concurrent Execution (3 functions, 3 threads)
    Parallel Execution (3 functions, 3 threads, 2 cores)
    core 1
    core 2

    View full-size slide

  13. root@kingschat-api-c8f8d6b76-4j65j:/app# nproc
    12
    root@tahmeel-api-prod-b5979bdc6-q5wz6:/# nproc
    1
    How many cores?

    View full-size slide

  14. Concurrent Execution (3 functions, 3 threads)
    Parallel Execution (3 functions, 3 threads, 2 cores)
    core 1
    core 2
    (by default) One erlang scheduler per core

    View full-size slide

  15. :observer_cli.start()

    View full-size slide

  16. Req1 Req2 Req3 Resp
    Req1
    Resp
    Req2
    Req3
    Sequential
    Concurrent
    Execution time Waiting time
    Req1 Resp
    Req2
    Req3
    Parallel

    View full-size slide

  17. Sequential execution

    View full-size slide

  18. Phoenix
    Request
    Req 1

    View full-size slide

  19. Phoenix
    Request
    Resp

    View full-size slide

  20. Phoenix
    Request
    Req 2

    View full-size slide

  21. Phoenix
    Request
    Resp

    View full-size slide

  22. Phoenix
    Request
    Req 3

    View full-size slide

  23. Phoenix
    Request
    Resp

    View full-size slide

  24. Concurrent execution

    View full-size slide

  25. Phoenix
    Request

    View full-size slide

  26. Phoenix
    Request
    Task 1
    Task 2
    Task 3

    View full-size slide

  27. Phoenix
    Request
    Task 1
    Task 2
    Task 3
    Req 1
    Req 2
    Req 3

    View full-size slide

  28. Phoenix
    Request
    Task 1
    Task 2
    Task 3
    Resp
    Resp
    Resp

    View full-size slide

  29. Phoenix
    Request
    Task 1
    Task 2
    Task 3

    View full-size slide

  30. R1
    APP Server DB Server (3 cores)
    R2
    R1
    R2
    Time
    Execution time Waiting time

    View full-size slide

  31. R1
    APP Server DB Server (3 cores)
    Send resp
    R2 R3
    R1 R2
    R3
    Time
    Execution time Waiting time

    View full-size slide

  32. How much can we gain?

    View full-size slide

  33. Amdahl’s Law

    View full-size slide

  34. Amdahl’s Law

    View full-size slide

  35. Amdahl’s Law in a nutshell
    The more synchronisation,
    the less benefit from multiple cores

    View full-size slide

  36. R1
    APP Server
    Send resp
    R2 R3
    R1 R2
    R3
    Time
    Execution time Waiting time
    Almost 100% parallel
    (almost no synchronisation)
    DB Server (3 cores)

    View full-size slide

  37. R1
    APP Server
    Send resp
    R2 R3
    R1 R2
    R3
    Time
    Execution time Waiting time
    This is not constant
    DB Server (3 cores)

    View full-size slide

  38. R1
    APP Server
    Send resp
    R2 R3
    R1 R2
    R3
    Time
    Execution time Waiting time
    This is not infinite
    DB Server (3 cores)

    View full-size slide

  39. R1
    APP Server
    R2 R3
    R1 R2
    R3
    R4
    Time
    Execution time Waiting time
    DB Server (3 cores)

    View full-size slide

  40. R1
    APP Server
    R2 R3
    R1 R2
    R3
    R4
    Time
    Execution time Waiting time
    DB Server (3 cores)

    View full-size slide

  41. R1
    APP Server
    R2 R3
    R1 R2
    R3
    R4
    R4
    Time
    Execution time Waiting time
    DB Server (3 cores)

    View full-size slide

  42. R1
    APP Server
    R2 R3
    R1 R2
    R3
    R4
    R4
    Time
    Execution time Waiting time
    DB Server (3 cores)

    View full-size slide

  43. R1
    APP Server
    R2 R3
    R1 R2
    R3
    R4
    R4
    Time
    Execution time Waiting time
    DB Server (3 cores)

    View full-size slide

  44. R1
    APP Server
    Send resp
    R2 R3
    R1 R2
    R3
    R4
    R4
    Time
    Execution time Waiting time
    DB Server (3 cores)

    View full-size slide

  45. R1
    APP Server
    R2 R3
    R1 R2
    R3
    R4
    R4
    Time
    Execution time Waiting time
    R5 R6 R7
    R5
    R6
    R7
    DB Server (3 cores)

    View full-size slide

  46. Phoenix
    Request
    Task 1
    Task 2
    Task 3
    Req 1
    Req 2
    Req 3
    Remember this?

    View full-size slide

  47. This isn’t exactly true

    View full-size slide

  48. Connection pool
    (Prevents from overworking the DB)

    View full-size slide

  49. Pool Manager
    (Blocks until a free worker is available)

    View full-size slide

  50. Pool Manager
    (Blocks until a free worker is available)

    View full-size slide

  51. It gets worse

    View full-size slide

  52. Pool Manager
    Mailbox
    Has to be synchronised

    View full-size slide

  53. Pool Manager
    Message Passing
    Is just copying data in shared memory

    View full-size slide

  54. Pool Manager
    Remember semaphores?

    View full-size slide

  55. Logger Metrics Sentry

    View full-size slide

  56. Network stack

    View full-size slide

  57. Network stack

    View full-size slide

  58. Network stack

    View full-size slide

  59. Network stack
    Sentry
    Metrics

    View full-size slide

  60. OS Threads
    (Garbage Collection)
    Data Bus
    Virtual Machines
    Memory characteristics (e.g. processor caches)

    Other synchronisation points

    View full-size slide

  61. That’s hard

    View full-size slide

  62. That’s REALLY hard

    View full-size slide

  63. That’s REALLY hard
    Seriously, people spend their entire careers on this

    View full-size slide

  64. So, what to do?

    View full-size slide

  65. Measure
    Measure

    View full-size slide

  66. Measure
    Measure
    Measure

    View full-size slide

  67. Measure ON PRODUCTION

    View full-size slide

  68. Measure ON PRODUCTION
    You WILL get false results on staging/locally

    View full-size slide

  69. Measure Entire system
    You WILL get false results for single functions

    View full-size slide

  70. Measure ONLY IF YOU HAVE TRAFFIC

    View full-size slide

  71. “premature optimization is the root of all evil”

    View full-size slide

  72. If something takes X ms, it will always
    take X ms.

    View full-size slide

  73. Async execution cannot “remove” this time
    It can only hide it

    View full-size slide

  74. BACK PRESSURE

    View full-size slide

  75. Producent
    Consumer
    Consumer

    View full-size slide

  76. Producent
    Consumer
    Consumer

    View full-size slide

  77. Producent
    Consumer
    Consumer

    View full-size slide

  78. Producent
    Consumer
    Consumer

    View full-size slide

  79. Producent
    Consumer
    Consumer

    View full-size slide

  80. Producent
    Consumer
    Consumer

    View full-size slide

  81. Producent
    Consumer
    Consumer

    View full-size slide

  82. Producent
    Consumer
    Consumer

    View full-size slide

  83. Producent
    Consumer
    Consumer

    View full-size slide

  84. Producent
    Consumer
    Consumer

    View full-size slide

  85. Producent
    Consumer
    Consumer

    View full-size slide

  86. Producent
    Consumer
    Consumer

    View full-size slide

  87. Producent
    Consumer
    Consumer

    View full-size slide

  88. Producent
    Consumer
    Consumer
    Stop

    View full-size slide

  89. Producent
    Consumer
    Consumer

    View full-size slide

  90. Producent
    Consumer
    Consumer

    View full-size slide

  91. Producent
    Consumer
    Consumer

    View full-size slide

  92. Producent
    Consumer
    Consumer

    View full-size slide

  93. Producent
    Consumer
    Consumer
    OK, give me more

    View full-size slide

  94. Producent
    Consumer
    Consumer

    View full-size slide

  95. Back pressure

    View full-size slide