High Availability

ElixirConf Europe

Transcript

1. iex(1)> high_availability
   @sasajuric
   aircloak.com
   

   View Slide

2. Erlang 

   (the language)
   Elixir LFE
   Erlang VM (BEAM)
   

   View Slide

3. highly_available ≈ works_almost_always
   

   View Slide

4. optional?(highly_available) # false
   

   View Slide

5. easy?(highly_available) # false
   

   View Slide

6. View Slide

7. highly_available do
   # the code here magically works
   end
   #doesntworklikethat
   

   View Slide

8. fault-tolerance
   scalability
   + responsiveness
   = high availability
   

   View Slide

9. A
   B
   D
   E
   F
   C
   

   View Slide

10. send(pid, message)
    A
    B
    message = receive ...
    

    View Slide

11. def loop(state) do
    receive do
    message ->
    new_state = handle(state, message)
    loop(new_state)
    end
    end
    

    View Slide

12. bank account
    balance
    {:deposit, amount}
    {:withdraw, amount}
    :balance
    amount
    

    View Slide

13. OS process or thread
    Erlang process
    

    View Slide

14. db connection
    background job
    request handler
    state
    request handler
    request handler
    request handler
    request handler
    request handler state
    state
    state
    background job
    background job
    background job
    db connection db connection
    db connection
    

    View Slide

15. A
    B
    C
    D
    E
    F
    

    View Slide

16. A
    exit message
    B
    C
    D
    E
    F
    

    View Slide

17. supervisor
    worker worker
    

    View Slide

18. S
    S S
    W
    W
    S
    W
    W
    W S
    W
    W
    

    View Slide

19. scheduler scheduler scheduler scheduler
    CPU CPU CPU CPU
    VM
    

    View Slide

20. VM
    VM
    VM
    VM
    VM
    

    View Slide

21. VM
    send(pid, message)
    VM 1
    send(pid, message)
    VM 2
    

    View Slide

22. shared-nothing concurrency
    process crash propagation
    + some VM magic
    = fault tolerance

    scalability

    responsiveness
    

    View Slide

23. View Slide

24. 3rd party data
    source
    transform data
    push server
    browsers
    bets validation
    

    View Slide

25. Data transformation
    1.Receive TCP stream"
    2.Assemble XML messages"
    3.Update the model"
    4.Store to database"
    5.Serialize the model"
    6.Dispatch
    

    View Slide

26. receive stream
    assemble messages
    update model
    store to database
    serialize data
    dispatch
    

    View Slide

27. receive stream
    assemble messages
    update model
    store to database
    serialize data
    dispatch
    message 6
    message 5
    message 4
    message 3
    message 2
    message 1
    

    View Slide

28. receive stream
    assemble messages
    update model
    store to database
    serialize data
    dispatch
    

    View Slide

29. storing to database
    queuing bet odds
    storing bet odds
    

    View Slide

30. match 1 match 3
    match 2
    

    View Slide

31. update model
    serialize data
    dispatch
    match 1 match 3
    match 2
    

    View Slide

32. View Slide

33. View Slide

34. View Slide

35. View Slide

36. View Slide