You ain't gonna need to write a GenServer

GenServers plays a central role in Elixir OTP applications. However, when you overuse it, it can become an anti-pattern and introduce bottlenecks in your system.

In this talk you'll learn:

* What's a GenServer
* When GenServer can harm
* GenServer FOMO (fear of missing out)
* How Elixir ecosystem get you covered

After this talk, you'll understand how putting a GenServer in the wrong place can slowdown in your entire system. You'll understand how the Elixir frameworks and libraries enable you to have reliable concurrency applications. You'll feel no shame of being an Elixir developer and never had to write yourself a GenServer.

Transcript

1. You ain't gonna need to
   write a GenServer
   by Ulisses Almeida
   

   View Slide

2. View Slide

3. View Slide

4. Summary
   ■ What is GenServer?
   ■ How can you harm your app by putting
   GenServer in the wrong place?
   ■ Why do people think that they need a
   GenServer all the time?
   ■ Considerations before writing a GenServer
   

   View Slide

5. I’m Ulisses Almeida
   ■ Elixir Developer at The
   Coingaming Group
   ■ Brazilian !
   ■ @ulissesalmeida
   

   View Slide

6. Learn Functional Programming with Elixir
   ■ 20% off with
   Elixir_Brazil_2019
   promo code
   ■ 3 e-books giveaways
   

   View Slide

7. Who 

   We Are
   The Coingaming Group are the
   crypto gaming pioneers who
   operate the world’s leading bitcoin
   gaming and betting brands.
   https://coingaming.io/
   

   View Slide

8. View Slide

9. View Slide

10. View Slide

11. Why Tallinn? "
    ★ Low renting costs
    ★ Free transportation for
    residents
    ★ E-estonia
    ★ Safe
    ★ Low taxes
    ★ Accessible travels
    ๏ Cold and Dark ☃
    ๏ Estonian is the first
    language "
    ๏ Russian is the second
    language %
    ๏ Eastern/nordic culture
    is different
    

    View Slide

12. Ok, GenServers
    A quick recap
    

    View Slide

13. https://twitter.com/ulissesalmeida/status/1125455706470088704
    

    View Slide

14. View Slide

15. View Slide

16. View Slide

17. View Slide

18. GenServer
    ■ Useful for: Client-Server
    relation
    ■ Keep state
    ■ Execute code
    asynchronously
    (handle_cast)
    ■ Synchronously too
    (handle_call)
    ■ Provide tracing and error
    reporting features
    ■ Can be supervised
    A process with a
    powerful set of
    features
    

    View Slide

19. Shades of a GenServer
    ■ Agent
    ■ Task
    ■ Task.Supervisor
    ■ Supervisor
    ■ DynamicSupervisor
    

    View Slide

20. CalculatorServer
    Baby
    1+1?
    1.
    2.
    3.
    From: Baby

    1+1?
    4.
    1+1=2
    5.
    

    View Slide

21. CalculatorServer
    Baby
    7.
    6.
    From: 

    MyCalcGenServer

    2
    8.
    

    View Slide

22. Live demo
    “…maybe I should have
    recorded a video”
    

    View Slide

23. View Slide

24. View Slide

25. View Slide

26. View Slide

27. View Slide

28. View Slide

29. View Slide

30. View Slide

31. View Slide

32. UÉ!?
    

    View Slide

33. hahahahah GenServer

    bottleneck

    omg what I have 

    done!?
    

    View Slide

34. Holy %#@!
    ~ 200 times slower

    ~ 10 times more
    memory
    120km/h
    1.6km/h
    ~75 times slower
    10 times bigger
    

    View Slide

35. What happened!?
    
    

    View Slide

36. CalculatorServer
    AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
    AAAAAAAAAAAAAAAA
    

    View Slide

37. Without GenServer…
    
    

    View Slide

38. CalculatorServer
    

    View Slide

39. CalculatorServer
    ☠
    
    
    
    ta pegando fogo bicho!
    !
    

    View Slide

40. Why people want it so
    much?
    Can we play just with
    functions?
    

    View Slide

41. Erlang OTP
    ■ Process hierarchy
    ■ Restarting strategies
    ■ Concurrency
    ■ GenServer plays the
    central role
    

    View Slide

42. wait, you said
    concurrency…
    

    View Slide

43. GenServer checklist
    ■ Model state accessed by multiple
    processes
    ■ Run multiple tasks concurrently
    ■ Communicate between servers
    ■ Gracefully handle clean startup and exit
    concerns
    https://pragprog.com/book/tvmelixir/adopting-elixir
    

    View Slide

44. CalculatorServer checklist
    ❌Model state accessed by multiple
    processes
    ❌Run multiple tasks concurrently
    ❌Communicate between servers
    ❌Gracefully handle clean startup and exit
    concerns
    https://pragprog.com/book/tvmelixir/adopting-elixir
    

    View Slide

45. CalculatorServer has no
    reason to be a GenServer
    A GenServer must
    never be used for
    code organization
    purposes
    https://hexdocs.pm/elixir/GenServer.html
    GenServer to organize 

    domain, business

    contexts
    GenServer to model 

    runtime properties

    of your system
    

    View Slide

46. Fear of missing
    out?
    Am I a proper Elixir
    developer If I'm not
    doing GenServers?
    

    View Slide

47. I have been working with Elixir for ~3 years
    ■ I never wrote a production GenServer
    ■ I wrote GenServers only in recruitment
    processes and for fun
    ■ I deleted some unnecessary production
    GenServers
    ■ I know excellent developers that never
    wrote one
    

    View Slide

48. Elixir developers, no
    shame in never had to
    create a GenServer.
    

    View Slide

49. Ecosystem got you
    covered
    Or… things that you should consider
    before writing a GenServer
    

    View Slide

50. Let's think about a typical
    web application
    

    View Slide

51. Phoenix
    Elixir
    Database
    Legacy
    Legacy of the Legacy
    Users
    

    View Slide

52. View Slide

53. Users & App
    

    View Slide

54. Users -> App checklist
    ✔ We need to handle multiple users
    connections concurrently
    ✔ Other processes might send messages to
    that connection
    ✔ We have to handle connections openings
    and closings gracefully.
    

    View Slide

55. GenServers!
    

    View Slide

56. However…
    ■ Each user’s request
    has a process that runs
    concurrently
    ■ You can focus on the
    functional part
    ■ (functional part?
    functional
    programming )
    Phoenix and
    Cowboy handles
    that for you
    

    View Slide

57. App &
    Database
    

    View Slide

58. App & Database checklist
    ✔Databases are an external service
    communication
    ✔We have to provide a limited number of
    connections to a lot of processes
    ✔These connections have to run
    ✔We need to handle connection openings
    and closings gracefully
    

    View Slide

59. GenServers!!!!
    

    View Slide

60. However…
    ■ Ecto supports a pool of
    connections
    ■ Ecto handle that
    concurrently
    ■ Work here is also done
    ■ You can focus on the
    business
    Ecto handles that
    for you
    

    View Slide

61. App &Other
    Services
    

    View Slide

62. App & Other Services checklist
    ✔ Usually, we communicate through a network
    ✔ We might need to offer a pool as a backpressure
    mechanism to not bring down weak services
    ✔ These pools have to run multiple requests
    concurrently
    ✔ Gracefully handle connections opening and
    closings
    

    View Slide

63. GenSeerveerss!!!
    

    View Slide

64. However…
    ■ You can start simple,
    for example, no pool
    ■ Watch out for big
    timeouts
    ■ Circuit breakers can
    help you out
    ■ Choose the libraries,
    and you can focus on
    features
    Hackney and other
    libraries handles
    that for you
    

    View Slide

65. NoSQL storage
    

    View Slide

66. Not everything is stored in SQL databases
    ■ Examples: caching,
    session, users
    presence
    ■ Transient or Persistent?
    ■ How big will it be?
    ■ Write vs Read
    ✔ This data will be
    accessible by multiple
    processes
    

    View Slide

67. GenSeeerveerS!!!!!
    

    View Slide

68. WITH ETS!!!!!!!
    

    View Slide

69. Phoenix
    Elixir
    Elixir
    ETS table
    Phoenix
    Elixir
    ETS table
    Phoenix
    Elixir
    ETS table
    User
    

    View Slide

70. However…
    ■ A better abstraction reveals
    intention, for example,
    Cachex vs GenServer + ETS
    ■ How does your deploy
    work?
    ■ Shutdown and booting the
    app will lose GenServer/
    ETS data
    ■ Redis for the rescue
    ■ Discuss the right tool for
    the job, compare the
    tradeoffs
    You might want to
    use a specific
    abstraction.
    

    View Slide

71. Serializable
    operations
    

    View Slide

72. Serializable
    operations
    Some resources
    can’t be changed at
    same time
    1
    1
    5
    I want to withdraw 2
    I want to withdraw 6
    Company Wallet
    ✔ Manage state accessed
    by multiple processes
    

    View Slide

73. …GenServer?
    

    View Slide

74. However…
    ■ You would need an
    Erlang cluster to run a
    global GenServer
    ■ What happens with
    network partition?
    ■ Use your database
    power
    ■ Transactions or locks in
    the database
    ■ Discuss with your team,
    compare the tradeoffs
    You environment
    might not allow.
    a.k.a Heroku
    

    View Slide

75. Background
    tasks
    

    View Slide

76. Tasks outside of a request/response cycle
    ■ Welcome e-mail
    ■ Push notifications
    ■ Scheduled tasks
    ✔ We need to spawn or
    schedule these
    background tasks
    ✔ We need to run the
    background tasks
    concurrently
    

    View Slide

77. Dynamic Supervisors 

    & Task!!!
    

    View Slide

78. However…
    ■ GenServers aren’t
    persistent
    ■ Shutting downs might
    lose data
    ■ RabbitMQ has persistent
    queues
    ■ Libraries that use Redis
    to store the jobs and
    their arguments
    ■ Discuss with your team,
    compare the tradeoffs
    How to handle the
    failures? Do you
    need some
    persistence?
    

    View Slide

79. We covered a lot 

    of considerations before 

    “let's write a GenServer” decision
    And they are still not
    over
    

    View Slide

80. GenServers are
    fundamental
    

    View Slide

81. Elixir frameworks and
    libraries cover most of
    the use cases that you
    would need to write one
    

    View Slide

82. But it doesn’t mean
    you should not learn
    about GenServer
    

    View Slide

83. You need to know
    GenServer to tune
    their configurations
    

    View Slide

84. When stuff goes
    wrong, you’ll need
    to debug and fix it.
    

    View Slide

85. You won’t might need
    to write a GenServer
    95% of time…
    

    View Slide

86. But that 5%…
    

    View Slide

87. Then, keep
    learning!
    

    View Slide

88. Remember
    

    View Slide

89. Putting a GenServer in
    the wrong place brings
    more harm than good
    

    View Slide

90. It’s good to compare
    and to discuss the
    tradeoffs with your
    team.
    

    View Slide

91. There’s no shame
    in never need to
    write a GenServer
    

    View Slide

92. Thank you
    

    View Slide

93. References
    ■ Adopting Elixir - https://pragprog.com/book/tvmelixir/adopting-
    elixir
    ■ Spawn or not spawn - https://www.theerlangelist.com/article/
    spawn_or_not
    ■ You may not need a GenServer - https://pragtob.wordpress.com/
    2019/04/24/you-may-not-need-genservers-and-supervision-trees
    ■ GenServer (Elixir Docs) - https://hexdocs.pm/elixir/GenServer.html
    ■ Elixir discussion forum - https://elixirforum.com/t/you-may-not-
    need-genservers-and-supervision-trees/12947
    

    View Slide