Upgrade to Pro — share decks privately, control downloads, hide ads and more …

You ain't gonna need to write a GenServer

You ain't gonna need to write a GenServer

GenServers plays a central role in Elixir OTP applications. However, when you overuse it, it can become an anti-pattern and introduce bottlenecks in your system.

In this talk you'll learn:

* What's a GenServer
* When GenServer can harm
* GenServer FOMO (fear of missing out)
* How Elixir ecosystem get you covered

After this talk, you'll understand how putting a GenServer in the wrong place can slowdown in your entire system. You'll understand how the Elixir frameworks and libraries enable you to have reliable concurrency applications. You'll feel no shame of being an Elixir developer and never had to write yourself a GenServer.

Ulisses Almeida

May 25, 2019
Tweet

More Decks by Ulisses Almeida

Other Decks in Programming

Transcript

  1. You ain't gonna need to
    write a GenServer
    by Ulisses Almeida

    View Slide

  2. View Slide

  3. View Slide

  4. Summary
    ■ What is GenServer?
    ■ How can you harm your app by putting
    GenServer in the wrong place?
    ■ Why do people think that they need a
    GenServer all the time?
    ■ Considerations before writing a GenServer

    View Slide

  5. I’m Ulisses Almeida
    ■ Elixir Developer at The
    Coingaming Group
    ■ Brazilian !
    ■ @ulissesalmeida

    View Slide

  6. Learn Functional Programming with Elixir
    ■ 20% off with
    Elixir_Brazil_2019
    promo code
    ■ 3 e-books giveaways

    View Slide

  7. Who 

    We Are
    The Coingaming Group are the
    crypto gaming pioneers who
    operate the world’s leading bitcoin
    gaming and betting brands.
    https://coingaming.io/

    View Slide

  8. View Slide

  9. View Slide

  10. View Slide

  11. Why Tallinn? "
    ★ Low renting costs
    ★ Free transportation for
    residents
    ★ E-estonia
    ★ Safe
    ★ Low taxes
    ★ Accessible travels
    ๏ Cold and Dark ☃
    ๏ Estonian is the first
    language "
    ๏ Russian is the second
    language %
    ๏ Eastern/nordic culture
    is different

    View Slide

  12. Ok, GenServers
    A quick recap

    View Slide

  13. https://twitter.com/ulissesalmeida/status/1125455706470088704

    View Slide

  14. View Slide

  15. View Slide

  16. View Slide

  17. View Slide

  18. GenServer
    ■ Useful for: Client-Server
    relation
    ■ Keep state
    ■ Execute code
    asynchronously
    (handle_cast)
    ■ Synchronously too
    (handle_call)
    ■ Provide tracing and error
    reporting features
    ■ Can be supervised
    A process with a
    powerful set of
    features

    View Slide

  19. Shades of a GenServer
    ■ Agent
    ■ Task
    ■ Task.Supervisor
    ■ Supervisor
    ■ DynamicSupervisor

    View Slide

  20. CalculatorServer
    Baby
    1+1?
    1.
    2.
    3.
    From: Baby

    1+1?
    4.
    1+1=2
    5.

    View Slide

  21. CalculatorServer
    Baby
    7.
    6.
    From: 

    MyCalcGenServer

    2
    8.

    View Slide

  22. Live demo
    “…maybe I should have
    recorded a video”

    View Slide

  23. View Slide

  24. View Slide

  25. View Slide

  26. View Slide

  27. View Slide

  28. View Slide

  29. View Slide

  30. View Slide

  31. View Slide

  32. UÉ!?

    View Slide

  33. hahahahah GenServer

    bottleneck

    omg what I have 

    done!?

    View Slide

  34. Holy %#@!
    ~ 200 times slower

    ~ 10 times more
    memory
    120km/h
    1.6km/h
    ~75 times slower
    10 times bigger

    View Slide

  35. What happened!?

    View Slide

  36. CalculatorServer
    AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
    AAAAAAAAAAAAAAAA

    View Slide

  37. Without GenServer…

    View Slide

  38. CalculatorServer

    View Slide

  39. CalculatorServer




    ta pegando fogo bicho!
    !

    View Slide

  40. Why people want it so
    much?
    Can we play just with
    functions?

    View Slide

  41. Erlang OTP
    ■ Process hierarchy
    ■ Restarting strategies
    ■ Concurrency
    ■ GenServer plays the
    central role

    View Slide

  42. wait, you said
    concurrency…

    View Slide

  43. GenServer checklist
    ■ Model state accessed by multiple
    processes
    ■ Run multiple tasks concurrently
    ■ Communicate between servers
    ■ Gracefully handle clean startup and exit
    concerns
    https://pragprog.com/book/tvmelixir/adopting-elixir

    View Slide

  44. CalculatorServer checklist
    ❌Model state accessed by multiple
    processes
    ❌Run multiple tasks concurrently
    ❌Communicate between servers
    ❌Gracefully handle clean startup and exit
    concerns
    https://pragprog.com/book/tvmelixir/adopting-elixir

    View Slide

  45. CalculatorServer has no
    reason to be a GenServer
    A GenServer must
    never be used for
    code organization
    purposes
    https://hexdocs.pm/elixir/GenServer.html
    GenServer to organize 

    domain, business

    contexts
    GenServer to model 

    runtime properties

    of your system

    View Slide

  46. Fear of missing
    out?
    Am I a proper Elixir
    developer If I'm not
    doing GenServers?

    View Slide

  47. I have been working with Elixir for ~3 years
    ■ I never wrote a production GenServer
    ■ I wrote GenServers only in recruitment
    processes and for fun
    ■ I deleted some unnecessary production
    GenServers
    ■ I know excellent developers that never
    wrote one

    View Slide

  48. Elixir developers, no
    shame in never had to
    create a GenServer.

    View Slide

  49. Ecosystem got you
    covered
    Or… things that you should consider
    before writing a GenServer

    View Slide

  50. Let's think about a typical
    web application

    View Slide

  51. Phoenix
    Elixir
    Database
    Legacy
    Legacy of the Legacy
    Users

    View Slide

  52. View Slide

  53. Users & App

    View Slide

  54. Users -> App checklist
    ✔ We need to handle multiple users
    connections concurrently
    ✔ Other processes might send messages to
    that connection
    ✔ We have to handle connections openings
    and closings gracefully.

    View Slide

  55. GenServers!

    View Slide

  56. However…
    ■ Each user’s request
    has a process that runs
    concurrently
    ■ You can focus on the
    functional part
    ■ (functional part?
    functional
    programming )
    Phoenix and
    Cowboy handles
    that for you

    View Slide

  57. App &
    Database

    View Slide

  58. App & Database checklist
    ✔Databases are an external service
    communication
    ✔We have to provide a limited number of
    connections to a lot of processes
    ✔These connections have to run
    ✔We need to handle connection openings
    and closings gracefully

    View Slide

  59. GenServers!!!!

    View Slide

  60. However…
    ■ Ecto supports a pool of
    connections
    ■ Ecto handle that
    concurrently
    ■ Work here is also done
    ■ You can focus on the
    business
    Ecto handles that
    for you

    View Slide

  61. App &Other
    Services

    View Slide

  62. App & Other Services checklist
    ✔ Usually, we communicate through a network
    ✔ We might need to offer a pool as a backpressure
    mechanism to not bring down weak services
    ✔ These pools have to run multiple requests
    concurrently
    ✔ Gracefully handle connections opening and
    closings

    View Slide

  63. GenSeerveerss!!!

    View Slide

  64. However…
    ■ You can start simple,
    for example, no pool
    ■ Watch out for big
    timeouts
    ■ Circuit breakers can
    help you out
    ■ Choose the libraries,
    and you can focus on
    features
    Hackney and other
    libraries handles
    that for you

    View Slide

  65. NoSQL storage

    View Slide

  66. Not everything is stored in SQL databases
    ■ Examples: caching,
    session, users
    presence
    ■ Transient or Persistent?
    ■ How big will it be?
    ■ Write vs Read
    ✔ This data will be
    accessible by multiple
    processes

    View Slide

  67. GenSeeerveerS!!!!!

    View Slide

  68. WITH ETS!!!!!!!

    View Slide

  69. Phoenix
    Elixir
    Elixir
    ETS table
    Phoenix
    Elixir
    ETS table
    Phoenix
    Elixir
    ETS table
    User

    View Slide

  70. However…
    ■ A better abstraction reveals
    intention, for example,
    Cachex vs GenServer + ETS
    ■ How does your deploy
    work?
    ■ Shutdown and booting the
    app will lose GenServer/
    ETS data
    ■ Redis for the rescue
    ■ Discuss the right tool for
    the job, compare the
    tradeoffs
    You might want to
    use a specific
    abstraction.

    View Slide

  71. Serializable
    operations

    View Slide

  72. Serializable
    operations
    Some resources
    can’t be changed at
    same time
    1
    1
    5
    I want to withdraw 2
    I want to withdraw 6
    Company Wallet
    ✔ Manage state accessed
    by multiple processes

    View Slide

  73. …GenServer?

    View Slide

  74. However…
    ■ You would need an
    Erlang cluster to run a
    global GenServer
    ■ What happens with
    network partition?
    ■ Use your database
    power
    ■ Transactions or locks in
    the database
    ■ Discuss with your team,
    compare the tradeoffs
    You environment
    might not allow.
    a.k.a Heroku

    View Slide

  75. Background
    tasks

    View Slide

  76. Tasks outside of a request/response cycle
    ■ Welcome e-mail
    ■ Push notifications
    ■ Scheduled tasks
    ✔ We need to spawn or
    schedule these
    background tasks
    ✔ We need to run the
    background tasks
    concurrently

    View Slide

  77. Dynamic Supervisors 

    & Task!!!

    View Slide

  78. However…
    ■ GenServers aren’t
    persistent
    ■ Shutting downs might
    lose data
    ■ RabbitMQ has persistent
    queues
    ■ Libraries that use Redis
    to store the jobs and
    their arguments
    ■ Discuss with your team,
    compare the tradeoffs
    How to handle the
    failures? Do you
    need some
    persistence?

    View Slide

  79. We covered a lot 

    of considerations before 

    “let's write a GenServer” decision
    And they are still not
    over

    View Slide

  80. GenServers are
    fundamental

    View Slide

  81. Elixir frameworks and
    libraries cover most of
    the use cases that you
    would need to write one

    View Slide

  82. But it doesn’t mean
    you should not learn
    about GenServer

    View Slide

  83. You need to know
    GenServer to tune
    their configurations

    View Slide

  84. When stuff goes
    wrong, you’ll need
    to debug and fix it.

    View Slide

  85. You won’t might need
    to write a GenServer
    95% of time…

    View Slide

  86. But that 5%…

    View Slide

  87. Then, keep
    learning!

    View Slide

  88. Remember

    View Slide

  89. Putting a GenServer in
    the wrong place brings
    more harm than good

    View Slide

  90. It’s good to compare
    and to discuss the
    tradeoffs with your
    team.

    View Slide

  91. There’s no shame
    in never need to
    write a GenServer

    View Slide

  92. Thank you

    View Slide

  93. References
    ■ Adopting Elixir - https://pragprog.com/book/tvmelixir/adopting-
    elixir
    ■ Spawn or not spawn - https://www.theerlangelist.com/article/
    spawn_or_not
    ■ You may not need a GenServer - https://pragtob.wordpress.com/
    2019/04/24/you-may-not-need-genservers-and-supervision-trees
    ■ GenServer (Elixir Docs) - https://hexdocs.pm/elixir/GenServer.html
    ■ Elixir discussion forum - https://elixirforum.com/t/you-may-not-
    need-genservers-and-supervision-trees/12947

    View Slide