Upgrade to Pro — share decks privately, control downloads, hide ads and more …

De Python à Erlang

De Python à Erlang

Retour d'expérience de érome Renard (ingénieur chez Swelen) sur la ré-écriture complète de Python à Erlang d'un adserver par 2 personnes qui ne connaissaient rien à Erlang, pourquoi ils ont chois Erlang et le chemin qui nous a amené jusqu'à la mise en production finale.

Meetup 14/10/2015 @ leboncoin.fr

erlang.paris

October 14, 2015
Tweet

More Decks by erlang.paris

Other Decks in Programming

Transcript

  1. From P..... to Erlang
    by Jérôme Renard (swiss army knife @ Swelen)

    View full-size slide

  2. What is Swelen ?
    Mobile advertising (iOS, Android,
    WebMobile)
    RTB

    View full-size slide

  3. The adserver team (2 people)
    https://github.com/jeromer/
    https://github.com/teh-cmc/
    no clue about Erlang
    no clue about functional
    programming

    View full-size slide

  4. Before the rewrite

    View full-size slide

  5. Adserver was
    written in P..... (+ MySQL, + Redis, +
    Nginx, + uwsgi, + ...)
    by a math genius (initial author,
    left the company)
    with piles of "temporary fixes"
    without any test
    (what could possibly go wrong ?)

    View full-size slide

  6. in the end
    a nigthmare to maintain
    costly to add new features
    (exponential)

    View full-size slide

  7. We could not keep up with the
    business

    View full-size slide

  8. New adserver requirements
    fast (< 50ms max to return an ad)
    scalable (10x traffic anytime)
    maintainable
    observable
    no HTTP caching
    synchronous reads
    asynchronous writes

    View full-size slide

  9. We had to pick a language
    Python (legacy adserver)
    PHP (hell no !)
    C (too complex to do well)
    Go (lots of interesting concepts)
    Rust (no stable version available)
    Erlang

    View full-size slide

  10. Why we chose Erlang
    designed for fault-tolerance
    built-in scalability
    built-in observability (observer,
    etop)
    no locks :) (huge win)
    competition used Erlang

    View full-size slide

  11. With Erlang, you not only have a
    language.
    You also get the entire platform
    for free !

    View full-size slide

  12. Time to learn
    LYSE (thank you @mononcqc)
    ton of manual pages (erldocs)
    lots of Erlang Factory videos
    Cowboy manual & source code
    Trial and (lots of) errors

    View full-size slide

  13. Erlang, a week later
    1st real world feature ready
    sent the entire trafic on a single
    node
    worked well, scaled well

    View full-size slide

  14. Time to write the rest

    View full-size slide

  15. We ♥ ETS
    blazingly fast
    super simple to use
    manage 10 different cache layers
    no longer need Redis

    View full-size slide

  16. We ♥ pattern matching
    extremely powerful
    code much safer, with much less
    error checks (let it crash)
    much easier to maintain

    View full-size slide

  17. We ♥ releases
    trivial deployements
    git push
    make release
    tar + scp
    restart the adserver
    done
    (we do no do hot upgrades)

    View full-size slide

  18. We ♥ observability (aka WTF is going on ?)
    remote shell on any node (shell
    history support ?)
    direct access to our API
    ♥ erl -name jerome -setcookie foo
    -hidden -remsh x.com -s observer

    View full-size slide

  19. Stupid mistakes learned the hard way 1/2
    store BIG terms in ETS -> ETS
    performance
    avoid ets:select if possible -> full
    table scan ->
    ETS table is lost when process
    crashes -> create a dedicated
    supervisor

    View full-size slide

  20. Stupid mistakes learned the hard way 2/2
    create an OTP app for anything ->
    message passing
    pass BIG terms between process
    -> message passing

    View full-size slide

  21. We went from
    unmaitainable adserver (by 2
    people)
    complex infrastructure
    high response time
    unable to keep up with the
    business

    View full-size slide

  22. To
    single release (aka tarball)
    response time is ~25ms (p90)
    can do trafic x8 without even
    sweating
    simplified infrastructure
    one maintainer (me)
    adding a feature takes 2/3 days in
    avg (w/ tests)

    View full-size slide

  23. Future plans
    switch to Mnesia
    use dialyzer

    View full-size slide

  24. Thoughts about Erlang
    Erlang not hard to learn, real
    motivation is required though
    Thinking in Erlang challenging
    sometimes
    Documentation complete, could
    be more accessible
    Unicode support could be more
    "natural"

    View full-size slide

  25. Thank you Ericsson & the
    Erlang community

    View full-size slide

  26. Thank you
    [email protected]
    @jeromerenard
    http://github.com/jeromer/

    View full-size slide