De Python à Erlang - Speaker Deck

Slide 1

Slide 1 text

From P..... to Erlang by Jérôme Renard (swiss army knife @ Swelen)

Slide 2

Slide 2 text

What is Swelen ? Mobile advertising (iOS, Android, WebMobile) RTB

Slide 3

Slide 3 text

The adserver team (2 people) https://github.com/jeromer/ https://github.com/teh-cmc/ no clue about Erlang no clue about functional programming

Slide 4

Slide 4 text

Before the rewrite

Slide 5

Slide 5 text

Adserver was written in P..... (+ MySQL, + Redis, + Nginx, + uwsgi, + ...) by a math genius (initial author, left the company) with piles of "temporary fixes" without any test (what could possibly go wrong ?)

Slide 6

Slide 6 text

in the end a nigthmare to maintain costly to add new features (exponential)

Slide 7

Slide 7 text

We could not keep up with the business

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

New adserver requirements fast (< 50ms max to return an ad) scalable (10x traffic anytime) maintainable observable no HTTP caching synchronous reads asynchronous writes

Slide 11

Slide 11 text

We had to pick a language Python (legacy adserver) PHP (hell no !) C (too complex to do well) Go (lots of interesting concepts) Rust (no stable version available) Erlang

Slide 12

Slide 12 text

Why we chose Erlang designed for fault-tolerance built-in scalability built-in observability (observer, etop) no locks :) (huge win) competition used Erlang

Slide 13

Slide 13 text

With Erlang, you not only have a language. You also get the entire platform for free !

Slide 14

Slide 14 text

Time to learn LYSE (thank you @mononcqc) ton of manual pages (erldocs) lots of Erlang Factory videos Cowboy manual & source code Trial and (lots of) errors

Slide 15

Slide 15 text

Erlang, a week later 1st real world feature ready sent the entire trafic on a single node worked well, scaled well

Slide 16

Slide 16 text

WIN !

Slide 17

Slide 17 text

Time to write the rest

Slide 18

Slide 18 text

We ♥ ETS blazingly fast super simple to use manage 10 different cache layers no longer need Redis

Slide 19

Slide 19 text

We ♥ pattern matching extremely powerful code much safer, with much less error checks (let it crash) much easier to maintain

Slide 20

Slide 20 text

We ♥ releases trivial deployements git push make release tar + scp restart the adserver done (we do no do hot upgrades)

Slide 21

Slide 21 text

We ♥ observability (aka WTF is going on ?) remote shell on any node (shell history support ?) direct access to our API ♥ erl -name jerome -setcookie foo -hidden -remsh x.com -s observer

Slide 22

Slide 22 text

Stupid mistakes learned the hard way 1/2 store BIG terms in ETS -> ETS performance avoid ets:select if possible -> full table scan -> ETS table is lost when process crashes -> create a dedicated supervisor

Slide 23

Slide 23 text

Stupid mistakes learned the hard way 2/2 create an OTP app for anything -> message passing pass BIG terms between process -> message passing

Slide 24

Slide 24 text

Summary

Slide 25

Slide 25 text

We went from unmaitainable adserver (by 2 people) complex infrastructure high response time unable to keep up with the business

Slide 26

Slide 26 text

To single release (aka tarball) response time is ~25ms (p90) can do trafic x8 without even sweating simplified infrastructure one maintainer (me) adding a feature takes 2/3 days in avg (w/ tests)

Slide 27

Slide 27 text

Future plans switch to Mnesia use dialyzer

Slide 28

Slide 28 text

Thoughts about Erlang Erlang not hard to learn, real motivation is required though Thinking in Erlang challenging sometimes Documentation complete, could be more accessible Unicode support could be more "natural"