Lonestar ElixirConf 2017 talk about vutuv and Phoenix

Slide 1

Slide 1 text

Stefan Wintermeyer @wintermeyer

Slide 2

Slide 2 text

Today I talk about •vutuv •WebPerformance •Phoenix for Newbies

Slide 3

Slide 3 text

Business Network Think LinkedIn but open-source (MIT), free and fast. Bonus: Less annoying emails.

Slide 4

Slide 4 text

free = big challenge In the ﬁrst 3 months we got > 200,000 new accounts.

Slide 5

Slide 5 text

There are many different ways of using Phoenix.

Slide 6

Slide 6 text

@chris_mccord @wintermeyer The Two Extremes

Slide 7

Slide 7 text

@chris_mccord Push Messages • > 2 Million concurrent connections on a single server • JavaScript • Heavy Load • Speed

Slide 8

Slide 8 text

@wintermeyer vutuv • No message pushing • No JavaScript • Heavy Load • Speed • No money

Slide 9

Slide 9 text

vutuv differentiates itself from other business networks by speed.

Slide 10

Slide 10 text

WebPerformance Fully Loaded on the browser: 700 ms (DSL in Germany)

Slide 11

Slide 11 text

Loading Time from Dulles, VA https://www.webpagetest.org/video/compare.php?tests=170131_QQ_BHG,170131_9F_BHH,170131_49_BHJ DSL

Slide 12

Slide 12 text

Why is WebPerformance important?

Slide 13

Slide 13 text

Jakob Nielsen’s book „Usability Engineering“ from 1993 Delay User reaction 0 - 100 ms Instant 100 - 300 ms Feels sluggish 300 - 1000 ms Machine is working… 1 s+ Mental context switch 10 s+ I’ll come back later…

Slide 14

Slide 14 text

Web Search Delay Experiment Type of Delay Delay (ms) Duration (weeks) Impact on Avg. Daily Searches Pre-header 100 4 -0.20 % Pre-header 200 6 -0.59% Post-header 400 6 0.59% Post-ads 200 4 0.30% Source: https://www.igvita.com/slides/2012/webperf-crash-course.pdf

Slide 15

Slide 15 text

Numbers by SOASTA

Slide 16

Slide 16 text

Numbers by SOASTA

Slide 17

Slide 17 text

Network latency is our biggest problem. Why?

Slide 18

Slide 18 text

TCP Source: http://en.wikipedia.org/wiki/Transmission_Control_Protocol Latency

Slide 19

Slide 19 text

Download a 58 kB file from a US east cost server to a client in Frankfurt. Protocol: HTTP Roundtrip Time: 80 ms Zeit client server 0 ms SYN --> <-- SYN,ACK 80 ms ACK --> GET --> <-- 10 TCP Segmente (14.600 Bytes) 160 ms ACK --> <-- 20 TCP Segmente (29.200 Bytes) 240 ms ACK --> <-- 40 TCP Segmente (15.592 Bytes) 320 ms ACK --> Time to download a 58 kB file: 320 ms

Slide 20

Slide 20 text

TCP Slow-Start KB 0 55 110 165 220 Roundtrip 1. 2. 3. 4. 213 99 42 14 114 57 28 14

Slide 21

Slide 21 text

SSL adds at least   one roundtrip.    HTTP/2.0 needs SSL.

Slide 22

Slide 22 text

HTTPS KB 0 50 100 150 200 Roundtrip 1. 2. 3. 4. 199 85 28 0 114 57 28 14 SSL

Slide 23

Slide 23 text

Above the fold target: 28 KB KB 0 50 100 150 200 Roundtrip 1. 2. 3. 4. 199 85 28 0 114 57 28 14 SSL

Slide 24

Slide 24 text

Latency is king! Bandwidth is not important for normal websites.

Slide 25

Slide 25 text

Frankfurt, Germany Sydney, Australia Sydney: https://www.webpagetest.org/result/170130_3H_S13/  Frankfurt: https://www.webpagetest.org/result/170130_YJ_SRS/ DSL

Slide 26

Slide 26 text

Frankfurt, Germany Sydney, Australia Sydney: https://www.webpagetest.org/result/170130_3H_S13/  Frankfurt: https://www.webpagetest.org/result/170130_YJ_SRS/ DSL

Slide 27

Slide 27 text

Frankfurt: https://www.webpagetest.org/result/170131_CX_B7D/  Sydney: https://www.webpagetest.org/result/170131_XZ_B7P/ 3G Frankfurt, Germany Sydney, Australia

Slide 28

Slide 28 text

Frankfurt: https://www.webpagetest.org/result/170131_CX_B7D/  Sydney: https://www.webpagetest.org/result/170131_XZ_B7P/ 3G Frankfurt, Germany Sydney, Australia

Slide 29

Slide 29 text

How do we tackle WebPerformance?

Slide 30

Slide 30 text

Set a time budget and stick to it! Our time budget is 1 second within Germany.

Slide 31

Slide 31 text

Start Render Document Complete

Slide 32

Slide 32 text

Important: We can not control the network!

Slide 33

Slide 33 text

What we can control: • Transfer Protocol (e.g. HTTP/1.1 vs. HTTP/2.0) • Compression (e.g. gzip vs. zopﬂi vs. brotli) • Amount of ﬁles • File size • Time the server needs to generate the HTML • Content (e.g. HTML, CSS, JavaScript, Images)

Slide 34

Slide 34 text

No content

Slide 35

Slide 35 text

File size Time on the server

Slide 36

Slide 36 text

The Servers

Slide 37

Slide 37 text

We use NGINX, MySQL and Phoenix Framework. Bare metal. Debian Linux.

Slide 38

Slide 38 text

Phoenix is about   10 times faster   than Ruby on Rails. Yes, I did a proof of concept of vutuv with Ruby on Rails.

Slide 39

Slide 39 text

But just a fast programming language is not good enough! Remember: We have to run vutuv.de on a dime.

Slide 40

Slide 40 text

We avoid serving freshly generated HTML when possible. Hard Drives are much cheaper than CPU.

Slide 41

Slide 41 text

NGINX is our gatekeeper. It checks for cookies and routes accordingly. I did think of Varnish but hard drive is cheaper than RAM.

Slide 42

Slide 42 text

/var/www/vutuv/ cache - index.html - index.html.gz users - xyz.html - xyz.html.gz No Cookie = Static Content Cross-Site Request Forgery (CSRF)

Slide 43

Slide 43 text

1,000,000 users x 28 KB = 26 GB

Slide 44

Slide 44 text

Rendering a user’s proﬁle: 50 ms. Serving a static ﬁle: less than 1 ms.

Slide 45

Slide 45 text

The Avatars A little bit more than just https://github.com/stavro/arc

Slide 46

Slide 46 text

No content

Slide 47

Slide 47 text

No content

Slide 48

Slide 48 text

No content

Slide 49

Slide 49 text

No content

Slide 50

Slide 50 text

No content

Slide 51

Slide 51 text

No content

Slide 52

Slide 52 text

guetzli -quality 75 a.jpg /tmp/q75-a.jpg convert a.jpg /tmp/q75-a.jpg -fx 'hypot(65-i, 65-j) < 65 ? u : v' new_a.jpg https://github.com/google/guetzli

Slide 53

Slide 53 text

By that we save 15-20% of ﬁle size. We use https://github.com/stavro/arc for uploading and initial conversion and optimize during off peak times.

Slide 54

Slide 54 text

Compression

Slide 55

Slide 55 text

/var/www/vutuv/ cache - index.html - index.html.gz users - xyz.html - xyz.html.gz Provide a compressed version of a static ﬁle.

Slide 56

Slide 56 text

html-miniﬁer --case-sensitive --collapse-boolean-attributes --collapse-whitespace --remove-comments --remove-optional-tags --remove-tag-whitespace --sort-attributes --sort-class-name --collapse-inline-tag-whitespace --conservative-collapse a.html > b.html Optimize before compress

Slide 57

Slide 57 text

Use zopﬂi instead of gzip. https://github.com/google/zopﬂi

Slide 58

Slide 58 text

For Phoenix live content use brotli in NGINX. https://github.com/google/brotli

Slide 59

Slide 59 text

Inlining

Slide 60

Slide 60 text

HTTP caching is good but for our time budget not good enough.

Slide 61

Slide 61 text

We inline a lot.

Slide 62

Slide 62 text

We inline all the CSS. Obviously we don’t use Twitter Bootstrap.

Slide 63

Slide 63 text

We inline many images.

Slide 64

Slide 64 text

No content

Slide 65

Slide 65 text

No content

Slide 66

Slide 66 text

Fill the Steps KB 0 50 100 150 200 Roundtrip 1. 2. 3. 4. 199 85 28 0 114 57 28 14 SSL

Slide 67

Slide 67 text

Prefetch

Slide 68

Slide 68 text

Analyse your workﬂow and prefetch stuff that you’ll need in the next step. […] <%= if @conn.assigns[:prefetch] do %> <% end %>

Slide 69

Slide 69 text

No content

Slide 70

Slide 70 text

Now I’d like to talk about Phoenix.

Slide 71

Slide 71 text

As a Rails developer it took me forever and many attempts to like the Phoenix Framework.

Slide 72

Slide 72 text

Phoenix does not offer a quick reward for newbies! By now everybody should understand why speed is important if you want somebody to use something.

Slide 73

Slide 73 text

Rails offers Quick Rewards Let’s create a mini blog application on macOS.

Slide 74

Slide 74 text

Rails Blog Example

Slide 75

Slide 75 text

rails new blog cd blog rails g scaffold post subject body rails db:migrate rails s 0:30 min

Slide 76

Slide 76 text

rails new blog cd blog rails g scaffold post subject body rails db:migrate rails s Same Commands on all Operating Systems

Slide 77

Slide 77 text

Let’s compare that with Phoenix The same mini blog application on macOS.

Slide 78

Slide 78 text

Phoenix Blog Example

Slide 79

Slide 79 text

mix phoenix.new blog Y cd blog vim config/dev.exs brew install postgres brew services start postgres createuser -W --createdb blog demo mix ecto.create mix phoenix.gen.html Post posts subject body vim web/router.ex resources "/posts", PostController mix ecto.migrate mix phoenix.server 3:30 min

Slide 80

Slide 80 text

Slide 81

Slide 81 text

Rails uses SQLite to make it easy for a Newbie. Phoenix should do the same.

Slide 82

Slide 82 text

Rails sets the routes for a scaffold. Phoenix should do the same.

Slide 83

Slide 83 text

mix phoenix.new blog cd blog mix ecto.create mix phoenix.gen.html Post posts subject body mix ecto.migrate mix phoenix.server This should be all

Slide 84

Slide 84 text

Thank you! www.vutuv.de https://github.com/vutuv/vutuv @wintermeyer