Lonestar ElixirConf 2017 talk about vutuv and Phoenix

Transcript

1. Stefan Wintermeyer
   @wintermeyer
   

   View Slide

2. Today I talk about
   •vutuv
   •WebPerformance
   •Phoenix for Newbies
   

   View Slide

3. Business Network
   Think LinkedIn but open-source (MIT), free and fast.
   Bonus: Less annoying emails.
   

   View Slide

4. free = big challenge
   In the first 3 months we got > 200,000 new accounts.
   

   View Slide

5. There are many different
   ways of using Phoenix.
   

   View Slide

6. @chris_mccord @wintermeyer
   The Two Extremes
   

   View Slide

7. @chris_mccord
   Push Messages
   • > 2 Million concurrent
   connections on a
   single server
   • JavaScript
   • Heavy Load
   • Speed
   

   View Slide

8. @wintermeyer
   vutuv
   • No message pushing
   • No JavaScript
   • Heavy Load
   • Speed
   • No money
   

   View Slide

9. vutuv differentiates itself
   from other business
   networks by speed.
   

   View Slide

10. WebPerformance
    Fully Loaded on the browser: 700 ms (DSL in Germany)
    

    View Slide

11. Loading Time from Dulles, VA
    https://www.webpagetest.org/video/compare.php?tests=170131_QQ_BHG,170131_9F_BHH,170131_49_BHJ
    DSL
    

    View Slide

12. Why is WebPerformance
    important?
    

    View Slide

13. Jakob Nielsen’s book
    „Usability Engineering“ from 1993
    Delay User reaction
    0 - 100 ms Instant
    100 - 300 ms Feels sluggish
    300 - 1000 ms Machine is working…
    1 s+ Mental context switch
    10 s+ I’ll come back later…
    

    View Slide

14. Web Search Delay
    Experiment
    Type of Delay Delay (ms)
    Duration
    (weeks)
    Impact on Avg.
    Daily Searches
    Pre-header 100 4 -0.20 %
    Pre-header 200 6 -0.59%
    Post-header 400 6 0.59%
    Post-ads 200 4 0.30%
    Source: https://www.igvita.com/slides/2012/webperf-crash-course.pdf
    

    View Slide

15. Numbers by SOASTA
    

    View Slide

16. Numbers by SOASTA
    

    View Slide

17. Network latency is our
    biggest problem.
    Why?
    

    View Slide

18. TCP
    Source: http://en.wikipedia.org/wiki/Transmission_Control_Protocol
    Latency
    

    View Slide

19. Download a 58 kB file from a US east cost server to a
    client in Frankfurt. Protocol: HTTP
    Roundtrip Time: 80 ms
    Zeit client server
    0 ms SYN -->
    <-- SYN,ACK
    80 ms ACK -->
    GET -->
    <-- 10 TCP Segmente (14.600 Bytes)
    160 ms ACK -->
    <-- 20 TCP Segmente (29.200 Bytes)
    240 ms ACK -->
    <-- 40 TCP Segmente (15.592 Bytes)
    320 ms ACK -->
    Time to download a 58 kB file: 320 ms
    

    View Slide

20. TCP Slow-Start
    KB
    0
    55
    110
    165
    220
    Roundtrip
    1. 2. 3. 4.
    213
    99
    42
    14
    114
    57
    28
    14
    

    View Slide

21. SSL adds at least 

    one roundtrip.

    

    HTTP/2.0 needs SSL.
    

    View Slide

22. HTTPS
    KB
    0
    50
    100
    150
    200
    Roundtrip
    1. 2. 3. 4.
    199
    85
    28
    0
    114
    57
    28
    14
    SSL
    

    View Slide

23. Above the fold target: 28 KB
    KB
    0
    50
    100
    150
    200
    Roundtrip
    1. 2. 3. 4.
    199
    85
    28
    0
    114
    57
    28
    14
    SSL
    

    View Slide

24. Latency is king!
    Bandwidth is not important
    for normal websites.
    

    View Slide

25. Frankfurt, Germany Sydney, Australia
    Sydney: https://www.webpagetest.org/result/170130_3H_S13/

    Frankfurt: https://www.webpagetest.org/result/170130_YJ_SRS/
    DSL
    

    View Slide

26. Frankfurt, Germany Sydney, Australia
    Sydney: https://www.webpagetest.org/result/170130_3H_S13/

    Frankfurt: https://www.webpagetest.org/result/170130_YJ_SRS/
    DSL
    

    View Slide

27. Frankfurt: https://www.webpagetest.org/result/170131_CX_B7D/

    Sydney: https://www.webpagetest.org/result/170131_XZ_B7P/
    3G
    Frankfurt, Germany Sydney, Australia
    

    View Slide

28. Frankfurt: https://www.webpagetest.org/result/170131_CX_B7D/

    Sydney: https://www.webpagetest.org/result/170131_XZ_B7P/
    3G
    Frankfurt, Germany Sydney, Australia
    

    View Slide

29. How do we tackle
    WebPerformance?
    

    View Slide

30. Set a time budget
    and stick to it!
    Our time budget is 1 second within Germany.
    

    View Slide

31. Start Render
    Document Complete
    

    View Slide

32. Important:
    We can not control the
    network!
    

    View Slide

33. What we can control:
    • Transfer Protocol (e.g. HTTP/1.1 vs. HTTP/2.0)
    • Compression (e.g. gzip vs. zopfli vs. brotli)
    • Amount of files
    • File size
    • Time the server needs to generate the HTML
    • Content (e.g. HTML, CSS, JavaScript, Images)
    

    View Slide

34. View Slide

35. File size
    Time on the server
    

    View Slide

36. The Servers
    

    View Slide

37. We use NGINX, MySQL
    and Phoenix Framework.
    Bare metal. Debian Linux.
    

    View Slide

38. Phoenix is about 

    10 times faster 

    than Ruby on Rails.
    Yes, I did a proof of concept of vutuv with Ruby on Rails.
    

    View Slide

39. But just a fast
    programming language is
    not good enough!
    Remember: We have to run vutuv.de on a dime.
    

    View Slide

40. We avoid serving
    freshly generated HTML
    when possible.
    Hard Drives are much cheaper than CPU.
    

    View Slide

41. NGINX is our gatekeeper.
    It checks for cookies and
    routes accordingly.
    I did think of Varnish but hard drive is cheaper than RAM.
    

    View Slide

42. /var/www/vutuv/
    cache
    - index.html
    - index.html.gz
    users
    - xyz.html
    - xyz.html.gz
    No Cookie = Static Content
    Cross-Site Request Forgery (CSRF)
    

    View Slide

43. 1,000,000 users x 28 KB
    = 26 GB
    

    View Slide

44. Rendering a user’s profile: 50 ms.
    Serving a static file: less than 1 ms.
    

    View Slide

45. The Avatars
    A little bit more than just https://github.com/stavro/arc
    

    View Slide

46. View Slide

47. View Slide

48. View Slide

49. View Slide

50. View Slide

51. View Slide

52. guetzli -quality 75 a.jpg /tmp/q75-a.jpg
    convert a.jpg /tmp/q75-a.jpg -fx
    'hypot(65-i, 65-j) < 65 ? u : v' new_a.jpg
    https://github.com/google/guetzli
    

    View Slide

53. By that we save
    15-20% of file size.
    We use https://github.com/stavro/arc for uploading and
    initial conversion and optimize during off peak times.
    

    View Slide

54. Compression
    

    View Slide

55. /var/www/vutuv/
    cache
    - index.html
    - index.html.gz
    users
    - xyz.html
    - xyz.html.gz
    Provide a compressed
    version of a static file.
    

    View Slide

56. html-minifier --case-sensitive
    --collapse-boolean-attributes
    --collapse-whitespace
    --remove-comments
    --remove-optional-tags
    --remove-tag-whitespace
    --sort-attributes
    --sort-class-name
    --collapse-inline-tag-whitespace
    --conservative-collapse a.html > b.html
    Optimize before compress
    

    View Slide

57. Use zopfli instead of gzip.
    https://github.com/google/zopfli
    

    View Slide

58. For Phoenix live content
    use brotli in NGINX.
    https://github.com/google/brotli
    

    View Slide

59. Inlining
    

    View Slide

60. HTTP caching is good
    but for our time budget
    not good enough.
    

    View Slide

61. We inline a lot.
    

    View Slide

62. We inline all the CSS.
    Obviously we don’t use Twitter Bootstrap.
    

    View Slide

63. We inline many images.
    

    View Slide

64. View Slide

65. View Slide

66. Fill the Steps
    KB
    0
    50
    100
    150
    200
    Roundtrip
    1. 2. 3. 4.
    199
    85
    28
    0
    114
    57
    28
    14
    SSL
    

    View Slide

67. Prefetch
    

    View Slide

68. Analyse your workflow and
    prefetch stuff that you’ll
    need in the next step.
    
    […]
    <%= if @conn.assigns[:prefetch] do %>
    
    <% end %>
    
    

    View Slide

69. View Slide

70. Now I’d like to talk
    about Phoenix.
    

    View Slide

71. As a Rails developer it took
    me forever and many attempts
    to like the Phoenix Framework.
    

    View Slide

72. Phoenix does not offer a
    quick reward for newbies!
    By now everybody should understand why speed is
    important if you want somebody to use something.
    

    View Slide

73. Rails offers
    Quick Rewards
    Let’s create a mini blog application on macOS.
    

    View Slide

74. Rails Blog Example
    

    View Slide

75. rails new blog
    cd blog
    rails g scaffold post subject body
    rails db:migrate
    rails s
    0:30 min
    

    View Slide

76. rails new blog
    cd blog
    rails g scaffold post subject body
    rails db:migrate
    rails s
    Same Commands on all
    Operating Systems
    

    View Slide

77. Let’s compare that
    with Phoenix
    The same mini blog application on macOS.
    

    View Slide

78. Phoenix Blog Example
    

    View Slide

79. mix phoenix.new blog
    Y
    cd blog
    vim config/dev.exs
    brew install postgres
    brew services start postgres
    createuser -W --createdb blog
    demo
    mix ecto.create
    mix phoenix.gen.html Post posts subject body
    vim web/router.ex
    resources "/posts", PostController
    mix ecto.migrate
    mix phoenix.server
    3:30 min
    

    View Slide

80. mix phoenix.new blog
    Y
    cd blog
    vim config/dev.exs
    brew install postgres
    brew services start postgres
    createuser -W --createdb blog
    demo
    mix ecto.create
    mix phoenix.gen.html Post posts subject body
    vim web/router.ex
    resources "/posts", PostController
    mix ecto.migrate
    mix phoenix.server
    Different Operating System?
    

    View Slide

81. Rails uses SQLite to
    make it easy for a
    Newbie.
    Phoenix should do the same.
    

    View Slide

82. Rails sets the routes
    for a scaffold.
    Phoenix should do the same.
    

    View Slide

83. mix phoenix.new blog
    cd blog
    mix ecto.create
    mix phoenix.gen.html Post posts subject body
    mix ecto.migrate
    mix phoenix.server
    This should be all
    

    View Slide

84. Thank you!
    www.vutuv.de
    https://github.com/vutuv/vutuv @wintermeyer
    

    View Slide