FrOSCon 2017 talk about vutuv

The talk was about some of the business and technical challenges (e.g. WebPerformance) creating and hosting https://www.vutuv.de . https://programm.froscon.de/2017/events/1960.html

Transcript

1. fast, free and secure business networking
   https://www.vutuv.de
   

   View Slide

2. A Business Network
   Think LinkedIn or XING but open-source (MIT).
   Free, fast and secure.
   

   View Slide

3. Why would one need an
   other business network?
   

   View Slide

4. Nobody likes LinkedIn or XING.
   Too many annoying emails and pop-ups.
   Nobody likes to pay for premium accounts.
   But those are First World problems!
   

   View Slide

5. LinkedIn and XING only work well 

   in a few countries (but still slow).
   

   View Slide

6. The majority of potential
   business networking users
   use low bandwidth feature
   phones which don’t support
   JavaScript. They have no
   access to LinkedIn at all.
   

   And they can’t pay for a
   premium account.
   

   View Slide

7. Some users need
   extra security. Not
   just HTTPS but Tor.
   Think oppressive
   governments and
   companies which
   use proxies to sniff
   if their employees
   hunt for new jobs.
   

   View Slide

8. Unique Selling Points
   • vutuv accounts are free.

   Revenue is generated with ads and human resources features.
   • vutuv is available.

   LinkedIn can’t be used in most parts of the world. vutuv is part of internet.org and can cover the whole world.
   • vutuv is fast.

   3-8 times faster than LinkedIn.
   • vutuv is secure.

   Your boss or government can’t see if you are hunting for a new job. Not just https but Tor.

   Bonus: We don’t store passwords. We don’t use cookies for non logged in users.
   • vutuv profiles have a better Google ranking

   Because of the side’s speed an average profile gets a better Google ranking.

   For many users it’s there first business networking account at all which results in unique content.
   

   View Slide

9. Traction
   Number of accounts
   0
   250.000
   500.000
   750.000
   1.000.000
   Nov. 2016 Dec. 2016 Jan. 2017 Feb. 2017 Mar. 2017 Apr. 2017
   April: > 5.000.000 unique visitors per month
   

   View Slide

10. > 1,000,000 users?! WOW!
    Did you get VC money?

    Did you buy a new car?
    

    View Slide

11. No, I did not get any VC money.

    

    By now I have given up on it
    and don’t search for investors
    any more.
    

    View Slide

12. Technology Today
    • Phoenix Framework

    Runs on the Erlang-BEAM like WhatsApp.
    • MySQL
    • No JavaScript

    Runs on all devises!
    Source-Code: https://github.com/vutuv/vutuv/
    

    View Slide

13. vutuv differentiates itself
    from other business
    networks by speed.
    

    View Slide

14. WebPerformance
    Fully Loaded on the browser: 700 ms (DSL in Germany)
    

    View Slide

15. Loading Time from Dulles, VA
    https://www.webpagetest.org/video/compare.php?tests=170131_QQ_BHG,170131_9F_BHH,170131_49_BHJ
    DSL
    

    View Slide

16. Why is WebPerformance
    important?
    

    View Slide

17. Jakob Nielsen’s book
    „Usability Engineering“ from 1993
    Delay User reaction
    0 - 100 ms Instant
    100 - 300 ms Feels sluggish
    300 - 1000 ms Machine is working…
    1 s+ Mental context switch
    10 s+ I’ll come back later…
    

    View Slide

18. Web Search Delay
    Experiment
    Type of Delay Delay (ms)
    Duration
    (weeks)
    Impact on Avg.
    Daily Searches
    Pre-header 100 4 -0.20 %
    Pre-header 200 6 -0.59%
    Post-header 400 6 0.59%
    Post-ads 200 4 0.30%
    Source: https://www.igvita.com/slides/2012/webperf-crash-course.pdf
    

    View Slide

19. Numbers by SOASTA
    

    View Slide

20. Numbers by SOASTA
    

    View Slide

21. Network latency is our
    biggest problem.
    Why?
    

    View Slide

22. TCP
    Source: http://en.wikipedia.org/wiki/Transmission_Control_Protocol
    Latency
    

    View Slide

23. Download a 58 kB file from a US east cost server to a
    client in Frankfurt. Protocol: HTTP
    Roundtrip Time: 80 ms
    Zeit client server
    0 ms SYN -->
    <-- SYN,ACK
    80 ms ACK -->
    GET -->
    <-- 10 TCP Segmente (14.600 Bytes)
    160 ms ACK -->
    <-- 20 TCP Segmente (29.200 Bytes)
    240 ms ACK -->
    <-- 40 TCP Segmente (15.592 Bytes)
    320 ms ACK -->
    Time to download a 58 kB file: 320 ms
    

    View Slide

24. TCP Slow-Start
    KB
    0
    55
    110
    165
    220
    Roundtrip
    1. 2. 3. 4.
    213
    99
    42
    14
    114
    57
    28
    14
    

    View Slide

25. SSL adds at least 

    one roundtrip.

    

    HTTP/2.0 needs SSL.
    

    View Slide

26. HTTPS
    KB
    0
    50
    100
    150
    200
    Roundtrip
    1. 2. 3. 4.
    199
    85
    28
    0
    114
    57
    28
    14
    SSL
    

    View Slide

27. Above the fold target: 28 KB
    KB
    0
    50
    100
    150
    200
    Roundtrip
    1. 2. 3. 4.
    199
    85
    28
    0
    114
    57
    28
    14
    SSL
    

    View Slide

28. Latency is king!
    Bandwidth is not important
    for normal websites.
    

    View Slide

29. Frankfurt, Germany Sydney, Australia
    Sydney: https://www.webpagetest.org/result/170130_3H_S13/

    Frankfurt: https://www.webpagetest.org/result/170130_YJ_SRS/
    DSL
    

    View Slide

30. Frankfurt, Germany Sydney, Australia
    Sydney: https://www.webpagetest.org/result/170130_3H_S13/

    Frankfurt: https://www.webpagetest.org/result/170130_YJ_SRS/
    DSL
    

    View Slide

31. Frankfurt: https://www.webpagetest.org/result/170131_CX_B7D/

    Sydney: https://www.webpagetest.org/result/170131_XZ_B7P/
    3G
    Frankfurt, Germany Sydney, Australia
    

    View Slide

32. Frankfurt: https://www.webpagetest.org/result/170131_CX_B7D/

    Sydney: https://www.webpagetest.org/result/170131_XZ_B7P/
    3G
    Frankfurt, Germany Sydney, Australia
    

    View Slide

33. How do we tackle
    WebPerformance?
    

    View Slide

34. Set a time budget
    and stick to it!
    Our time budget is 1 second within Germany.
    

    View Slide

35. Start Render
    Document Complete
    

    View Slide

36. Important:
    We can not control the
    network!
    

    View Slide

37. What we can control:
    • Transfer Protocol (e.g. HTTP/1.1 vs. HTTP/2.0)
    • Compression (e.g. gzip vs. zopfli vs. brotli)
    • Amount of files
    • File size
    • Time the server needs to generate the HTML
    • Content (e.g. HTML, CSS, JavaScript, Images)
    

    View Slide

38. View Slide

39. File size
    Time on the server
    

    View Slide

40. The Servers
    

    View Slide

41. We use NGINX, MySQL
    and Phoenix Framework.
    Bare metal. Debian Linux.
    

    View Slide

42. Phoenix is about 

    10 times faster 

    than Ruby on Rails.
    Yes, I did a proof of concept of vutuv with Ruby on Rails.
    

    View Slide

43. But just a fast
    programming language is
    not good enough!
    Remember: We have to run vutuv.de on a dime.
    

    View Slide

44. We avoid serving
    freshly generated HTML
    when possible.
    Hard Drives are much cheaper than CPU.
    

    View Slide

45. NGINX is our gatekeeper.
    It checks for cookies and
    routes accordingly.
    I did think of Varnish but hard drive is cheaper than RAM.
    

    View Slide

46. /var/www/vutuv/
    cache
    - index.html
    - index.html.gz
    users
    - xyz.html
    - xyz.html.gz
    No Cookie = Static Content
    Cross-Site Request Forgery (CSRF)
    

    View Slide

47. 1,000,000 users x 28 KB
    = 26 GB
    

    View Slide

48. Rendering a user’s profile: 50 ms.
    Serving a static file: less than 1 ms.
    

    View Slide

49. The Avatars
    A little bit more than just https://github.com/stavro/arc
    

    View Slide

50. View Slide

51. View Slide

52. View Slide

53. View Slide

54. View Slide

55. View Slide

56. guetzli -quality 75 a.jpg /tmp/q75-a.jpg
    convert a.jpg /tmp/q75-a.jpg -fx
    'hypot(65-i, 65-j) < 65 ? u : v' new_a.jpg
    https://github.com/google/guetzli
    

    View Slide

57. By that we save
    15-20% of file size.
    We use https://github.com/stavro/arc for uploading and
    initial conversion and optimize during off peak times.
    

    View Slide

58. Compression
    

    View Slide

59. /var/www/vutuv/
    cache
    - index.html
    - index.html.gz
    users
    - xyz.html
    - xyz.html.gz
    Provide a compressed
    version of a static file.
    

    View Slide

60. html-minifier --case-sensitive
    --collapse-boolean-attributes
    --collapse-whitespace
    --remove-comments
    --remove-optional-tags
    --remove-tag-whitespace
    --sort-attributes
    --sort-class-name
    --collapse-inline-tag-whitespace
    --conservative-collapse a.html > b.html
    Optimize before compress
    

    View Slide

61. Use zopfli instead of gzip.
    https://github.com/google/zopfli
    

    View Slide

62. For Phoenix live content
    use brotli in NGINX.
    https://github.com/google/brotli
    

    View Slide

63. Inlining
    

    View Slide

64. HTTP caching is good
    but for our time budget
    not good enough.
    

    View Slide

65. We inline a lot.
    

    View Slide

66. We inline all the CSS.
    Obviously we don’t use Twitter Bootstrap.
    

    View Slide

67. We inline many images.
    

    View Slide

68. View Slide

69. View Slide

70. Fill the Steps
    KB
    0
    50
    100
    150
    200
    Roundtrip
    1. 2. 3. 4.
    199
    85
    28
    0
    114
    57
    28
    14
    SSL
    

    View Slide

71. Prefetch
    

    View Slide

72. Analyse your workflow and
    prefetch stuff that you’ll
    need in the next step.
    
    […]
    <%= if @conn.assigns[:prefetch] do %>
    
    <% end %>
    
    

    View Slide

73. View Slide

74. The future of vutuv?
    

    View Slide

75. Obviously money is the biggest problem.
    Because we don’t use JavaScript we
    can’t use the classic Google Ads system
    to earn money. And charging for
    accounts is a no go either.
    

    View Slide

76. Potential Future Technology
    • Phoenix Framework

    Runs on the Erlang-BEAM like WhatsApp.
    • CouchDB

    Syncs over continents. Super fast anywhere.
    • No JavaScript

    Runs on all devises!
    • Cheap servers

    The world is our data center (blue dots). We are not affected by catastrophes. DNS will reroute to other continents.
    • Short distance to the browser

    This ensures the best browsing experience.
    

    View Slide

77. You can help!
    Please create an account
    and tell your friends about
    www.vutuv.de
    

    View Slide

78. www.wintermeyer-consulting.de
    Ruby on Rails, Phoenix und WebPerformance.
    twitter.com/wintermeyer
    

    View Slide