Techniques for Building Disruptive Software For a Concurrent World

How do you build for Scale? How about if you could write software that can run forever, survive crashes and self-heal? A software that is always online without downtime and provides service to their clients even when faced with unexpected circumstances.

Why you should care: we are in a highly connected age of internet of things with multiple connected devices, social networks, multiplayer games, high traffic web servers etc. To meet the demand and to always provide uninterrupted services to user, regardless of unexpected bugs, hardware or software failures, Software systems have to resilient and highly available

Transcript

1. eNigeria 2018 Masterclass Session Shuaib Afegbua |> @afegbuas LET IT CRASH: TECHNIQUES FOR BUILDING DISRUPTIVE SOFTWARE For a Concurrent World
2. SHUAIB AFEGBUA @afegbuas Software Craftsman Long love affair with Java. Also writes Elixir, Scala and others
3. My past, present and possible future • Computer Science major (FUTMINNA and ABU) • Worked at Mott Macdonald/Cambridge Education for a long time and some other places • Got tired, quit and stayed at home. @afegbuas
4. My past, present and possible future • Got bored and started #buildingTheFuture (what the hell is that?) at ECTORIUM • I love building things - comvers, spaces.ng, powershop.ng and some others I can’t list here • Amateur writer • I have an addiction bleeding edge technology • Got another addiction - READING @afegbuas
5. Some stuff I built or still building with friends
6. WWW.POWERSHOP.NG
7. WWW.SPACES.NG
8. Sometimes I write https://medium.com/@afegbua
9. WECODE Community http://bit.ly/graphql-saturday
10. In the beginning was? @afegbuas
11. IDEA You have this great idea that will change the world
12. Prototyping You start small, make wireframes and to test your ideas
13. An MVP Something you can ship out to user. A Minimum Valuable, Viable, Validating, Product.
14. Launch & Feedback You test the market and get feedback from users, sometimes cruel responses
15. Iterate
16. Traction Your idea, product, etc. gains popularity or acceptance.
17. Scale {:ok, “my app is highly available”} OR {:error, “I don die”}
18. What the heck is this talk about? @afegbuas
19. How and why should you care about scaling? @afegbuas
20. Let’s understand some terms I will throw around @afegbuas
21. Program @afegbuas Software operating system OS process Concurrency Parallelism Go JVM Scheduler OTP Erlang Actor model
22. Why should you I build for this? @afegbuas
23. Networked age @afegbuas
24. 1980s and early 90s Mostly desktops @afegbuas
25. 1989 Tim Berners Lee @afegbuas
26. 2005 1 BILLION USERS @afegbuas
27. 2017 3.4 BILLION USERS @afegbuas
28. @afegbuas 60 billion Whatsapp messages Over 2.5 quintillion bytes of data daily (2.5 x 1018 or 2 trillion million) => 10 million blu-ray disc 24/7 connected systems Social networks Web servers Multi players game
29. What becomes imperative? “The system must be responsive, regardless of the number of connected clients. The impact of unexpected errors must be as minimal as possible, instead of affecting the entire system. It’s acceptable if an occasional request fails due to a bug, but it’s a major problem when the entire system becomes completely unavailable. Ideally, the system should never crash or be taken down, not even during a software upgrade. It should always be up and running, providing the service to its clients.” ... SAS ̌ A JURIC ́
30. What ? @afegbuas
31. What is a highly available system A software that is always online without downtime and provides service to their clients even when faced with unexpected circumstances. Hence it has to be fault tolerant, scalable, responsive and distributed. @afegbuas
32. Fault tolerance • A system has to keep working when something unforeseen happens. • Minimise, isolate, and recover from the runtime errors effects to keep the system up @afegbuas
33. Scalability • Handle increase in load • Additional hardware without change in code @afegbuas
34. Distributed • Run on multiple computers and handle failed • Horizontal scaling @afegbuas
35. Responsiveness • System reasonably fast and responsive • No prolonged request handling, even if the load increases or unexpected errors happen • Lengthy tasks shouldn’t block the rest of the system or have a significant effect on performance. @afegbuas
36. Live updating • Pushing update code shouldn’t require without restarting any servers @afegbuas
37. Real-life Use Cases
38. WhatsApp • 55 Billion messages per day • 35 engineers at 450M user, 50 at 900M users • 1 Billion active daily users • 1.3 billion active monthly users • 1 billion videos per day • 4.5 Billion photos per day • 60 supported languages
39. BleacherReport • World 2nd largest sports website, gets 1.5 billion page views per month and 250,000 users at its peak • Initially on Ruby on Rails, but later, they experienced issues with scaling. • They moved from 150 to 5 AWS servers. • From 50 engineers to 15
40. Pinterest • 250 million monthly active users as of October 2018 • Rewrote their api: 50 percent response time is around 500 microseconds with a 90 percent response time of 800 microseconds., microseconds! • 90% reduction in lines of code 10,000 to 1000
41. Moz • An analytic company that helps clients to be more visible in search engines • They adopted Elixir to create a database-free solution for analytics That became faster than the database solution
42. BET 365 • They handle over 6 million HTTP requests and more than 500,000 database transactions per second! • They deliver live online experiences in 17 languages to over 11 million customers worldwide.
43. CDFIPB Recruitment • You all know tales from online recruitment in Nigeria • 41K concurrent users at peak period. An average of 1K users on low period • We had two commodity servers hosted on linode
44. Aeternity • A blockchain company • Need to scale their distributed system and peer-to-peer network by providing fault tolerant systems capable of scaling to billions of potential users.
45. Others
46. {:ok, “that’s enough”} @afegbuas
47. How to handle a billion users? @afegbuas
48. The Erlang/OTP BEAM as a case study @afegbuas
49. Concurrency Many things happening at a time Parallelism Two or more things happening simultaneously @afegbuas
50. Concurrency models @afegbuas
51. Shared memory concurrency • Components communicate by altering content of memory locations. • Mainly handled by the OS • Uses locks to to communicate between threads • C#, Java, C, C++, .
52. Message based concurrency • Communicate by sharing messages between processes (Actors) • A process is an independent and usually different from OS process • Erlang, Elixir, LFE, Akka (Scala and Java), Dart, occam.
53. How and why do this in Erlang/OTP ? @afegbuas
54. Erlang Erlang is platform for building highly available systems. It consisting of four distinct parts: • the language, • the virtual machine, • the framework • the tools.
55. How Erlang? A little bit of story of the tortoise fell from heaven and cracked it shell Conceived by Ericsson team led by Joe Armstrong, in the 80s to solve their telecom needs Sas Juric: “A telephone network should always operate regardless of the number of simultaneous calls, unexpected bugs, or hardware and software upgrades taking place.” @afegbuas
56. Design goal To provide 99.9999999% availability = 1 second downtime every 32 years Erlang is sometimes called a concurrency-oriented language.. • Compiled • Immutable state • Functional paradigm • Abstracted Processes • Message based concurrency • Supervision tools • Distributed @afegbuas
57. Erlang Concurrency Model @afegbuas
58. Process • The process is the basic unit of concurrency • Lightweight and cheap - about 1kb/2kb initial memory footprint. Can grow and shrink as required • No shared state/memory thus never need to compete for locks for access to shared data. • They communicate by sending messages. The receiver gets a read only copy of the sender’s message (transparent distribution)
59. ERLANG VM (BEAM) A single operating system process SCHEDULER THREAD CPU Image credits: https://pragmaticstudio.com/
60. Multi Core computer Image credits: https://pragmaticstudio.com/
61. So how does the BEAM handles this? @afegbuas
62. Fault Tolerance • Erlang processes are completely isolated from each other • They crash alone • Supervisors are present to detects and manage failures
63. Distribution • Erlang processes are completely isolated from each other • They crash alone • They communicate by sending messages. The receiver gets a read only copy of the sender’s message (transparent distribution)
64. Scalability • No shared state/memory thus never need to compete for locks for access to shared data. • No locks, mutexes, or semaphores • Erlang processes are completely isolated from each other • The virtual machine can efficiently parallelize the execution of processes as much as possible • Erlang programs take advantage of all available cores to execute programs
65. Distribution • Same mode of communication regardless of the machine where the process is running be it single or remote machine; local and remote • Handles failures across machines
66. Responsiveness • Excellent scheduler preemption • I/0 operations are delegated to separate threads internally • Excellent garbage collection
67. Erlang code can be difficult and verbose So you can write nicer erlang: Elixir @afegbuas
68. So WTF is Elixir ? @afegbuas
69. Elixir Elixir is a dynamic, functional language designed for building scalable and maintainable applications and runs on the Erlang virtual machine (BEAM). @afegbuas
70. Elixir ? All the goods of Erlang plus • Concise - Erlang is verbose • Metaprogramming with Macros. • Polymorphism using Protocols • Better tools - mix and hex • Friendlier syntax • Seamless interoperability @afegbuas
71. Abstracting process Agents and Task • Start a process • A recursive function in an infinite loop passing the state for each function call • A way to handle message • How to respond to these messages @afegbuas
72. OTP framework and behaviours @afegbuas
73. OTP behaviours GenServer • Generic Server behaviour is an abstraction of client/server functionality GenEvent • A behaviour module that Provides event-handling support Supervisor • Provides error handling and recovery in concurrent systems Application • Generic implementation of components and libraries @afegbuas
74. So what next? • Build fast and build well • Think design, test fast and iterate. • Adopt the right tools. • Create appealing user experience • Create a knowledge base company • Avoid over bloated engineering team. Recruit for your need
75. 1. https://moz.com/devblog/moz-analytics-db-free/ 2. https://www.netguru.co/blog/10-companies-use-elixir 3. https://www.manning.com/books/elixir-in-action 4. https://www.erlang-solutions.com/resources/case-studies.html#aol-marketplace-rea l-time-bidding-and-platform-powered-by-erlang-9-doc 5. https://www.zdnet.com/article/whatsapp-now-one-billion-people-send-55-billion- messages-per-day/ 6.
76. Thank you @afegbuas