Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Simply Distributed
Search
Nugroho Herucahyono
October 22, 2015
Technology
0
120
Simply Distributed
Nugroho Herucahyono
October 22, 2015
Tweet
Share
More Decks by Nugroho Herucahyono
See All by Nugroho Herucahyono
Choosing the right technology
xinuc
0
180
This Talk is so Meta
xinuc
1
130
A Tale of a Happy Programmer
xinuc
0
150
Rails on Wiradipa - Jakarta.rb Februari 2012 - Hafiz Badrie Lubiz
xinuc
1
160
Why Ruby? - View from business aspect - Jakarta.rb Februari 2012 - Fajrin Rasyid
xinuc
1
350
Other Decks in Technology
See All in Technology
ネットワーク保護はどう変わるのか?re:Inforce 2025最新アップデート解説
tokushun
0
210
american aa airlines®️ USA Contact Numbers: Complete 2025 Support Guide
aaguide
0
180
開発生産性を組織全体の「生産性」へ! 部門間連携の壁を越える実践的ステップ
sudo5in5k
2
7.2k
生成AI開発案件におけるClineの業務活用事例とTips
shinya337
0
260
タイミーのデータモデリング事例と今後のチャレンジ
ttccddtoki
6
2.4k
使いたいMCPサーバーはWeb APIをラップして自分で作る #QiitaBash
bengo4com
0
1.9k
Yahoo!しごとカタログ 新しい境地を創るエンジニア募集!
lycorptech_jp
PRO
0
110
SmartNewsにおける 1000+ノード規模 K8s基盤 でのコスト最適化 – Spot・Gravitonの大規模導入への挑戦
vsanna2
0
140
Flutter向けPDFビューア、pdfrxのpdfium WASM対応について
espresso3389
0
130
品質と速度の両立:生成AI時代の品質保証アプローチ
odasho
1
360
生まれ変わった AWS Security Hub (Preview) を紹介 #reInforce_osaka / reInforce New Security Hub
masahirokawahara
0
470
LLM時代の検索
shibuiwilliam
2
170
Featured
See All Featured
A Modern Web Designer's Workflow
chriscoyier
695
190k
Practical Orchestrator
shlominoach
189
11k
Producing Creativity
orderedlist
PRO
346
40k
Site-Speed That Sticks
csswizardry
10
690
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
48
2.9k
BBQ
matthewcrist
89
9.7k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
251
21k
Building a Modern Day E-commerce SEO Strategy
aleyda
42
7.4k
Writing Fast Ruby
sferik
628
62k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
357
30k
How to Ace a Technical Interview
jacobian
278
23k
A Tale of Four Properties
chriscoyier
160
23k
Transcript
Simply Distributed KNIF 2015, Bandung
Who? Nugroho Herucahyono @xinuc Programmer @Bukalapak
Keandalan Sistem dalam Mendukung Penyediaan Layanan
“Andal" => reliable & scalable
reliable: fault tolerant scalable: able to grow
How a reliable & scalable system built?
Most systems start small
Typical web application Webserver Database Client
Typical web application • Need more features • Serve more
users • Need to be more reliable
More features Add more code Split the system
More users Need to scale machine limitation add more machines
More business value Need more reliable System should be fault
tolerant Self healing Backup, redundancy
How we do it? Current “Best practice”: • Split system
into smaller services • Communicate with http • Scale independently • Gracefully handle failure
How we do it? Load Balancer Search Engine Client Authentication
Content Management Search Scheduler Transaction Database 2 Database 1 Job Queue
How we do it? Current “Best practice” apparently is not
the best: • Requires massive change to our system • Manual load balancing, replication • Manual resource management • Inefficient communication (http? really?)
How we do it? Load Balancer Search Engine Client Authentication
Content Management Search Scheduler Transaction Database 2 Database 1 Job Queue Too Complicated!!
What would a good computer scientist do?
Introduce a new layer of abstraction!
A new layer of abstraction • Handle resource management •
Handle load balancing • Handle service communication • Handle service failure • Handle replication
A new layer of abstraction We need “Operating System” of
a cluster
A new layer of abstraction Cluster Operating system Operating System
Pod Application Hardware Operating System Pod Application Hardware Operating System Pod Application Hardware
Cluster Operating System • Build in interprocess communication • Build
in monitoring & supervision • Automatic load balancing • Automatic resource management • Scale with little / no system modification
What do we have now? • Erlang VM & OTP
• Docker, Kubernetes
Erlang VM & OTP node 1 erlang vm erlang processes
node 2 erlang vm erlang processes
Erlang VM & OTP Supervisor Supervisor Worker Worker Worker Worker
Worker OTP Supervision Tree
Erlang VM & OTP • Build in interprocess communication √
• Build in monitoring & supervision √ • Automatic load balancing X • Automatic resource management X • Scale with little / no system modification √
Erlang VM & OTP • The building block is too
low level? (erlang processes) • Your application need to be written in erlang (or other erlang vm languages)
Docker • Like virtual machine, but much lighter • Encapsulate
our application into single “executable” • Remove dependencies, development vs production headache
Docker Host OS Docker Container Container Container Server
Kubernetes • Manages & monitors containers • Resource allocation between
containers
Kubernetes Host OS Docker Container Container Pod Host OS Docker
Container Container Pod Node 1 Node 2 Kubernetes
Docker & Kubernetes • Build in interprocess communication X •
Build in monitoring & supervision √ • Automatic load balancing √ • Automatic resource management √ • Scale with little / no system modification X
Docker & Kubernetes • No build in interprocess communication •
Still have to modify the system (split into smaller services) • Too complicated
Can we do better?
Let’s zoom out a bit • Service vs Process •
Node vs Core They’re conceptually the same
Maybe we can push down the abstraction layer?
What if, our “Cluster operating system” is a real Operating
System?
We need a real “Distributed Operating System”
Distributed Operating System Operating System Application Hardware Hardware Hardware
Distributed Operating System • Encapsulate multiple machines as a single
node • Transparent from user / application point of view • Handle load balancing, replication & distribution automatically • Better yet, if we can add more machine on the fly
Is it possible? I have no idea.
We’ve done something similar • Raid • Multiple disk, single
volume • Transparent from applications • Automatic failure handling & replication
We need Raid for CPU & Memory
Or maybe, we can push it down further, to the
hardware level?
We need a real “Distributed Motherboard” :D
Distributed Operating System Operating System Application Hardware
Distributed Motherboard • Node 1, 32 Cores, 32 GB RAM
• Node 2, 32 Cores, 32 GB RAM • Detected by operating system as 1 Node, 64 Cores, 64 GB RAM
Distributed Motherboard • We can add more node, on the
fly • Motherboard will communicate between each other • Abstract their resources as a SINGLE NODE
Again, is it possible? I have no idea.
We’ve done that too • Hardware Raid Controller • Multiple
Disk, detected as a single hardware • Transparent from operating system & application
Too much wishful thinking?
Why does it matter?
Why does it matter? Scalable & Reliable system is a
SOLVED problem We already have Google, Facebook, etc as a prove
Why does it matter? • Scalable & Reliable system is
not easy & cheap • Need a group of highly skilled experts to build
Case Study: WhatsApp
Case Study: WhatsApp • WhatsApp use Erlang VM & OTP
• They can scale it without adding too much complexity
Case Study: WhatsApp
Case Study: WhatsApp We need more companies like WhatsApp
Small Startups? Can 4-fresh-graduate startup create a product used by
a billion users?
Non profits? Can we create non profit system than serve
billons of users?
./bukalapak
more research on this, please :)
Thank you