Raft Consesus in Distributed System

https://yusufs.medium.com/creating-distributed-kv-database-by-implementing-raft-consensus-using-golang-d0884eef2e28

Transcript

1. Raft Consesus in Distributed System Yusuf Syaifudin - 7 May 2021
2. What we will discuss? • What is Consensus Algorithm in Distributed System? • Why do we need Consensus Algorithm? • Current usage of Consensus Algorithm IRL • Introducing Raft • Demo
3. What is Consensus Algorithm in Distributed System? A fundamental problem in distributed computing and multi-agent systems is to achieve overall system reliability in the presence of a number of faulty processes. - https://en.wikipedia.org/wiki/Consensus_(computer_science)
4. Why do we need Consensus Algorithm? Consensus is in anywhere near us: • Have you ever done discussing about where and what you’ll eat at lunch with your friends? • Picking a time with your teammates to schedule a meeting?
5. Why do we need Consensus Algorithm? CAP theorem CAP Theorem is a concept that a distributed database system can only have 2 of the 3: • Consistency – The data is same in all the server nodes(leader or follower), implying the system has nearly instantaneous sync capabilities • Availability – Every request gets a response(success/failure). It requires the system to be operational 100% of the time to serve requests, and • Partition Tolerance – The system continues to respond, even after some of the server nodes fail. This implies that the system maintains all the requests/responses function somehow.
6. Current usage of Consensus Algorithm IRL MongoDB: in Replicaset it actually just one “master” and the rest of them is “follower” (similar to Raft, will discuss later). Write operation only can be done in master node, while follower only serve read operation. https://github.com/mongodb/mongo-go-driver /blob/v1.5.1/mongo/collection.go#L94-L97 https://github.com/mongodb/mongo-go-driver /blob/v1.5.1/mongo/description/server_selecto r.go#L108-L125 https://docs.mongodb.com/manual/reference/ glossary/#std-term-driver
7. Current usage of Consensus Algorithm IRL • Consul: https://www.consul.io/docs/architecture/consensus#raft-protocol-overview
8. Current usage of Consensus Algorithm IRL • Zookeeper https://zookeeper.apache.org/doc/r3.5.1-alpha/zookeeperOver.html
9. Introducing Raft Go to: http://thesecretlivesofdata.com/raft/