ETCD

Transcript

1. ETCD
2. What is ETCD? ETCD is a distributed key value store produced by CoreOS (Red Hat) ETCD is conceptually a HashMap that is persisted and replicated across a set of machines. It is semantically similar to other distributed lock stores and service discovery systems (eg; Apache Zookeeper and HashiCorp Consul)
3. What is ETCD? ETCD provides strongly consistent storage distributed across a set of nodes ETCD implements the raft consensus algorithm (as opposed to Paxos) to achieve consensus on the value committed to a particular key Popular use cases included anything involving distributed coordination Distributing configuration/metadata to multiple machines, Coordinating access to resources, distributed locking, service discovery etc.
4. Example: Distributed Locking Entry: myapp.com.au - Machine01 - Machine02 - …
5. Example: Kubernetes
6. When not to use ETCD - Do not use ETCD if you need SQL style querying - Do not use ETCD if you need to store > 8 GB - If you need full featured service discovery, do not use ETCD.
7. KeyValueAPI
8. KeyValueAPI Infrastructure
9. ETCD-Proxy Flows
10. Shortcomings - There is no managed backup for any data stored in etcd. - ETCD is only deployed to OpenStack Cloud 2. - Reconfiguring roles and access is not dynamic. ~1 month lead time - You cannot store sensitive data in either key-path or values
11. Be Prepared... ...Over time, we found that the failures of the global instance of Chubby consistently generated service outages, many of which were visible to end users. As it turns out, true global Chubby outages are so infrequent that service owners began to add dependencies to Chubby assuming that it would never go down. Its high reliability provided a false sense of security because the services could not function appropriately when Chubby was unavailable, however rarely that occurred...
12. ...(Google) SRE makes sure that global Chubby meets, but does not significantly exceed, its service level objective. In any given quarter, if a true failure has not dropped availability below the target, a controlled outage will be synthesized by intentionally taking down the system. In this way, we are able to flush out unreasonable dependencies on Chubby shortly after they are added. Doing so forces service owners to reckon with the reality of distributed systems sooner rather than later...