akka-cluster-etcd

Presented at Scalar conference 2016

Transcript

1. akka-cluster-etcd Maciej Biłas
2. Who am I? • Maciej Biłas • Software engineer at Adform • much data, some plumbing • Scala for ~4 years
3. Agenda • Application deployment • Akka Cluster construction • Deployment demo
4. Application deployment a subjective point of view
5. A long time ago in a galaxy far, far away…
6. ftp://
7. A couple of years later…
8. The Old Way • Servers, VMs • Chefs, Ansibles, … • System libraries, packages, … • logging, monitoring, … • Host-centric infrastructure
10. Forget about docker running on your host for a moment
11. … forget about hosts
12. think about containers
14. The New Way
15. +
16. Mesos + Marathon Kubernetes Docker Swarm +
17. Kubernetes • Resource utilisation • Self-healing • Horizontal scaling
18. Kubernetes • Where do my containers live? • What are the IP addresses of their machines?
19. What about
 Akka Cluster?
20. What is Akka Cluster anyway? actors over the network
21. akka-cluster • fault-tolerant • decentralized • peer-to-peer
22. akka-cluster • fault-tolerant • decentralized • peer-to-peer
23. Seed nodes akka.cluster.seed-nodes = [ "akka.tcp://ClusterSystem@host1:2552", “akka.tcp://ClusterSystem@host2:2552" ]
24. akka-cluster-etcd • Service discovery • Akka Extension • github.com/rkrzewski/akka-cluster-etcd
25. Cluster discovery import pl.caltha.akka.cluster.ClusterDiscovery 
 ClusterDiscovery(system).start()
26. etcd • /etc distributed • distributed key-value store • strongly-consistent • Raft consensus protocol
27. akka.cluster { discovery.etcd { host = etcd port = 2379 } }
28. Deployment demo cluster-monitor application
29. + Running an Akka Cluster on Kubernetes
31. Kubernetes • Replication controller • Service
32. gcloud container clusters create “hello-scalar” \ --machine-type=$MACHINE_TYPE \ --num-nodes=3
33. Deployment • etcd • cloud-monitor frontend • cloud-monitor backend
34. Deployment kubectl create -f etcd-rc.yml \ 
 -f etcd-srv.yml \
 -f frontend-rc.yml \
 -f frontend-srv.yml \
 -f backend-rc.yml
35. Deployment kubectl create -f etcd-rc.yml \ 
 -f etcd-srv.yml \
 -f frontend-rc.yml \
 -f frontend-srv.yml \
 -f backend-rc.yml
36. frontend-rc.yml apiVersion: v1 kind: ReplicationController metadata: name: cluster-monitor-frontend spec: replicas: 1 selector: app: cluster-monitor-frontend template: metadata: labels: app: cluster-monitor-frontend spec: containers: - name: cluster-monitor image: eu.gcr.io/sandbox-maciejb/akka-cluster-etcd/monitor:latest imagePullPolicy: IfNotPresent env: - name: CLUSTER_ROLE value: frontend
37. frontend-srv.yml apiVersion: v1 kind: Service metadata: name: cluster-monitor-frontend spec: ports: - port: 8080 selector: app: cluster-monitor-frontend type: LoadBalancer
38. DEMO
39. DEMO BACKUP
41. Horizonal scaling
42. ➜ kubectl get rc NAME DESIRED CURRENT AGE cloud-monitor-backend 1 1 9m cloud-monitor-frontend 1 1 9m etcd 1 1 15m ➜ kubectl scale rc/cloud-monitor-backend --replicas=3 replicationcontroller "cloud-monitor-backend" scaled
43. ➜ kubectl get rc NAME DESIRED CURRENT AGE cloud-monitor-backend 1 1 9m cloud-monitor-frontend 1 1 9m etcd 1 1 15m ➜ kubectl scale rc/cloud-monitor-backend --replicas=3 replicationcontroller "cloud-monitor-backend" scaled
45. ➜ kubectl scale rc/cloud-monitor-backend --replicas=2
47. Self-healing
48. ➜ kubectl get pods NAME READY STATUS RESTARTS AGE cloud-monitor-backend-2zpds 1/1 Running 0 2m cloud-monitor-backend-cxj7r 1/1 Running 0 4m cloud-monitor-frontend-biw43 1/1 Running 0 3m etcd-tmpz6 1/1 Running 0 23m ➜ kubectl delete pods/cloud-monitor-backend-2zpds pod "cloud-monitor-backend-2zpds" deleted ➜ kubectl get pods NAME READY STATUS RESTARTS AGE cloud-monitor-backend-cxj7r 1/1 Running 0 5m cloud-monitor-backend-dqpvd 1/1 Running 0 20s cloud-monitor-frontend-biw43 1/1 Running 0 3m etcd-tmpz6 1/1 Running 0 24m
49. ➜ kubectl get pods NAME READY STATUS RESTARTS AGE cloud-monitor-backend-2zpds 1/1 Running 0 2m cloud-monitor-backend-cxj7r 1/1 Running 0 4m cloud-monitor-frontend-biw43 1/1 Running 0 3m etcd-tmpz6 1/1 Running 0 23m ➜ kubectl delete pods/cloud-monitor-backend-2zpds pod "cloud-monitor-backend-2zpds" deleted ➜ kubectl get pods NAME READY STATUS RESTARTS AGE cloud-monitor-backend-cxj7r 1/1 Running 0 5m cloud-monitor-backend-dqpvd 1/1 Running 0 20s cloud-monitor-frontend-biw43 1/1 Running 0 3m etcd-tmpz6 1/1 Running 0 24m
50. ➜ kubectl get pods NAME READY STATUS RESTARTS AGE cloud-monitor-backend-2zpds 1/1 Running 0 2m cloud-monitor-backend-cxj7r 1/1 Running 0 4m cloud-monitor-frontend-biw43 1/1 Running 0 3m etcd-tmpz6 1/1 Running 0 23m ➜ kubectl delete pods/cloud-monitor-backend-2zpds pod "cloud-monitor-backend-2zpds" deleted ➜ kubectl get pods NAME READY STATUS RESTARTS AGE cloud-monitor-backend-cxj7r 1/1 Running 0 5m cloud-monitor-backend-dqpvd 1/1 Running 0 20s cloud-monitor-frontend-biw43 1/1 Running 0 3m etcd-tmpz6 1/1 Running 0 24m
51. ➜ kubectl get pods NAME READY STATUS RESTARTS AGE cloud-monitor-backend-2zpds 1/1 Running 0 2m cloud-monitor-backend-cxj7r 1/1 Running 0 4m cloud-monitor-frontend-biw43 1/1 Running 0 3m etcd-tmpz6 1/1 Running 0 23m ➜ kubectl delete pods/cloud-monitor-backend-2zpds pod "cloud-monitor-backend-2zpds" deleted ➜ kubectl get pods NAME READY STATUS RESTARTS AGE cloud-monitor-backend-cxj7r 1/1 Running 0 5m cloud-monitor-backend-dqpvd 1/1 Running 0 20s cloud-monitor-frontend-biw43 1/1 Running 0 3m etcd-tmpz6 1/1 Running 0 24m
52. Recap • Application deployment • akka-cluster + etcd • Kubernetes
53. Q&A
54. Thank you!