akka-cluster-etcd - Speaker Deck

Slide 1

Slide 1 text

akka-cluster-etcd Maciej Biłas

Slide 2

Slide 2 text

Who am I? • Maciej Biłas • Software engineer at Adform • much data, some plumbing • Scala for ~4 years

Slide 3

Slide 3 text

Agenda • Application deployment • Akka Cluster construction • Deployment demo

Slide 4

Slide 4 text

Application deployment a subjective point of view

Slide 5

Slide 5 text

A long time ago in a galaxy far, far away…

Slide 6

Slide 6 text

ftp://

Slide 7

Slide 7 text

A couple of years later…

Slide 8

Slide 8 text

The Old Way • Servers, VMs • Chefs, Ansibles, … • System libraries, packages, … • logging, monitoring, … • Host-centric infrastructure

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

Forget about docker running on your host for a moment

Slide 11

Slide 11 text

… forget about hosts

Slide 12

Slide 12 text

think about containers

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

The New Way

Slide 15

Slide 15 text

Slide 16

Slide 16 text

Mesos + Marathon Kubernetes Docker Swarm +

Slide 17

Slide 17 text

Kubernetes • Resource utilisation • Self-healing • Horizontal scaling

Slide 18

Slide 18 text

Kubernetes • Where do my containers live? • What are the IP addresses of their machines?

Slide 19

Slide 19 text

What about  Akka Cluster?

Slide 20

Slide 20 text

What is Akka Cluster anyway? actors over the network

Slide 21

Slide 21 text

akka-cluster • fault-tolerant • decentralized • peer-to-peer

Slide 22

Slide 22 text

akka-cluster • fault-tolerant • decentralized • peer-to-peer

Slide 23

Slide 23 text

Seed nodes akka.cluster.seed-nodes = [ "akka.tcp://ClusterSystem@host1:2552", “akka.tcp://ClusterSystem@host2:2552" ]

Slide 24

Slide 24 text

akka-cluster-etcd • Service discovery • Akka Extension • github.com/rkrzewski/akka-cluster-etcd

Slide 25

Slide 25 text

Cluster discovery import pl.caltha.akka.cluster.ClusterDiscovery   ClusterDiscovery(system).start()

Slide 26

Slide 26 text

etcd • /etc distributed • distributed key-value store • strongly-consistent • Raft consensus protocol

Slide 27

Slide 27 text

akka.cluster { discovery.etcd { host = etcd port = 2379 } }

Slide 28

Slide 28 text

Deployment demo cluster-monitor application

Slide 29

Slide 29 text

+ Running an Akka Cluster on Kubernetes

Slide 30

Slide 30 text

No content

Slide 31

Slide 31 text

Kubernetes • Replication controller • Service

Slide 32

Slide 32 text

gcloud container clusters create “hello-scalar” \ --machine-type=$MACHINE_TYPE \ --num-nodes=3

Slide 33

Slide 33 text

Deployment • etcd • cloud-monitor frontend • cloud-monitor backend

Slide 34

Slide 34 text

Deployment kubectl create -f etcd-rc.yml \   -f etcd-srv.yml \  -f frontend-rc.yml \  -f frontend-srv.yml \  -f backend-rc.yml

Slide 35

Slide 35 text

Deployment kubectl create -f etcd-rc.yml \   -f etcd-srv.yml \  -f frontend-rc.yml \  -f frontend-srv.yml \  -f backend-rc.yml

Slide 36

Slide 36 text

frontend-rc.yml apiVersion: v1 kind: ReplicationController metadata: name: cluster-monitor-frontend spec: replicas: 1 selector: app: cluster-monitor-frontend template: metadata: labels: app: cluster-monitor-frontend spec: containers: - name: cluster-monitor image: eu.gcr.io/sandbox-maciejb/akka-cluster-etcd/monitor:latest imagePullPolicy: IfNotPresent env: - name: CLUSTER_ROLE value: frontend

Slide 37

Slide 37 text

frontend-srv.yml apiVersion: v1 kind: Service metadata: name: cluster-monitor-frontend spec: ports: - port: 8080 selector: app: cluster-monitor-frontend type: LoadBalancer

Slide 38

Slide 38 text

DEMO

Slide 39

Slide 39 text

DEMO BACKUP

Slide 40

Slide 40 text

No content

Slide 41

Slide 41 text

Horizonal scaling

Slide 42

Slide 42 text

➜ kubectl get rc NAME DESIRED CURRENT AGE cloud-monitor-backend 1 1 9m cloud-monitor-frontend 1 1 9m etcd 1 1 15m ➜ kubectl scale rc/cloud-monitor-backend --replicas=3 replicationcontroller "cloud-monitor-backend" scaled

Slide 43

Slide 43 text

Slide 44

Slide 44 text

No content

Slide 45

Slide 45 text

➜ kubectl scale rc/cloud-monitor-backend --replicas=2

Slide 46

Slide 46 text

No content

Slide 47

Slide 47 text

Self-healing

Slide 48

Slide 48 text

➜ kubectl get pods NAME READY STATUS RESTARTS AGE cloud-monitor-backend-2zpds 1/1 Running 0 2m cloud-monitor-backend-cxj7r 1/1 Running 0 4m cloud-monitor-frontend-biw43 1/1 Running 0 3m etcd-tmpz6 1/1 Running 0 23m ➜ kubectl delete pods/cloud-monitor-backend-2zpds pod "cloud-monitor-backend-2zpds" deleted ➜ kubectl get pods NAME READY STATUS RESTARTS AGE cloud-monitor-backend-cxj7r 1/1 Running 0 5m cloud-monitor-backend-dqpvd 1/1 Running 0 20s cloud-monitor-frontend-biw43 1/1 Running 0 3m etcd-tmpz6 1/1 Running 0 24m