Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Elastic scaling in a (micro)service oriented architecture

Elastic scaling in a (micro)service oriented architecture

Splitting an application up into multiple independent services can be a good way to keep it scaling and ensure stability and developer productivity in larger, growing teams. But just splitting the codebase, creating APIs and deploying the code on some servers is not enough, somehow your services need to know where and how other services are accessible. Classical approaches like hardcoding everything in every service or having a central load-balancer can quickly lead to problems in terms of scalability and maintainability. In this talk I'll show how we at ResearchGate tackled this challenge. With the help of tools like Consul and haproxy we created a setup that allows us to quickly boot and shutdown services. This ensures that all servers are utilized optimally and load spikes can be reacted upon quickly and automatically.

Bastian Hofmann

July 01, 2017
Tweet

More Decks by Bastian Hofmann

Other Decks in Programming

Transcript

  1. Scaling
    microservices
    with Kubernetes
    @BastianHofmann

    View full-size slide

  2. Microservices

    View full-size slide

  3. http://blog.philipphauer.de/microservices-nutshell-pros-cons/
    Monolith Microservices

    View full-size slide

  4. Stricter separation
    of concerns

    View full-size slide

  5. Diverse technology
    stacks

    View full-size slide

  6. Things that you
    don’t want to do in
    language X

    View full-size slide

  7. Learning Curves

    View full-size slide

  8. Technical scalability

    View full-size slide

  9. Cultural scalability

    View full-size slide

  10. This is the story how
    we are trying to
    solve these

    View full-size slide

  11. 13 million users

    View full-size slide

  12. 193 countries

    View full-size slide

  13. ~1800 request/s

    View full-size slide

  14. lots of data

    View full-size slide

  15. ~ 140 components

    View full-size slide

  16. ~ 400 repositories

    View full-size slide

  17. https://www.flickr.com/photos/npobre/2601582256/

    View full-size slide

  18. Key parts in a
    scalable micro
    service architecture

    View full-size slide

  19. Configuration
    Management

    View full-size slide

  20. Service Discovery

    View full-size slide

  21. Diverse technology
    stacks

    View full-size slide

  22. The same for every
    service

    View full-size slide

  23. How to get the
    services on our
    servers?

    View full-size slide

  24. One Click
    Deployment

    View full-size slide

  25. Availability

    View full-size slide

  26. Zero Downtime
    Deployments

    View full-size slide

  27. Server
    Server Server
    Server

    View full-size slide

  28. Server
    Server Server
    Server

    View full-size slide

  29. Server
    Server Server
    Server

    View full-size slide

  30. Server
    Server Server
    Server

    View full-size slide

  31. Server
    Server Server
    Server

    View full-size slide

  32. Server
    Server Server
    Server

    View full-size slide

  33. Fast deployments

    View full-size slide

  34. Fast rollbacks

    View full-size slide

  35. •Ansible
    •Capistrano
    •Saltstack
    •Custom
    •….

    View full-size slide

  36. https://www.flickr.com/photos/40987321@N02/5580348753/

    View full-size slide

  37. Different libraries,
    packages, web
    servers,
    configurations,
    versions

    View full-size slide

  38. •Chef
    •Puppet
    •Ansible
    •Docker
    •….

    View full-size slide

  39. https://www.docker.com/

    View full-size slide

  40. Configuration
    Management

    View full-size slide

  41. How do I
    synchronize
    configuration over
    services?

    View full-size slide

  42. [
    "db_user": "user",
    "db_pw": "pw",
    "serviceA": "serviceA.local:8018"
    ]

    View full-size slide

  43. Config file on disk

    View full-size slide

  44. Inconsistencies

    View full-size slide

  45. Consul
    https://www.consul.io/

    View full-size slide

  46. Consul
    Server
    Consul
    Server
    Consul
    Server
    Consul
    Agent
    ver
    Consul
    Agent
    Server
    Consul
    Agent
    Server
    Co
    Ag
    Server

    View full-size slide

  47. Key/Value Store

    View full-size slide

  48. $kv->put('test/foo/bar', 'bazinga');
    $kv->get('test/foo/bar', ['raw' => true]);
    $kv->delete('test/foo/bar');

    View full-size slide

  49. $kv->put('test/db/pw', 'secret_pw');

    View full-size slide

  50. https://www.vaultproject.io/

    View full-size slide

  51. Cycling of
    credentials

    View full-size slide

  52. Service Discovery

    View full-size slide

  53. How does one
    service know where
    another service is?

    View full-size slide

  54. Server
    Service A
    Server
    Service B
    Service C Service C

    View full-size slide

  55. Central load
    balancer

    View full-size slide

  56. HAproxy
    http://www.haproxy.org/

    View full-size slide

  57. Server
    Service A
    Server
    Service B
    Service C Service C
    Load balancer

    View full-size slide

  58. Health checks

    View full-size slide

  59. GET /health HTTP/1.1
    Host: serviceA.local
    HTTP/1.1 200 OK

    View full-size slide

  60. Scalability?

    View full-size slide

  61. Load balancer

    View full-size slide

  62. Load balancer

    View full-size slide

  63. Consul
    https://www.consul.io/

    View full-size slide

  64. Consul
    Server
    Consul
    Server
    Consul
    Server
    Consul
    Agent
    ver
    Consul
    Agent
    Server
    Consul
    Agent
    Server
    Co
    Ag
    Server

    View full-size slide

  65. Consul for Service
    Discovery

    View full-size slide

  66. Consul
    Agent
    Server
    Service A
    Registration
    Health check

    View full-size slide

  67. Load balance
    directly in the client

    View full-size slide

  68. $ curl http://localhost:8500/v1/catalog/service/refind-
    service
    [
    {
    "ServicePort": 10780,
    "ServiceAddress": "",
    "ServiceTags": [
    "env:rg_dev",
    "protocol:http"
    ],
    "ServiceName": "refind-service",
    "ServiceID": "refind-service",
    "Address": "172.20.4.61",
    "Node": "refind-1.ipbl.rgoffice.net"
    },
    {
    "ServicePort": 10780,
    "ServiceAddress": "",
    "ServiceTags": [
    "env:rg_dev",
    "protocol:http"

    View full-size slide

  69. $ dig -p 8600 @localhost refind-
    service.service.rgoffice.consul. ANY
    ; <<>> DiG 9.9.5-3ubuntu0.11-Ubuntu <<>> -p 8600 @localhost
    refind-service.service.rgoffice.consul. ANY
    ; (1 server found)
    ;; global options: +cmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 19315
    ;; flags: qr aa rd; QUERY: 1, ANSWER: 2, AUTHORITY: 0,
    ADDITIONAL: 0
    ;; WARNING: recursion requested but not available
    ;; QUESTION SECTION:
    ;refind-service.service.rgoffice.consul. IN ANY
    ;; ANSWER SECTION:
    refind-service.service.rgoffice.consul. 0 IN A 172.20.4.61
    refind-service.service.rgoffice.consul. 0 IN A 172.20.4.58

    View full-size slide

  70. Flexible routing
    options

    View full-size slide

  71. Circuit breakers

    View full-size slide

  72. Server
    Service A
    Server
    Service B
    Service C Service C
    Linkerd Consul

    View full-size slide

  73. Single Point of
    Failure

    View full-size slide

  74. Server
    Service A
    Server
    Service B
    Service C Service C
    Linkerd Consul Linkerd Consul

    View full-size slide

  75. So we found
    solutions to these
    challenges

    View full-size slide

  76. Lots of manual work

    View full-size slide

  77. Vendor lock in

    View full-size slide

  78. Very Powerful

    View full-size slide

  79. Learning curve

    View full-size slide

  80. Solves a lot of the
    before mentioned
    challenges out of
    the box

    View full-size slide

  81. Google Cloud
    Platform

    View full-size slide

  82. Kubernetes
    Cluster

    View full-size slide

  83. Image
    • A docker image
    built from a
    Dockerfile that
    contains everything
    a service needs to
    run

    View full-size slide

  84. • A container runs a
    docker image.
    • Only 1 process
    can run inside of a
    container
    Container

    View full-size slide

  85. Pod
    • A group of 1 or more
    containers
    • Same port space
    • Ports are not
    accessible from
    outside of the pod

    View full-size slide

  86. Replica Set
    • Defines and
    manages how
    many instances
    of a pod should
    run

    View full-size slide

  87. Running our
    services

    View full-size slide

  88. Deployment
    • Manages
    updates and
    rollbacks of
    replica sets

    View full-size slide

  89. Service
    • Makes a port
    of a pod
    accessible to
    other pods

    View full-size slide

  90. Ingress
    • Makes a
    service
    accessible to
    the outside
    of
    Kubernetes

    View full-size slide

  91. Service discovery

    View full-size slide

  92. Node
    • A physical server
    • Containers get
    distributed
    automatically

    View full-size slide

  93. ConfigMaps
    & Secrets
    • Configuration that
    can be mounted
    inside of a
    container

    View full-size slide

  94. Volumes
    • Volumes can be
    mounted into a
    container to access
    a ConfigMap, Secret
    or a folder on the
    host

    View full-size slide

  95. Configuration
    management

    View full-size slide

  96. Namespaces
    • Dedicated
    environment
    to deploy
    services in

    View full-size slide

  97. PHP-FPM
    NGINX
    LINKERD
    STATSD
    MEM

    CACHED
    MONGO

    ROUTER
    PHP Application POD

    View full-size slide

  98. PHP-FPM
    NGINX
    LINKERD
    STATSD
    MEM

    CACHED
    MONGO

    ROUTER
    PHP Application POD
    ReplicaSet: 2 instances
    PHP-FPM
    NGINX
    LINKERD
    STATSD
    MEM

    CACHED
    MONGO

    ROUTER
    PHP Application POD

    View full-size slide

  99. PHP-FPM
    NGINX
    LINKERD
    STATSD
    MEM

    CACHED
    MONGO

    ROUTER
    ReplicaSet: 2 instances
    PHP-FPM
    NGINX
    LINKERD
    STATSD
    MEM

    CACHED
    MONGO

    ROUTER
    CONFIG
    WEB :80
    PHP Application POD PHP Application POD

    View full-size slide

  100. PHP-FPM
    NGINX
    LINKERD
    STATSD
    MEM

    CACHED
    MONGO

    ROUTER
    ReplicaSet: 2 instances
    PHP-FPM
    NGINX
    LINKERD
    STATSD
    MEM

    CACHED
    MONGO

    ROUTER
    CONFIG
    WEB :80
    https://php-app.k8s.foo.com:443/
    PHP Application POD PHP Application POD

    View full-size slide

  101. A bit of code

    View full-size slide

  102. FROM node:7
    WORKDIR /opt/appmiral
    ADD . /opt/appmiral
    RUN apt-get install -y curl git && \
    npm install bower@latest -g && npm
    install grunt@latest -g && \
    npm install && bower install --allow-
    root && grunt build
    EXPOSE 9012
    CMD node /opt/appmiral/dist/server.js

    View full-size slide

  103. docker build -t your-registry/researchgate/
    appmiral .
    docker push your-registry/researchgate/
    apprmial

    View full-size slide

  104. ApiVersion: extensions/v1beta1
    kind: Deployment
    metadata:
    name: appmiral
    spec:
    replicas: 2
    template:
    spec:
    containers:
    -
    name: appmiral
    image: your-registry/researchgate/appmiral
    resources:
    requests:
    cpu: 1
    memory: 200Mi
    env:
    - name: NODE_ENV
    value: "production"
    ports:
    -
    containerPort: 9012
    livenessProbe:
    httpGet:
    path: /health
    port: 9012

    View full-size slide

  105. -
    name: appmiral
    image: your-registry/researchgate/appmiral
    resources:
    requests:
    cpu: 1
    memory: 200Mi
    env:
    - name: NODE_ENV
    value: "production"
    ports:
    -
    containerPort: 9012
    livenessProbe:
    httpGet:
    path: /health
    port: 9012

    View full-size slide

  106. kind: Service
    apiVersion: v1
    metadata:
    name: appmiral
    spec:
    ports:
    -
    name: http
    port: 9012
    targetPort: 9012
    protocol: TCP
    selector:
    app: appmiral

    View full-size slide

  107. apiVersion: extensions/v1beta1
    kind: Ingress
    metadata:
    name: appmiral-ing
    spec:
    rules:
    - host: appmiral.kluster-01.rgoffice.net
    http:
    paths:
    - path: /
    backend:
    serviceName: appmiral
    servicePort: 9012

    View full-size slide

  108. kubectl create -f k8s_appmiral.yaml

    View full-size slide

  109. Rolling
    Deployments

    View full-size slide

  110. Service Discovery

    View full-size slide

  111. Service Virtual IP
    address

    View full-size slide

  112. Environment
    Variables

    View full-size slide

  113. APPMIRAL_SERVICE_HOST=10.0.162.149
    APPMIRAL_SERVICE_PORT=9012

    View full-size slide

  114. $ nslookup appmiral
    Server: 10.0.0.10
    Address 1: 10.0.0.10
    Name: appmiral
    Address 1: 10.0.162.149

    View full-size slide

  115. LinkerD in
    Kubernetes

    View full-size slide

  116. PHP-FPM
    NGINX
    LINKERD
    STATSD
    MEM

    CACHED
    MONGO

    ROUTER
    PHP Application POD

    View full-size slide

  117. Helm
    The package manager for Kubernetes
    https://helm.sh/

    View full-size slide

  118. Draft
    Smart scaffolding
    https://github.com/Azure/draft

    View full-size slide

  119. Manual Scaling

    View full-size slide

  120. kubectl scale
    --replicas=3
    deployment/my-app

    View full-size slide

  121. https://kubernetes.io/docs/user-guide/horizontal-pod-
    autoscaling/

    View full-size slide

  122. https://www.flickr.com/photos/darkdwarf/19701555974/

    View full-size slide

  123. http://speakerdeck.com/u/bastianhofmann

    View full-size slide

  124. http://twitter.com/BastianHofmann
    http://lanyrd.com/people/BastianHofmann
    http://speakerdeck.com/u/bastianhofmann
    [email protected]

    View full-size slide