Erstellung hochverfügbarer Dienste mit Consul

Consul ist ein verteiltes Service Discovery Tool. Das in Go geschriebene Programm ist für viele Plattformen verfügbar, unter anderem für Solaris 11 bzw. Illumos. In diesem Vortrag will ich aufzeigen, wie Consul bei der Erstellung von hochverfügbaren Diensten verwendet werden kann. Das Ziel ist es Gateways, Loadbalancer und auch Datenbanken hochverfügbar zu halten.

Transcript

1. © 2017 stefan.husch@qutic.com
   Erstellung hochverfügbarer
   Dienste mit Consul
   

   View Slide

2. Stefan Husch
   @qutic
   ‣ Inhaber ‚qutic development‘
   ‣ Beratung, Entwicklung und Hosting von Webapplikationen
   ‣ Rechenzentrums-Dienstleistungen
   

   View Slide

3. Was ist das Ziel?
   

   View Slide

4. Hochverfügbarkeit von
   Gateways,
   Loadbalancer,
   Datenbankserver,
   DNS-Auth-Server,
   etc.
   

   View Slide

5. Was ist Consul?
   

   View Slide

6. ‣Service Discovery
   ‣Automatic Leader Election
   ‣HA Key/Value Store
   ‣Multi Datacenter
   

   View Slide

7. Service Discovery
   

   View Slide

8. ‣Health Check
   ‣DNS und HTTP JSON API
   ‣command-line interface
   

   View Slide

9. Service bekanntgeben
   

   View Slide

10. Health Check
    

    View Slide

11. DNS-API
    $ dig @127.0.0.1 \
    -p 8600 \
    database.service.dc1.consul \
    +short
    10.10.10.30
    10.10.10.31
    

    View Slide

12. HTTP-API
    $ curl \
    127.0.0.1:8500/v1/status/leader
    "10.10.10.11:8300"
    $ curl \
    127.0.0.1:8500/v1/health/state/critical
    []
    

    View Slide

13. Leader Election
    

    View Slide

14. ‣ Intern und Extern
    ‣ Raft Protocol
    ‣ Extern: HTTP JSON API
    

    View Slide

15. HA Key/Value Store
    

    View Slide

16. View Slide

17. Multi Datacenter
    

    View Slide

18. Hochverfügbarkeit
    

    View Slide

19. Consul
    

    View Slide

20. View Slide

21. View Slide

22. DNS-Auth-Server
    

    View Slide

23. View Slide

24. Loadbalancer
    

    View Slide

25. View Slide

26. View Slide

27. 1. Consul client check health of service by executing check_lb script
    1. check if loadbalancer is online
    2. store ip addresses of all virtual interfaces as key-value in Consul
    2. If check succceeds marks as healthy, if not mark as unhealthy,
    communicate result with Consul leader
    3. If service is unhealthy, Consul will execute watch-script check_lb
    on the fallback loadbalancer
    1. Get virtual interfaces and their ip out of consul key-value store
    2. Create virtual interfaces with their ip
    3. Update arp-table by sending out a gratuitous arp for all
    4. Add route
    5. Start failover loadbalancer service
    

    View Slide

28. Watcher Skript bekanntgeben
    

    View Slide

29. Gateway
    

    View Slide

30. ... wie beim Loadbalancer,
    nur mit interner IP-Adresse
    

    View Slide

31. MySQL Client Access
    

    View Slide

32. View Slide

33. View Slide

34. 1. Consul client check health of service by executing check_mysql script
    1. check if mysql is online
    2. If check succceeds marks as healthy, if not mark as unhealthy,
    communicate result with Consul leader
    3. If service is unhealthy, Consul will execute watch-script check_mysql
    on all mysql-proxy instances
    1. Rewrite mysql-proxy configuration (or use consul-template !)
    2. Restart mysql-proxy Service
    

    View Slide

35. ?
    

    View Slide

36. Quellen
    ‣ https://www.consul.io
    ‣ https://github.com/hashicorp/consul
    ‣ https://www.youtube.com/watch?v=rA4hyJ-pccU
    ‣ http://www.thekelleys.org.uk/dnsmasq/doc.html
    ‣ https://unbound.net
    ‣ https://www.consul.io/docs/internals/consensus.html
    ‣ https://www.consul.io/docs/guides/leader-election.html
    ‣ https://stripe.com/blog/service-discovery-at-stripe
    ‣ https://github.com/eBay/fabio
    ‣ https://github.com/hashicorp/consul-template
    ‣ https://github.com/darkcrux/consul-skipper
    

    View Slide