Getting Organized with Service Discovery for Microservices

Microservices are small and enable you to go fast. But you’ll need to get them organized: this is where Service Discovery comes into play. There are several open source solutions available: HashiCorp Consul, CoreOS etcd and Netflix Eureka. But how should we handle security, multiple datacenter and monitoring? And how to look up the services from your services? This session will gives an overview of the tools available and their utilities with practical examples.

Transcript

1. .consulting .solutions .partnership Getting Organized with Service Discovery for Microservices Alexander Schwartz, Principal IT Consultant Continuous Lifecycle London 2016 – 4 May 2016
2. Getting Organized with Service Discovery for Microservices © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 2 Everyday life in the cloud 1 etcd: Central Configuration Store 2 Consul: Service Discovery, Configuration and Monitoring 3 Eureka: Availability First! 4 Service Discovery Comparison 5
3. My sponsor and employer – msg systems ag © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 3 Founded in 1980 More than 5.500 employees 727 M € (573 M £) turnover in 2015 Offices in 24 countries Headquarter based near Munich in Germany
4. That‘s me – Alexander Schwartz © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 4 14 years of Java 7 years of PL/SQL 7 years of consumer finance 3.5 years direct banking 1 wife 2 children 471 found Geocaches @ahus1de
5. Getting Organized with Service Discovery for Microservices © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 5 Everyday life in the cloud 1 etcd: Central Configuration Store 2 Consul: Service Discovery, Configuration and Monitoring 3 Eureka: Availability First! 4 Service Discovery Comparison 5
6. Everyday life in the cloud A simple Cloud Setup © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 6 Where are my application servers? Where is my database?
7. Everyday life in the cloud A simple Cloud Setup © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 7
8. Everyday life in the cloud A simple Cloud Setup © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 8
9. Everyday life in the cloud Configuration at runtime saves you the hassle – and maintenance windows © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 9 • Change parameters at runtime  Target-IPs of servers  Names of input and output folders  Timeouts and other business parameters • Central database for  Configuration parameters  Service-Discovery for Clients  Service-Discovery for Load Balancers
10. Getting Organized with Service Discovery for Microservices 10 © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 Everyday life in the cloud 1 etcd: Central Configuration Store 2 Consul: Service Discovery, Configuration and Monitoring 3 Eureka: Availability First! 4 Service Discovery Comparison 5
11. CoreOS etcd etcd is a Central Configuration Store © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 11 Suitable for configuration and service discovery • Simple: HTTP/REST w/ JSON • Reliable: highly available and consistent • Secure: optionally with SSL-Client-Certificates • Fast: several thousand write operations per second Operations: GET/PUT/DELETE/CAS/WAIT/TTL # PUT curl -L -X PUT http://127.0.0.1:4001/v2/keys/foo-service/container1 –d value="localhost:1111" # GET curl -L http://127.0.0.1:4001/v2/keys/foo-service/container1 # WAIT curl -L http://127.0.0.1:4001/v2/keys/foo-service?wait=true\&recursive=true...
12. CoreOS etcd + Hightower´s confd When values change, confd will update your Configuration Files © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 12 confd
13. CoreOS etcd + Hightower´s confd When values change, confd will update your Configuration Files © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 13 • Service templates (*.toml) • Configuration templates (*.tmpl) # nginx.toml [template] src = "nginx.tmpl" dest = "/etc/nginx/sites-enabled/app.conf" keys = [ "/foo-service" ] check_cmd = "/usr/sbin/nginx -t" reload_cmd = "/usr/sbin/service nginx reload" # nginx.tmpl upstream app_pool { {{ range getvs "/foo-service/*" }} server {{ . }}; {{ end }} } confd
14. CoreOS etcd + Jurmous’ Java Client Use Jurmos’ Java Client to talk to etcd © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 14 • Java-API for all of etcd’s API calls • Implemented on top of Netty (for asynchronous, non-blocking IO) // Simple put etcd.put("/foo-service/container1","localhost:1111").send(); // Wait for next change on foo EtcdResponsePromise<EtcdKeysResponse> promise = etcd.getDir("/foo-service/") .recursive().waitForChange().send(); // Java 8 lambda construction promise.addListener(promise -> { // do something on change });
15. CoreOS etcd Pros and Cons etcd © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 15 Pro: • Easy access (REST), simple installation (Go-Binary), transport security (SSL) • Changes are published to the clients immediately • Generic structure for flexible usage • Used by CoreOS and Kubernetes Contra: • No access controls for on a per client level Add-ons: • Vulcan (HTTP Load Balancer, configuration stored directly in etcd) • SkyDNS (Service discovery via DNS, data stored in etcd)
16. Getting Organized with Service Discovery for Microservices 16 © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 Everyday life in the cloud 1 etcd: Central Configuration Store 2 Consul: Service Discovery, Configuration and Monitoring 3 Eureka: Availability First! 4 Service Discovery Comparison 5
17. Hashicorp Consul Consul © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 17 Built for Service Discovery, Configuration and Monitoring • Easy to use: HTTP/REST w/ JSON • Reliable: highly available and consistent • Secure: built in Access Control Lists and Encryption Main Sponsor: HashiCorp (the ones also building Vagrant) Homepage: https://www.consul.io/
18. Netflix Ribbon Ribbon © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 18 Built for client side load balancing • Ribbon is well integrated into Feign: type safe web APIs Main-Sponsor: Netflix Homepage: https://github.com/Netflix/ribbon/
19. Consul and Ribbon The Client contacts available Services directly © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 19 Consul-Server (for Key Value Store und Service Catalogue) Services Client
20. Services with Consul Every Consul Instance knows about its local Services © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 20 "service": { "name": "web", "tags": ["master"], "port": 80 }
21. Services with Consul Every Consul Instance checks its local Services © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 21 "check": { "id": "http", "name": "HTTP API on port 80", "http": "http://localhost:80", "service_id": "web", "interval": "10s", "timeout": "1s" } "check": { "id": "script", "name": "Script Check", "script": "/usr/local/bin/check_mem.py", "service_id": "web", "interval": "10s" } "check": { "id": "ttl", "name": "Web Status", "service_id": "web", "ttl": "30s" }
22. Services with Consul Consul UI © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 22
23. Services with Consul Lookup Services using DNS SRV Records © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 23 $ dig @127.0.0.1 -p 8600 web.service.dc1.consul. SRV web.service.dc1.consul. 0 IN SRV 1 1 80 web2.node.dc1.consul. web.service.dc1.consul. 0 IN SRV 1 1 80 web1.node.dc1.consul. web1.node.dc1.consul. 0 IN A 192.168.23.21 web2.node.dc1.consul. 0 IN A 192.168.23.22
24. Consul and Ribbon Application Code for Ribbon © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 24 MyService api = Feign.builder() .contract(new JAXRSContract()) .encoder(new JacksonEncoder()) .decoder(new JacksonDecoder()) .client(RibbonClient.builder() .lbClientFactory(new ConsulLbFactory()) .build()) .target(MyService.class, "http://web"); api.call();
25. public class ConsulLbFactory implements LBClientFactory { private Map<String, LBClient> clients = new HashMap<>(); public synchronized LBClient create(String clientName) { LBClient client = clients.get(clientName); if(client == null) { ILoadBalancer lb = new ConsulLoadBalancer(clientName); IClientConfig clientConfig = new DefaultClientConfigImpl(); clientConfig.loadDefaultValues(); client = LBClient.create(lb, clientConfig); clients.put(clientName, client); } return client; } } Consul and Ribbon Java-API for Consul © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 25 public class ConsulLoadBalancer extends BaseLoadBalancer { public ConsulLoadBalancer(String serviceName) { Consul consul = Consul.builder().build(); HealthClient client = consul.healthClient(); ServiceHealthCache svHealth = ServiceHealthCache.newCache(client, serviceName); svHealth.addListener(map -> { List<Server> newServers = new ArrayList<>(); map.forEach((hostAndPort, serviceHealth) -> newServers.add(new Server(hostAndPort.getHostText(), hostAndPort.getPort()))); setServersList(newServers); }); /* ... */ } }
26. Consul and Ribbon Pros and Cons Service Discovery with Consul © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 26 Pro: • Contact to Services without a Single Point of Failure • Ready to use Libraries available • Decentralized monitoring of services • GUI for key value store and monitoring • Semantic of services instead of key value store only Add-ons: • eBay Fabio (HTTP Load Balancer, configuration directly in consul) • consul-template (writes configuration files like confd) • Nomad (scheduler that registers service automatically with consul)
27. Getting Organized with Service Discovery for Microservices 27 © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 Everyday life in the cloud 1 etcd: Central Configuration Store 2 Consul: Service Discovery, Configuration and Monitoring 3 Eureka: Availability First! 4 Service Discovery Comparison 5
28. Netflix Eureka Eureka © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 28 Build for Service Discovery inside Amazon Web Services (AWS) • REST: you choose from JSON and XML • Simple: Client-Pull • Reliable: highly available, availability before consistency Main Sponsor: Netflix Homepage: https://github.com/Netflix/eureka
29. Netflix Eureka Brewers CAP Theorem © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 29 „A distributed system can fulfil only two out of three availability modes at a time.“ Consistency Here: After a service registered, clients can find it (Linearizability). Availability Here: Service registration and lookup available for clients. Partition Tolerance Hier: Service registration and lookup work even in the presence of networks being partly unavailable.
30. Netflix Eureka Eureka – Availability first (1/2) © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 30 Etcd: • Registration: CP • Lookup: AP (Standard) or CP („?quorum=true“) Consul: • Registration : CP for Key-Value-Store AP for Service-Registration and Health via local Agents („Anti-Entropy“) CP for combining the Information in the Service-Registry • Lookup via REST API: either CP („?consistent“), with the currently known leader, or AP („?stale“) • Lookup via DNS: with the currently known leader, alternatively AP with an upper time limit „stale reads“
31. Netflix Eureka Eureka – Availability first (2/2) © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 31 Eureka: • Registration of Services: AP (a Client registers with the server instance and publishes its status every 30 seconds) • Lookup of Services: AP (a client receives the view of the one server it connects to) • „Self-Preservation“ if a network partition is suspected between Server/Server or Clients/Server: The server keeps the current status of all known services
32. Netflix Eureka Eureka – Pull and Cache instead of Query and Push (1/2) © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 32 Etcd: • Service-Lookup with HTTP GET as synchronous call • Waiting for changes with HTTP GET blocking query („?wait=true&waitIndex=X“) Consul: • Service-Lookup with HTTP GET as synchronous call • Service-Lookup with HTTP GET as blocking query („?wait=Xs&index=Y“)
33. Netflix Eureka Eureka – Pull and Cache instead of Query and Push (1/2) © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 33 Eureka: • Clients pull every 30 seconds the delta from the service registry of their server • Server pull every 30 seconds the delta from other servers • Service lookup is a client-local operation • A status change can take up to 2 minutes to be published to all participants (Lookup with HTTP GET as a synchronous call is available via API, but rarely used)
34. Getting Organized with Service Discovery for Microservices 34 © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 Everyday life in the cloud 1 etcd: Central Configuration Store 2 Consul: Service Discovery, Configuration and Monitoring 3 Eureka: Availability First! 4 Service Discovery Comparison 5
35. Service Discovery Comparison Capability Matrix © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 35 Etcd Consul Eureka Configurations Yes Yes No Service Discovery via REST-API Yes (via KV-Store) Yes Yes Write configuration files Yes (via confd) Yes (native and confd) No Dynamic DNS Yes (via SkyDNS) Yes (native) No Topology Support No Yes (Data center & WAN) Yes (Region & Zone) Access Controls SSL Client-Certificates Detailed (ACLs) Servlet-Security Health Checks for Services No Yes No Edge Service (Vulcan) (Fabio) (Zuul) GUI No Yes No
36. Service Discovery Comparison Who’s your favourite? © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 36 Etcd – highly available Key-Value-Store • Part of CoreOS • Easy to use REST-API Consul – Services as „First Class Citizen“ in addition to Key-Value-Store • Active Monitoring of Services • Built in Security: Transport encryption und ACLs Eureka – Built for Amazon Cloud • Very good integration into the Amazon Infrastructure • Schedule-based replication of the complete service catalogue to the clients (Client Pull) • Availability first – Consistency second @ahus1de
37. Service Discovery Comparison Links © msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 37 HashiCorps Consul für Service Discovery, Monitoring und Konfigurationsmanagement http://heise.de/-3040847 eBay Fabio: https://github.com/eBay/fabio Nomad: https://www.nomadproject.io/ Saltstack and Consul Examples https://github.com/ahus1/saltconsul-examples @ahus1de CoreOS etcd: https://github.com/coreos/etcd Jurnous etcdj: https://github.com/jurmous/etcd4j Vulcan: http://www.vulcanproxy.com/ SkyDNS: https://github.com/skynetservices/skydns Netflix Eureka: https://github.com/Netflix/eureka Apache Zookeeper: https://zookeeper.apache.org/ Netflix exhibitor: https://github.com/Netflix/exhibitor HashiCorp Consul: http://consul.io/
38. .consulting .solutions .partnership Alexander Schwartz Principal IT Consultant +49 171 5625767 alexander.schwartz@msg-systems.com @ahus1de msg systems ag (Headquarters) Robert-Buerkle-Str. 1, 85737 Ismaning Germany www.msg-systems.com