Getting Organized with Service Discovery for Microservices

.consulting .solutions .partnership
Getting Organized with
Service Discovery for Microservices
Alexander Schwartz, Principal IT Consultant
Continuous Lifecycle London 2016 – 4 May 2016

View Slide

Getting Organized with Service Discovery for Microservices
© msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016 2
Everyday life in the cloud
1
etcd: Central Configuration Store
2
Consul: Service Discovery, Configuration and Monitoring
3
Eureka: Availability First!
4
Service Discovery Comparison
5

View Slide

My sponsor and employer – msg systems ag
Founded in 1980
More than 5.500 employees
727 M € (573 M £)
turnover in 2015
Offices in 24 countries
Headquarter based
near Munich in Germany

View Slide

That‘s me – Alexander Schwartz
14 years of Java
7 years of PL/SQL
7 years of
consumer finance
3.5 years direct banking
1 wife
2 children
471 found
Geocaches
@ahus1de

View Slide

1
2
3
4
5

View Slide

A simple Cloud Setup
Where are my application servers?
Where is my database?

View Slide


View Slide

Configuration at runtime saves you the hassle – and maintenance windows
• Change parameters at runtime
 Target-IPs of servers
 Names of input and output folders
 Timeouts and other business parameters
• Central database for
 Configuration parameters
 Service-Discovery for Clients
 Service-Discovery for Load Balancers

View Slide

10
© msg | Getting Organized with Service Discovery for Microservices | Alexander Schwartz | Continuous Lifecycle London 2016 | 4 May 2016
1
2
3
4
5

View Slide

CoreOS etcd
etcd is a Central Configuration Store
Suitable for configuration and service discovery
• Simple: HTTP/REST w/ JSON
• Reliable: highly available and consistent
• Secure: optionally with SSL-Client-Certificates
• Fast: several thousand write operations per second
Operations: GET/PUT/DELETE/CAS/WAIT/TTL
# PUT
curl -L -X PUT http://127.0.0.1:4001/v2/keys/foo-service/container1 –d value="localhost:1111"
# GET
curl -L http://127.0.0.1:4001/v2/keys/foo-service/container1
# WAIT
curl -L http://127.0.0.1:4001/v2/keys/foo-service?wait=true\&recursive=true...

View Slide

CoreOS etcd + Hightower´s confd
When values change, confd will update your Configuration Files
confd

View Slide

CoreOS etcd + Hightower´s confd
When values change, confd will update your Configuration Files
• Service templates (*.toml)
• Configuration templates (*.tmpl)
# nginx.toml
[template]
src = "nginx.tmpl"
dest = "/etc/nginx/sites-enabled/app.conf"
keys = [ "/foo-service" ]
check_cmd = "/usr/sbin/nginx -t"
reload_cmd = "/usr/sbin/service nginx reload"
# nginx.tmpl
upstream app_pool {
{{ range getvs "/foo-service/*" }}
server {{ . }};
{{ end }}
}
confd

View Slide

CoreOS etcd + Jurmous’ Java Client
Use Jurmos’ Java Client to talk to etcd
• Java-API for all of etcd’s API calls
• Implemented on top of Netty (for asynchronous, non-blocking IO)
// Simple put
etcd.put("/foo-service/container1","localhost:1111").send();
// Wait for next change on foo
EtcdResponsePromise promise = etcd.getDir("/foo-service/")
.recursive().waitForChange().send();
// Java 8 lambda construction
promise.addListener(promise -> {
// do something on change
});

View Slide

CoreOS etcd
Pros and Cons etcd
Pro:
• Easy access (REST), simple installation (Go-Binary), transport security (SSL)
• Changes are published to the clients immediately
• Generic structure for flexible usage
• Used by CoreOS and Kubernetes
Contra:
• No access controls for on a per client level
Add-ons:
• Vulcan (HTTP Load Balancer, configuration stored directly in etcd)
• SkyDNS (Service discovery via DNS, data stored in etcd)

View Slide

16
1
2
3
4
5

View Slide

Hashicorp Consul
Consul
Built for Service Discovery, Configuration and Monitoring
• Easy to use: HTTP/REST w/ JSON
• Reliable: highly available and consistent
• Secure: built in Access Control Lists and Encryption
Main Sponsor: HashiCorp (the ones also building Vagrant)
Homepage: https://www.consul.io/

View Slide

Netflix Ribbon
Ribbon
Built for client side load balancing
• Ribbon is well integrated into Feign: type safe web APIs
Main-Sponsor: Netflix
Homepage: https://github.com/Netflix/ribbon/

View Slide

Consul and Ribbon
The Client contacts available Services directly
Consul-Server
(for Key Value Store
und Service Catalogue)
Services
Client

View Slide

Services with Consul
Every Consul Instance knows about its local Services
"service": {
"name": "web",
"tags": ["master"],
"port": 80
}

View Slide

Every Consul Instance checks its local Services
"check": {
"id": "http",
"name": "HTTP API on port 80",
"http": "http://localhost:80",
"service_id": "web",
"interval": "10s",
"timeout": "1s"
}
"check": {
"id": "script",
"name": "Script Check",
"script": "/usr/local/bin/check_mem.py",
"interval": "10s"
}
"check": {
"id": "ttl",
"name": "Web Status",
"ttl": "30s"
}

View Slide

Consul UI

View Slide

Lookup Services using DNS SRV Records
$ dig @127.0.0.1 -p 8600 web.service.dc1.consul. SRV
web.service.dc1.consul. 0 IN SRV 1 1 80 web2.node.dc1.consul.
web.service.dc1.consul. 0 IN SRV 1 1 80 web1.node.dc1.consul.
web1.node.dc1.consul. 0 IN A 192.168.23.21
web2.node.dc1.consul. 0 IN A 192.168.23.22

View Slide

Consul and Ribbon
Application Code for Ribbon
MyService api = Feign.builder()
.contract(new JAXRSContract())
.encoder(new JacksonEncoder())
.decoder(new JacksonDecoder())
.client(RibbonClient.builder()
.lbClientFactory(new ConsulLbFactory())
.build())
.target(MyService.class, "http://web");
api.call();

View Slide

public class ConsulLbFactory implements LBClientFactory {
private Map clients = new HashMap<>();
public synchronized LBClient create(String clientName) {
LBClient client = clients.get(clientName);
if(client == null) {
ILoadBalancer lb = new ConsulLoadBalancer(clientName);
IClientConfig clientConfig = new DefaultClientConfigImpl();
clientConfig.loadDefaultValues();
client = LBClient.create(lb, clientConfig);
clients.put(clientName, client);
}
return client;
}
}
Consul and Ribbon
Java-API for Consul
public class ConsulLoadBalancer extends BaseLoadBalancer {
public ConsulLoadBalancer(String serviceName) {
Consul consul = Consul.builder().build();
HealthClient client = consul.healthClient();
ServiceHealthCache svHealth = ServiceHealthCache.newCache(client, serviceName);
svHealth.addListener(map -> {
List newServers = new ArrayList<>();
map.forEach((hostAndPort, serviceHealth) ->
newServers.add(new Server(hostAndPort.getHostText(),
hostAndPort.getPort())));
setServersList(newServers);
});
/* ... */
}
}

View Slide

Consul and Ribbon
Pros and Cons Service Discovery with Consul
Pro:
• Contact to Services without a Single Point of Failure
• Ready to use Libraries available
• Decentralized monitoring of services
• GUI for key value store and monitoring
• Semantic of services instead of key value store only
Add-ons:
• eBay Fabio (HTTP Load Balancer, configuration directly in consul)
• consul-template (writes configuration files like confd)
• Nomad (scheduler that registers service automatically with consul)

View Slide

27
1
2
3
4
5

View Slide

Netflix Eureka
Eureka
Build for Service Discovery inside Amazon Web Services (AWS)
• REST: you choose from JSON and XML
• Simple: Client-Pull
• Reliable: highly available, availability before consistency
Main Sponsor: Netflix
Homepage: https://github.com/Netflix/eureka

View Slide

Netflix Eureka
Brewers CAP Theorem
„A distributed system can fulfil only two out of three availability modes at a time.“
Consistency
Here: After a service registered, clients can find it (Linearizability).
Availability
Here: Service registration and lookup available for clients.
Partition Tolerance
Hier: Service registration and lookup work even in the presence of networks being partly unavailable.

View Slide

Netflix Eureka
Eureka – Availability first (1/2)
Etcd:
• Registration: CP
• Lookup: AP (Standard) or CP („?quorum=true“)
Consul:
• Registration :
CP for Key-Value-Store
AP for Service-Registration and Health via local Agents („Anti-Entropy“)
CP for combining the Information in the Service-Registry
• Lookup via REST API: either CP („?consistent“),
with the currently known leader, or AP („?stale“)
• Lookup via DNS: with the currently known leader,
alternatively AP with an upper time limit „stale reads“

View Slide

Netflix Eureka
Eureka – Availability first (2/2)
Eureka:
• Registration of Services: AP (a Client registers with the server instance and publishes its status
every 30 seconds)
• Lookup of Services: AP (a client receives the view of the one server it connects to)
• „Self-Preservation“ if a network partition is suspected between Server/Server or Clients/Server:
The server keeps the current status of all known services

View Slide

Netflix Eureka
Eureka – Pull and Cache instead of Query and Push (1/2)
Etcd:
• Service-Lookup with HTTP GET as synchronous call
• Waiting for changes with HTTP GET blocking query („?wait=true&waitIndex=X“)
Consul:
• Service-Lookup with HTTP GET as synchronous call
• Service-Lookup with HTTP GET as blocking query („?wait=Xs&index=Y“)

View Slide

Netflix Eureka
Eureka – Pull and Cache instead of Query and Push (1/2)
Eureka:
• Clients pull every 30 seconds the delta from the service registry of their server
• Server pull every 30 seconds the delta from other servers
• Service lookup is a client-local operation
• A status change can take up to 2 minutes to be published to all participants
(Lookup with HTTP GET as a synchronous call is available via API, but rarely used)

View Slide

34
1
2
3
4
5

View Slide

Capability Matrix
Etcd Consul Eureka
Configurations Yes Yes No
Service Discovery
via REST-API
Yes
(via KV-Store)
Yes Yes
Write configuration files Yes
(via confd)
Yes
(native and confd)
No
Dynamic DNS Yes
(via SkyDNS)
Yes
(native)
No
Topology Support No Yes (Data center
& WAN)
Yes
(Region & Zone)
Access Controls SSL Client-Certificates Detailed (ACLs) Servlet-Security
Health Checks for Services No Yes No
Edge Service (Vulcan) (Fabio) (Zuul)
GUI No Yes No

View Slide

Who’s your favourite?
Etcd – highly available Key-Value-Store
• Part of CoreOS
• Easy to use REST-API
Consul – Services as „First Class Citizen“ in addition to Key-Value-Store
• Active Monitoring of Services
• Built in Security: Transport encryption und ACLs
Eureka – Built for Amazon Cloud
• Very good integration into the Amazon Infrastructure
• Schedule-based replication of the complete service catalogue to the clients (Client Pull)
• Availability first – Consistency second
@ahus1de

View Slide

Links
HashiCorps Consul für Service Discovery,
Monitoring und Konfigurationsmanagement
http://heise.de/-3040847
eBay Fabio:
https://github.com/eBay/fabio
Nomad:
https://www.nomadproject.io/
Saltstack and Consul Examples
https://github.com/ahus1/saltconsul-examples
@ahus1de
CoreOS etcd: https://github.com/coreos/etcd
Jurnous etcdj: https://github.com/jurmous/etcd4j
Vulcan: http://www.vulcanproxy.com/
SkyDNS: https://github.com/skynetservices/skydns
Netflix Eureka: https://github.com/Netflix/eureka
Apache Zookeeper: https://zookeeper.apache.org/
Netflix exhibitor: https://github.com/Netflix/exhibitor
HashiCorp Consul: http://consul.io/

View Slide

.consulting .solutions .partnership
Alexander Schwartz
Principal IT Consultant
+49 171 5625767
[email protected]
@ahus1de
msg systems ag (Headquarters)
Robert-Buerkle-Str. 1, 85737 Ismaning
Germany
www.msg-systems.com

View Slide

Getting Organized with Service Discovery for Microservices

Getting Organized with Service Discovery for Microservices

Alexander Schwartz

More Decks by Alexander Schwartz

Other Decks in Technology

Featured

Transcript