Tweet
Tweet

Slide 1

Slide 1 text

Wir unternehmen IT. Managing fabric8 with Apache ZooKeeper München, 29 September 2014

Slide 2

Slide 2 text

Seite JBoss OneDayTalk 2014 How to make a REST service scalable and fault tolerant without changing code. 14.10.2014 www.consol.de 2

Slide 3

Slide 3 text

Seite JBoss OneDayTalk 2014 14.10.2014 www.consol.de 3 What makes a small REST service interesting?

Slide 4

Slide 4 text

Seite Monolithic Service 14.10.2014 www.consol.de 4

Slide 5

Slide 5 text

Seite Microservices 14.10.2014 www.consol.de 5

Slide 6

Slide 6 text

Seite Microservices are not for everyone 14.10.2014 www.consol.de 6 YOU MUST BE THIS TALL TO USE MICROSERVICES Monitoring Provisioning Rapid Application Development (Martin Fowler)

Slide 7

Slide 7 text

Seite Example: Uptime Assume a Monolithic Service with 99.99% availability. What if you have… ~30 Microservices (each with 99.99% SLA)? 14.10.2014 www.consol.de 7 http://www.slideshare.net/stonse/microservices-at-netflix

Slide 8

Slide 8 text

Seite Netflix Simian Army 14.10.2014 www.consol.de 8

Slide 9

Slide 9 text

Seite How would I do that? 14.10.2014 www.consol.de 9 How Would I Do That?

Slide 10

Slide 10 text

Seite How would I do that? 14.10.2014 www.consol.de 10 implements most of it this guy

Slide 11

Slide 11 text

Seite Fabric8 • Open source integration platform • Originally part of JBoss Fuse, now independent open source project • Utilizes a lot of Apache projects • Makes it really easy to provision, automate, configure, and manage from a central location with a consistent UI and UX. • It also gives you non-functional requirements like configuration management, service discovery, failover, load balancing, centralized monitoring, automation, and more! 14.10.2014 www.consol.de 11

Slide 12

Slide 12 text

Seite Fabric8 • Open source integration platform • Originally part of JBoss Fuse, now independent open source project • Utilizes a lot of Apache projects • Makes it really easy to provision, automate, configure, and manage from a central location with a consistent UI and UX. • It also gives you non-functional requirements like configuration management, service discovery, failover, load balancing, centralized monitoring, automation, and more! 14.10.2014 www.consol.de 12 Focus of this talk

Slide 13

Slide 13 text

Seite Fabric8 and ZooKeeper How to make a REST service scalable and fault tolerant without changing code. 14.10.2014 www.consol.de 13

Slide 14

Slide 14 text

Seite Build Fabric8 1.1.0.CR5 from Source 14.10.2014 www.consol.de 14 fabian@ubuntu:~$ git clone https://github.com/fabric8io/fabric8.git … fabian@ubuntu:~$ cd fabric8 fabian@ubuntu:~/fabric8$ git checkout -b fabric8-1.1.0.CR5 fabric8- 1.1.0.CR5 … fabian@ubuntu:~/fabric8$ mvn clean install -DskipTests … fabian@ubuntu:~/fabric8$ cd ..

Slide 15

Slide 15 text

Seite Install and Run Fabric8 14.10.2014 www.consol.de 15 fabian@ubuntu:~$ tar xfz ~/.m2/repository/io/fabric8/fabric8- karaf/1.1.0.CR5/fabric8-karaf-1.1.0.CR5.tar.gz fabian@ubuntu:~$ cd fabric8-karaf-1.1.0.CR5/ fabian@ubuntu:~/fabric8-karaf-1.1.0.CR5$ ./bin/fabric8 … Fabric8:karaf@root>

Slide 16

Slide 16 text

Seite List of Fabric8 Maven Archetypes 14.10.2014 www.consol.de 16 fabian@ubuntu:~$ mvn archetype:generate -Dfilter=io.fabric8: … 31: remote -> io.fabric8.archetypes:karaf-rest-archetype (Creates a new RESTful WebService Example using JAX-RS with JBoss Fuse) …

Slide 17

Slide 17 text

Seite Create Project from Archetype 14.10.2014 www.consol.de 17 fabian@ubuntu:~$ mvn archetype:generate \ -DarchetypeGroupId=io.fabric8.archetypes \ -DarchetypeArtifactId=karaf-rest-archetype \ -DarchetypeVersion=1.1.0.CR5 \ -DgroupId=de.consol.research.fabric8 \ -DartifactId=demo \ -Dversion=1.0-SNAPSHOT \ -Dpackage=de.consol.research.fabric8.demo

Slide 18

Slide 18 text

Seite Set fabric8.profile in pom.xml 14.10.2014 www.consol.de 18 ... de.consol.research.fabric8 demo 1.0-SNAPSHOT bundle ... ... demo feature-cxf cxf-jaxrs swagger ...

Slide 19

Slide 19 text

Seite Deploy as new Profile in Fabric8 14.10.2014 www.consol.de 19 fabian@ubuntu:~/demo$ mvn clean install ... fabian@ubuntu:~/demo$ mvn fabric8:deploy ...

Slide 20

Slide 20 text

Seite Run Containers from Profile 14.10.2014 www.consol.de 20 Fabric8:karaf@root> profile-list | grep demo ... Fabric8:karaf@root> container-create-child --profile demo root container1 ... Fabric8:karaf@root> container-create-child --profile demo root container2 ... Fabric8:karaf@root> container-list ...

Slide 21

Slide 21 text

Seite View Containers in Browser 14.10.2014 www.consol.de 21

Slide 22

Slide 22 text

Seite Runtime APIs 14.10.2014 www.consol.de 22

Slide 23

Slide 23 text

Seite Example Call 14.10.2014 www.consol.de 23

Slide 24

Slide 24 text

Seite One VM running per Child Container 14.10.2014 www.consol.de 24 fabian@ubuntu:~$ ps aux | grep instances

Slide 25

Slide 25 text

Seite View Service Endpoints in ZooKeeper 14.10.2014 www.consol.de 25 Fabric8:karaf@root> profile-edit --features fabric-zookeeper-commands \ fabric ... Fabric8:karaf@root> zk:list --recursive ...

Slide 26

Slide 26 text

Seite Start container from HTTP Gateway Profile 14.10.2014 www.consol.de 26 Fabric8:karaf@root> profile-list | grep gateway ... Fabric8:karaf@root> container-create-child --profile gateway-http \ root demogateway ... Fabric8:karaf@root> container-list ...

Slide 27

Slide 27 text

Seite Gateway Container Running 14.10.2014 www.consol.de 27

Slide 28

Slide 28 text

Seite Gateway Configuration via Wiki 14.10.2014 www.consol.de 28

Slide 29

Slide 29 text

Seite Example Call Through Gateway 14.10.2014 www.consol.de 29

Slide 30

Slide 30 text

Seite View Call in Logs 14.10.2014 www.consol.de 30

Slide 31

Slide 31 text

Seite Demo Summary 14.10.2014 www.consol.de 31 implements most of it already this guy

Slide 32

Slide 32 text

Seite Fabric8 Terminology • Container: Running Instance. – Child Container: Same host – SSH Container: Other host – Cloud Container: Other host • Fabric: Collection of containers sharing a ZooKeeper registry 14.10.2014 www.consol.de 32 Container Container Container Container Fabric

Slide 33

Slide 33 text

Seite Fabric8 Terminology • Container: Running Instance. – Child Container: Same host – SSH Container: Other host – Cloud Container: Other host • Fabric: Collection of containers sharing a ZooKeeper registry • Fabric Server: Container serving the ZooKeeper registry 14.10.2014 www.consol.de 33 Container Container Container Fabric Server

Slide 34

Slide 34 text

Seite Fabric8 Terminology • Container: Running Instance. – Child Container: Same host – SSH Container: Other host – Cloud Container: Other host • Fabric: Collection of containers sharing a ZooKeeper registry • Fabric Server: Container serving the ZooKeeper registry • Ensemble: Collection of Fabric Servers maintaining a distributed ZooKeeper registry 14.10.2014 www.consol.de 34 Container Fabric Server Server Server

Slide 35

Slide 35 text

Seite Create an Ensemble 14.10.2014 www.consol.de 35 Fabric8:karaf@root> container-list ... Fabric8:karaf@root> ensemble-add container1 container2 ... Fabric8:karaf@root> container-list ...

Slide 36

Slide 36 text

Seite ZooKeeper • Maintains distributed registry • No single point of failure • Consistent as long as majority of nodes in ensemble is up 14.10.2014 www.consol.de 36 ✓ ✓ ✓

Slide 37

Slide 37 text

Seite ZooKeeper Hierarchal namespace (like a file system). Each znode has data and children. Data is read and written in its entirety. 14.10.2014 www.consol.de 37 fabric | +--registry (runtime registry) | | | +--containers | | | +----root | +--configs (configuration registry) | +--versions | | | +--1.0 | | | +--profiles | | | +--default | +--containers

Slide 38

Slide 38 text

Seite ZooKeeper Flags for create: • Ephemeral: znode deleted when creator fails or explicitly deleted • Sequence: append a monotonically increasing counter to znode name 14.10.2014 www.consol.de 38 String create(path, data, acl, flags); void delete(path, expectedVersion); Stat setData(path, data, expectedVersion); (data, Stat) getData(path, watch); Stat exists(path, watch); String[] getChildren(path, watch); void sync();

Slide 39

Slide 39 text

Seite ZooKeeper 14.10.2014 www.consol.de 39 public final class FabricCxfRegistrationHandler ... { protected void registerApiEndpoint(...) { ... ZooKeeperUtils.setData(..., path, json, CreateMode.EPHEMERAL); ... } } public enum ZkPath { ... API_REST_ENDPOINTS ("/fabric/registry/clusters/apis/rest/{name}/{version}/{container}") ... }

Slide 40

Slide 40 text

Seite ZooKeeper • Two Protocols: –Zab: ZooKeeper Atomic Broadcast –Leader Election 14.10.2014 www.consol.de 40

Slide 41

Slide 41 text

Seite Zab – ZooKeeper Atomic Broadcast 14.10.2014 www.consol.de 41 Server Server Server Server Leader read, watch read, watch read, watch read, watch client client client client • Super scalable • High throughput • Guaranteed order of events • Upates may be delayed on some servers

Slide 42

Slide 42 text

Seite Zab – ZooKeeper Atomic Broadcast 14.10.2014 www.consol.de 42 Server Server Server Server Leader write client client client client • Writes are serialized by the leader • Leader implements two phase commit protocol (propose, accept, commit) • Commit phase done when majority of servers commit • ZooKeeper is optimized for read performance, not for write performance write write write

Slide 43

Slide 43 text

Seite Leader Election 14.10.2014 www.consol.de 43 Server Server Server Server Leader • Quorum protocol • Leader with latest state wins. • An elected leader has committed all transactions that will ever be committed from previous epochs before it starts broadcasting new transactions; before the sync phase completes, a new leader does not broadcast new state changes.

Slide 44

Slide 44 text

Seite Summary How to make a REST service scalable and fault tolerant without changing code. 14.10.2014 www.consol.de 44

Slide 45

Slide 45 text

Seite Summary 14.10.2014 www.consol.de 45 Use an integration platform, like fabric8. The platform takes care of it.

Slide 46

Slide 46 text

Seite 14.10.2014 www.consol.de 46

Slide 47

Slide 47 text

Seite Backup Slide: Fabric8 and Karaf 14.10.2014 www.consol.de 47 OSGi Bundle Contains Java classes and resources Karaf Feature Contains OSGi bundles, references to repositories and other features, configuration Fabric8 Profile Fabric8‘s Unit of deployment, contains OSGi bundles, Karaf features, and configuration

Slide 48

Slide 48 text

Seite Backup Slide: ZooKeeper Recipies What are ZooKeeper Recipies: • Higher-level Constructs with ZooKeeper Examples of Recipies: • Name Service • Configuration • Group Membership • Barriers • Queues • Locks • Two-phased Commit • Leader Election Apache Curator Framework implements some of these recipies. References • https://www.usenix.org/conference/usenix-atc-10/zookeeper-wait-free-coordination-internet-scale-systems • http://zookeeper.apache.org/doc/trunk/recipes.html 14.10.2014 www.consol.de 48