DockerGrunn 20150303 - Docker Clustering en Service Discovery met CoreOS

Presentatie tijdens DockerGrunn meetup van 3 maart 2015.

Transcript

1. Docker'Clustering'en'Service'Discovery'met'CoreOS DockerGrunn*meetup,*3*maart*2015
2. Introduc)e • @MarcelHarkema • Ik-werk-bij-TKP-als-so7ware-architect • Zo'n-100-collega's-op-de-afdeling-ICT • Agile,-Scrum,-DevOps • www.tkppensioen.nl/werkenMbijMtkp/ict
3. Vanavond • Achtergrond: • Van/applica3es/op/centrale/Java/applica3eservers • .../naar/applica3es/met/embedded/serverlogica • .../en/services • Hoe/zou/een/pla=orm/voor/services/er/uit/kunnen/zien? • Proof/of/concept • Demo
4. Achtergrond
5. Van$applica)es$op$Java$Applica)eservers$... • GlassFish)(vgl.)JBoss)AS,)WebLogic)&)WebSphere) • Specifieke)Java)EE)versie)en)bibliotheken • Meerdere)applicaDes • Centraal)ingericht)en)beheerd • Clusters • Hardware)loadIbalancer)over)cluster)instances • Vrij)staDsch
6. ..."naar"applica*es"met"embedded"serverlogica • All$in$one)JAR)applica/es • Embedded)Je5y • Alle)a7ankelijkheden)meebundelen)(dus)ook)die)normaal)in)de) Java)EE)applica/eserver)zi5en) • Applica/es)draaien)op)systemen)die)een)Java)VM)(JRE))hebben • Hoe)zit)het)dan)met)clustering,)load$balancing,)deployment,)e.d.) van)de)applica/es)?
7. ..."en"services • Monolieten)opdelen)in)meerdere)services • N.B.)Service)hoe6)niet)al8jd)REST)of)WS?*)(SOAP))te)zijn Kan)bijv.)ook)een)AngularJS/NodeJS)applica8e)zijn • Laat)meer)varia8e)toe)wat)betre6)invulling)(meerdere) programmeertalen,)bijv.) • Microservices)architectural)style • mar8nfowler.com/ar8cles/microservices.html
8. Ingrediënten*van*zo'n*pla2orm
9. Boodschappenlijstje • Packaging"van"de"services"met"a.ankelijkheden • Deployment"van"de"services • Plek"om"ze"te"laten"landen • Meervoudig"vanwege"beschikbaarheid • Nieuwe"versie"neerze<en"terwijl"de"winkel"openblij> • Service3discovery:"Ontdekken"waar"de"services"draaien • Load"balancing • Health"checks"(niet"gezonde"services"uit"de"roulaDe"nemen) • Service3registry:"Plek"om"vast"te"leggen"welke"services"waar"draaien • Service3announcement:"Service"maakt"zich"bekend"aan"de"omgeving
10. Proof%of%concept
11. Proof%of%concept • Packaging)van)service)met)z'n)a0ankelijkheden)bundelen)in)een)Docker) image • Deployment)met/op)CoreOS)als)host)voor)Docker)containers • etcd)(onderdeel)van)CoreOS))als)service)registry • Fleet)(onderdeel)van)CoreOS))voor)het)deployen)van)services)(m.b.v.) systemd1units))als)geclusterde)Docker)containers • Service)announcement)met)Registrator • Service)discovery)en)loadDbalancing)met)vulcand
12. Packaging:)Docker • Packaging)van)een)service)met)z'n)a0ankelijkheden • Docker)image)=)bijv.)een)all;in;one)JAR)+)Java)8)runBme)(JRE))+)BusyBox • Generieke)oplossing)(i.t.t.)centrale)Java)appserver),)ook)applicaBes)gemaakt) met)NodeJS,)PHP,)Ruby,)etc. • Docker)images)gemaakt)door)CI)server)(bijv.)Jenkins)of)Travis) • Developers)verantwoordelijk)voor)de)Docker)images • Operators)kunnen)developers)evt.)ondersteunen • Operators)verantwoordelijk)voor)plaVorm)waar)de)containers)op)draaien
13. OS#voor#containers:#CoreOS • Minimaal(Linux(OS(gemaakt(voor( containers((Docker,(systemd<nspawn,( Rocket) • "Secure(the(internet" • Atomic(automaDc(updates((acDve/ passive(dual<parDDon,(rolling(reboots) • Omaha(update(protocol(en(SDK(van( Google's(Chrome(OS • Geen(package(manager(dus((gebruik(je( containers(voor) • coreos.com/using<coreos
14. Service'Registry:'etcd • Distributed+key+value+store • Shared+configura8on,+service+discovery+ &+locking • Expiring+keys+via+TTL • Onderdeel+van+CoreOS • Ook+gebruikt+buiten+CoreOS+(bijv.+ Kubernetes) • coreos.com/usingIcoreos/etcd
15. Deployment:+systemd • Systeem(en(service(manager • Zit(tegenwoordig(in(veel(Linux(distributes,(ook(in(CoreOS • Unit(files [Unit] Description=Hello World Service Requires=docker.service After=docker.service [Service] ExecStart=/usr/bin/docker run busybox /bin/sh -c "while true; do echo Hello World; sleep 1; done" [Install] WantedBy=multi-user.target
16. Clustering+en+Deployment:+Fleet • Gedistribueerd+init+systeem+(clustering),+bovenop+systemd+en+ etcd • Met+fleetctl+uitrollen+van+systemd+units+(van+Docker+ containers)+op+het+cluster • Je+kan+dus+containers+meervoudig+neerze?en • Ik+gebruik+zelf+een+versienummer+in+de+naam+van+de+container • Meerdere+versies+naast+elkaar+draaien
17. Clustering+en+Deployment:+Fleet • Resource)scheduling:)"least4loaded")scheduling)algorithm,)#aantal)units • Sturen)van)de)schedulingbeslissing)/)dependency)scheduling • Op)specifieke)machine)(MachineID))/)machines)met)specifieke)metadata) (MachineMetadata))/)neerzeDen)bij)specifieke)container)(MachineOf))/) of)juist)niet)(Conflicts))/)op)alle)machines)in)cluster)(Global) • Opnieuw)schedulen)als)machine)faalt)(of)reboot) • Fleet)kent)alleen)vrij)eenvoudige)resource)scheduling)(zie)issue)#1055) • coreos.com/using4coreos/clustering
18. Clustering+en+Deployment:+Fleet
19. Service'Announcement:'ExecStartPost'&'ExecStopPost [Unit] Description=My Advanced Service After=etcd.service After=docker.service [Service] TimeoutStartSec=0 ExecStartPre=-/usr/bin/docker kill apache1 ExecStartPre=-/usr/bin/docker rm apache1 ExecStartPre=/usr/bin/docker pull coreos/apache ExecStart=/usr/bin/docker run --name apache1 -p 80:80 coreos/apache /usr/sbin/apache2ctl -D FOREGROUND ExecStartPost=/usr/bin/etcdctl set /domains/example.com/10.10.10.123:8081 running ExecStop=/usr/bin/docker stop apache1 ExecStopPost=/usr/bin/etcdctl rm /domains/example.com/10.10.10.123:8081 [Install] WantedBy=multi-user.target Zie$'Advanced$Unit$Files'.
20. Service'Announcement:'Sidekick'containers [Unit] Description=Announce nginx1.service # Binds this unit and nginx1 together. # When nginx1 is stopped, this unit will be stopped too. BindsTo=nginx1.service [Service] ExecStart=/bin/sh -c "while true; do etcdctl set /services/website/nginx1 \ '{ \"host\": \"%H\", \"port\": 8080, \"version\": \"52c7248a14\" }' \ --ttl 60;sleep 45;done" ExecStop=/usr/bin/etcdctl delete /services/website/nginx1 [X-Fleet] # This unit will always be colocated with nginx1.service MachineOf=nginx1.service Zie$'Run$a$Simple$Sidekick'$en$service4discovery.md.
21. Service'Announcement:'Registrator • In$eigen$Docker$container$op$elke$CoreOS$machine • Luistert$naar$Docker$lifecycle$events$(via$Docker$remote$API) • Registreert$services$bij$een$bepaalde$service$registry • Ondersteunt$meerdere$service$registries$(backends),$w.o.$etcd,$ Consul$en$SkyDNS • Pluggable$service$registry$backends • github.com/gliderlabs/registrator
22. Service'Discovery:'vulcand • Load&balancer • Servers,1Backends1en1Frontends • Circuit1breakers1&1failover1predicates1 (wanneer1en1hoe1vaak1opnieuw1 proberen?) • github.com/mailgun/vulcand
23. Registrator*backend*voor*vulcand • Backend)voor)vulcand)v2)API)toevoegen)aan)Registrator • Maakt)gebruik)van)de)vulcand)v2)API)(registreert)niet)direct)in) etcd) • Voert)health)checks)uit)op)services • github.com/MarcelHarkema/registrator/tree/feature/ dockergrunnDvulcandDv2DapiDhealthDchecks)branch)op)GitHub
24. Enkele&alterna*even • Service(discovery • Consul(met(consul3template(of(confd((beide(werken(bijv.(met(HAProxy(of(Nginx) • SmartStack:(Synapse((met(HAProxy)(&(Nerve • Nginx(proxy • Clustering • Docker(Swarm • Kubernetes • Mesos
25. Demo
26. Demo%op%Digital%Ocean • 3#CoreOS#hosts#met#global#units#(i.e.#op#elke#host)#registrator#en#vulcand • Een#NodeJS#demo#applica?e,#deze#print#de#Docker#containerCid#en#applica?eversie • Meerdere#versies#draaien#naast#elkaar#(v0.1.0#en#v0.2.0) • Van#elke#versie#een#aantal#instan?es#(containers) • Switchen#van#vulcand#frontend#van#v0.1.0#backend#naar#v0.2.0#backend#(blue/green#deployment) • /health#URL#voor#health#check • REST#API#voor#beinvloeden#health#status#(simuleren#van#een#falende#health#check) • N.B.#Voor#deze#demo#gebruik#ik#roundCrobin#DNS#en#geen#firstChop#redundancy#/#elas?c#IP#(zie# daarvoor#bijv.#CARP#/#VRRP,#AWS#Route#53,#e.d.) • Zie#github.com/MarcelHarkema/DockerGrunnCMeetupC20150303
27. Re Re Re External LB Docker container met vulcand Docker container met Demo applicatie Legenda CoreOS-1 CoreOS-2 CoreOS-3 De Vu Vu De De Vu etcd etcd etcd Verzoeken vanuit vulcand op CoreOS-1 Verzoeken vanuit vulcand op CoreOS-2 Verzoeken vanuit vulcand op CoreOS-3 demo-application.qahwah.io Docker container met Registrator
28. Wat$draait$er? $"fleetctl"list*units UNIT MACHINE ACTIVE SUB demo-application-v0.1.0@1.service e9436653.../10.133.191.90 active running demo-application-v0.1.0@2.service 1f264d10.../10.133.191.84 active running demo-application-v0.2.0@1.service 1f264d10.../10.133.191.84 active running demo-application-v0.2.0@2.service 9b655596.../10.133.191.87 active running registrator-vulcand.service 1f264d10.../10.133.191.84 active running registrator-vulcand.service 9b655596.../10.133.191.87 active running registrator-vulcand.service e9436653.../10.133.191.90 active running vulcand.service 1f264d10.../10.133.191.84 active running vulcand.service 9b655596.../10.133.191.87 active running vulcand.service e9436653.../10.133.191.90 active running
29. systemd'unit'bij'demo'applica2e [Unit] Description=Demo application v0.2.0 After=docker.service Requires=docker.service [Service] ... Environment=IMAGE=marcelharkema/demo-application:0.2.0 CONTAINER=demo-application-v0.2.0 ... ExecStart=/usr/bin/docker run --rm --name=${CONTAINER} \ -e SERVICE_ID=${CONTAINER}@%i \ -e SERVICE_NAME=${CONTAINER} \ -e SERVICE_FRONTEND_HOST_LOAD_BALANCER=${CONTAINER}.${DNS_DOMAIN} \ -e SERVICE_FRONTEND_HOST_CONTAINER=instance-%i.${CONTAINER}.${DNS_DOMAIN} \ -e SERVICE_CHECK_HTTP=/health \ -e SERVICE_CHECK_INTERVAL=3s \ -P ${IMAGE} ... [X-Fleet] Conflicts=${CONTAINER}*.service
30. vulcand(keys(in(etcd:(alle(backends(en(servers $"etcdctl"ls"))recursive"/vulcand/backends /vulcand/backends/demo-application-v0.2.0 /vulcand/backends/demo-application-v0.2.0/backend /vulcand/backends/demo-application-v0.2.0/servers /vulcand/backends/demo-application-v0.2.0/servers/demo-application-v0.2.0@1 /vulcand/backends/demo-application-v0.2.0/servers/demo-application-v0.2.0@2 /vulcand/backends/demo-application-v0.2.0@1 /vulcand/backends/demo-application-v0.2.0@1/backend /vulcand/backends/demo-application-v0.2.0@1/servers /vulcand/backends/demo-application-v0.2.0@1/servers/demo-application-v0.2.0@1 /vulcand/backends/demo-application-v0.2.0@2 /vulcand/backends/demo-application-v0.2.0@2/backend /vulcand/backends/demo-application-v0.2.0@2/servers /vulcand/backends/demo-application-v0.2.0@2/servers/demo-application-v0.2.0@2
31. vulcand(keys(in(etcd:(alle(frontends $"etcdctl"ls"))recursive"/vulcand/frontends /vulcand/frontends/demo-application-v0.2.0 /vulcand/frontends/demo-application-v0.2.0/frontend /vulcand/frontends/demo-application-v0.2.0@1 /vulcand/frontends/demo-application-v0.2.0@1/frontend /vulcand/frontends/demo-application-v0.2.0@2 /vulcand/frontends/demo-application-v0.2.0@2/frontend N.B.$frontends$...@1$en$...@2$ontsluiten$specifieke$instances.$Dit$kan$handig$zijn$ voor$bijv.$debugging.$Te$configureren$via de$SERVICE_FRONTEND_HOST_CONTAINER$environment$variabele$(en$uit$te$ze?en$ door$die$variabele$niet$te$ze?en).
32. vulcand(keys(in(etcd:(een(backend $"etcdctl"get"/vulcand/backends/demo3applica6on3v0.2.0/backend"|"./jq". { "Id": "demo-application-v0.2.0", "Type": "http", "Settings": { "Timeouts": { "Read": "", "Dial": "", "TLSHandshake": "" }, "KeepAlive": { "Period": "", "MaxIdleConnsPerHost": 0 } } }
33. vulcand(keys(in(etcd:(een(server $"etcdctl"get"/vulcand/backends/demo3applica6on3v0.2.0/servers/ demo3applica6on3v0.2.0@1"|"./jq". { "Id": "demo-application-v0.2.0@1", "URL": "http://10.133.191.84:49154" }
34. vulcand(keys(in(etcd:(een(frontend $"etcdctl"get"/vulcand/frontends/demo3applica6on3v0.2.0/frontend"|"./jq". { "Id": "demo-application-v0.2.0", "Route": "Host(\"demo-application-v0.2.0.qahwah.io\") && PathRegexp(\".*\")", "Type": "http", "BackendId": "demo-application-v0.2.0", "Settings": { "Limits": { "MaxMemBodyBytes": 0, "MaxBodyBytes": 0 }, "FailoverPredicate": "", "Hostname": "", "TrustForwardHeader": false } }
35. vulcand(keys(in(etcd:(nog(een(frontend $"etcdctl"get"/vulcand/frontends/demo3applica6on3v0.2.0@1/frontend"|"./jq". { "Id": "demo-application-v0.2.0@1", "Route": "Host(\"instance-1.demo-application-v0.2.0.qahwah.io\") && PathRegexp(\".*\")", "Type": "http", "BackendId": "demo-application-v0.2.0@1", "Settings": { "Limits": { "MaxMemBodyBytes": 0, "MaxBodyBytes": 0 }, "FailoverPredicate": "", "Hostname": "", "TrustForwardHeader": false } }
36. Simuleren)niet)healthy)applica0e $"etcdctl"get"/vulcand/backends/demo3applica6on3v0.2.0/servers/ demo3applica6on3v0.2.0@1"|"./jq". { "Id": "demo-application-v0.2.0@1", "URL": "http://10.133.191.84:49154" } $"curl"'H""Content'Type:"applica5on/json""'d"'{"health":false}'"\ ""h@p:/ /10.133.191.84:49154/api/health De#container#wordt#uit#roula/e#genomen#na#een#falende#health#check: Mar 03 15:17:06 coreos-2.qahwah.io Health check demo-application-v0.2.0@1 (http://10.133.191.84:49154/health) result HTTP code 200 Mar 03 15:17:09 coreos-2.qahwah.io Health check demo-application-v0.2.0@1 (http://10.133.191.84:49154/health) result HTTP code 500
37. Meerdere%versies%naast%elkaar%en%switchen demo%applica+on.qahwah.io1frontend1naar1servers1van1backend1 demo%applica+on%v0.2.01wijzen: $ docker exec vulcand vctl frontend upsert \ -id demo-application.qahwah.io \ -b demo-application-v0.2.0 \ -route 'Host(`demo-application.qahwah.io`) && PathRegexp(`.*`)' OK: frontend upserted
38. Vragen?