Tweet
Tweet

Slide 1

Slide 1 text

© 2016 Mesosphere, Inc. All Rights Reserved. AGILE APPLICATION DEVELOPMENT FOR CLOUD-NATIVE APPS 1 Michael Hausenblas, Distributed Systems Jester | 2016-10-21 | Agile Tour London @mhausenblas

Slide 2

Slide 2 text

© 2016 Mesosphere, Inc. All Rights Reserved. sys admin/SRE appops developer architect QA/test engineer data engineer

Slide 3

Slide 3 text

© 2016 Mesosphere, Inc. All Rights Reserved. CHALLENGE 3

Slide 4

Slide 4 text

© 2016 Mesosphere, Inc. All Rights Reserved. CHALLENGE 4

Slide 5

Slide 5 text

© 2016 Mesosphere, Inc. All Rights Reserved. WHAT THE HECK IS CLOUD NATIVE? 5

Slide 6

Slide 6 text

© 2016 Mesosphere, Inc. All Rights Reserved. 6 application orchestration container orchestration job scheduling containerization resource management base provisioning CloudFoundry, Heroku, OpenShift Kubernetes, Marathon, Swarm, Nomad, Firmament Chronos, Kubernetes, Nomad Docker, OCI/runc, rkt/appc, Apache Mesos Apache Mesos Vagrant, Puppet, Chef, Ansible, Salt machine management AWS, Azure, Google Cloud, OpenStack, vSphere

Slide 7

Slide 7 text

© 2016 Mesosphere, Inc. All Rights Reserved. CONTAINER
 101 7 • Containers are not VMs • app-level dependency management • lightweight (startup time, footprint, average runtime) • isolation & resource consumption/accounting

Slide 8

Slide 8 text

© 2016 Mesosphere, Inc. All Rights Reserved. CONTAINER
 101 8 • namespaces (resource isolation) • CLONE_NEWNS (Linux 2.4.19) filesystem mount points • CLONE_NEWUTS (Linux 2.6.19) hostname & NIS domain name • CLONE_NEWIPC (Linux 2.6.19) interprocess communication • CLONE_NEWPID (Linux 2.6.24) process ID space • CLONE_NEWNET (Linux 2.6.29) network system resources (network devices, IP routing tables, port numbers, etc.) • CLONE_NEWUSER (Linux 3.8) user & group ID space • cgroups (resource consumption throttling & accounting)

Slide 9

Slide 9 text

© 2016 Mesosphere, Inc. All Rights Reserved. DOCKER 9 https://articles.microservices.com/containers-are-a-lie-2521afda1f81

Slide 10

Slide 10 text

© 2016 Mesosphere, Inc. All Rights Reserved. ARTEFACTS LAYERING 10 examples layer source artifacts .scala, .go, .py, .sh, etc.
 and config files packaging Dockerfile, OCI, rkt runtime Marathon app spec, Kubernetes RC, Docker compose

Slide 11

Slide 11 text

© 2016 Mesosphere, Inc. All Rights Reserved. RELEASE CYCLE 11

Slide 12

Slide 12 text

© 2016 Mesosphere, Inc. All Rights Reserved. RELEASE CYCLE 12 dev CI/CD CI/CD time QA prod dev QA prod release

Slide 13

Slide 13 text

© 2016 Mesosphere, Inc. All Rights Reserved. GOOD PRACTICES 13

Slide 14

Slide 14 text

© 2016 Mesosphere, Inc. All Rights Reserved. CI/CD PIPELINES 14

Slide 15

Slide 15 text

© 2016 Mesosphere, Inc. All Rights Reserved. CI/CD PIPELINES 15

Slide 16

Slide 16 text

© 2016 Mesosphere, Inc. All Rights Reserved. VERSION CONTROL ALL THE THINGZ! 16 • code and infrastructure • ideally: use a DVCS (GitHub, Gitlab) • immutability (see also http://p24e.io) • be careful with sensitive bits (credentials)

Slide 17

Slide 17 text

© 2016 Mesosphere, Inc. All Rights Reserved. HOW TO BUILD STUFF 17 1st generation • Jenkins
 https://jenkins.io/ • Bamboo
 https://www.atlassian.com/software/bamboo • TeamCity
 https://www.jetbrains.com/teamcity/ 2nd generation • Travis
 https://travis-ci.org/ • CircleCI
 https://circleci.com/ • concourse
 https://concourse.ci/

Slide 18

Slide 18 text

© 2016 Mesosphere, Inc. All Rights Reserved. REGISTRIES: WHERE TO PUT YOUR BUILD ARTEFACTS 18 • Docker Hub
 https://hub.docker.com/ • Google Cloud
 https://cloud.google.com/tools/container-registry/ • AWS
 https://aws.amazon.com/ecr/ • CoreOS
 https://quay.io/ • SUSE Portus
 http://port.us.org/ • JFrog Artifactory
 https://www.jfrog.com/artifactory/ • Run your own
 https://docs.docker.com/registry/deploying/

Slide 19

Slide 19 text

© 2016 Mesosphere, Inc. All Rights Reserved. A WORD ON TESTING 19

Slide 20

Slide 20 text

© 2016 Mesosphere, Inc. All Rights Reserved. CONTINUOUS DEPLOYMENT 20

Slide 21

Slide 21 text

© 2016 Mesosphere, Inc. All Rights Reserved. CONTAINER ORCHESTRATION 21

Slide 22

Slide 22 text

© 2016 Mesosphere, Inc. All Rights Reserved. WHAT DOES A SCHEDULER DO? 22

Slide 23

Slide 23 text

© 2016 Mesosphere, Inc. All Rights Reserved. SERVICE DISCOVERY 23 DNS-based easy to integrate SRV records no health checks TTL Proxy-based no port conflicts fast failover no UDP management of VIPs or service ports Application-aware developer fully in control and full-feature implementation effort requires distributed state management (ZK, etcd or Consul)

Slide 24

Slide 24 text

© 2016 Mesosphere, Inc. All Rights Reserved. ZERO-DOWNTIME DEPLOYMENTS 24 rolling deployment bring up new version 
 and terminate old ones until all old are gone blue-green deployment launch a new stack and switch traffic from old to new when new instances are healthy canary deployment bring up a new version, start by routing a small portion of traffic to the new app, and slowly increase

Slide 25

Slide 25 text

© 2016 Mesosphere, Inc. All Rights Reserved. ZERO-DOWNTIME DEPLOYMENTS 25 https://github.com/mhausenblas/zdd-lab

Slide 26

Slide 26 text

© 2016 Mesosphere, Inc. All Rights Reserved. LOGGING AND MONITORING 26

Slide 27

Slide 27 text

© 2016 Mesosphere, Inc. All Rights Reserved. SCOPES 27 container host & intra-host service (app/business)

Slide 28

Slide 28 text

© 2016 Mesosphere, Inc. All Rights Reserved. MONITORING 28 host container service* *) service: app-level + business-level collectd event router storage dashboard alerting collectd collectd node

Slide 29

Slide 29 text

© 2016 Mesosphere, Inc. All Rights Reserved. MONITORING 29 • Event router • Riemann http://riemann.io/ • fluentd http://www.fluentd.org/ • (Kafka, logstash, Flume) • Storage • KairosDB https://kairosdb.github.io/ • InfluxDB https://influxdata.com/time-series-platform/ influxdb/ • (Local FS, HDFS)

Slide 30

Slide 30 text

© 2016 Mesosphere, Inc. All Rights Reserved. MONITORING 30 • Multi-role tools • Sysdig* https://sysdig.com/blog/monitoring-mesos/ • Prometheus https://prometheus.io/ • DataDog* https://www.datadoghq.com/ • Ruxit* http://www.dynatrace.com/en/ruxit/ • Sensu https://sensuapp.org/ 
 *) available via DC/OS Universe

Slide 31

Slide 31 text

© 2016 Mesosphere, Inc. All Rights Reserved. BUT WHAT ABOUT HUMANS? 31

Slide 32

Slide 32 text

© 2016 Mesosphere, Inc. All Rights Reserved. 32 appops The person who writes an app is also the person responsible for operating the app in prod.

Slide 33

Slide 33 text

© 2016 Mesosphere, Inc. All Rights Reserved. 33 It's not about provisioning
 a VM or installing a DC/OS cluster or replacing a faulty HDD …
 
 … this would be on the infrastructure team. appops

Slide 34

Slide 34 text

© 2016 Mesosphere, Inc. All Rights Reserved. 34 appops speakerdeck.com/charity/devops-for-developers-building-an-effective-ops-org-1

Slide 35

Slide 35 text

© 2016 Mesosphere, Inc. All Rights Reserved. CASE STUDIES 35

Slide 36

Slide 36 text

© 2016 Mesosphere, Inc. All Rights Reserved. ISV: AUTODESK 36 http://cloudengineering.autodesk.com/blog/2016/04/autodesk-is-forging-ahead-with-dcos.html

Slide 37

Slide 37 text

© 2016 Mesosphere, Inc. All Rights Reserved. ONLINE RETAILER: OTTO 37 https://dev.otto.de/2015/09/30/on-monoliths-and-microservices/

Slide 38

Slide 38 text

© 2016 Mesosphere, Inc. All Rights Reserved. LEARNING RESOURCES 38

Slide 39

Slide 39 text

© 2016 Mesosphere, Inc. All Rights Reserved. 39 WHERE CAN
 I LEARN MORE? 39 http://itrevolution.com/books/phoenix-project-devops-book/

Slide 40

Slide 40 text

© 2016 Mesosphere, Inc. All Rights Reserved. 40 WHERE CAN
 I LEARN MORE? 40 https://dcos.io

Slide 41

Slide 41 text

© 2016 Mesosphere, Inc. All Rights Reserved. Q & A 41 • @mhausenblas • mhausenblas.info • [email protected] https://dcos.io