The DevOps Awakens: Moving to a Multi-Cloud Architecture

The DevOps Awakens: Moving to a Multi-Cloud Architecture

82056a07bb46dc7ee72fb51747a36ef1?s=128

Jonathan Stacks

September 21, 2017
Tweet

Transcript

  1. The DevOps Awakens: Moving to a Multi-Cloud Architecture jonstacks @stacks_overflow

  2. None
  3. Organizers

  4. What is

  5. The Problem

  6. The Solution

  7. Collaborative Cash Flow Optimization

  8. The Roles We Play Provide innovative, reliable, easy-to-use applications that

    help to liberate working capital for the benefit of both the buyers and suppliers using C2FO.
  9. None
  10. Our Journey to +

  11. Q4 2016 Build Registry Deploy 1 Build 2 Push Images

    4 Stop/Start Blue 5 Start/Stop Green 3 Pull Images
  12. Q4 2016 Full Stack Staging User Acceptance Testing Production

  13. Q1 2017

  14. None
  15. New Requirements • Multi-Cloud • Scale Globally in the Future

    • Go Live in 5-6 months • “Keep the Lights On”
  16. None
  17. More New Requirements • ... • Deploy 3 new applications

  18. Q1 2017 - Fleet to be deprecated... Fleet

  19. None
  20. Q1 2017 - Decisions... Fleet Kubernetes

  21. None
  22. Q1 2017 • Started getting infrastructure stood up on GCP

    • Began deploying some VMs and running tests • Stood up our first K8s cluster ◦ Deployed some simple apps ◦ See what we could do to break it
  23. Q2 2017 • Automated deployments to kubernetes using helm. •

    Got another DevOps Engineer! ◦ Instrumentation! ◦ Monitoring! • Integration with our current systems
  24. Q3 2017 • Staging & UAT environment up and running

    • Deadline push ◦ Solidify process ◦ Lots of load testing ◦ Few bug fixes • We got prod up!
  25. Q3 2017 Average time to spin up a new environment:

    ~ 10 mins Environment Deployments Pods FSS 47 111 UAT 47 110 Production 47 110 Cluster Pods Nodes Pods/Node Testing 19 4 4.75 Non-Prod 273 14 19.5 Production 152 10 15.2
  26. Why?

  27. Open Source Software +

  28. Open Source Software • Kuberntes ◦ Reduced our Infrastructure footprint

    ◦ Lots of features ▪ Rolling Deploys ▪ Healthchecks ▪ Namespaced DNS ▪ Service Discovery ◦ No pets or snowflakes. ◦ Declarative syntax • Helm ◦ Release management ◦ Version our Kubernetes deployments ◦ Specify our application dependencies
  29. Processes • We value automated, repeatable deployments. • Anyone can

    deploy a development environment. • Working on more blameless post mortems. • Processes are very flexible to change. • Quick all-hands meetings
  30. Culture • We hire really good people • No “sacred

    cows” • We trust our employees ◦ We have the freedom to innovate and make changes ◦ We are allowed to take calculated risks
  31. Lessons Learned

  32. We could have collaborated more often & tried to deploy

    DEV OPS FEB. AUG. GCP, K8S POC, Deployment Integration App Architecture & Code Changes X
  33. Not all software is created Equal • Some software took

    more effort to containerize properly for kubernetes. • Best results from apps that follow the 12-factor pattern. • Frustrations with Consul ◦ Doesn’t handle changes in IP very well.
  34. None
  35. !=

  36. Implementing Devops processes is hard • Hindsight is 20/20 •

    Mastery takes time & effort • Well worth it
  37. Thank you!

  38. is hiring!

  39. References 1. By GitHub - https://github.com/github/octicons, MIT, https://commons.wikimedia.org/w/index.php?curid=33440579 2. https://kubernetes.io/docs/tasks/administer-cluster/out-of-resource/#node-oom-behavior

    3. https://brorlandi.github.io/StarWarsIntroCreator/ 4. https://12factor.net/ 5. https://github.com/kubernetes/charts/issues/1892 6. https://github.com/hashicorp/consul/issues/1580