Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Moving from Mesos to Kubernetes without anyone...

Anubhav Mishra
December 10, 2017

Moving from Mesos to Kubernetes without anyone noticing

At Hootsuite, we’ve been using Mesos and Marathon as our microservices platform for over two years but last year, we made the decision to bet on Kubernetes as its replacement. Eight months later, a small team of three operations engineers had migrated our first microservice from Mesos to Kubernetes. All without developers making any code changes. This was possible by architecting our applications with the proper set of abstractions. Fast-forward three months later and we have almost 20 microservices running on Kubernetes in production.

In this session, we’ll do a live demo of migrating a service from Mesos to Kubernetes, just like how we did it at Hootsuite! We will cover why architecting your infrastructure with the “right” abstractions helps you do these huge migrations with ease and how Kubernetes already contains these abstractions. We will explore how having a service mesh helps routing between two platforms while doing the migration. Also, how a mature CI/CD pipeline can help you deploy to two platforms with ease. To conclude we will explore the differences in running a service in Mesos and Kubernetes.

Anubhav Mishra

December 10, 2017
Tweet

More Decks by Anubhav Mishra

Other Decks in Technology

Transcript

  1. vs

  2. vs

  3. Agenda • Hootsuite’s Journey from Mesos to Kubernetes • Microservices

    pipeline ◦ Mesos and Marathon ◦ Kubernetes • Migration without major disruption • Live demo! • Lessons learned/Conclusion
  4. I want to build a microservice Oh! A “microservice”? Hmm..

    seems to be the new thing huh. Yep, just create a JIRA ticket.
  5. Ok! Now I only need Java, Sensu checks and a

    Jenkins pipeline top deploy to the servers
  6. { upstream service-1 { server 10.0.10.1:5041; .... } } service-2

    10.0.10.2 curl https://10.0.10.1:5041/service/service-1/endpoint service-1 10.0.10.1
  7. { upstream service-1 { server 10.0.10.1:5041; .... } } service-2

    10.0.10.2 service-1 10.0.10.1 localhost:8080
  8. ?

  9. Microservices on Mesos and Marathon • Project Skeleton ◦ Golang

    or Scala • Pipeline as Code ◦ Jenkinsfile ◦ Makefile • Docker images for packaging • API on top of Marathon • Dynamic service discovery • Fat middleware using Consul and NGINX
  10. Microservices on Kubernetes • Project Skeleton ◦ Golang or Scala

    • Pipeline as Code ◦ Jenkinsfile ◦ Makefile • Docker images for packaging • API on top of Marathon • Dynamic service discovery • Fat middleware using Consul and NGINX • Documentation for getting started • ./mesos2k8s
  11. service-2 { upstream service-1 { } upstream bridge-1 { server

    10.0.20.1:5041; .... } } bridge-1 (multi-dc aware)
  12. service-2 curl http://localhost:5040/service/service-1/endpoint { upstream service-1 { } upstream bridge-1

    { server 10.0.20.1:5041; .... } } curl https://10.0.20.1:5041/service/service-1/endpoint bridge-1 (multi-dc aware)
  13. service-2 curl http://localhost:5040/service/service-1/endpoint { upstream service-1 { } upstream bridge-1

    { server 10.0.20.1:5041; .... } } curl https://10.0.30.1:5041/service/service-1/endpoint bridge-1 (multi-dc aware)
  14. foo 10.0.10.4 foo 10.0.17.100 service-1 172.10.0.10 10.0.30.10 apiVersion: v1 kind:

    Service metadata: name: foo labels: app: foo spec: ports: - port: 5040 protocol: TCP name: http selector: app: nginx-skyline-router
  15. foo 10.0.10.4 foo 10.0.17.100 service-1 172.10.0.10 10.0.30.10 # match on:

    # service.namespace.svc.cluster.local # service.namespace # service server_name REGEX …. location / { rewrite ^/(.*)$ /service/$service/$1 break; proxy_pass https://egress_bridge; }
  16. Things fail , Let’s talk about it…. • “The bad

    config outage” • “The classic security group fail”
  17. • Migrating Container Schedulers: http://code.hootsuite.com/migrating-container-orchestrators-mesos-kubernetes-n omad/ • Abstracting Marathon Deployment

    Details from Microservices: http://code.hootsuite.com/abstracting-marathon-deployment-details-from-microse rvices/ • Consul: https://www.consul.io/ Links
  18. Developer Advocate - HashiCorp @anubhavm Anubhav Mishra I am joining

    https://medium.com/@anubhavmishra/i-am-joining-hashicorp-5a38e0977867