Deploying Django On Kubernetes

Cf0e8bd777c70655decff2855964668d?s=47 Bill Prin
January 27, 2016

Deploying Django On Kubernetes

This was a presentation at the Django SF meetup. It walks through deploying Django, PostgreSQL, and Redis into a Kubernetes (Container Engine) cluster, and compares it to other deployment options.

Cf0e8bd777c70655decff2855964668d?s=128

Bill Prin

January 27, 2016
Tweet

Transcript

  1. None
  2. @briandorsey Hello, Django Start with Django app • AngularJS Guestbook

    application (create and read) • Using local PostgreSQL and Redis Inspired by: http://instagram-engineering.tumblr.com/post/13649370142/what-powers- instagram-hundreds-of-instances
  3. @briandorsey Go Django Go Multiple ways to deploy • PaaS

    • Pros: Simple and easy, lots of useful features • Cons: Limited flexibility • Traditional VM/physical machine without containers • Pros: Familiar and proven • Cons: Poor isolation or resource usage, complex and heavy to scale • Containers • Pros: Efficiency, isolation, portability, immutability • Cons: Less proven, less familiar, “rough edges” (storage, security)
  4. @briandorsey “It’s The Future” This presentation is talking about containers.

    http://blog.circleci.com/its-the-future/ -No, it’s really easy. You just set up a Kubernetes cluster. I need a cluster? -Kubernetes cluster. It’ll manage the deployments of all your services. I only have one service. -What do you mean? You have an app right, so you gotta have at least 8-12 services?
  5. @briandorsey Django on Docker Let’s build a Docker image for

    the application • The Dockerfile adds my application code and dependencies to the image • Starts Django with a Gunicorn server • docker build . -t grc.io/my-project/guestbook
  6. @briandorsey Challenges Now I want to run my image on

    a cluster of nodes (VMs)
  7. @briandorsey Challenges Now I want to run my image on

    a cluster of nodes (VMs) • How does it get scheduled onto an appropriate node? • How does it discover other services like a database or another team’s microservice? • How do end-users or a client microservice know how to find it? • How do I scale it up and down? Manually and automatically? • How do I safely update it? Roll it back? A/B or canary test it? • Where do I store secrets like passwords and or SSL certificates? • How can I namespace my containers?
  8. @briandorsey Enter Kubernetes Greek for “Helmsman”; also the root of

    the word “Governor” • Container orchestrator • Runs containers • Supports multiple cloud and bare- metal environments • Inspired and informed by Google’s experiences and internal systems • Open source, written in Go Manage applications, not machines
  9. @briandorsey Hello, Django Follow along at: https://github.com/waprin/kubernetes_django_postgres_redis Code or documentation

    problems? Submit a GibHub Issue or PR, mention @waprin. README.md is aimed at Google Container Engine (managed Kubernetes), but configs and guide should should work with slight modification on cloud providers that support external load balancers and persistent volumes (AWS, OpenStack, GCE). gcloud commands (Google Cloud SDK CLI) should be replaced by their equivalents.
  10. @briandorsey kubectl Main CLI To Interact With Cluster Some useful

    commands: kubectl get <resource_type> # Gets a list of a given resource (Pods, Services) kubectl describe <resource_type> <resource_name> # Describe a resource kubectl logs <pod> # Get logs from a pod # Run Django Migrations from a Pod kubectl exec -it <frontend-pod> -- python /app/manage.py migrate
  11. @briandorsey Django on Kubernetes First, let’s just deploy the Django

    container kubectl create -f kubernetes_config/frontend.yaml
  12. @briandorsey Django on Kubernetes First, let’s just deploy the Django

    container kubectl create -f kubernetes_config/frontend.yaml This does a couple of things: • Creates a Replication Controller that will control how many Pods are run • Each Pod runs a container using the image we provide. • Creates a Service that maps an external IP to those Pods
  13. #kubernetes @kubernetesio Node 1 Node 2 Node 3 Internet frontend

    frontend frontend Service with External Load Balancer frontend RC
  14. @briandorsey Kubernetes Concepts Replication Controllers scale Pods that match a

    given Label. kubectl scale rc frontend --replicas=5 kubectl rolling-update rc frontend --image=frontend:v2 Every Pod comes with a Virtual IP on a private network. Within a Kubernetes cluster, a Service will round-robin to a Pod that matches its Label.
  15. @briandorsey PostgreSQL and Redis We still need PostgreSQL and Redis

  16. @briandorsey PostgreSQL and Redis We still need PostgreSQL and Redis

    • Could use external services • But let’s deploy them to our cluster
  17. @briandorsey PostgreSQL and Redis • kubectl create -f redis_cluster.yaml •

    redis-master and redis-slave Services created with their respective Replication Controllers • kubectl create -f db_password.yaml • Creates a Secret with our database passwords • kubectl create -f postgres.yaml • Mounts a Volume (GCE disk) and Secret (database passwords) • Not a highly available setup
  18. #kubernetes @kubernetesio Node 1 Node 2 Node 3 Internet redis-slave

    redis-slave redis-master frontend frontend postgres frontend Service with External Load Balancer frontend RC redis-master RC redis-slave RC postgres RC postgres redis-master redis-slave
  19. @briandorsey Cool Use Cases • Continuous Integration/Continuous Delivery • Two

    critical things for CI/CD: isolation and scalability. • Docker gives you isolation • Kubernetes gives you scalability • https://github.com/GoogleCloudPlatform/jenkernetes • A/B or Canary Testing • Have one Service map to two Replication Controllers • Experiment with changing the ratio of each Pod type backing the Service
  20. @briandorsey Cool Use Cases Scale up lots of load testing

    client Pods to simulate CPU usage and trigger Pod and Node autoscaling (beta feature). https://cloud.google.com/solutions/distributed-load-testing-using- kubernetes
  21. Horizontal Pod Autoscaling [Beta] apiVersion: extensions/v1beta1 kind: HorizontalPodAutoscaler metadata: name:

    frontend spec: scaleRef: kind: ReplicationController name: frontend minReplicas: 1 maxReplicas: 10 cpuUtilization: targetPercentage: 50 https://www.flickr.com/photos/davedehetre/4440211085
  22. @briandorsey Django On Kubernetes When You Should Deploy Django on

    Kubernetes • You love the portability, flexibility, composability, and immutability of containers • You want to deploy Django in a microservices environment • You want a toolkit that solves the distributed systems problems and lets you design the platform
  23. @briandorsey Django On Kubernetes When You Should Deploy Django on

    Kubernetes • You love the portability, flexibility, composability, and immutability of containers • You want to deploy Django in a microservices environment • You want a toolkit that solves the distributed systems problems and lets you design the platform When You Should Not Deploy Django on Kubernetes • You are looking for a fully-fledged PaaS (though many are built on Kubernetes) • Containers and microservices offer more complexity than benefit for your use cases
  24. @briandorsey Popular Alternatives • Docker Swarm • Advantages: Built by

    Docker, similar toolchain • Disadvantages: Fewer features and abstractions, less of a cluster- oriented approach • Mesos • Advantages: Proven in production at scale, can schedule non- containerized workflows • Disadvantages: More resource-intensive, high complexity • Running Kubernetes on Mesos is possible
  25. @briandorsey Community Some interesting companies relating to Kubernetes: • Deis

    (http://deis.com) is an open source PaaS with Django support built on Kubernetes • Gondor (http://gondor.io) is a Django-focused PaaS built on Kubernetes • Flocker is a ClusterHQ (http://clusterhq.com) product providing additional tooling for storage on Kubernetes. • Sysdig (http://sysdig.org) provides container analytics products
  26. Google confidential │ Do not distribute Kubernetes status & plans

    Open sourced in June, 2014 v1.0 in July, 2015, v1.1 Nov, 2015, v1.2 Jan, 2016 Google Container Engine (GKE) • hosted Kubernetes - don’t think about cluster setup • GA in August, 2015 PaaSes: • RedHat OpenShift, Deis, Stratos Distros: • CoreOS Tectonic, Mirantis Murano (OpenStack), RedHat Atomic, Mesos
  27. #kubernetes @kubernetesio Kubernetes is Open Source We want your help!

    http://kubernetes.io https://github.com/kubernetes/kubernetes/ Kubernetes Slack Community: http://slack.kubernetes.io @kubernetesio
  28. #kubernetes @kubernetesio • GCP Next 2016, March 23-24 San Francisco

    http://goo.gl/lNPpwr • GCP Slack Community: https://gcp-slack.appspot.com • http://cloud.google.com/python • http://cloud.google.com/python/django(coming soon)
  29. #kubernetes @kubernetesio • Jon Parrott • Sandeep Dinesh • Eli

    Bixby • Amy Unruh • Jeff Mendoza • Aja Hammerly • Jason Hansen • Rudy Mutter • EventBrite Thanks for the help!
  30. end..