deployments, scaling and management and other operational burden of applications. Kubernetes was designed from the ground-up as a loosely coupled collection of components centered around these key aspects. It abstracts away the underlying hardware of the nodes and provides a uniform interface for applications to be both deployed and consume the shared pool of resources. Intro - What is Kubernetes? Evolution of Kubernetes - Google’s Borg
configuration management tools for DevOps Teams. Deploying new services to production required rigorous work, - Boot up a dedicated VM instance with required OS & resources (RAM, CPU, Disk) after careful considerations (With the preferred cloud provider or on-premise) - Write custom multiple chef recipes/puppet scripts to automate processes like Initial setup, Configuration, Deployments and Shutdown. Eg., AWS OpsWorks - For each deployments, the chef recipe should contain steps to pull latest application code, install dependencies and other custom configurations based on the operating system and the application framework. - After testing the chef recipes, setup service discovery and Health checks for monitoring. It’s completely manual and time-consuming process to deploy services. Before containers and kubernetes Evolution of Kubernetes - Google’s Borg
running hundreds of thousands of jobs. 2013: Transitions from Borg to Omega For clusters with large scale compute 2014: Google Introduces Kubernetes 2015: The year of Kube v1.0 & CNCF 2016: The Year Kubernetes Goes Mainstream! 2017: The Year of Enterprise Adoption & Support - GKE, EKS, AKS 2018: Vernacular.ai embraces Kubernetes Evolution of Kubernetes - Google’s Borg
runs hundreds of thousands of jobs, from many thousands of different applications, across a number of clusters each with up to tens of thousands of machines. Google’s Borg maintains the lifecycle of all the applications. It Admits, Schedules, Starts, Restarts and Monitors the application. Hides the internal details of resource management and failure handling so it's users can focus on application development. Operates with very high reliability and availability, and supports applications that do the same Evolution of Kubernetes - Google’s Borg
application and services. User submits their work as tasks that run as one or more jobs in a Borg Cell. Google’s Borg - User perspective Evolution of Kubernetes - Google’s Borg
master, centralized controller - Borglet, in each worker machine. Borg Master - Handles the client RPC calls and mutates the state of the workloads. Borg master process is logically one, but replicated five times. Each replica maintains an in-memory copy of the state of the cell and this state is also recorded in a highly available Paxos based store on the replica's local store. When a job is submitted, the Borgmaster records it persistently in the Paxos store and adds the job’s tasks to the pending queue. Google’s Borg - Master Evolution of Kubernetes - Google’s Borg
assigns tasks to machines if there are sufficient available resources that meet the job’s constraints. The scheduling algorithm has two parts: - Feasibility checking, to find machines on which the task could run - Scoring, which picks one of the feasible machines. E-PVM approach, minimizes the change in cost when placing a task ends up spreading the tasks in multiple machines at the expense of large tasks that require more resources, "worst fit" algorithm. "Best fit" algorithm tries to pack the worker machines as much tightly as possible, leaving other machines empty of jobs. Borg’s scoring model is a hybrid one that tries to reduce the amount of stranded resources – ones that cannot be used. It provides about 3–5% better packing efficiency Google’s Borg - Scheduler Evolution of Kubernetes - Google’s Borg
agent that is present on every machine in a cell. It manages the workloads and reports the state of the machine to the Borg master and other monitoring systems. The Borg master polls each Borglet every few seconds to retrieve the machine’s current state and send it any outstanding requests. If a Borglet does not respond to several poll messages its machine is marked as down and any tasks it was running are rescheduled on other machines. Evolution of Kubernetes - Google’s Borg
so Borg caches the scores until the properties of the machine or task change Equivalence classes Tasks in a Borg job usually have identical requirements and constraints, so rather than determining feasibility for every pending task on every machine, and scoring all the feasible machines, Borg only does feasibility and scoring for one task per equivalence class Relaxed randomization It is wasteful to calculate feasibility and scores for all the machines in a large cell, so the scheduler examines machines in a random order until it has found “enough” feasible machines to score, and then selects the best within that set. Google’s Borg - Scalability Evolution of Kubernetes - Google’s Borg
Borg’s primary goals is to make efficient use of Google’s fleet of machines, which represents a significant financial investment, increasing utilization can save millions of dollars. Evaluation methodology Cell compaction technique, given a workload, find out how small a cell it could be fitted into by removing machines until the workload no longer fitted. Experiment with same set of workload in different cells and then plot min and max machines needed in a graph allot 90%ile resources for better policies that promises scheduling and cost benefits. Cell Sharing Running both production and non-production workloads in the same cell increases resource utilization. Large cells Google runs large cells to run large compute and to avoid cell fragmentations
Resource limits - Resource usage Borg users request CPU in units of milli-cores, and memory and disk space in bytes. Google’s Borg - Utilization Evolution of Kubernetes - Google’s Borg
mithunarunan
hiracky16
taketakekaho
gogutan
nealle
yut0naga1_fa
stomk
jtakeda
kairim0
akitoshiga
teamlab
iktakahiro
usanchuu
lara
kimpetersen
zakiyama
bermonpainter
davidcarrasco
frankvandijk
palkan
codingconduct
helmedeiros
nwiizo
geoffreycrofte
bkeepers
