SpringOne 2GX - Writing a Kubernetes Autoscaler - How To

@saturnism @kubernetesio @googlecloud Writing a Kubernetes Autoscaler Kuberntes API -
In Depth

@saturnism @kubernetesio @googlecloud SPRINGONE2GX WASHINGTON, DC Unless otherwise indicated, these
slides are © 2013-2015 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Writing a Kubernetes Autoscaler Ray Tsang @saturnism

@saturnism @kubernetesio @googlecloud Ray Tsang Developer Advocate [email protected] @saturnism |
+RayTsang

@saturnism @kubernetesio @googlecloud Ray Tsang Developer Architect Traveler Photographer flickr.com/saturnism

@saturnism @kubernetesio @googlecloud What we are scaling today? Highly scalable
distributed in-memory cache

@saturnism @kubernetesio @googlecloud Distributed in-memory cache Key-value NoSQL store Wonderful
Elastic Property Infinispan

@saturnism @kubernetesio @googlecloud Node 1 Cache Node 2 Cache Node
3 Cache A B C

3 Cache A A B B C C N-Copies for Redundancy

3 Cache A A B B C C

@saturnism @kubernetesio @googlecloud Node 1 Cache Node 3 Cache A
A B C C B

@saturnism @kubernetesio @googlecloud Node 1 Cache A B Node 2
Cache B C Node 3 Cache A Node 4 Cache More Capacity! C

@saturnism @kubernetesio @googlecloud Node 1 Cache A B Node 2
Cache B C Node 3 Cache A Node 4 Cache C Rebalanced!

@saturnism @kubernetesio @googlecloud Wouldn’t it be nice to… Autoscale this?

@saturnism @kubernetesio @googlecloud Horizontal Scaling Vertical Scaling

@saturnism @kubernetesio @googlecloud Stateless Workload Scale Out System Metrics Easy
Stuff

@saturnism @kubernetesio @googlecloud Scale In Custom Metrics Stateful Workload Difficult
Stuff

@saturnism @kubernetesio @googlecloud Today - Difficult Stuff! Autoscale Infinispan, with
target # of entries per node, out and in!

@saturnism @kubernetesio @googlecloud Keep in Mind I’ve never written an
autoscaler before!

@saturnism @kubernetesio @googlecloud Oh, By The Way Running in Kubernetes!

@saturnism @kubernetesio @googlecloud Enter Kubernetes Greek for “Helmsman”; also the
root of the word “Governor” • Container orchestrator • Runs containers • Supports multiple cloud and bare- metal environments • Inspired and informed by Google’s experiences and internal systems • Open source, written in Go Manage applications, not machines

@saturnism @kubernetesio @googlecloud Developer View

@saturnism @kubernetesio @googlecloud web browsers Scheduler kubectl web browsers scheduler
Kubelet Kubelet Kubelet Kubelet Config file Kubernetes Master Container Image Developer View What just happened?

@saturnism @kubernetesio @googlecloud Kubernetes Exposes REST API

@saturnism @kubernetesio @googlecloud Let’s See It !

@saturnism @kubernetesio @googlecloud Keep in Mind I’ve never written an
autoscaler before!

@saturnism @kubernetesio @googlecloud Autoscaling 101 What I learned

@saturnism @kubernetesio @googlecloud Node Your App Metrics Collector JMX Metrics
Node Your App Metrics Collector JMX Metrics Metrics Server API Autoscaler Scaling Config Metrics Store Aggregated Metrics Actuator Desired # of Nodes Image / Template Your App Metrics Collector Read Instantiate a new instance Autoscaling Pattern

@saturnism @kubernetesio @googlecloud Autoscaler Configuration Autoscaler Scaling Config Metric Target
utilization Image / Template Min # of instances Max # of instances CPU, Memory, QPS, etc. 80% or 100 QPS To start a new instance No less than this many No more than this many

@saturnism @kubernetesio @googlecloud Push Metrics Metric Metric Name Utilization /
Value Instance Name Cluster Name Timestamp CPU, Memory, QPS, etc. 80% or 100 QPS node-1431 Infinispan Cluster 12:00:34.123

@saturnism @kubernetesio @googlecloud Instance Name Metric Name Metric Value Timestamp
node-1 numberOfEntries 0 23:01:00 node-2 numberOfEntries 0 23:01:00 node-1 numberOfEntries 100 23:01:10 node-2 numberOfEntries 100 23:01:10 node-1 numberOfEntries 150 23:01:20 node-2 numberOfEntries 150 23:01:20 node-1 numberOfEntries 200 23:01:30 node-2 numberOfEntries 200 23:01:30 Example Metrics

@saturnism @kubernetesio @googlecloud Instance Name Metric Name Metric Value Timestamp
node-1 numberOfEntries 0 23:01:00 node-2 numberOfEntries 0 23:01:00 node-1 numberOfEntries 100 23:01:10 node-2 numberOfEntries 100 23:01:10 node-1 numberOfEntries 150 23:01:20 node-2 numberOfEntries 150 23:01:20 node-1 numberOfEntries 200 23:01:30 node-2 numberOfEntries 200 23:01:30 Example Metrics Instance Name Average Metric Value node-1 150 node-2 150 Sum 300

@saturnism @kubernetesio @googlecloud Autoscaler Algorithm target # instances = sum(utilization)
target utilization ceil( )

@saturnism @kubernetesio @googlecloud Autoscaler Algorithm sum(300) 100 target # instances
= target utilization = 100 ceil( )

@saturnism @kubernetesio @googlecloud Autoscaler Algorithm min < target # instances
< max

@saturnism @kubernetesio @googlecloud It’s expensive to scale out Delay the
scaling in Watch out for noises

@saturnism @kubernetesio @googlecloud Let’s write an autoscaler! The real stuff

@saturnism @kubernetesio @googlecloud Let’s see some code

@saturnism @kubernetesio @googlecloud I used Spring Boot and Groovy @SpringBootApplication
@ComponentScan("org.a8r") @EnableAutoConfiguration @EnableAsync @EnableScheduling class Application { static void main(String[] args) { SpringApplication.run(Application, args) } }

@saturnism @kubernetesio @googlecloud Metrics Store - Used a Tree Cache

@saturnism @kubernetesio @googlecloud Microservices - Yay! @RestController @RequestMapping("/a8r/autoscaler") @Slf4j class
AutoscalerService { }

@saturnism @kubernetesio @googlecloud class AutoscalerDefintion { @NotEmpty String replicationControllerId @NotEmpty
String metricName @NotNull Double threshold @NotNull Integer duration Integer minReplicas = 1 @NotNull Integer maxReplicas } @RestController @RequestMapping("/a8r/autoscaler") @Slf4j class AutoscalerService { ... }

@saturnism @kubernetesio @googlecloud I Love @Slf4J! @Slf4j class AutoscalerService {
… log.info "Updated autoscaler for $fqn, {}", definition … }

@saturnism @kubernetesio @googlecloud Wake up periodically @Scheduled(fixedRateString = "\${autoscaler.wakeupInterval}") void
autoscale() { autoscalerCache.root.children.each { scale(it.data as AutoscalerDefintion) } } @Async void scale(AutoscalerDefintion definition) { … }

@saturnism @kubernetesio @googlecloud Autoscaler Algorithm

@saturnism @kubernetesio @googlecloud Talking to Kubernetes API

@saturnism @kubernetesio @googlecloud SSLSocketFactory with CA Certificate @Bean SSLSocketFactory sslSocketFactory(
@Value("#{environment.KUBERNETES_CA_CERT_FILE}") String caCertFile) throws Exception { ... }

@saturnism @kubernetesio @googlecloud RestTemplate with Headers and SSLSocketFactory restTemplate.setRequestFactory(new SimpleClientHttpRequestFactory()
{ protected void prepareConnection(HttpURLConnection conn, String httpMethod) throws IOException { if (headers["Authorization"]) { conn.setRequestProperty("Authorization", headers.getFirst("Authorization")) } if (sslSocketFactory && conn instanceof HttpsURLConnection) { ((HttpsURLConnection) conn).setSSLSocketFactory(sslSocketFactory) } super.prepareConnection(conn, httpMethod); };

@saturnism @kubernetesio @googlecloud Certificate Authority… Ugh! Let’s see the code

@saturnism @kubernetesio @googlecloud http://github.com/jmxtrans JMXTrans to the rescue!

@saturnism @kubernetesio @googlecloud { "servers" : [ { "url": "service:jmx:http-remoting-jmx://${infinispan.host}:${infinispan.port}",
"queries" : [ { "obj": "jboss.infinispan:type=Cache,component=Statistics,name=\"namedCache(dist_sync)\",*", "attr": ["numberOfEntries"], "outputWriters": [ { "@class": "com.googlecode.jmxtrans.model.output.A8RWriter", "metricName": "custom.cloudmonitoring.googleapis.com/infinispan/namedCache/numberOfEntries" } ] } ] } ] } JMX → Output Writer

@saturnism @kubernetesio @googlecloud Live Demo! Demo time

@saturnism @kubernetesio @googlecloud Try Kubernetes Today!

@saturnism @kubernetesio @googlecloud In the Roadmap - Native Autoscaling in
Kubernetes Nodes Pods Horizontal # of nodes # of pods Vertical resources for a node resources for a pod

@saturnism @kubernetesio @googlecloud Horizontal Pod Autoscaling https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/autoscaling.md Autoscaler is First
Class Citizen Autoscaler Controller and Resource Using Pod CPU / Memory Utilization

@saturnism @kubernetesio @googlecloud Vertical Pod Autoscaling https://github.com/kubernetes/kubernetes/issues/10782 “vertical auto-sizer sets
the compute resource limits and request for pods which do not have them set, and periodically adjust them based on demand signals”

@saturnism @kubernetesio @googlecloud Horizontal Node Autoscaling https://github.com/kubernetes/kubernetes/issues/11748 Pretty much done
on Google Compute Engine! Using Node CPU / Memory Utilization

@saturnism @kubernetesio @googlecloud Try out Google Container Engine https://cloud.google.com/container-engine/

@saturnism @kubernetesio @googlecloud Thanks! Images by Connie Zhou http://kubernetes.io http://bit.ly/1QLg5E1

SpringOne 2GX - Writing a Kubernetes Autoscaler - How To

SpringOne 2GX - Writing a Kubernetes Autoscaler - How To

More Decks by Ray Tsang

Other Decks in Technology

Featured

Transcript