SpringOne 2GX - Writing a Kubernetes Autoscaler - How To

This talk is intended to show case the Kubernetes API and how to access it from the Java platform.

The autoscaler example code can be found here: https://github.com/saturnism/a8r

Infinispan container source to be autoscaled is here:
https://github.com/saturnism/infinite-infinispan-demo

Transcript

1. @saturnism @kubernetesio @googlecloud Writing a Kubernetes Autoscaler Kuberntes API - In Depth
2. @saturnism @kubernetesio @googlecloud SPRINGONE2GX WASHINGTON, DC Unless otherwise indicated, these slides are © 2013-2015 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Writing a Kubernetes Autoscaler Ray Tsang @saturnism
3. @saturnism @kubernetesio @googlecloud Ray Tsang Developer Advocate rayt@google.com @saturnism | +RayTsang
4. @saturnism @kubernetesio @googlecloud Ray Tsang Developer Architect Traveler Photographer flickr.com/saturnism
5. @saturnism @kubernetesio @googlecloud What we are scaling today? Highly scalable distributed in-memory cache
6. @saturnism @kubernetesio @googlecloud Distributed in-memory cache Key-value NoSQL store Wonderful Elastic Property Infinispan
7. @saturnism @kubernetesio @googlecloud Node 1 Cache Node 2 Cache Node 3 Cache A B C
8. @saturnism @kubernetesio @googlecloud Node 1 Cache Node 2 Cache Node 3 Cache A A B B C C N-Copies for Redundancy
9. @saturnism @kubernetesio @googlecloud Node 1 Cache Node 2 Cache Node 3 Cache A A B B C C
10. @saturnism @kubernetesio @googlecloud Node 1 Cache Node 3 Cache A A B C C B
11. @saturnism @kubernetesio @googlecloud Node 1 Cache A B Node 2 Cache B C Node 3 Cache A Node 4 Cache More Capacity! C
12. @saturnism @kubernetesio @googlecloud Node 1 Cache A B Node 2 Cache B C Node 3 Cache A Node 4 Cache C Rebalanced!
13. @saturnism @kubernetesio @googlecloud Wouldn’t it be nice to… Autoscale this?
14. @saturnism @kubernetesio @googlecloud Horizontal Scaling Vertical Scaling
15. @saturnism @kubernetesio @googlecloud Stateless Workload Scale Out System Metrics Easy Stuff
16. @saturnism @kubernetesio @googlecloud Scale In Custom Metrics Stateful Workload Difficult Stuff
17. @saturnism @kubernetesio @googlecloud Today - Difficult Stuff! Autoscale Infinispan, with target # of entries per node, out and in!
18. @saturnism @kubernetesio @googlecloud Keep in Mind I’ve never written an autoscaler before!
19. @saturnism @kubernetesio @googlecloud Oh, By The Way Running in Kubernetes!
20. @saturnism @kubernetesio @googlecloud Enter Kubernetes Greek for “Helmsman”; also the root of the word “Governor” • Container orchestrator • Runs containers • Supports multiple cloud and bare- metal environments • Inspired and informed by Google’s experiences and internal systems • Open source, written in Go Manage applications, not machines
21. @saturnism @kubernetesio @googlecloud Developer View
22. @saturnism @kubernetesio @googlecloud web browsers Scheduler kubectl web browsers scheduler Kubelet Kubelet Kubelet Kubelet Config file Kubernetes Master Container Image Developer View What just happened?
23. @saturnism @kubernetesio @googlecloud Kubernetes Exposes REST API
24. @saturnism @kubernetesio @googlecloud Let’s See It !
25. @saturnism @kubernetesio @googlecloud Keep in Mind I’ve never written an autoscaler before!
26. @saturnism @kubernetesio @googlecloud Autoscaling 101 What I learned
27. @saturnism @kubernetesio @googlecloud Node Your App Metrics Collector JMX Metrics Node Your App Metrics Collector JMX Metrics Metrics Server API Autoscaler Scaling Config Metrics Store Aggregated Metrics Actuator Desired # of Nodes Image / Template Your App Metrics Collector Read Instantiate a new instance Autoscaling Pattern
28. @saturnism @kubernetesio @googlecloud Autoscaler Configuration Autoscaler Scaling Config Metric Target utilization Image / Template Min # of instances Max # of instances CPU, Memory, QPS, etc. 80% or 100 QPS To start a new instance No less than this many No more than this many
29. @saturnism @kubernetesio @googlecloud Push Metrics Metric Metric Name Utilization / Value Instance Name Cluster Name Timestamp CPU, Memory, QPS, etc. 80% or 100 QPS node-1431 Infinispan Cluster 12:00:34.123
30. @saturnism @kubernetesio @googlecloud Instance Name Metric Name Metric Value Timestamp node-1 numberOfEntries 0 23:01:00 node-2 numberOfEntries 0 23:01:00 node-1 numberOfEntries 100 23:01:10 node-2 numberOfEntries 100 23:01:10 node-1 numberOfEntries 150 23:01:20 node-2 numberOfEntries 150 23:01:20 node-1 numberOfEntries 200 23:01:30 node-2 numberOfEntries 200 23:01:30 Example Metrics
31. @saturnism @kubernetesio @googlecloud Instance Name Metric Name Metric Value Timestamp node-1 numberOfEntries 0 23:01:00 node-2 numberOfEntries 0 23:01:00 node-1 numberOfEntries 100 23:01:10 node-2 numberOfEntries 100 23:01:10 node-1 numberOfEntries 150 23:01:20 node-2 numberOfEntries 150 23:01:20 node-1 numberOfEntries 200 23:01:30 node-2 numberOfEntries 200 23:01:30 Example Metrics
32. @saturnism @kubernetesio @googlecloud Instance Name Metric Name Metric Value Timestamp node-1 numberOfEntries 0 23:01:00 node-2 numberOfEntries 0 23:01:00 node-1 numberOfEntries 100 23:01:10 node-2 numberOfEntries 100 23:01:10 node-1 numberOfEntries 150 23:01:20 node-2 numberOfEntries 150 23:01:20 node-1 numberOfEntries 200 23:01:30 node-2 numberOfEntries 200 23:01:30 Example Metrics Instance Name Average Metric Value node-1 150 node-2 150 Sum 300
33. @saturnism @kubernetesio @googlecloud Autoscaler Algorithm target # instances = sum(utilization) target utilization ceil( )
34. @saturnism @kubernetesio @googlecloud Autoscaler Algorithm sum(300) 100 target # instances = target utilization = 100 ceil( )
35. @saturnism @kubernetesio @googlecloud Autoscaler Algorithm min < target # instances < max
36. @saturnism @kubernetesio @googlecloud It’s expensive to scale out Delay the scaling in Watch out for noises
37. @saturnism @kubernetesio @googlecloud Let’s write an autoscaler! The real stuff
38. @saturnism @kubernetesio @googlecloud Let’s see some code
39. @saturnism @kubernetesio @googlecloud I used Spring Boot and Groovy @SpringBootApplication @ComponentScan("org.a8r") @EnableAutoConfiguration @EnableAsync @EnableScheduling class Application { static void main(String[] args) { SpringApplication.run(Application, args) } }
40. @saturnism @kubernetesio @googlecloud Metrics Store - Used a Tree Cache
41. @saturnism @kubernetesio @googlecloud Microservices - Yay! @RestController @RequestMapping("/a8r/autoscaler") @Slf4j class AutoscalerService { }
42. @saturnism @kubernetesio @googlecloud class AutoscalerDefintion { @NotEmpty String replicationControllerId @NotEmpty String metricName @NotNull Double threshold @NotNull Integer duration Integer minReplicas = 1 @NotNull Integer maxReplicas } @RestController @RequestMapping("/a8r/autoscaler") @Slf4j class AutoscalerService { ... }
43. @saturnism @kubernetesio @googlecloud I Love @Slf4J! @Slf4j class AutoscalerService { … log.info "Updated autoscaler for $fqn, {}", definition … }
44. @saturnism @kubernetesio @googlecloud Wake up periodically @Scheduled(fixedRateString = "\${autoscaler.wakeupInterval}") void autoscale() { autoscalerCache.root.children.each { scale(it.data as AutoscalerDefintion) } } @Async void scale(AutoscalerDefintion definition) { … }
45. @saturnism @kubernetesio @googlecloud Autoscaler Algorithm
46. @saturnism @kubernetesio @googlecloud Talking to Kubernetes API
47. @saturnism @kubernetesio @googlecloud SSLSocketFactory with CA Certificate @Bean SSLSocketFactory sslSocketFactory( @Value("#{environment.KUBERNETES_CA_CERT_FILE}") String caCertFile) throws Exception { ... }
48. @saturnism @kubernetesio @googlecloud RestTemplate with Headers and SSLSocketFactory restTemplate.setRequestFactory(new SimpleClientHttpRequestFactory() { protected void prepareConnection(HttpURLConnection conn, String httpMethod) throws IOException { if (headers["Authorization"]) { conn.setRequestProperty("Authorization", headers.getFirst("Authorization")) } if (sslSocketFactory && conn instanceof HttpsURLConnection) { ((HttpsURLConnection) conn).setSSLSocketFactory(sslSocketFactory) } super.prepareConnection(conn, httpMethod); };
49. @saturnism @kubernetesio @googlecloud Certificate Authority… Ugh! Let’s see the code
50. @saturnism @kubernetesio @googlecloud http://github.com/jmxtrans JMXTrans to the rescue!
51. @saturnism @kubernetesio @googlecloud { "servers" : [ { "url": "service:jmx:http-remoting-jmx://${infinispan.host}:${infinispan.port}", "queries" : [ { "obj": "jboss.infinispan:type=Cache,component=Statistics,name=\"namedCache(dist_sync)\",*", "attr": ["numberOfEntries"], "outputWriters": [ { "@class": "com.googlecode.jmxtrans.model.output.A8RWriter", "metricName": "custom.cloudmonitoring.googleapis.com/infinispan/namedCache/numberOfEntries" } ] } ] } ] } JMX → Output Writer
52. @saturnism @kubernetesio @googlecloud Live Demo! Demo time
53. @saturnism @kubernetesio @googlecloud Try Kubernetes Today!
54. @saturnism @kubernetesio @googlecloud In the Roadmap - Native Autoscaling in Kubernetes Nodes Pods Horizontal # of nodes # of pods Vertical resources for a node resources for a pod
55. @saturnism @kubernetesio @googlecloud Horizontal Pod Autoscaling https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/autoscaling.md Autoscaler is First Class Citizen Autoscaler Controller and Resource Using Pod CPU / Memory Utilization
56. @saturnism @kubernetesio @googlecloud Vertical Pod Autoscaling https://github.com/kubernetes/kubernetes/issues/10782 “vertical auto-sizer sets the compute resource limits and request for pods which do not have them set, and periodically adjust them based on demand signals”
57. @saturnism @kubernetesio @googlecloud Horizontal Node Autoscaling https://github.com/kubernetes/kubernetes/issues/11748 Pretty much done on Google Compute Engine! Using Node CPU / Memory Utilization
58. @saturnism @kubernetesio @googlecloud Try out Google Container Engine https://cloud.google.com/container-engine/
59. @saturnism @kubernetesio @googlecloud Thanks! Images by Connie Zhou http://kubernetes.io http://bit.ly/1QLg5E1