Alice and the lost pod: practical guide to Kubernetes in Scala

"Following Alice’s adventure to the world of pods and higher-order functions. Now Alice is a professional Scala developer and over the years she has forgotten her trip to the world of pods and higher-order functions. But this time a new adventure found her. Alice needs to bring back the pod home using her knowledge of Scala. Will she be able to discover the link between Scala and Kubernetes and save her friend? You will find it out in this talk."
Presented at Scala Love in the City 2021 - virtually
ScalaCon 2021 - virtually

Transcript

1. Alice and the lost pod: practical guide to Kubernetes in Scala
2. Roksolana Diachuk • Big Data Developer at Captify • Diversity & Inclusion ambassador for Captify Kyiv o ff ice • Women Who Code Kyiv Data Engineering Lead and Mentor • Speaker and traveller
3. DISCLAIMER
4. In previous episode talk…
5. Functional forest
6. “Why these houses look like constructor details?” - asks Alice. “This is one of our interesting features. All of our objects constitute type system and they are immutable”
7. TYPE SYSTEM IMMUTABLE DECLARATIVE DSL
8. magic-db- cluster-0
9. magic-db- cluster-0
10. 2 years later
11. Alice is a successful Scala developer working in a big city. Getting tired of work s h e d e c i d e d t o spend some time in the countryside in her uncle’s house.
12. Reading time
13. On one of the days Alice was walking in the forest around the house. It kept a lot of memories of childhood she held dear.
16. cave@k8s % _
17. cave@k8s % kubectl get al l
18. cave@k8s % kubectl get al l NAME DESIRED CURRENT AGE pv/magic-db 1 1 2y120h
19. cave@k8s % kubectl describe pv magic-db- p v
20. cave@k8s % kubectl describe pv magic-db- p v Name: magic-db-pv Namespace: default StorageClass: hostpath Status: Running Volume: Labels: <none>
21. magic-db- cluster-0 Long time no see! !
22. magic-db- cluster-0
23. magic-db- cluster-0 “I’m so happy to see you!”
25. magic-db- cluster-0
26. magic-db- cluster-0
27. magic-db- cluster-0
28. “But how can I help you?” magic-db- cluster-0 You can use the power of your Scala knowledge to help me.
29. “Ok, I can do this”
30. java-k8s-client fabric8io skuber
31. skuber
32. Data model Object kinds List kinds Pods, ReplicationController ObjectResource Pod.Spec, ReplicationController.Status ListResource[Pod], ListResource[Node]
33. val k8s = client.init( client.defaultK8sCon fi g.currentContext)
34. val container = Container( name = “magic-link”, image = “magic-link:1.7.9”) val labels = “app" -> "magic-link" val template = Pod.Spec() .addContainer(container).addLabel(labels) k8s.create(Pod(“magic-link-pod”, template))
35. apiVersion: v1 kind: Pod metadata: name: magic-link-pod labels: app: magic-link spec: containers: - name: magic-link image: magic-link:1.7.9
36. kubectl get all
37. NAME READY STATUS AGE …. magic-db- cluster-0
38. kubectl get all
39. YA NAME READY STATUS AGE pod/magic-link-pod-djf8 0/1 ContainerCreating 5s pod/magic-link-pod-djf8 1/1 Running 37s magic-db- cluster-0 magic-svc -djf8 YAY!
40. k8s.delete[Pod](“magic-link-pod”)
41. Magic-link -djf8
42. magic-link -djf8
43. “We need more than that. But what exactly?”
44. “We need more pods!” magic-db- cluster-0 What? !
45. val container = Container( name = “magic-link”, image = “magic-link:1.7.9”) val labels = “app" -> "magic-link" val template = Pod.Template.Spec .addContainer(container) .addLabel(labels)
46. container.exposePort(80) val deployment = Deployment(“magic-link“) .withReplicas(3) .withTemplate(template) val created = k8s.create(deployment)
47. NAME READY STATUS AGE deployment/magic-link 1/1 Running 32s NAME DESIRED CURRENT AGE replicaset/magic-link 1 1 27s
48. val container = Container( name = “magic-link”, image = “magic-link:1.8”) container.exposePort(80) val deployment = Deployment(“magic-link“) .withReplicas(3) .withTemplate(template) val created = k8s.create(deployment)
49. created.recoverWith { case ex: K8SException if (ex.status.code.contains(409)) => { println("Deployment object already exists”) (k8s get[Deployment] deployment.name) fl atMap { currentDep => println("retrieved latest deployment, now updating”)
 val updated = deployment.withResourceVersion( currentDep.metadata.resourceVersion) k8s update updated> }
50. NAME READY STATUS AGE pod/magic-link-wjnr 1/1 Running 15s pod/magic-link-eh8f 1/1 Running 11s pod/magic-link-eu2r 1/1 Running 7s magic-link- eu2r magic-link- eh8f magic-link- wjnr
51. val scale = k8s.getScale[Deployment] (“magic-link”) val upscale = scale.withSpecReplicas(6) k8s.updateScale[Deployment] (“magic-link”, upscale)
52. NAME READY STATUS AGE pod/magic-link-7wjr 1/1 Running 32s pod/magic-link-eh8f 1/1 Running 28s pod/magic-link-eu2r 1/1 Running 23s pod/magic-link-uew 1/1 Running 12s pod/magic-link-omr8 1/1 Running 8s pod/magic-link-yt8p 1/1 Running 5s
53. private def listPods(pods: List[Pod]) = { logger.info(“POD NAMESPACE STATUS”) … logger.info(s"$name $ns $phase”) }
54. … pods.map { pod: Pod => val name = pod.name val ns = pod.namespace val phaseOpt = for { status <- pod.status phase <- status.phase } yield phase } …
55. val podsList: Future[PodList] = k8s.listInNamespace[PodList](“default”) val printPods = podsList.map( pods => listPods(pods.items))
56. NAME NAMESPACE STATUS pod/magic-link-7wjr default Running pod/magic-link-eh8f default Running pod/magic-link-eu2r default Running pod/magic-link-uew default Running pod/magic-link-omr8 default Running pod/magic-link-yt8p default Running
57. val service = Service(“magic-svc”, labels, 80) .exposeOnNodePort(30001 -> 80) k8s.create(service)
58. NAME TYPE PORT(S) AGE svc/magic-svc NodePort 30001 27s
59. magic-svc: df-np: Is that really you? We thought we’ve lost you! magic-svc: It’s magic-db-cluster-0 df-np: Who’s that? Hello!
60. magic-db- cluster-0 Oh my god! I can talk to someone from home
61. “It’s good but there should be another way” magic-db- cluster-0 What do you mean?!
62. “But I really don’t know what else I can build help you” magic-db- cluster-0
63. But Alice, you can build anything you want magic-db- cluster-0 “But I really don’t know what else I can build help you”
64. “You’re a genius! And I know exactly what to build for you.” Something that will let me fly back home magic-db- cluster-0
65. Custom object Service Namespace Volume Deployment StatefulSet ReplicaSet DaemonSet Pod
67. case class CustomResource[Sp, St] ( override val kind: String, override val apiVersion: String, override val metadata: String,
 spec: Sp,
 status: Option[St] extends ObjectResource { …
68. … def withMetadata(metadata: ObjectMetadata) def withName(name: String) def withGenerateName(generateName: String) def withNamespace(namespace: String) def withLabels(labels: Tuple2[String, String]*) def withAnnotations(generateName: Tuple2[String, String]*) def withFinalizers(namespace: String*) def withStatus(status: St) }
69. object CustomResource { def statusMethodEnabler[C <: CustomResource[_,_]] (implicit rd: ResourceDe fi nition[C]: HasStatusSubresource[C]) = { if (!rd.spec.subresources.map(_.status).isDe fi ned) throw new K8sException(Status(message = Some(“Status sub resource must be de fi ned”)))
 new HasStatusSubresource[C] {} } …
71. val crd = CustomResourceDe fi nition( name = “launcher.io”, kind = “launcher”, scope = Scope.Cluster) implicit val system = ActorSystem() implicit val dispatcher = system.dispatcher val saved = saveCrd(crd)
72. def saveCrd(crd: CustomResourceDe fi nition) = { k8s.create(crd).recoverWith { case notFound: K8sException if (notFound.status.code.contains(404)) => { println(“Unable to create CRD”) throw notFound } …
73. case alreadyExists: K8sException if (alreadyExists.status.code.contains(404)) => { (k8s get[CustomResourceDe fi nition] crd.name) fl atMap { existing => val currentVersion = existing.metadata.resourceVersion val newMeta = crd.metadata.copy( resourceVersion = currentVersion) val updatedObj = crd.copy(metadata = newMeta) k8s update updatedObj> }
74. NAME READY STATUS AGE launcher-crd 1/1 Running 33s magic-db- cluster-0
75. “Yay! You’re fl ying home!” magic-db- cluster-0
76. Thank you! Will see you again someday! magic-db- cluster-0
78. Now Alice knew that her knowledge of Scala could help her to build a link between Scala and Kubernetes. And she could build anything she wanted!
80. My contact info dead_flowers22 roksolana-d roksolanadiachuk roksolanad