Alice and the return to the world of pods and higher-order functions

"Alice discovered in the past records that all the pods had disappeared. Alice must return to the world of pods and higher-order functions in order to save her friends. To build a bridge with that world, Alice will rely on the power of ZIO K8s.

Join Alice in her exploration of this library and her rescue mission."

Presented at Functional Scala 2022

Transcript

1. Alice and the return to the world of pods and higher-order functions
2. Roksolana Diachuk • Big Data Developer at Captify • Women Who Code Kyiv Data Engineering Lead • Member of Ukrainian Scala community • Speaker
3. In previous episodes talks…
4. Functional forest
5. magic-db- cluster-0
6. 2 years later
7. NAME READY STATUS AGE launcher-crd 1/1 Running 33s magic-db- cluster-0
8. 3 months passed
9. { “predictions”: [ [ “B-natural phenomenon”, “O”, “B-geographical entity”, “B-time indicator”, ] ] }
10. { “instances”: [Pods and higher- order functions are in danger]} Test
11. 5 Days passed
12. “Oh, don’t be rude, Alice!
 I will only entertain you a bit with my riddle” “Like it explains anything”
13. The past is the key to the future
14. 1 Day LAter
15. Version Timestamp User Operation Comment 9 2021-11-01 Alice CREATE New pods created 10 2021-11-01 System DELETE All pods lost val deltaTable = DeltaTable.forPath(spark, “/magic-db”) deltaTable.history()
17. 3 DayS LAter
18. Alice has been thinking of the ways to help the pods but nothing came up.She decided to distract herself from anxious thoughts by the work tasks.
19. anonym% There’s a way to help your friends Alice% What? How? anonym% You need to travel to their world Alice% But I don’t know how to get there anonym% There’s a passage close to you anonym% Follow my instructions Alice% But I don’t know how to get there anonym% You don’t Alice% How will I know that you’re telling the truth?
21. “What is that?”
22. “Oh my …”
23. kafka-1 kafka-0 “Hello, Alice!”
24. kafka-1 kafka-0 “What happened here?” “It’s a long story. Follow us”
25. A few months ago new moment got started psql-0 “Freedom to the pods!!” And then pods started to disappear
26. On one of the days someone set the factory on fi re Some pods found shelter in functional forest kafka-1 kafka-0
27. “And what can I do from here?” “Actually, a lot!” kafka-1 kafka-2
28. “Never used it, but I can fi gure it out” “You’ll need to use Zio-k8s” kafka-1 kafka-2
29. Code generator support for custom resources Client for the Kubernetes API Support for operators
30. Hand-written base code Generated code ZIO-k8s internals
31. libraryDependencies ++= Seq( 
 "com.coralogix" %% "zio-k8s-client" % “1.4.8”, "com.softwaremill.sttp.client3" %% "async-http-client-backend-zio" % “3.1.1", “com.softwaremill.sttp.client3" %% "slf4j-backend" % “3.1.1" ) ZIO-k8s setup “Looks easy”
32. case class Config(k8s: K8sClusterConfig) val configDesc = ConfigDescriptor.nested(“k8s") (clusterConfigDescriptor).to[Config] val config = TypesafeConfig.fromDefaultLoader[Config](configDesc) val pods = (client ++ cluster) >>> Pods.live
33. def getPod(namespace: String, name: String): IO[ContextInfoFailure, Pod] = { for { ns <- namespace maybePodName <- system .env(name) .mapError(reason => ContextInfoFailure.PodNameMissing(Some(reason))) … } } yield result }
34. …. result <- maybePodName match { case Some(podName) => pods .get(podName, ns) case None => ZIO.fail(ContextInfoFailure.PodNameMissing(None)) } } yield result }
35. getPod("default", “magic-db-cluster-1") “Let’s get this pod!”
36. “But where is this pod?” “We think they’re in another namespace” kafka-1 kafka-2
37. getPod(“pods-freedom“, "magic-db-cluster-1") “Hmm, what about this namespace?”
38. getPod(“pods-freedom“, "magic-db-cluster-1") NAME READY STATUS AGE pod/magic-db-cluster-1 1/1 Running 2y10d
39. private def tailLogs(podName: String, containerName: Option[String]): ZIO[Pods with Console, K8sFailure, Unit] = pods .getLog(podName, K8sNamespace.pods-freedom, containerName, follow = Some(true)) .tap { line => console.putStrLn(line).ignore } .runDrain
40. tailLogs(“magic-db-cluster-1", Some("magic-db")) 5 Nov 17:23:41 * ERROR! Can’t fi nd data storage location
 5 Nov 17:25:13 * ERROR! Can’t fi nd data storage location 5 Nov 17:28:03 * ERROR! Can’t fi nd data storage location 5 Nov 17:31:22 * ERROR! Can’t fi nd data storage location
41. “What if we can move this pod back?”
42. val updatedPod = magicDbPod .modify(_.metadata.each.namespace).setTo(“default") println(updatedPod)
43. Code generation! Les boilerplate code Access control
44. “Then let’s build something to move the pod” “???” kafka-1 kafka-2
45. Custom object Service Namespace Volume Deployment StatefulSet ReplicaSet DaemonSet Pod
46. apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: name: nstrains.transport namespace: default spec: group: transport versions: - name: v1 nstrain.yaml
47. properties: spec: properties: image: magic-train replicas: 2 scope: Namespaced names: plural: nstrains singular: nstrain kind: NsTrain nstrain.yaml
48. //sbt.plugins addSbtPlugin("com.coralogix" %% "zio-k8s-crd" % "1.4.8") //build.sbt externalCustomResourceDefinitions := Seq(file("crds/nstrain.yaml")) enablePlugins(K8sCustomResourceCodegenPlugin) CRD setup
49. import com.coralogix.zio.k8s.client.com.example.stable.v1.nstrain nstrain.get("namespace-train", K8sNamespace.pods-freedom) “This should work!”
51. “What’s happening?” kafka-1 kafka-2 “Some kind of blackout”
52. import com.coralogix.zio.k8s.client.com.example.stable.v1.nstrain nstrain.get("namespace-train", K8sNamespace.pods-freedom)
55. To be continued…
56. Thank you for attention
57. My contact info dead_flowers22 roksolana-d roksolanadiachuk roksolanad
58. Stand with Ukraine