Alice in the world of machine learning

"Alice traveled to the world of pods and higher-order functions and all the knowledge she acquired motivated her to become a Scala developer. Part of her work now includes helping out her fellow data scientists colleagues. And there’s one task which only she can do with her knowledge of Kubernetes. Will Alice be able to gather her knowledge and deploy ML models with Kubeflow? You will find out in this talk."
Presented at Data Love 2021 - virtually

Transcript

1. Alice in the world of machine learning
2. Roksolana Diachuk • Big Data Developer at Captify • Diversity & Inclusion ambassador for Captify Kyiv o ff ice • Women Who Code Kyiv Data Engineering Lead and Mentor • Speaker and traveller
3. In previous episodes talks…
4. Functional forest
5. TYPE SYSTEM IMMUTABLE DECLARATIVE DSL
6. magic-db- cluster-0
7. 2 years later
8. magic-db- cluster-0 Long time no see! !
9. NAME READY STATUS AGE launcher-crd 1/1 Running 33s magic-db- cluster-0
11. 3 months passed
12. Alice came back from vacation and decided to change something in her life. She realised that she got bored working as backend engineer and decided to switch to big data engineering.
13. "Do you know how to deploy machine learning models?”
14. “But what this model does?” “This model extracts speci fi c entities out of search queries“
15. “Sounds nice. But why me?” “I’ve heard that you have some experience with Kubernetes”
16. “So we really need your help with deploying these models”
17. Model Output data
18. A few days later
19. Kube fl ow ML fl ow Clipper Self-hosted Seldon
22. Central dashboard Istio Kube fl ow pipelines Argo Katib TFJobs KFServing, Seldon,
 TFServing Data ingestion Data analysis Data transformation Data validation Trainer Model evaluation and validation Serving Logging Jupiter MinIO
23. Central dashboard Istio Kube fl ow pipelines Argo Katib TFJobs KFServing, Seldon,
 TFServing Data ingestion Data analysis Data transformation Data validation Trainer Model evaluation and validation Serving Logging Shared utilities for garbage collection, data access control Pipeline storage Shared con fi guration framework and job orchestration Integrated frontend for job management, monitoring, debugging, data/model visualisation Jupiter MinIO Tuner
24. “I wonder how it works with Kubernetes”
25. Central dashboard Istio Kube fl ow pipelines Argo Jupiter Katib MinIO TFJobs KFServing, Seldon,
 TFServing Data ingestion Data analysis Data transformation Data validation Trainer Model evaluation and validation Serving Logging Kubernetes operators Persistent volume Argo Work fl ows Ingress gateway Service account
26. No installations
27. gcloud services enable \ compute.googleapis.com \ container.googleapis.com \ iam.googleapis.com \ servicemanagement.googleapis.com \ cloudresourcemanagemer.googleapis.com \ ml.googleapis.com \ meshcon fi g.googleapis.com
29. “Can’t wait to see what I can do with that!”
30. Pipeline Model deploy Model training Data pre-processing
31. Model deploy Model training Data pre-processing Pipeline
32. CRD CRD Pod Pod API server Pipeline service DSL compiler component.py component.yaml Argo controllers Kube fl ow pipeline execution
33. Component is one step in Kube fl ow pipeline
34. Component structure
35. “But what should I do next?”
36. Docker image Docker registry Bash script
37. image_name = gcr.io/$PROJECT_ID/kube fl ow/train image_tag = latest
 full_image_name = ${image_name}:${image_tag} cd “$(dirname “$0”)” docker build —build-arg -t “${full_image_name}” . docker push “$full_image_name”
38. deploy preprocess train Name Tags latest latest latest
40. name: train description: Trains entity recognition model. inputs: - {name: Input x URI, type: GCSPath} - {name: Input y URI, type: GCSPath} - {name: Input job dir URI, type: GCSPath} - {name: Input tags, type: Integer} - {name: Input dropout, type: Integer} - {name: Output model URI template, type: GCSPath} …
41. outputs: - name: Output model URI - type: GCSPath implementation: container: image: gcr.io/<PROJECT-ID>/kube fl ow/train:latest …
42. command: [ python3, /pipelines/component/src/train.py, —input-x-path, {inputValue: Input x URI}, —input-job-dir, {inputValue: Input job dir URI}, —input-y-path, {inputValue: Input y URI}, —input-tags, {inputValue: Input tags, type: Integer}, —input-words, {inputValue: Input dropout}, —input-dropout, {inputValue: Output model URI template}, —output-model-path- fi le, {outputPath: Output model URI} ]
43. Kube fl ow UI
45. train_operation = kfp.components.load_component_from_url (‘https://storage.googleapis.com/{}/train/preprocess/ component.yaml'.format(BUCKET)) Load component
46. deploy preprocess train Name
47. training component.yaml Container Registry
48. Create pipeline Compile pipeline Create Experiment Run pipeline
49. @dsl.pipeline( name=‘DS team Pipeline’, description='Performs preprocessing, training and deployment' ) def pipeline(): …
 Creating pipeline
50. pipeline_func = pipeline pipeline_ fi lename = pipeline_func._name_ + ‘.pipeline.zip’ Import kfp.compiler as compiler compiler.Compiler().compile(pipeline_func, pipeline_ fi lename)
 Compiling pipeline
51. client = kfp.Client() try: experiment = client.get_experiment(experiment_name=EXPERIMENT_NAME) experiment: experiment = client.create_experiment(EXPERIMENT_NAME)
 Creating experiment
52. arguments = {} run_name = pipeline_func.__name__ + ‘run' run_result = client.run_pipeline(experiment.id, run_name, pipeline_ fi lename,
 arguments) Running pipeline
53. Preprocess Train Deploy
55. Preprocess Train Deploy
56. pipeline run 1 pipeline run 2 pipeline run 3 Run name Status Accuracy score 98,4% 98,86% 97,65% Training metrics
57. Test your model with sample input data { “instances”: [London on Monday evening]} Test { “instances”: [weather in London today]}
58. { “predictions”: [ [ “B-natural phenomenon”, “O”, “B-geographical entity”, “B-time indicator”, ] ] }
60. “It looks great! Thank you very much!” “You’re welcome!”
63. { “instances”: [Pods and higher- order functions are in danger]} Test
64. To be continued…
66. Resources
67. My contact info dead_flowers22 roksolana-d roksolanadiachuk roksolanad