Kubeflowを動かしたい人生だった

[Shinjukuもくもくプログラミング #43 - connpass](https://shinjuku-moku.connpass.com/event/125338/)での発表内容です

Transcript

1. KubeflowΛಈ͔͍ͨ͠ਓੜͩͬͨ shinjuku-mokumoku #43
2. औ૊಺༰ Kubeflowͷ؀ڥߏங
3. Minikube for Kubeflow Quick Setup • ηοτΞοϓεΫϦϓτ͕ఏڙ͞Ε͍ͯΔ $ export KUBEFLOW_TAG=v0.5.0 $ curl -O https://raw.githubusercontent.com/kubeflow/kubeflow/${KUBEFLOW_TAG}/scripts/setup-minikube.sh chmod +x setup-minikube.sh $ ./setup-minikube.sh
4. $ curl -O https://raw.githubusercontent.com/kubeflow/kubeflow/${KUBEFLOW_TAG}/scripts/setup-minikube.sh % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 6051 100 6051 0 0 21994 0 --:--:-- --:--:-- --:--:-- 21923 $ ./setup-minikube.sh Assign CPUs between 3..6 [4]: 6 Assign memory (in GB) between 8..11 [8]: 8 Assign disk space (in GB) between 40..46 [50]: 50 Choose a virtualizer/hypervisor installed on your system (virtualbox, vmwarefusion, kvm2, hyperkit): virtualbox If you'd like to access a local directory in Jupyter, please enter the full path: ./setup-minikube.sh: line 42: ks: command not found Installing ksonnet... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 620 0 620 0 0 1104 0 --:--:-- --:--:-- --:--:-- 1103 100 14.6M 100 14.6M 0 0 3991k 0 0:00:03 0:00:03 --:--:-- 6754k [OK] ! Deleting "minikube" from virtualbox ... " Failed to kill mount process: os: process already finished # The "minikube" cluster has been deleted. $ minikube v1.0.0 on darwin (amd64) % Downloading Kubernetes v1.14.0 images in the background ... ! Creating virtualbox VM (CPUs=6, Memory=8192MB, Disk=50000MB) ... & "minikube" IP address is 192.168.99.108 ' Configuring Docker as the container runtime ... ' Version of container runtime is 18.06.2-ce ⌛ Waiting for image downloads to complete ... ✨ Preparing Kubernetes environment ...
5. ೤͍invalid memory addressΤϥʔ INFO Using context "minikube" from kubeconfig file "/Users/asei/.kube/config" INFO Creating environment "default" with namespace "kubeflow", pointing to cluster at address "https://192.168.99.108:8443" panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1057c42] goroutine 1 [running]: (continue)
6. ಉ͡ΤϥʔΛใࠂͯ͠ ͍ΔIssueΛൃݟ Kubeflow getting panic: runtime error: invalid memory address or nil pointer dereference when kfctl.sh generate k8s · Issue #883 · ksonnet/ksonnet
7. ksonnetͷόʔδϣϯ ͕ݹ͍ʁ v0.13.1Ͱfixͨ͠Β͍͠
8. ͔ͨ͠ʹݹ͍ (v0.11.0) install_ks_kubectl_minikube() { # Installing ksonnet if needed KS_VERSION=$(ks version | grep 'ksonnet version' | awk '{print $3}') if [[ $KS_VERSION != "0.11.0" ]]; then echo -e "${YELLOW}Installing ksonnet...${NC}" if [[ ${PLATFORM} == $OSX ]]; then curl -OL https://github.com/ksonnet/ksonnet/releases/download/v0.11.0/ks_0.11.0_darwin_amd64.tar.gz tar zxf ks_0.11.0_darwin_amd64.tar.gz export PATH=$PATH:$(pwd)/ks_0.11.0_darwin_amd64 elif [[ ${PLATFORM} == $LINUX ]]; then curl -OL https://github.com/ksonnet/ksonnet/releases/download/v0.11.0/ks_0.11.0_linux_amd64.tar.gz tar zxf ks_0.11.0_linux_amd64.tar.gz export PATH=$PATH:$(pwd)/ks_0.11.0_linux_amd64 fi echo -e "${GREEN}[OK]${NC}" fi
9. मਖ਼ͯ͠ಈ͔͢ $ ./setup-minikube-patched.sh Assign CPUs between 3..6 [4]: 6 Assign memory (in GB) between 8..11 [8]: 8 Assign disk space (in GB) between 40..46 [50]: 50 Choose a virtualizer/hypervisor installed on your system (virtualbox, vmwarefusion, kvm2, hyperkit): virtualbox If you'd like to access a local directory in Jupyter, please enter the full path: /Users/asei/Documents/TIL/python/Kubeflow ./setup-minikube-patched.sh: line 42: ks: command not found Installing ksonnet... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 620 0 620 0 0 1029 0 --:--:-- --:--:-- --:--:-- 1028 100 21.8M 100 21.8M 0 0 2840k 0 0:00:07 0:00:07 --:--:-- 4459k [OK] ! Deleting "minikube" from virtualbox ... " Failed to kill mount process: os: process already finished # The "minikube" cluster has been deleted. $ minikube v1.0.0 on darwin (amd64) % Downloading Kubernetes v1.14.0 images in the background ... ! Creating virtualbox VM (CPUs=6, Memory=8192MB, Disk=50000MB) ...
10. No resources found. INFO Using context "minikube" from kubeconfig file "/Users/asei/.kube/config" INFO Creating environment "default" with namespace "kubeflow", \ pointing to "version:v1.14.0" cluster at address "https://192.168.99.109:8443" + popd ~/Documents/TIL/python/Kubeflow/localapp/ks_app \ ~/Documents/TIL/python/Kubeflow/localapp + [[ minikube != \m\i\n\i\k\u\b\e ]] + popd ~/Documents/TIL/python/Kubeflow/localapp + set +x Getting kubeflow namespace ready...[OK] Bringing kubeflow services up.No resources found. No resources found. (repeat...)
11. εΫϦϓτʹཔΒͳ͍
12. ύε͕௕͗͢Τϥʔ $ export KFAPP=~/Documents/TIL/python/Kubeflow/minikube-kubeflow $ KUBEFLOW_REPO=${KUBEFLOW_SRC} ${KUBEFLOW_SRC}/scripts/kfctl.sh \ init ${KFAPP} --platform minikube + ENV_FILE=env.sh : : Name /Users/asei/Documents/TIL/python/Kubeflow/minikube-kubeflow \ should not be longer than 25 characters + exit 1
13. ແ৘
14. Minikube͡Όͳͯ͘ GCPͰ ͋Ε͚ͩએ఻͍ͯ͠ΔͷͳΒ͖ͬͱͺͺͬ ͱͰ͖Δ͸ͣ
15. exit status 1 $ kfctl apply all -V INFO[0001] deploying kubeflow application filename="cmd/apply.go:35" couldn't apply KfApp: \ (kubeflow.error): Code 500 with message: coordinator Apply failed for gcp: \ (kubeflow.error): Code 400 with message: Error when running gcloud container clusters get-credentials:\ exit status 1
16. (༨ஊ) Ksonnet͕ফ͑ͦ͏ Welcoming Heptio Open Source Projects to VMware - Cloud Native Apps Prior to the acquisition, Heptio had been shifting focus and resources away from ksonnet; with the acquisition, we felt it was the right time to rethink our investment in ksonnet. As a result, work on ksonnet will end and the GitHub repositories will be archived.
17. Recap 1.minikubeͰKubeflowΛಈ͔͢εΫϦϓτ͸ո͍͠ 2.Kubeflow͸ಈ͔ͳ͍ 3.ksonnet͕ফ͑ͦ͏