機械学習開発のためのコンテナ入門 / Container for ML

CA 1Day Youth Boot Campでの発表資料です

Transcript

1. Container for ML YouthBootCamp20211203 Yuki Iwazaki@chck / CyberAgent
2. Today’s Goal 2
3. 今日から使える Portableな実験環境 Scalableな推論App 3
4. Prerequisites 4
5. Yuki Iwazaki 2014...Backend Engineer in DSP └2017...ML/DS in Inhouse └2018-...Research Engineer in AI Lab Multimedia (Vision & Language) 5 chck
6. 難解な環境構築 通らないpip install 競合するlibrary version 引き継ぎできない実験コード 6
7. コンテナ技術の世界へ 7
8. Docker for ML MLのためのDockerについて 8
9. Docker Docker is an open platform for developing, shipping, and running applications. Docker enables you to separate your applications from your infrastructure so you can deliver software quickly. With Docker, you can manage your infrastructure in the same ways you manage your applications. By taking advantage of Docker’s methodologies for shipping, testing, and deploying code quickly, you can significantly reduce the delay between writing code and running it in production. https://docs.docker.com/get-started/overview/ 9
10. Why Docker? • コードを動かすための依存をDockerさえ入っていれば動く状態にまでPackagingできる ◦ CondaやPoetry, RStudioではだめ? ▪ 自分のPCでだけ動かすならOK 。移行や共有時どうする? ▪ OSへの直Installや別途Downloadが必要な依存がある場合は? • 実験コードを動かすための長く複雑な手順を Dockerで隠蔽できる ◦ READMEにはDocker commandを書くだけ • Kubernetesを始めとする強力なContainer ServiceにML AppをDeployできる ◦ 2021年現在、一定規模以上のAppの運用を考えると業界標準に 10
11. https://www.datadoghq.com/docker-adoption/ 11
12. LiveDemo ➔ git clone https://github.com/chck/container4ml.git ➔ cd container4ml ➔ ls .git .gitignore 1-simple 2-jupyter 3-fastapi README.md 12
13. Story 1 • 既にあるPython ScriptをDocker化したい • Docker化のProcess ◦ Python Scriptを動かすための依存を含んだDocker Imageを作成 ◦ Docker Containerとして実行し、挙動を確認 ◦ DockerfileのCommitやDocker RegistryにImageをUploadして完了 13
14. Docker Image An image is a read-only template with instructions for creating a Docker container. Often, an image is based on another image, with some additional customization. https://docs.docker.com/get-started/overview/ 14
15. Docker Container A container is a runnable instance of an image. You can create, start, stop, move, or delete a container using the Docker CLI. It is a standard unit of software that packages up code and all its dependencies. https://www.docker.com/resources/what-container 15
16. Docker Container A container is a runnable instance of an image. You can create, start, stop, move, or delete a container using the Docker CLI. It is a standard unit of software that packages up code and all its dependencies. https://www.docker.com/resources/what-container MacBook, ThinkPad, EC2, GCE macOS, Windows10, Ubuntu JupyterLab, Flask, MySQL, Original App 16
17. Story 1: 再掲 • 既にあるPython ScriptをDocker化したい • Docker化のProcess ◦ Python Scriptを動かすための依存を含んだDocker Imageを作成 ◦ Docker Containerとして実行し、挙動を確認 ◦ DockerfileのCommitやDocker RegistryにImageをUploadして完了 17
18. 1. BaseとなるImageを決める “$keyword docker hub”で検索 18
19. 2. Dockerfileを書く ➔ cd 1-simple ➔ ls .dockerignore Dockerfile Makefile pyproject.toml docker-compose.yml main.py poetry.lock ➔ vi Dockerfile -------------------------- 19
20. 2. Dockerfileを書く ➔ vi Dockerfile -------------------------- FROM python:3.8-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends curl \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] 20
21. 2. Dockerfileを書く ➔ vi Dockerfile -------------------------- FROM python:3.8-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends curl \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] 21
22. 2. Dockerfileを書く ➔ vi Dockerfile -------------------------- FROM python:3.8-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends curl \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] Base Imageを指定 22
23. 2. Dockerfileを書く ➔ vi Dockerfile -------------------------- FROM python:3.8-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends curl \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] 環境変数の定義 23
24. 2. Dockerfileを書く ➔ vi Dockerfile -------------------------- FROM python:3.8-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends curl \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] 環境構築の実行コマンドを記述 24
25. 2. Dockerfileを書く ➔ vi Dockerfile -------------------------- FROM python:3.8-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends curl \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] Image内でのDirectory移動 25
26. 2. Dockerfileを書く ➔ vi Dockerfile -------------------------- FROM python:3.8-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends curl \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] local://${pwd}/main.py docker://app/main.py COPY main.py . 26
27. 2. Dockerfileを書く ➔ vi Dockerfile -------------------------- FROM python:3.8-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends curl \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] 起動時に実行されるコマンド 27
28. 3. Docker Imageの作成 ➔ docker build . -t container4ml-simple:1.0 28
29. 3. Docker Imageの作成 ➔ docker build . -t container4ml-simple:1.0 コンテキストの指定. Dockerに見てほしいpath. COPY句でどこを起点にするか . 基本的にDockerfileのある場所でOK イメージ名. お作法的にはowner/image_name イメージタグ. イメージは更新されゆくのでいわゆる versioning 29
30. 3. Docker Imageの作成 ➔ docker build . -t container4ml-simple:1.0 ➔ docker images REPOSITORY TAG IMAGE ID CREATED SIZE container4ml-simple 1.0 be5c53cd3ad6 7 seconds ago 437MB Image sizeはなるべく小さくする 重いとImageのupload/downloadに時間がかかりML Lifecycleの効率も落ちる 30
31. 4. Docker Containerの起動 ➔ docker run container4ml-simple:1.0 scikit-learn: 1.0.1 ➔ docker ps -a CONTAINER ID IMAGE COMMAND ... NAMES 30f773835fbc container4ml-simple:1.0 "python main.py" $(CONTAINER_NAME) ➔ cat main.py import sklearn print(f"scikit-learn: {sklearn.__version__}") 31
32. 4. Docker Containerの実行上書き ➔ docker run --rm container4ml-simple:1.0 ls -lh total 4.0K -rw-r--r-- 1 root root 62 Dec 2 14:54 main.py 実行コマンドは上書きできる 同Imageで挙動だけ変えたい時等で活用 例えばTraining/Servingを同Imageにするとか 32
33. 5. Docker ImageのUpload/Download ➔ docker tag container4ml-simple:1.0 chck/container4ml-simple:1.0 ➔ docker images REPOSITORY TAG IMAGE ID CREATED SIZE chck/container4ml-simple 1.0 c1575db34e5d 2 hours ago 437MB container4ml-simple 1.0 c1575db34e5d 2 hours ago 437MB ➔ docker push chck/container4ml-simple:1.0 ➔ open https://hub.docker.com/repository/docker/chck/container4ml-simple ➔ docker rmi chck/container4ml-simple:1.0 ➔ docker pull chck/container4ml-simple:1.0 ➔ docker images 33
34. 5. Docker ImageのUpload/Download ➔ docker tag container4ml-simple:1.0 chck/container4ml-simple:1.0 ➔ docker images REPOSITORY TAG IMAGE ID CREATED SIZE chck/container4ml-simple 1.0 c1575db34e5d 2 hours ago 437MB container4ml-simple 1.0 c1575db34e5d 2 hours ago 437MB ➔ docker push chck/container4ml-simple:1.0 ➔ open https://hub.docker.com/repository/docker/chck/container4ml-simple ➔ docker rmi chck/container4ml-simple:1.0 ➔ docker pull chck/container4ml-simple:1.0 ➔ docker images tagはAliasとして働くので同じImage IDを持ち, Diskも重複消費しない 指定registryにimageがuploadされる 34
35. 5. Docker ImageのUpload/Download ➔ docker tag container4ml-simple:1.0 chck/container4ml-simple:1.0 ➔ docker images REPOSITORY TAG IMAGE ID CREATED SIZE chck/container4ml-simple 1.0 c1575db34e5d 2 hours ago 437MB container4ml-simple 1.0 c1575db34e5d 2 hours ago 437MB ➔ docker push chck/container4ml-simple:1.0 ➔ open https://hub.docker.com/repository/docker/chck/container4ml-simple ➔ docker rmi chck/container4ml-simple:1.0 ➔ docker pull chck/container4ml-simple:1.0 ➔ docker images 指定したDocker Imageを削除. Registryに上げていれば安心 指定RegistryからImageをDownload これはDocker Hubの参照例 35
36. Tips: Image Tagは複数付与できる ➔ docker tag container4ml-simple:1.0 chck/container4ml-simple:1.0 ➔ docker tag container4ml-simple:1.0 chck/container4ml-simple:latest ➔ docker push chck/container4ml-simple:1.0 ➔ docker push chck/container4ml-simple:latest 36
37. Tips: slim? alpine? ➔ docker pull python:3.8-alpine ➔ docker pull python:3.8-slim ➔ docker pull python:3.8 ➔ docker images REPOSITORY TAG IMAGE ID CREATED SIZE python 3.8-alpine d314e28e240c 2 days ago 43.2MB python 3.8-slim 64458f531a7e 2 weeks ago 122MB python 3.8 67ec76d9f73b 2 weeks ago 909MB Debian baseのfull image (無印) > 軽量版 (slim) > 最軽量版 (alpine) 依存も少なくなっていくのでトレードオフ 最初はslimがおすすめ 37
38. Docker Architecture 38 Docker Client: CUIとしてコマンドで操作 https://docs.docker.com/get-started/overview/#docker-architecture
39. Docker Architecture 39 Docker Host: 仮想環境の本体 DockerのContainerやImage, NetworkやVolumeを管理 Clientのコマンドも待受 https://docs.docker.com/get-started/overview/#docker-architecture
40. Docker Architecture 40 Docker Registry: Docker Imageの共有先 PublicとPrivateを使い分け https://docs.docker.com/get-started/overview/#docker-architecture
41. Docker Architecture 41 今回は 手元のPCがこの部分 https://docs.docker.com/get-started/overview/#docker-architecture
42. Docker Architecture 42 こっちは Remote Server Docker Hubや GCR, ECRが相当 https://docs.docker.com/get-started/overview/#docker-architecture
43. Story 2 • 学習コードの乗ったJupyterLabをDocker上で動かしたい • Docker Compose化のProcess ◦ Dockerfileを書くところまで同じ ◦ docker-compose.ymlを作成 ◦ Docker Composeによる実行、挙動の確認 43
44. Docker Compose Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application’s services. Then, with a single command, you create and start all the services from your configuration. 44 https://docs.docker.com/compose/
45. Docker Compose Docker: Docker Compose: 45 ➔ docker build . -t container4ml ➔ docker run --rm -p 8888:8888 -v ${PWD}:/app container4ml (docker-compose.ymlがある状態で) ➔ docker compose up
46. 1. Dockerfileを書く ➔ cd 2-jupyter ➔ ls .dockerignore docker-compose.yml Makefile pyproject.toml .gitignore Dockerfile poetry.lock train.ipynb ➔ vi Dockerfile -------------------------- 46
47. 1. Dockerfileを書く 47 ➔ vi Dockerfile -------------------------- FROM python:3.8-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends git build-essential \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY *.bin . EXPOSE 8888 CMD ["jupyter", "lab", "--allow-root", "--ip=0.0.0.0", "--no-browser", "--ServerApp.allow_origin=*", "--ServerApp.token=", "--ServerApp.password="]
48. 1-simpleのDockerfileと比較 ➔ vi Dockerfile -------------------------- FROM python:3.8-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends curl \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] 48
49. 1. Dockerfileを書く 49 ➔ vi Dockerfile -------------------------- FROM python:3.8-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends git build-essential \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY *.bin . EXPOSE 8888 CMD ["jupyter", "lab", "--allow-root", "--ip=0.0.0.0", "--no-browser", "--ServerApp.allow_origin=*", "--ServerApp.token=", "--ServerApp.password="]
50. 2. docker-compose.ymlを書く ➔ vi docker-compose.yml -------------------------- services: jupyter: image: chck/container4ml-jupyter:1.0 build: . ports: - "8888:8888" volumes: - ${PWD}:/app 50
51. 2. docker-compose.ymlを書く ➔ vi docker-compose.yml -------------------------- services: jupyter: image: chck/container4ml-jupyter:1.0 build: . ports: - "8888:8888" volumes: - ${PWD}:/app 51 任意のService名 Dockerfileを参照しながら 指定Image名でdocker build portやvolume optionを付与してdocker runされ jupyter containerが起動
52. 3. Docker Containerの起動 (via docker compose) ➔ docker compose up Jupyter Server 1.12.1 is running at:... ➔ docker images ➔ docker ps CONTAINER ID IMAGE ... PORTS ... 47ac8f57c169 chck/container4ml-jupyter:1.0 ... 0.0.0.0:8888->8888/tcp … ➔ open http://localhost:8888 # train.ipynbを実行、model.binが作られたことを確認後、 docker compose build (次ページ) ➔ Ctrl-C Gracefully stopping... 52
53. UA Classifier Simpleな学習モデルの例として、 UserAgent StringからPC or Mobileを 分類するタスクを考える 53
54. UserAgent https://towardsdatascience.com/still-parsing-user-agent-strings-for-your-machine-learning-models-use-this-instead-8928c0e7e74f Client (e.g. ブラウザ) がServerアクセス時に付与するOS等の情報をまとめた文字列 Web広告の文脈では擬似的な個人情報として使えたりするため昨今規制が厳しい 54
55. Story 3 • JupyterLab上で学習したModelを乗せたFastAPI製推論AppをDockerに乗せたい • Docker Compose化のProcess ◦ 今までと同じ 55
56. 1. Dockerfileを書く ➔ cd 3-fastapi ➔ ls .env Dockerfile Makefile test-scale.sh .envrc infra poetry.lock test-switch.sh docker-compose.yml main.py pyproject.toml ➔ vi Dockerfile -------------------------- 56
57. 1. Dockerfileを書く 57 ➔ vi Dockerfile -------------------------- FROM chck/container4ml-jupyter:1.0 AS trainer FROM python:3.8-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends git build-essential \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR /models COPY --from=trainer /app/*.bin . WORKDIR ${APP_HOME} COPY main.py . EXPOSE 80 CMD ["gunicorn", "main:app", "--bind=0.0.0.0:80", "--workers=1", "--threads=8", "--timeout=0", "--worker-class=uvicorn.workers.UvicornWorker"]
58. 1. Dockerfileを書く 58 ➔ vi Dockerfile -------------------------- FROM chck/container4ml-jupyter:1.0 AS trainer FROM python:3.8-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends git build-essential \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR /models COPY --from=trainer /app/*.bin . WORKDIR ${APP_HOME} COPY main.py . EXPOSE 80 CMD ["gunicorn", "main:app", "--bind=0.0.0.0:80", "--workers=1", "--threads=8", "--timeout=0", "--worker-class=uvicorn.workers.UvicornWorker"]
59. 2. docker-compose.ymlを書く ➔ vi docker-compose.yml -------------------------- services: myapp: image: chck/container4ml-fastapi:1.0 build: . ports: - "2222:80" environment: - PYTHONUNBUFFERED=1 - DEBUG=true - REDISHOST=myredis - MODEL_PATH=/models/ua_classifier.bin volumes: - ${PWD}:/app depends_on: - myredis myredis: image: redis:alpine 59
60. 2. docker-compose.ymlを書く ➔ vi docker-compose.yml -------------------------- services: myapp: image: chck/container4ml-fastapi:1.0 build: . ports: - "2222:80" environment: - PYTHONUNBUFFERED=1 - DEBUG=true - REDISHOST=myredis - MODEL_PATH=/models/ua_classifier.bin volumes: - ${PWD}:/app depends_on: - myredis myredis: image: redis:alpine 60 任意のService名 任意のService名 Dockerfileを参照しながら 指定Image名でdocker build 指定Imageをdocker pullで用意 portやvolume, env optionを付与してdocker runされ jupyter containerが起動 myredis containerの起動を待ってから myappを起動
61. 2. docker-compose.ymlを書く ➔ vi docker-compose.yml -------------------------- services: myapp: image: chck/container4ml-fastapi:1.0 build: . ports: - "2222:80" environment: - PYTHONUNBUFFERED=1 - DEBUG=true - REDISHOST=myredis - MODEL_PATH=/models/ua_classifier.bin volumes: - ${PWD}:/app depends_on: - myredis myredis: image: redis:alpine 61 w/o docker compose ➔ docker network create mynwk ➔ docker run -p 2222:80 -e PYTHONUNBUFFERED=1 -e DEBUG=true -e REDISHOST=myredis -e MODEL_PATH=/models/ua_classifier.bin -v ${PWD}:/app --net mynwk chck/container4ml-fastapi:1.0 ➔ docker run --net mynwk redis:alpine w/ docker compose ➔ docker compose up Container間通信(mynwk) myapp myredis
62. 3. Docker Containerの起動 (via docker compose) ➔ docker compose up -d ➔ docker compose logs -f ➔ docker ps CONTAINER ID IMAGE ... PORTS ... NAMES b36811e9f286 chck/container4ml-fastapi:1.0 ... 0.0.0.0:2222->80/tcp 3-fastapi-myapp-1 7304fab5c081 redis:alpine ... 6379/tcp 3-fastapi-myredis-1 ➔ open http://localhost:2222 ➔ open http://localhost:2222/stats ➔ cat main.py ➔ docker compose down ➔ # docker-compose.ymlのenvironmentsにMODEL_NAME=Bを追加して docker compose up --build -d 62
63. 時間が余った時用 • docker system prune -a • 1 Image: 1 Appの理由 • Docker化が嬉しい場面 • Dataなどの重たいファイルはvolumeでsyncしておく話 • ModelをStorageに持つかImageに持つか • Docker container のDebug方法 • 変数によってbuildの振る舞いを変えたい時 • Training/Servingは同じImageか分けるか • Securityの話 63
64. 質問 & 休憩 hh:mm - hh:mm 64
65. Container Deployment ML AppのServingについて 65
66. Why Container Deployment? https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/ 66
67. https://www.datadoghq.com/docker-adoption/ 67
68. https://www.datadoghq.com/container-report-2020/ 68
69. Container Orchestration • Containerの展開や状態の管理を 担うService • Docker ImageをどうServingする かの部分 • Kubernetesを基とする Managed Serviceが複数展開 https://www.datadoghq.com/container-report-2020/ 69
70. DockerとKubernetesの関係 https://docs.bytemark.co.uk/article/kubernetes-terminology-glossary/ 70
71. https://www.docker.com/blog/top-questions-docker-kubernetes-competitors-or-together/docker-kubernetes-together/ 71
72. Kubernetes Architecture 72
73. Cloud Run Archirecture 73
74. Cloud Run • GCPが提供する Managed Container Service • k8sの細かい所をwrapしてApp のCodingに集中できるようにし たもの • Managing Level ◦ GCE < GKE < Cloud Run https://www.idkrtm.com/5-kinds-of-compute-on-google-cloud/ 74
75. Kubernetes vs Cloud Run https://medium.com/google-cloud-jp/cloudrun-buildpacks-110438eccb35 75
76. Story 3-2 • LocalのDocker Composeで動作確認のできたUA Estimatorを Cloud RunにDeployしたい • DeployのProcess ◦ Docker ImageをRegistryにUpload ◦ Cloud Run Serviceを構築・起動 ◦ Cloud Run ServiceにUploadしたImageをApply 76
77. 1. RegistryにImageをUpload ➔ docker tag chck/container4ml-fastapi:1.0 gcr.io/$(GCP_PROJECT)/container4ml/fastapi:1.0 ➔ docker tag gcr.io/$(GCP_PROJECT)/container4ml/fastapi:1.0 gcr.io/$(GCP_PROJECT)/container4ml/fastapi:latest ➔ docker push gcr.io/$(GCP_PROJECT)/container4ml/fastapi:1.0 ➔ docker push gcr.io/$(GCP_PROJECT)/container4ml/fastapi:latest ➔ open https://console.cloud.google.com/gcr/images 77
78. 2. Cloud Run Serviceを構築・起動 https://console.cloud.google.com/run 78
79. 3. UploadしたImageを Cloud RunにApply https://console.cloud.google.com/run/deploy 79
80. Local to Cloud Run 80 docker push
81. Autoscaling https://github.com/rakyll/hey WebAppの負荷試験ができるheyでbenchmarkを取ってみる ➔ brew install hey ➔ sh ./test-scale.sh ➔ sh ./test-scale.sh https://${CLOUDRUN_DOMAIN} Cloud Runのmetricsやheyのbenchmarkを観察 81
82. Blue-green deployment 既存のApp (blue) 稼働中の裏に 新版のApp (green) をTrafficが来ない状態で Deployし、Standbyになったタイミングで Router内部の向き先をgreenに変えることで 無停止Deployを行う仕組み Cloud RunではRevisionによってこれを達成 ➔ sh ./test-switch.sh https://${CLOUDRUN_DOMAIN} ➔ # Deploy New Revision in Cloud Run 82 https://candost.blog/the-blue-green-deployment-strategy/
83. Traffic Splitting Manage trafficで50%(A):50%(B)に Requestを分配してみる この仕組みでSimpleなA/B Testも可 能 83
84. Simple A/A Testing 同じImageでTrafficを分け、Metricsを計測 有意差がないことを確認する 84 Version:1.0 Version:1.0 Routing 50% 50%
85. Simple A/B Testing Image A とImage BでTrafficを分け、Metricsを計測 有意差を検証 85 Version:1.0 Version:1.1 Routing 50% 50%
86. Today’s Goal • Portableな実験環境 ◦ -> Docker, Docker-Compose • Scalableな推論App ◦ -> Cloud Run • まずは研究コードのDocker化から始めてみる 86
87. Thank you! 質問などあれば 🙌 or slack: @chck 88
88. 質問 & 休憩 hh:mm - 17:00 89