Tweet
Tweet

Slide 1

Slide 1 text

Container for ML YouthBootCamp20211203 Yuki Iwazaki@chck / CyberAgent

Slide 2

Slide 2 text

Today’s Goal 2

Slide 3

Slide 3 text

今日から使える Portableな実験環境 Scalableな推論App 3

Slide 4

Slide 4 text

Prerequisites 4

Slide 5

Slide 5 text

Yuki Iwazaki 2014...Backend Engineer in DSP └2017...ML/DS in Inhouse └2018-...Research Engineer in AI Lab Multimedia (Vision & Language) 5 chck

Slide 6

Slide 6 text

難解な環境構築 通らないpip install 競合するlibrary version 引き継ぎできない実験コード 6

Slide 7

Slide 7 text

コンテナ技術の世界へ 7

Slide 8

Slide 8 text

Docker for ML MLのためのDockerについて 8

Slide 9

Slide 9 text

Docker Docker is an open platform for developing, shipping, and running applications. Docker enables you to separate your applications from your infrastructure so you can deliver software quickly. With Docker, you can manage your infrastructure in the same ways you manage your applications. By taking advantage of Docker’s methodologies for shipping, testing, and deploying code quickly, you can significantly reduce the delay between writing code and running it in production. https://docs.docker.com/get-started/overview/ 9

Slide 10

Slide 10 text

Why Docker? • コードを動かすための依存をDockerさえ入っていれば動く状態にまでPackagingできる ○ CondaやPoetry, RStudioではだめ? ■ 自分のPCでだけ動かすならOK 。移行や共有時どうする? ■ OSへの直Installや別途Downloadが必要な依存がある場合は? • 実験コードを動かすための長く複雑な手順を Dockerで隠蔽できる ○ READMEにはDocker commandを書くだけ • Kubernetesを始めとする強力なContainer ServiceにML AppをDeployできる ○ 2021年現在、一定規模以上のAppの運用を考えると業界標準に 10

Slide 11

Slide 11 text

https://www.datadoghq.com/docker-adoption/ 11

Slide 12

Slide 12 text

LiveDemo ➔ git clone https://github.com/chck/container4ml.git ➔ cd container4ml ➔ ls .git .gitignore 1-simple 2-jupyter 3-fastapi README.md 12

Slide 13

Slide 13 text

Story 1 • 既にあるPython ScriptをDocker化したい • Docker化のProcess ○ Python Scriptを動かすための依存を含んだDocker Imageを作成 ○ Docker Containerとして実行し、挙動を確認 ○ DockerfileのCommitやDocker RegistryにImageをUploadして完了 13

Slide 14

Slide 14 text

Docker Image An image is a read-only template with instructions for creating a Docker container. Often, an image is based on another image, with some additional customization. https://docs.docker.com/get-started/overview/ 14

Slide 15

Slide 15 text

Docker Container A container is a runnable instance of an image. You can create, start, stop, move, or delete a container using the Docker CLI. It is a standard unit of software that packages up code and all its dependencies. https://www.docker.com/resources/what-container 15

Slide 16

Slide 16 text

Docker Container A container is a runnable instance of an image. You can create, start, stop, move, or delete a container using the Docker CLI. It is a standard unit of software that packages up code and all its dependencies. https://www.docker.com/resources/what-container MacBook, ThinkPad, EC2, GCE macOS, Windows10, Ubuntu JupyterLab, Flask, MySQL, Original App 16

Slide 17

Slide 17 text

Story 1: 再掲 • 既にあるPython ScriptをDocker化したい • Docker化のProcess ○ Python Scriptを動かすための依存を含んだDocker Imageを作成 ○ Docker Containerとして実行し、挙動を確認 ○ DockerfileのCommitやDocker RegistryにImageをUploadして完了 17

Slide 18

Slide 18 text

1. BaseとなるImageを決める “$keyword docker hub”で検索 18

Slide 19

Slide 19 text

2. Dockerfileを書く ➔ cd 1-simple ➔ ls .dockerignore Dockerfile Makefile pyproject.toml docker-compose.yml main.py poetry.lock ➔ vi Dockerfile -------------------------- 19

Slide 20

Slide 20 text

2. Dockerfileを書く ➔ vi Dockerfile -------------------------- FROM python:3.8-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends curl \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] 20

Slide 21

Slide 21 text

2. Dockerfileを書く ➔ vi Dockerfile -------------------------- FROM python:3.8-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends curl \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] 21

Slide 22

Slide 22 text

2. Dockerfileを書く ➔ vi Dockerfile -------------------------- FROM python:3.8-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends curl \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] Base Imageを指定 22

Slide 23

Slide 23 text

2. Dockerfileを書く ➔ vi Dockerfile -------------------------- FROM python:3.8-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends curl \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] 環境変数の定義 23

Slide 24

Slide 24 text

2. Dockerfileを書く ➔ vi Dockerfile -------------------------- FROM python:3.8-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends curl \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] 環境構築の実行コマンドを記述 24

Slide 25

Slide 25 text

2. Dockerfileを書く ➔ vi Dockerfile -------------------------- FROM python:3.8-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends curl \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] Image内でのDirectory移動 25

Slide 26

Slide 26 text

2. Dockerfileを書く ➔ vi Dockerfile -------------------------- FROM python:3.8-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends curl \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] local://${pwd}/main.py docker://app/main.py COPY main.py . 26

Slide 27

Slide 27 text

2. Dockerfileを書く ➔ vi Dockerfile -------------------------- FROM python:3.8-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends curl \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] 起動時に実行されるコマンド 27

Slide 28

Slide 28 text

3. Docker Imageの作成 ➔ docker build . -t container4ml-simple:1.0 28

Slide 29

Slide 29 text

3. Docker Imageの作成 ➔ docker build . -t container4ml-simple:1.0 コンテキストの指定. Dockerに見てほしいpath. COPY句でどこを起点にするか . 基本的にDockerfileのある場所でOK イメージ名. お作法的にはowner/image_name イメージタグ. イメージは更新されゆくのでいわゆる versioning 29

Slide 30

Slide 30 text

3. Docker Imageの作成 ➔ docker build . -t container4ml-simple:1.0 ➔ docker images REPOSITORY TAG IMAGE ID CREATED SIZE container4ml-simple 1.0 be5c53cd3ad6 7 seconds ago 437MB Image sizeはなるべく小さくする 重いとImageのupload/downloadに時間がかかりML Lifecycleの効率も落ちる 30

Slide 31

Slide 31 text

4. Docker Containerの起動 ➔ docker run container4ml-simple:1.0 scikit-learn: 1.0.1 ➔ docker ps -a CONTAINER ID IMAGE COMMAND ... NAMES 30f773835fbc container4ml-simple:1.0 "python main.py" $(CONTAINER_NAME) ➔ cat main.py import sklearn print(f"scikit-learn: {sklearn.__version__}") 31

Slide 32

Slide 32 text

4. Docker Containerの実行上書き ➔ docker run --rm container4ml-simple:1.0 ls -lh total 4.0K -rw-r--r-- 1 root root 62 Dec 2 14:54 main.py 実行コマンドは上書きできる 同Imageで挙動だけ変えたい時等で活用 例えばTraining/Servingを同Imageにするとか 32

Slide 33

Slide 33 text

5. Docker ImageのUpload/Download ➔ docker tag container4ml-simple:1.0 chck/container4ml-simple:1.0 ➔ docker images REPOSITORY TAG IMAGE ID CREATED SIZE chck/container4ml-simple 1.0 c1575db34e5d 2 hours ago 437MB container4ml-simple 1.0 c1575db34e5d 2 hours ago 437MB ➔ docker push chck/container4ml-simple:1.0 ➔ open https://hub.docker.com/repository/docker/chck/container4ml-simple ➔ docker rmi chck/container4ml-simple:1.0 ➔ docker pull chck/container4ml-simple:1.0 ➔ docker images 33

Slide 34

Slide 34 text

5. Docker ImageのUpload/Download ➔ docker tag container4ml-simple:1.0 chck/container4ml-simple:1.0 ➔ docker images REPOSITORY TAG IMAGE ID CREATED SIZE chck/container4ml-simple 1.0 c1575db34e5d 2 hours ago 437MB container4ml-simple 1.0 c1575db34e5d 2 hours ago 437MB ➔ docker push chck/container4ml-simple:1.0 ➔ open https://hub.docker.com/repository/docker/chck/container4ml-simple ➔ docker rmi chck/container4ml-simple:1.0 ➔ docker pull chck/container4ml-simple:1.0 ➔ docker images tagはAliasとして働くので同じImage IDを持ち, Diskも重複消費しない 指定registryにimageがuploadされる 34

Slide 35

Slide 35 text

5. Docker ImageのUpload/Download ➔ docker tag container4ml-simple:1.0 chck/container4ml-simple:1.0 ➔ docker images REPOSITORY TAG IMAGE ID CREATED SIZE chck/container4ml-simple 1.0 c1575db34e5d 2 hours ago 437MB container4ml-simple 1.0 c1575db34e5d 2 hours ago 437MB ➔ docker push chck/container4ml-simple:1.0 ➔ open https://hub.docker.com/repository/docker/chck/container4ml-simple ➔ docker rmi chck/container4ml-simple:1.0 ➔ docker pull chck/container4ml-simple:1.0 ➔ docker images 指定したDocker Imageを削除. Registryに上げていれば安心 指定RegistryからImageをDownload これはDocker Hubの参照例 35

Slide 36

Slide 36 text

Tips: Image Tagは複数付与できる ➔ docker tag container4ml-simple:1.0 chck/container4ml-simple:1.0 ➔ docker tag container4ml-simple:1.0 chck/container4ml-simple:latest ➔ docker push chck/container4ml-simple:1.0 ➔ docker push chck/container4ml-simple:latest 36

Slide 37

Slide 37 text

Tips: slim? alpine? ➔ docker pull python:3.8-alpine ➔ docker pull python:3.8-slim ➔ docker pull python:3.8 ➔ docker images REPOSITORY TAG IMAGE ID CREATED SIZE python 3.8-alpine d314e28e240c 2 days ago 43.2MB python 3.8-slim 64458f531a7e 2 weeks ago 122MB python 3.8 67ec76d9f73b 2 weeks ago 909MB Debian baseのfull image (無印) > 軽量版 (slim) > 最軽量版 (alpine) 依存も少なくなっていくのでトレードオフ 最初はslimがおすすめ 37

Slide 38

Slide 38 text

Docker Architecture 38 Docker Client: CUIとしてコマンドで操作 https://docs.docker.com/get-started/overview/#docker-architecture

Slide 39

Slide 39 text

Docker Architecture 39 Docker Host: 仮想環境の本体 DockerのContainerやImage, NetworkやVolumeを管理 Clientのコマンドも待受 https://docs.docker.com/get-started/overview/#docker-architecture

Slide 40

Slide 40 text

Docker Architecture 40 Docker Registry: Docker Imageの共有先 PublicとPrivateを使い分け https://docs.docker.com/get-started/overview/#docker-architecture

Slide 41

Slide 41 text

Docker Architecture 41 今回は 手元のPCがこの部分 https://docs.docker.com/get-started/overview/#docker-architecture

Slide 42

Slide 42 text

Docker Architecture 42 こっちは Remote Server Docker Hubや GCR, ECRが相当 https://docs.docker.com/get-started/overview/#docker-architecture

Slide 43

Slide 43 text

Story 2 • 学習コードの乗ったJupyterLabをDocker上で動かしたい • Docker Compose化のProcess ○ Dockerfileを書くところまで同じ ○ docker-compose.ymlを作成 ○ Docker Composeによる実行、挙動の確認 43

Slide 44

Slide 44 text

Docker Compose Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application’s services. Then, with a single command, you create and start all the services from your configuration. 44 https://docs.docker.com/compose/

Slide 45

Slide 45 text

Docker Compose Docker: Docker Compose: 45 ➔ docker build . -t container4ml ➔ docker run --rm -p 8888:8888 -v ${PWD}:/app container4ml (docker-compose.ymlがある状態で) ➔ docker compose up

Slide 46

Slide 46 text

1. Dockerfileを書く ➔ cd 2-jupyter ➔ ls .dockerignore docker-compose.yml Makefile pyproject.toml .gitignore Dockerfile poetry.lock train.ipynb ➔ vi Dockerfile -------------------------- 46

Slide 47

Slide 47 text

1. Dockerfileを書く 47 ➔ vi Dockerfile -------------------------- FROM python:3.8-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends git build-essential \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY *.bin . EXPOSE 8888 CMD ["jupyter", "lab", "--allow-root", "--ip=0.0.0.0", "--no-browser", "--ServerApp.allow_origin=*", "--ServerApp.token=", "--ServerApp.password="]

Slide 48

Slide 48 text

1-simpleのDockerfileと比較 ➔ vi Dockerfile -------------------------- FROM python:3.8-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends curl \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] 48

Slide 49

Slide 49 text

1. Dockerfileを書く 49 ➔ vi Dockerfile -------------------------- FROM python:3.8-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends git build-essential \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY *.bin . EXPOSE 8888 CMD ["jupyter", "lab", "--allow-root", "--ip=0.0.0.0", "--no-browser", "--ServerApp.allow_origin=*", "--ServerApp.token=", "--ServerApp.password="]

Slide 50

Slide 50 text

2. docker-compose.ymlを書く ➔ vi docker-compose.yml -------------------------- services: jupyter: image: chck/container4ml-jupyter:1.0 build: . ports: - "8888:8888" volumes: - ${PWD}:/app 50

Slide 51

Slide 51 text

2. docker-compose.ymlを書く ➔ vi docker-compose.yml -------------------------- services: jupyter: image: chck/container4ml-jupyter:1.0 build: . ports: - "8888:8888" volumes: - ${PWD}:/app 51 任意のService名 Dockerfileを参照しながら 指定Image名でdocker build portやvolume optionを付与してdocker runされ jupyter containerが起動

Slide 52

Slide 52 text

3. Docker Containerの起動 (via docker compose) ➔ docker compose up Jupyter Server 1.12.1 is running at:... ➔ docker images ➔ docker ps CONTAINER ID IMAGE ... PORTS ... 47ac8f57c169 chck/container4ml-jupyter:1.0 ... 0.0.0.0:8888->8888/tcp … ➔ open http://localhost:8888 # train.ipynbを実行、model.binが作られたことを確認後、 docker compose build (次ページ) ➔ Ctrl-C Gracefully stopping... 52

Slide 53

Slide 53 text

UA Classifier Simpleな学習モデルの例として、 UserAgent StringからPC or Mobileを 分類するタスクを考える 53

Slide 54

Slide 54 text

UserAgent https://towardsdatascience.com/still-parsing-user-agent-strings-for-your-machine-learning-models-use-this-instead-8928c0e7e74f Client (e.g. ブラウザ) がServerアクセス時に付与するOS等の情報をまとめた文字列 Web広告の文脈では擬似的な個人情報として使えたりするため昨今規制が厳しい 54

Slide 55

Slide 55 text

Story 3 • JupyterLab上で学習したModelを乗せたFastAPI製推論AppをDockerに乗せたい • Docker Compose化のProcess ○ 今までと同じ 55

Slide 56

Slide 56 text

1. Dockerfileを書く ➔ cd 3-fastapi ➔ ls .env Dockerfile Makefile test-scale.sh .envrc infra poetry.lock test-switch.sh docker-compose.yml main.py pyproject.toml ➔ vi Dockerfile -------------------------- 56

Slide 57

Slide 57 text

1. Dockerfileを書く 57 ➔ vi Dockerfile -------------------------- FROM chck/container4ml-jupyter:1.0 AS trainer FROM python:3.8-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends git build-essential \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR /models COPY --from=trainer /app/*.bin . WORKDIR ${APP_HOME} COPY main.py . EXPOSE 80 CMD ["gunicorn", "main:app", "--bind=0.0.0.0:80", "--workers=1", "--threads=8", "--timeout=0", "--worker-class=uvicorn.workers.UvicornWorker"]

Slide 58

Slide 58 text

1. Dockerfileを書く 58 ➔ vi Dockerfile -------------------------- FROM chck/container4ml-jupyter:1.0 AS trainer FROM python:3.8-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends git build-essential \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR /models COPY --from=trainer /app/*.bin . WORKDIR ${APP_HOME} COPY main.py . EXPOSE 80 CMD ["gunicorn", "main:app", "--bind=0.0.0.0:80", "--workers=1", "--threads=8", "--timeout=0", "--worker-class=uvicorn.workers.UvicornWorker"]

Slide 59

Slide 59 text

2. docker-compose.ymlを書く ➔ vi docker-compose.yml -------------------------- services: myapp: image: chck/container4ml-fastapi:1.0 build: . ports: - "2222:80" environment: - PYTHONUNBUFFERED=1 - DEBUG=true - REDISHOST=myredis - MODEL_PATH=/models/ua_classifier.bin volumes: - ${PWD}:/app depends_on: - myredis myredis: image: redis:alpine 59

Slide 60

Slide 60 text

2. docker-compose.ymlを書く ➔ vi docker-compose.yml -------------------------- services: myapp: image: chck/container4ml-fastapi:1.0 build: . ports: - "2222:80" environment: - PYTHONUNBUFFERED=1 - DEBUG=true - REDISHOST=myredis - MODEL_PATH=/models/ua_classifier.bin volumes: - ${PWD}:/app depends_on: - myredis myredis: image: redis:alpine 60 任意のService名 任意のService名 Dockerfileを参照しながら 指定Image名でdocker build 指定Imageをdocker pullで用意 portやvolume, env optionを付与してdocker runされ jupyter containerが起動 myredis containerの起動を待ってから myappを起動

Slide 61

Slide 61 text

2. docker-compose.ymlを書く ➔ vi docker-compose.yml -------------------------- services: myapp: image: chck/container4ml-fastapi:1.0 build: . ports: - "2222:80" environment: - PYTHONUNBUFFERED=1 - DEBUG=true - REDISHOST=myredis - MODEL_PATH=/models/ua_classifier.bin volumes: - ${PWD}:/app depends_on: - myredis myredis: image: redis:alpine 61 w/o docker compose ➔ docker network create mynwk ➔ docker run -p 2222:80 -e PYTHONUNBUFFERED=1 -e DEBUG=true -e REDISHOST=myredis -e MODEL_PATH=/models/ua_classifier.bin -v ${PWD}:/app --net mynwk chck/container4ml-fastapi:1.0 ➔ docker run --net mynwk redis:alpine w/ docker compose ➔ docker compose up Container間通信(mynwk) myapp myredis

Slide 62

Slide 62 text

3. Docker Containerの起動 (via docker compose) ➔ docker compose up -d ➔ docker compose logs -f ➔ docker ps CONTAINER ID IMAGE ... PORTS ... NAMES b36811e9f286 chck/container4ml-fastapi:1.0 ... 0.0.0.0:2222->80/tcp 3-fastapi-myapp-1 7304fab5c081 redis:alpine ... 6379/tcp 3-fastapi-myredis-1 ➔ open http://localhost:2222 ➔ open http://localhost:2222/stats ➔ cat main.py ➔ docker compose down ➔ # docker-compose.ymlのenvironmentsにMODEL_NAME=Bを追加して docker compose up --build -d 62

Slide 63

Slide 63 text

時間が余った時用 • docker system prune -a • 1 Image: 1 Appの理由 • Docker化が嬉しい場面 • Dataなどの重たいファイルはvolumeでsyncしておく話 • ModelをStorageに持つかImageに持つか • Docker container のDebug方法 • 変数によってbuildの振る舞いを変えたい時 • Training/Servingは同じImageか分けるか • Securityの話 63

Slide 64

Slide 64 text

質問 & 休憩 hh:mm - hh:mm 64

Slide 65

Slide 65 text

Container Deployment ML AppのServingについて 65

Slide 66

Slide 66 text

Why Container Deployment? https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/ 66

Slide 67

Slide 67 text

https://www.datadoghq.com/docker-adoption/ 67

Slide 68

Slide 68 text

https://www.datadoghq.com/container-report-2020/ 68

Slide 69

Slide 69 text

Container Orchestration • Containerの展開や状態の管理を 担うService • Docker ImageをどうServingする かの部分 • Kubernetesを基とする Managed Serviceが複数展開 https://www.datadoghq.com/container-report-2020/ 69

Slide 70

Slide 70 text

DockerとKubernetesの関係 https://docs.bytemark.co.uk/article/kubernetes-terminology-glossary/ 70

Slide 71

Slide 71 text

https://www.docker.com/blog/top-questions-docker-kubernetes-competitors-or-together/docker-kubernetes-together/ 71

Slide 72

Slide 72 text

Kubernetes Architecture 72

Slide 73

Slide 73 text

Cloud Run Archirecture 73

Slide 74

Slide 74 text

Cloud Run • GCPが提供する Managed Container Service • k8sの細かい所をwrapしてApp のCodingに集中できるようにし たもの • Managing Level ○ GCE < GKE < Cloud Run https://www.idkrtm.com/5-kinds-of-compute-on-google-cloud/ 74

Slide 75

Slide 75 text

Kubernetes vs Cloud Run https://medium.com/google-cloud-jp/cloudrun-buildpacks-110438eccb35 75

Slide 76

Slide 76 text

Story 3-2 • LocalのDocker Composeで動作確認のできたUA Estimatorを Cloud RunにDeployしたい • DeployのProcess ○ Docker ImageをRegistryにUpload ○ Cloud Run Serviceを構築・起動 ○ Cloud Run ServiceにUploadしたImageをApply 76

Slide 77

Slide 77 text

1. RegistryにImageをUpload ➔ docker tag chck/container4ml-fastapi:1.0 gcr.io/$(GCP_PROJECT)/container4ml/fastapi:1.0 ➔ docker tag gcr.io/$(GCP_PROJECT)/container4ml/fastapi:1.0 gcr.io/$(GCP_PROJECT)/container4ml/fastapi:latest ➔ docker push gcr.io/$(GCP_PROJECT)/container4ml/fastapi:1.0 ➔ docker push gcr.io/$(GCP_PROJECT)/container4ml/fastapi:latest ➔ open https://console.cloud.google.com/gcr/images 77

Slide 78

Slide 78 text

2. Cloud Run Serviceを構築・起動 https://console.cloud.google.com/run 78

Slide 79

Slide 79 text

3. UploadしたImageを Cloud RunにApply https://console.cloud.google.com/run/deploy 79

Slide 80

Slide 80 text

Local to Cloud Run 80 docker push

Slide 81

Slide 81 text

Autoscaling https://github.com/rakyll/hey WebAppの負荷試験ができるheyでbenchmarkを取ってみる ➔ brew install hey ➔ sh ./test-scale.sh ➔ sh ./test-scale.sh https://${CLOUDRUN_DOMAIN} Cloud Runのmetricsやheyのbenchmarkを観察 81

Slide 82

Slide 82 text

Blue-green deployment 既存のApp (blue) 稼働中の裏に 新版のApp (green) をTrafficが来ない状態で Deployし、Standbyになったタイミングで Router内部の向き先をgreenに変えることで 無停止Deployを行う仕組み Cloud RunではRevisionによってこれを達成 ➔ sh ./test-switch.sh https://${CLOUDRUN_DOMAIN} ➔ # Deploy New Revision in Cloud Run 82 https://candost.blog/the-blue-green-deployment-strategy/

Slide 83

Slide 83 text

Traffic Splitting Manage trafficで50%(A):50%(B)に Requestを分配してみる この仕組みでSimpleなA/B Testも可 能 83

Slide 84

Slide 84 text

Simple A/A Testing 同じImageでTrafficを分け、Metricsを計測 有意差がないことを確認する 84 Version:1.0 Version:1.0 Routing 50% 50%

Slide 85

Slide 85 text

Simple A/B Testing Image A とImage BでTrafficを分け、Metricsを計測 有意差を検証 85 Version:1.0 Version:1.1 Routing 50% 50%

Slide 86

Slide 86 text

Today’s Goal • Portableな実験環境 ○ -> Docker, Docker-Compose • Scalableな推論App ○ -> Cloud Run • まずは研究コードのDocker化から始めてみる 86

Slide 87

Slide 87 text

Thank you! 質問などあれば 🙌 or slack: @chck 88

Slide 88

Slide 88 text

質問 & 休憩 hh:mm - 17:00 89