CyberAgent AI事業本部MLOps研修Container編 / Container for MLOps

事業部研修のMLOps - Container編の資料です

Transcript

1. ©2023 CyberAgent Inc. Distribution prohibited MLOps Container編 AI Lab 岩崎 祐貴
2. ©2023 CyberAgent Inc. Distribution prohibited Today’s Schedule 10:00 - 12:30 … Container編 13:45 - 14:45 … MLOps基礎編 15:00 - 18:30 … MLOps応用編
3. ©2023 CyberAgent Inc. Distribution prohibited Today’s Schedule 10:00 - 12:30 … Container編 13:45 - 14:45 … MLOps基礎編 15:00 - 18:30 … MLOps応用編
4. ©2023 CyberAgent Inc. Distribution prohibited Today’s Goal
5. ©2023 CyberAgent Inc. Distribution prohibited 今日から使える Portableな実験環境 Scalableな推論App
6. ©2023 CyberAgent Inc. Distribution prohibited Prerequisites 6 1. Docker Desktop 2. Docker Hub ユーザ登録 3. pyenv 4. poetry 5. awscli (認証含む) 6. hey 7. direnv 8. Terraform
7. ©2023 CyberAgent Inc. Distribution prohibited Yuki Iwazaki 2014...Backend Engineer in DSP └2017...ML/DS in Inhouse └2018-...Research Engineer in AI Lab Multimedia (Vision & Language) 7 @chck
8. ©2023 CyberAgent Inc. Distribution prohibited 難解な環境構築 通らないpip install 競合するlibrary version 引き継ぎできない実験コード
9. ©2023 CyberAgent Inc. Distribution prohibited コンテナ技術の世界へ
10. ©2023 CyberAgent Inc. Distribution prohibited Docker for ML MLのためのDockerについて
11. ©2023 CyberAgent Inc. Distribution prohibited Docker Docker is an open platform for developing, shipping, and running applications. Docker enables you to separate your applications from your infrastructure so you can deliver software quickly. With Docker, you can manage your infrastructure in the same ways you manage your applications. By taking advantage of Docker’s methodologies for shipping, testing, and deploying code quickly, you can significantly reduce the delay between writing code and running it in production. https://docs.docker.com/get-started/overview/ 11
12. ©2023 CyberAgent Inc. Distribution prohibited Why Docker? コードを動かすための依存をDockerさえ入っていれば動く状態に • PipやConda、Virtualenvではだめ? • 移行や共有時どうする? • OSへの直Installや別途Downloadが必要な依存がある場合は? 実験コードを動かすための長く複雑な手順をDockerで隠蔽できる • READMEにはDocker commandを書くだけ Kubernetesを始めとする強力なContainer ServiceにML AppをDeployできる • 2023年現在、一定規模以上のAppの運用を考えると業界標準に 12
13. ©2023 CyberAgent Inc. Distribution prohibited https://www.datadoghq.com/docker-adoption/ 13
14. ©2023 CyberAgent Inc. Distribution prohibited Live Demo ➔ git clone https://github.com/chck/container4ml-aws.git ➔ cd container4ml-aws ➔ ls -a .dockerignore .envrc .github 2-jupyter infra Makefile .env.example .git 1-simple 3-fastapi LICENSE README.md ➔ cp .env.example .env 14
15. ©2023 CyberAgent Inc. Distribution prohibited 初期設定 (.env) ➔ $EDITOR .env -------------------------- AWS_BUCKET= AWS_REGION=ap-northeast-1 AWS_ACCOUNT_ID= USER= ➔ direnv allow . 用意したAWSアカウントID USERは任意のIDを入れる .envが環境変数に反映される 用意したS3 BUCKET
16. ©2023 CyberAgent Inc. Distribution prohibited 初期設定 (Terraform) ➔ make bucket ➔ cd infra ➔ terraform init ➔ terraform plan ➔ terraform apply ➔ cd .. 用意したAWSアカウントにTerraform用のbucketを作成 作られるResourceの確認 用意したAWSアカウントに対しTerraformを実行
17. ©2023 CyberAgent Inc. Distribution prohibited Story 1 既にあるPython ScriptをDocker化したい Docker化のProcess • Python Scriptを動かすための依存を含んだDocker Imageを作成 • Docker Containerとして実行し、挙動を確認 • DockerfileのCommitやDocker RegistryにImageをUploadして完了 17
18. ©2023 CyberAgent Inc. Distribution prohibited Docker Image An image is a read-only template with instructions for creating a Docker container. Often, an image is based on another image, with some additional customization. https://docs.docker.com/get-started/overview/ 18
19. ©2023 CyberAgent Inc. Distribution prohibited Docker Container A container is a runnable instance of an image. You can create, start, stop, move, or delete a container using the Docker CLI. It is a standard unit of software that packages up code and all its dependencies. https://www.docker.com/resources/what-container 19
20. ©2023 CyberAgent Inc. Distribution prohibited Docker Container A container is a runnable instance of an image. You can create, start, stop, move, or delete a container using the Docker CLI. It is a standard unit of software that packages up code and all its dependencies. https://www.docker.com/resources/what-container MacBook, ThinkPad, EC2, GCE macOS, Windows, Ubuntu JupyterLab, Flask, MySQL, Original App 20
21. ©2023 CyberAgent Inc. Distribution prohibited Story 1: 再掲 既にあるPython ScriptをDocker化したい Docker化のProcess • Python Scriptを動かすための依存を含んだDocker Imageを作成 • Docker Containerとして実行し、挙動を確認 • DockerfileのCommitやDocker RegistryにImageをUploadして完了 21
22. ©2023 CyberAgent Inc. Distribution prohibited 1. BaseとなるImageを決める “$keyword docker hub”で検索 22
23. ©2023 CyberAgent Inc. Distribution prohibited 2. Dockerfileを書く ➔ cd 1-simple ➔ ls .dockerignore Dockerfile Makefile pyproject.toml compose.yml main.py poetry.lock ➔ $EDITOR Dockerfile -------------------------- 23
24. ©2023 CyberAgent Inc. Distribution prohibited 2. Dockerfileを書く ➔ $EDITOR Dockerfile -------------------------- FROM python:3.9-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends build-essential \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] 24
25. ©2023 CyberAgent Inc. Distribution prohibited 2. Dockerfileを書く ➔ $EDITOR Dockerfile -------------------------- FROM python:3.9-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends build-essential \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] 25
26. ©2023 CyberAgent Inc. Distribution prohibited 2. Dockerfileを書く ➔ $EDITOR Dockerfile -------------------------- FROM python:3.9-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends build-essential \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] Base Imageを指定 26
27. ©2023 CyberAgent Inc. Distribution prohibited 2. Dockerfileを書く ➔ $EDITOR Dockerfile -------------------------- FROM python:3.9-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends build-essential \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] 環境変数の定義 27
28. ©2023 CyberAgent Inc. Distribution prohibited 2. Dockerfileを書く ➔ $EDITOR Dockerfile -------------------------- FROM python:3.9-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends build-essential \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] 環境構築の実行コマンドを記述 28
29. ©2023 CyberAgent Inc. Distribution prohibited 2. Dockerfileを書く ➔ $EDITOR Dockerfile -------------------------- FROM python:3.9-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends build-essential \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] Image内でのDirectory指定 29
30. ©2023 CyberAgent Inc. Distribution prohibited 2. Dockerfileを書く ➔ $EDITOR Dockerfile -------------------------- FROM python:3.9-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends build-essential \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] local://${pwd}/main.py docker://app/main.py 30
31. ©2023 CyberAgent Inc. Distribution prohibited 2. Dockerfileを書く ➔ $EDITOR Dockerfile -------------------------- FROM python:3.9-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends build-essential \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] 起動時に実行されるコマンド 31
32. ©2023 CyberAgent Inc. Distribution prohibited 3. Docker Imageの作成 ➔ docker build . -t container4ml-simple:1.0 32
33. ©2023 CyberAgent Inc. Distribution prohibited 3. Docker Imageの作成 ➔ docker build . -t container4ml-simple:1.0 コンテキストの指定. Dockerに見てほしいpath. COPY句でどこを起点にするか . 基本的にDockerfileのある場所でOK イメージ名. お作法的にはowner/image_name イメージタグ. イメージは更新されゆくのでいわゆる versioning 33
34. ©2023 CyberAgent Inc. Distribution prohibited 3. Docker Imageの作成 ➔ docker build . -t container4ml-simple:1.0 ➔ docker images REPOSITORY TAG IMAGE ID CREATED SIZE container4ml-simple 1.0 $(IMAGE_ID) 7 seconds ago 628MB Image sizeはなるべく小さくするべきだが、 慣れるまではゴリゴリに削らなくて OK 34
35. ©2023 CyberAgent Inc. Distribution prohibited 4. Docker Containerの起動 ➔ watch docker ps -a ➔ docker run container4ml-simple:1.0 scikit-learn: 1.2.2 ➔ docker run --rm container4ml-simple:1.0 ➔ cat main.py import sklearn print(f"scikit-learn: {sklearn.__version__}") 35 ←watchとは別窓で実行
36. ©2023 CyberAgent Inc. Distribution prohibited 4. Docker Containerの実行上書き ➔ tail -1 Dockerfile ➔ docker run --rm container4ml-simple:1.0 ls -lh total 4.0K -rw-r--r-- 1 root root 62 May 8 23:52 main.py 実行コマンドは上書きできる 同Imageで挙動だけ変えたい時等で活用 例えばTraining/ServingのImageを共通化するとか 36
37. ©2023 CyberAgent Inc. Distribution prohibited 5. Docker ImageのUpload/Download ➔ open https://hub.docker.com/ ➔ docker login ➔ docker tag container4ml-simple:1.0 $(HUB_ID)/container4ml-simple:1.0 ➔ docker images REPOSITORY TAG IMAGE ID CREATED SIZE chck/container4ml-simple 1.0 c1575db34e5d 2 hours ago 628MB container4ml-simple 1.0 c1575db34e5d 2 hours ago 628MB ➔ docker push $(HUB_ID)/container4ml-simple:1.0 ➔ open https://hub.docker.com/repository/docker/$(HUB_ID)/container4ml-simple ➔ docker rmi $(HUB_ID)/container4ml-simple:1.0 ➔ docker pull $(HUB_ID)/container4ml-simple:1.0 ➔ docker images 37
38. ©2023 CyberAgent Inc. Distribution prohibited 5. Docker ImageのUpload/Download ➔ open https://hub.docker.com/ ➔ docker login ➔ docker tag container4ml-simple:1.0 $(HUB_ID)/container4ml-simple:1.0 ➔ docker images REPOSITORY TAG IMAGE ID CREATED SIZE chck/container4ml-simple 1.0 c1575db34e5d 2 hours ago 628MB container4ml-simple 1.0 c1575db34e5d 2 hours ago 628MB 38
39. ©2023 CyberAgent Inc. Distribution prohibited 5. Docker ImageのUpload/Download ➔ open https://hub.docker.com/ ➔ docker login ➔ docker tag container4ml-simple:1.0 $(HUB_ID)/container4ml-simple:1.0 ➔ docker images REPOSITORY TAG IMAGE ID CREATED SIZE chck/container4ml-simple 1.0 c1575db34e5d 2 hours ago 628MB container4ml-simple 1.0 c1575db34e5d 2 hours ago 628MB 39 TagはAliasとして働くので同じImage IDを持ち, Diskも重複消費しない
40. ©2023 CyberAgent Inc. Distribution prohibited 5. Docker ImageのUpload/Download ➔ docker push $(HUB_ID)/container4ml-simple:1.0 ➔ open hub.docker.com/repository/docker/$(HUB_ID)/container4ml-simple ➔ docker rmi $(HUB_ID)/container4ml-simple:1.0 ➔ docker pull $(HUB_ID)/container4ml-simple:1.0 ➔ docker images 指定registryにimageをupload 40
41. ©2023 CyberAgent Inc. Distribution prohibited 5. Docker ImageのUpload/Download ➔ docker push $(HUB_ID)/container4ml-simple:1.0 ➔ open hub.docker.com/repository/docker/$(HUB_ID)/container4ml-simple ➔ docker rmi $(HUB_ID)/container4ml-simple:1.0 ➔ docker pull $(HUB_ID)/container4ml-simple:1.0 ➔ docker images 41 指定したDocker Imageを削除. Registryに上げていれば安心 指定RegistryからImageをDownload
42. ©2023 CyberAgent Inc. Distribution prohibited Tips: Image Tagは複数付与できる ➔ docker tag container4ml-simple:1.0 $(HUB_ID)/container4ml-simple:1.0 ➔ docker tag container4ml-simple:1.0 $(HUB_ID)/container4ml-simple:latest ➔ docker push $(HUB_ID)/container4ml-simple:1.0 ➔ docker push $(HUB_ID)/container4ml-simple:latest 42
43. ©2023 CyberAgent Inc. Distribution prohibited Tips: slim? alpine? ➔ docker pull python:3.9 ➔ docker pull python:3.9-slim ➔ docker pull python:3.9-alpine ➔ docker images REPOSITORY TAG IMAGE ID CREATED SIZE python 3.9 67ec76d9f73b 2 weeks ago 857MB python 3.9-slim 64458f531a7e 2 weeks ago 118MB python 3.9-alpine d314e28e240c 2 days ago 57.8MB 依存とサイズのトレードオフ 最初はslimがおすすめ 43 最軽量版 (alpine) 軽量版 (slim) Debian baseのfull image (無印)
44. ©2023 CyberAgent Inc. Distribution prohibited Docker Architecture 44 Docker Client: CUIとしてコマンドで操作 https://docs.docker.com/get-started/overview/#docker-architecture
45. ©2023 CyberAgent Inc. Distribution prohibited Docker Architecture 45 Docker Host: 仮想環境の本体 DockerのContainerやImage, NetworkやVolumeを管理 Clientのコマンドを待受 https://docs.docker.com/get-started/overview/#docker-architecture
46. ©2023 CyberAgent Inc. Distribution prohibited Docker Architecture 46 Docker Registry: Docker Imageの共有先 PublicとPrivateを使い分け https://docs.docker.com/get-started/overview/#docker-architecture
47. ©2023 CyberAgent Inc. Distribution prohibited Docker Architecture 47 今回は 手元のPCがこの部分 https://docs.docker.com/get-started/overview/#docker-architecture
48. ©2023 CyberAgent Inc. Distribution prohibited Docker Architecture 48 こっちは Remote Server Docker Hubや GCR, ECRが相当 https://docs.docker.com/get-started/overview/#docker-architecture
49. ©2023 CyberAgent Inc. Distribution prohibited Story 2 学習コードの乗ったJupyterLabをDocker上で動かしたい Docker Compose化のProcess • Dockerfileを書くところまで同じ • compose.ymlを作成 • Docker Composeによる実行、挙動の確認 49
50. ©2023 CyberAgent Inc. Distribution prohibited Docker Compose Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application’s services. Then, with a single command, you create and start all the services from your configuration. 50 https://docs.docker.com/compose/
51. ©2023 CyberAgent Inc. Distribution prohibited Docker vs Docker Compose Docker: Docker Compose: 51 ➔ docker build . -t container4ml ➔ docker run --rm -p 8888:8888 -v ${PWD}:/app container4ml (compose.ymlがある状態で) ➔ docker compose up ↓概念の説明なので実行しなくて OK
52. ©2023 CyberAgent Inc. Distribution prohibited 1. Dockerfileを書く ➔ cd 2-jupyter ➔ ls .dockerignore compose.yml Makefile pyproject.toml .gitignore Dockerfile poetry.lock train.ipynb ➔ $EDITOR Dockerfile -------------------------- 52
53. ©2023 CyberAgent Inc. Distribution prohibited 1. Dockerfileを書く 53 ➔ $EDITOR Dockerfile -------------------------- FROM python:3.9-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends build-essential \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY *.bin . EXPOSE 8888 CMD ["jupyter", "lab", "--allow-root", "--ip=0.0.0.0", "--no-browser", "--ServerApp.allow_origin=*", "--ServerApp.token=", "--ServerApp.password="]
54. ©2023 CyberAgent Inc. Distribution prohibited 1-simpleのDockerfileと比較 ➔ $EDITOR Dockerfile -------------------------- FROM python:3.9-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends build-essential \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] 54
55. ©2023 CyberAgent Inc. Distribution prohibited 1. Dockerfileを書く 55 ➔ $EDITOR Dockerfile -------------------------- FROM python:3.9-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends build-essential \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY *.bin . EXPOSE 8888 CMD ["jupyter", "lab", "--allow-root", "--ip=0.0.0.0", "--no-browser", "--ServerApp.allow_origin=*", "--ServerApp.token=", "--ServerApp.password="]
56. ©2023 CyberAgent Inc. Distribution prohibited 2. compose.ymlを書く ➔ $EDITOR compose.yml -------------------------- services: jupyter: image: …/container4ml-jupyter:${USER} build: . ports: - "8888:8888" volumes: - ${PWD}:/app 56
57. ©2023 CyberAgent Inc. Distribution prohibited 2. compose.ymlを書く ➔ $EDITOR compose.yml -------------------------- services: jupyter: image: …/container4ml-jupyter:${USER} build: . ports: - "8888:8888" volumes: - ${PWD}:/app 57 任意のService名 Dockerfileを参照しながら 指定Image名でdocker build portやvolume optionを付与してdocker runされ jupyter containerが起動
58. ©2023 CyberAgent Inc. Distribution prohibited 3. Docker Containerの起動 (via docker compose) ➔ docker compose up Jupyter Server 2.5.0 is running at:... ➔ docker images ➔ docker ps 58 ←docker compose upとは別窓で実行 ←docker buildとdocker runが両方行われる
59. ©2023 CyberAgent Inc. Distribution prohibited 3. Docker Containerの起動 (via docker compose) ➔ open http://127.0.0.1:8888 ➔ train.ipynbを開いてRun -> Run All Cells └ ua_classifier.binが作られたことを確認 59
60. ©2023 CyberAgent Inc. Distribution prohibited User Agent (UA) https://towardsdatascience.com/still-parsing-user-agent-strings-for-your-machine-learning-models-use-this-instead-8928c0e7e74f Client (e.g. ブラウザ) がServerアクセス時に付与するOS等の情報をまとめた文字列 Web広告の文脈では擬似的な個人情報として使えたりするため昨今規制が厳しい 60
61. ©2023 CyberAgent Inc. Distribution prohibited UA Classifier Simpleな学習モデルの例として、 UserAgent StringからPC or Mobileを分類するタスクを考える 61
62. ©2023 CyberAgent Inc. Distribution prohibited Tips: ModelをCloud Storageに保存しておく例 ➔ ts = !TZ=Asia/Tokyo date +"%Y%m%d%H%M" ➔ aws s3 cp ua_classifier.bin s3://${AWS_BUCKET}/models/ua_classifier/{ts[0]}.bin ➔ aws s3 cp ua_classifier.bin s3://${AWS_BUCKET}/models/ua_classifier/latest.bin ➔ aws s3 ls --human-readable s3://${AWS_BUCKET}/models/ua_classifier/ 2023-05-02 08:19:44 281.9 KiB 202305020819.bin 2023-05-02 08:19:50 281.9 KiB latest.bin ↓Tipsなので実行しなくてOK
63. ©2023 CyberAgent Inc. Distribution prohibited 3. Docker Containerの起動 (via docker compose) ➔ Ctrl-C └ Gracefully stopping... ➔ docker compose build ➔ docker run …/container4ml-jupyter:${USER} ls └ ua_classifier.bin ➔ aws ecr get-login-password --region ${AWS_REGION} | docker login -u AWS --password-stdin ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com ➔ docker push …/container4ml-jupyter:${USER} └ Amazon ECRのUIを確認 63
64. ©2023 CyberAgent Inc. Distribution prohibited Story 3 JupyterLab上で学習したModelを乗せたFastAPI製推論Appを Dockerに乗せたい Docker Compose化のProcess • 今までと同じ 64
65. ©2023 CyberAgent Inc. Distribution prohibited 1. Dockerfileを書く ➔ cd 3-fastapi ➔ ls bin Dockerfile main.py pyproject.toml compose.yml k8s poetry.lock setup.cfg ➔ $EDITOR Dockerfile -------------------------- 65
66. ©2023 CyberAgent Inc. Distribution prohibited 1. Dockerfileを書く 66 ➔ $EDITOR Dockerfile -------------------------- ARG AWS_ACCOUNT_ID ARG AWS_REGION=${AWS_REGION:-"ap-northeast-1"} ARG TRAINER_VERSION=${TRAINER_VERSION:-"latest"} FROM ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/container4ml-jupyter:${TRAINER_VERSION} AS trainer FROM python:3.9-slim ...(中略)... WORKDIR /models COPY --from=trainer /app/ua_classifier.bin . WORKDIR ${APP_HOME} COPY main.py . EXPOSE 80 CMD ["gunicorn", "main:app", "--bind=0.0.0.0:80", "--workers=1", "--threads=8", "--timeout=0", "--worker-class=uvicorn.workers.UvicornWorker"]
67. ©2023 CyberAgent Inc. Distribution prohibited 1. Dockerfileを書く 67 ➔ $EDITOR Dockerfile -------------------------- ARG AWS_ACCOUNT_ID ARG AWS_REGION=${AWS_REGION:-"ap-northeast-1"} ARG TRAINER_VERSION=${TRAINER_VERSION:-"latest"} FROM ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/container4ml-jupyter:${TRAINER_VERSION} AS trainer FROM python:3.9-slim ...(中略)... WORKDIR /models COPY --from=trainer /app/ua_classifier.bin . WORKDIR ${APP_HOME} COPY main.py . EXPOSE 80 CMD ["gunicorn", "main:app", "--bind=0.0.0.0:80", "--workers=1", "--threads=8", "--timeout=0", "--worker-class=uvicorn.workers.UvicornWorker"]
68. ©2023 CyberAgent Inc. Distribution prohibited 2. compose.ymlを書く ➔ $EDITOR compose.yml -------------------------- services: myapp: image: ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/container4ml-fastapi:latest build: args: - AWS_ACCOUNT_ID=${AWS_ACCOUNT_ID} ports: - "3333:80" environment: … - REDISHOST=myredis - MODEL_NAME=A volumes: - ${PWD}:/app depends_on: - myredis myredis: image: redis:alpine 68
69. ©2023 CyberAgent Inc. Distribution prohibited 2. compose.ymlを書く ➔ $EDITOR compose.yml -------------------------- services: myapp: image: …/container4ml-fastapi:latest build: args: - AWS_ACCOUNT_ID=${AWS_ACCOUNT_ID} ports: - "3333:80" environment: … - REDISHOST=myredis - MODEL_NAME=A volumes: - ${PWD}:/app depends_on: - myredis myredis: image: redis:alpine 69 任意のService名 Dockerfileを参照しながら 指定Image名でdocker build 指定Imageをdocker pullで用意 docker runのoption相当 myredis containerの起動を待ってから myappを起動
70. ©2023 CyberAgent Inc. Distribution prohibited 2. compose.ymlを書く ➔ $EDITOR compose.yml -------------------------- services: myapp: image: …/container4ml-fastapi:latest build: args: - AWS_ACCOUNT_ID=${AWS_ACCOUNT_ID} ports: - "3333:80" environment: … - REDISHOST=myredis - MODEL_NAME=A volumes: - ${PWD}:/app depends_on: - myredis myredis: image: redis:alpine 70 w/o docker compose ➔ docker network create mynwk ➔ docker run -p 3333:80 -e PYTHONUNBUFFERED=1 -e DEBUG=true -e REDISHOST=myredis -e MODEL_NAME=A -v ${PWD}:/app --net mynwk …/container4ml-fastapi:latest ➔ docker run --net mynwk redis:alpine w/ docker compose ➔ docker compose up Container間通信(mynwk) myapp myredis
71. ©2023 CyberAgent Inc. Distribution prohibited (option) main.pyの挙動確認 ➔ cat main.py
72. ©2023 CyberAgent Inc. Distribution prohibited 3. Docker Containerの起動 (via docker compose) ➔ aws ecr get-login-password --region ${AWS_REGION} | docker login -u AWS --password-stdin ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com ➔ docker compose up -d ➔ docker ps CONTAINER ID IMAGE ... PORTS ... NAMES ... .../container4ml-fastapi:1.0 ... 0.0.0.0:3333->80/tcp 3-fastapi-myapp-1 ... redis:alpine ... 6379/tcp 3-fastapi-myredis-1 ➔ open http://localhost:3333 ➔ open http://localhost:3333/stats ➔ docker compose down ➔ compose.ymlのMODEL_NAMEをAからBに変更後、 docker compose up --build -d 72
73. ©2023 CyberAgent Inc. Distribution prohibited 時間が余った時用 - docker system prune -a - M1/M2 MacのPython開発事情 - 1 Image : 1 Appの理由 - Docker化が嬉しい場面 - Dataなどの重たいファイルはvolumeでsyncしておく話 - ModelをStorageに持つかImageに持つか - Docker container のDebug方法 - 変数によってbuildの振る舞いを変えたい時 - Training/Servingは同じImageか分けるか - Securityの話 73
74. ©2023 CyberAgent Inc. Distribution prohibited Tips: Docker for Macの割当 Resourceを上げる Preferences -> Resources 74
75. ©2023 CyberAgent Inc. Distribution prohibited 質問 & 休憩 hh:mm - hh:mm
76. ©2023 CyberAgent Inc. Distribution prohibited Container Deployment ML AppのServingについて
77. ©2023 CyberAgent Inc. Distribution prohibited Why Container Deployment? https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/ 77
78. ©2023 CyberAgent Inc. Distribution prohibited https://www.datadoghq.com/docker-adoption/ 78
79. ©2023 CyberAgent Inc. Distribution prohibited https://www.datadoghq.com/container-report-2020/ 79
80. ©2023 CyberAgent Inc. Distribution prohibited Container Orchestration Containerの展開や状態の 管理を担うService Docker ImageをどうServing するかの部分 Kubernetesを基とする Managed Serviceが複数展開 https://www.datadoghq.com/container-report-2020/ 80
81. ©2023 CyberAgent Inc. Distribution prohibited DockerとKubernetesの関係 https://docs.bytemark.co.uk/article/kubernetes-terminology-glossary/ 81
82. ©2023 CyberAgent Inc. Distribution prohibited https://www.docker.com/blog/top-questions-docker-kubernetes-competitors-or-together/docker-kubernetes-together/ 82
83. ©2023 CyberAgent Inc. Distribution prohibited Docker Compose vs Kubernetes ➔ docker compose up -d ➔ docker compose ps ➔ docker compose down ➔ cd 3-fastapi/k8s ➔ kind create cluster --config=kind-config.yml ➔ kubectl cluster-info --context kind-kind ➔ kubectl apply -f deployment.yml ➔ kubectl apply -f service.yml ➔ kubectl get po,svc,deploy ➔ kubectl delete -f service.yml ➔ kubectl delete -f deployment.yml ➔ kind delete cluster
84. ©2023 CyberAgent Inc. Distribution prohibited Kubernetes Architecture 84
85. ©2023 CyberAgent Inc. Distribution prohibited AWS App Runner Archirecture 85 Managed
86. ©2023 CyberAgent Inc. Distribution prohibited AWS App Runner AWSが提供する Managed Container Service k8sの細かい所をwrapして AppのCodingに集中できる ようにしたもの 86
87. ©2023 CyberAgent Inc. Distribution prohibited Story 3-2 LocalのDocker Composeで動作確認のできたUA Classifierを App RunnerにDeployしたい DeployのProcess 1. Docker ImageをRegistryにUpload 2. UploadしたImageを指定してApp Runnerを構築・起動 87
88. ©2023 CyberAgent Inc. Distribution prohibited Local to AWS App Runner 88 docker push
89. ©2023 CyberAgent Inc. Distribution prohibited 1. RegistryにImageをUpload ➔ docker tag ${HUB_ID}/container4ml-fastapi:1.0 ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/container4ml-fastapi:1.0 ➔ docker tag ${HUB_ID}/container4ml-fastapi:latest ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/container4ml-fastapi:latest ➔ aws ecr get-login-password --region ap-northeast-1 | docker login --username AWS --password-stdin ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com ➔ docker push ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/container4ml-fastapi:1.0 ➔ docker push ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/container4ml-fastapi:latest ➔ open https://ap-northeast-1.console.aws.amazon.com/ecr/repositories 89
90. ©2023 CyberAgent Inc. Distribution prohibited 2. App Runnerを立ててみる https://ap-northeast-1.console.aws.amazon.com/apprunner/home └ 右側のCreate service -> 作成画面へ 90
91. ©2023 CyberAgent Inc. Distribution prohibited
92. ©2023 CyberAgent Inc. Distribution prohibited 3-fastapi/DockerfileのEXPOSEを見ながら設定 REDISHOSTはterraformで立てたものを指定 MODEL_NAMEはAから USERは任意のIDに .envにも指定した$USERを-suffixとして追加
93. ©2023 CyberAgent Inc. Distribution prohibited
94. ©2023 CyberAgent Inc. Distribution prohibited
95. ©2023 CyberAgent Inc. Distribution prohibited 作成されるまでしばし待つ 画面右下の　　　　　　　　から確認画面へ 確認画面右下の　　　　　　　　　　　　 をクリック
96. ©2023 CyberAgent Inc. Distribution prohibited Autoscaling https://github.com/rakyll/hey WebAppの負荷試験ができるheyでbenchmarkを取ってみる ➔ sh ./test-scale.sh ➔ sh ./test-scale.sh https://${APPRUNNER_DOMAIN} App Runnerのmetricsやheyのbenchmarkを観察 96
97. ©2023 CyberAgent Inc. Distribution prohibited GitHub ActionsによるDeployの自動化 いわゆるCI/CDのお話 基本的にGit repoにyamlを置くだけ このハンズオンのrepositoryでも設定済
98. ©2023 CyberAgent Inc. Distribution prohibited GitHub ActionsによるDeployの自動化 ➔ cat .github/workflows/deploy-container4ml.yml -------------------------- ... on: push: branches: [main] ... jobs: infra: runs-on: ubuntu-latest steps: - run: echo "container4ml deploy job triggered." changes: runs-on: ubuntu-latest needs: infra ... トリガ条件 jobs ∋ steps needsで実行順を制御できる 実際に実行されるのは stepsのコマンド
99. ©2023 CyberAgent Inc. Distribution prohibited ➔ cat .github/workflows/deploy-container4ml.yml -------------------------- ... on: push: branches: [main] ... jobs: infra: runs-on: ubuntu-latest steps: - run: echo "container4ml deploy job triggered." changes: runs-on: ubuntu-latest needs: infra ... GitHub ActionsによるDeployの自動化 GitHub repo -> Actionsから実行履歴が見れる
100. ©2023 CyberAgent Inc. Distribution prohibited GitHub ActionsによるDeployの自動化 トリガ条件: main branchにpushされたら 実行内容: paths-filterで更に分岐 └ trainerルート... '2-jupyter/**' の差分 └ S3からua_classifier.binをdownloadし、それを含めて 　　　　　 container4ml-jupyterをdocker build、ECRにpush └ predictorルート... '3-fastapi/**' の差分 └ container4ml-fastapiをdocker build、ECRにpush
101. ©2023 CyberAgent Inc. Distribution prohibited Blue-green deployment 既存のApp (Blue) 稼働中の裏に 新版のApp (Green) をTrafficが来ない 状態でDeployし、Standbyになった タイミングでRouter内部の向き先を Greenに変えることで無停止Deployを 行う仕組み App Runnerもこれに対応 ➔ sh ./test-switch.sh https://${APPRUNNER_DOMAIN} ➔ # Deploy New Version in App Runner 101 https://candost.blog/the-blue-green-deployment-strategy/
102. ©2023 CyberAgent Inc. Distribution prohibited Traffic Splitting BlueとGreenで m%(A):n%(B)にRequestを分配 この仕組みで 新機能のCanary Releaseや A/B Testも可能 102 https://dev.classmethod.jp/articles/alb-blue-green-deployment/
103. ©2023 CyberAgent Inc. Distribution prohibited Simple A/A Testing 同じVersionでTrafficを分け、Metricsを計測 有意差がないことを確認 103 Version:1.0 Version:1.0 Routing 50% 50%
104. ©2023 CyberAgent Inc. Distribution prohibited Simple A/B Testing Version A と BでTrafficを分け、Metricsを計測 有意差を確認 104 Version:1.0 Version:1.1 Routing 50% 50%
105. ©2023 CyberAgent Inc. Distribution prohibited A/B Testingの実装方法 Load balancerでSplitting Container A, Bの前段にLoad balancerを立ててm:nに分散 - Pros 👍 ??? - Cons 👎 ??? 105 Application内でSplitting Python Code内でrandomやuuidのhash値、 DB参照などのルールに基づいてif else等で分割 - Pros 👍 ??? - Cons 👎 ??? ざっくり2種類の方法がある
106. ©2023 CyberAgent Inc. Distribution prohibited A/B Testingの実装方法 Load balancerでSplitting Container A, Bの前段にLoad balancerを立ててm:nに分散 - Pros 👍 ModelとContainerが1:1で紐づくので Appの実装がシンプル - Cons 👎 Cache戦略によっては同じuuidでも Requestの度にA/Bを横断してしまう 106 Application内でSplitting Python Code内でrandomやuuidのhash値、 DB参照などのルールに基づいてif else等で分割 - Pros 👍 QueryのUUIDに応じて分割先のルール設計が簡単 - Cons 👎 App内で複数VersionのModelをLoadしておくResourceが必要 ざっくり2種類の方法がある
107. ©2023 CyberAgent Inc. Distribution prohibited Today’s Goal • Portableな実験環境 ◦ Docker, Docker-Compose • Scalableな推論App ◦ App Runner まずは実験コードのDocker化から始めてみる 107
108. ©2023 CyberAgent Inc. Distribution prohibited Thank you! 質問などあれば 🙌