CyberAgent AI事業本部MLOps研修Container編 / Container for MLOps

by chck

Slide 1

Slide 1 text

Slide 2

Slide 2 text

Slide 3

Slide 3 text

Slide 4

Slide 4 text

Slide 5

Slide 5 text

Slide 6

Slide 6 text

Slide 7

Slide 7 text

Slide 8

Slide 8 text

Slide 9

Slide 9 text

Slide 10

Slide 10 text

Slide 11

Slide 11 text

©2023 CyberAgent Inc. Distribution prohibited Docker Docker is an open platform for developing, shipping, and running applications. Docker enables you to separate your applications from your infrastructure so you can deliver software quickly. With Docker, you can manage your infrastructure in the same ways you manage your applications. By taking advantage of Docker’s methodologies for shipping, testing, and deploying code quickly, you can signiﬁcantly reduce the delay between writing code and running it in production. https://docs.docker.com/get-started/overview/ 11

Slide 12

Slide 12 text

©2023 CyberAgent Inc. Distribution prohibited Why Docker? コードを動かすための依存をDockerさえ入っていれば動く状態に ● PipやConda、Virtualenvではだめ? ● 移行や共有時どうする? ● OSへの直Installや別途Downloadが必要な依存がある場合は? 実験コードを動かすための長く複雑な手順をDockerで隠蔽できる ● READMEにはDocker commandを書くだけ Kubernetesを始めとする強力なContainer ServiceにML AppをDeployできる ● 2023年現在、一定規模以上のAppの運用を考えると業界標準に 12

Slide 13

Slide 13 text

Slide 14

Slide 14 text

©2023 CyberAgent Inc. Distribution prohibited Live Demo ➔ git clone https://github.com/chck/container4ml-aws.git ➔ cd container4ml-aws ➔ ls -a .dockerignore .envrc .github 2-jupyter infra Makefile .env.example .git 1-simple 3-fastapi LICENSE README.md ➔ cp .env.example .env 14

Slide 15

Slide 15 text

©2023 CyberAgent Inc. Distribution prohibited 初期設定 (.env) ➔ $EDITOR .env -------------------------- AWS_BUCKET= AWS_REGION=ap-northeast-1 AWS_ACCOUNT_ID= USER= ➔ direnv allow . 用意したAWSアカウントID USERは任意のIDを入れる .envが環境変数に反映される用意したS3 BUCKET

Slide 16

Slide 16 text

©2023 CyberAgent Inc. Distribution prohibited 初期設定 (Terraform) ➔ make bucket ➔ cd infra ➔ terraform init ➔ terraform plan ➔ terraform apply ➔ cd .. 用意したAWSアカウントにTerraform用のbucketを作成作られるResourceの確認用意したAWSアカウントに対しTerraformを実行

Slide 17

Slide 17 text

©2023 CyberAgent Inc. Distribution prohibited Story 1 既にあるPython ScriptをDocker化したい Docker化のProcess ● Python Scriptを動かすための依存を含んだDocker Imageを作成 ● Docker Containerとして実行し、挙動を確認 ● DockerﬁleのCommitやDocker RegistryにImageをUploadして完了 17

Slide 18

Slide 18 text

©2023 CyberAgent Inc. Distribution prohibited Docker Image An image is a read-only template with instructions for creating a Docker container. Often, an image is based on another image, with some additional customization. https://docs.docker.com/get-started/overview/ 18

Slide 19

Slide 19 text

©2023 CyberAgent Inc. Distribution prohibited Docker Container A container is a runnable instance of an image. You can create, start, stop, move, or delete a container using the Docker CLI. It is a standard unit of software that packages up code and all its dependencies. https://www.docker.com/resources/what-container 19

Slide 20

Slide 20 text

Slide 21

Slide 21 text

©2023 CyberAgent Inc. Distribution prohibited Story 1: 再掲既にあるPython ScriptをDocker化したい Docker化のProcess ● Python Scriptを動かすための依存を含んだDocker Imageを作成 ● Docker Containerとして実行し、挙動を確認 ● DockerﬁleのCommitやDocker RegistryにImageをUploadして完了 21

Slide 22

Slide 22 text

Slide 23

Slide 23 text

©2023 CyberAgent Inc. Distribution prohibited 2. Dockerﬁleを書く ➔ cd 1-simple ➔ ls .dockerignore Dockerfile Makefile pyproject.toml compose.yml main.py poetry.lock ➔ $EDITOR Dockerfile -------------------------- 23

Slide 24

Slide 24 text

©2023 CyberAgent Inc. Distribution prohibited 2. Dockerﬁleを書く ➔ $EDITOR Dockerfile -------------------------- FROM python:3.9-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends build-essential \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] 24

Slide 25

Slide 25 text

Slide 26

Slide 26 text

Slide 27

Slide 27 text

Slide 28

Slide 28 text

Slide 29

Slide 29 text

Slide 30

Slide 30 text

Slide 31

Slide 31 text

Slide 32

Slide 32 text

Slide 33

Slide 33 text

©2023 CyberAgent Inc. Distribution prohibited 3. Docker Imageの作成 ➔ docker build . -t container4ml-simple:1.0 コンテキストの指定. Dockerに見てほしいpath. COPY句でどこを起点にするか . 基本的にDockerﬁleのある場所でOK イメージ名. お作法的にはowner/image_name イメージタグ. イメージは更新されゆくのでいわゆる versioning 33

Slide 34

Slide 34 text

©2023 CyberAgent Inc. Distribution prohibited 3. Docker Imageの作成 ➔ docker build . -t container4ml-simple:1.0 ➔ docker images REPOSITORY TAG IMAGE ID CREATED SIZE container4ml-simple 1.0 $(IMAGE_ID) 7 seconds ago 628MB Image sizeはなるべく小さくするべきだが、慣れるまではゴリゴリに削らなくて OK 34

Slide 35

Slide 35 text

©2023 CyberAgent Inc. Distribution prohibited 4. Docker Containerの起動 ➔ watch docker ps -a ➔ docker run container4ml-simple:1.0 scikit-learn: 1.2.2 ➔ docker run --rm container4ml-simple:1.0 ➔ cat main.py import sklearn print(f"scikit-learn: {sklearn.__version__}") 35 ←watchとは別窓で実行

Slide 36

Slide 36 text

©2023 CyberAgent Inc. Distribution prohibited 4. Docker Containerの実行上書き ➔ tail -1 Dockerfile ➔ docker run --rm container4ml-simple:1.0 ls -lh total 4.0K -rw-r--r-- 1 root root 62 May 8 23:52 main.py 実行コマンドは上書きできる同Imageで挙動だけ変えたい時等で活用例えばTraining/ServingのImageを共通化するとか 36

Slide 37

Slide 37 text

©2023 CyberAgent Inc. Distribution prohibited 5. Docker ImageのUpload/Download ➔ open https://hub.docker.com/ ➔ docker login ➔ docker tag container4ml-simple:1.0 $(HUB_ID)/container4ml-simple:1.0 ➔ docker images REPOSITORY TAG IMAGE ID CREATED SIZE chck/container4ml-simple 1.0 c1575db34e5d 2 hours ago 628MB container4ml-simple 1.0 c1575db34e5d 2 hours ago 628MB ➔ docker push $(HUB_ID)/container4ml-simple:1.0 ➔ open https://hub.docker.com/repository/docker/$(HUB_ID)/container4ml-simple ➔ docker rmi $(HUB_ID)/container4ml-simple:1.0 ➔ docker pull $(HUB_ID)/container4ml-simple:1.0 ➔ docker images 37

Slide 38

Slide 38 text

Slide 39

Slide 39 text

©2023 CyberAgent Inc. Distribution prohibited 5. Docker ImageのUpload/Download ➔ open https://hub.docker.com/ ➔ docker login ➔ docker tag container4ml-simple:1.0 $(HUB_ID)/container4ml-simple:1.0 ➔ docker images REPOSITORY TAG IMAGE ID CREATED SIZE chck/container4ml-simple 1.0 c1575db34e5d 2 hours ago 628MB container4ml-simple 1.0 c1575db34e5d 2 hours ago 628MB 39 TagはAliasとして働くので同じImage IDを持ち, Diskも重複消費しない

Slide 40

Slide 40 text

©2023 CyberAgent Inc. Distribution prohibited 5. Docker ImageのUpload/Download ➔ docker push $(HUB_ID)/container4ml-simple:1.0 ➔ open hub.docker.com/repository/docker/$(HUB_ID)/container4ml-simple ➔ docker rmi $(HUB_ID)/container4ml-simple:1.0 ➔ docker pull $(HUB_ID)/container4ml-simple:1.0 ➔ docker images 指定registryにimageをupload 40

Slide 41

Slide 41 text

Slide 42

Slide 42 text

©2023 CyberAgent Inc. Distribution prohibited Tips: Image Tagは複数付与できる ➔ docker tag container4ml-simple:1.0 $(HUB_ID)/container4ml-simple:1.0 ➔ docker tag container4ml-simple:1.0 $(HUB_ID)/container4ml-simple:latest ➔ docker push $(HUB_ID)/container4ml-simple:1.0 ➔ docker push $(HUB_ID)/container4ml-simple:latest 42

Slide 43

Slide 43 text

©2023 CyberAgent Inc. Distribution prohibited Tips: slim? alpine? ➔ docker pull python:3.9 ➔ docker pull python:3.9-slim ➔ docker pull python:3.9-alpine ➔ docker images REPOSITORY TAG IMAGE ID CREATED SIZE python 3.9 67ec76d9f73b 2 weeks ago 857MB python 3.9-slim 64458f531a7e 2 weeks ago 118MB python 3.9-alpine d314e28e240c 2 days ago 57.8MB 依存とサイズのトレードオフ最初はslimがおすすめ 43 最軽量版 (alpine) 軽量版 (slim) Debian baseのfull image (無印)

Slide 44

Slide 44 text

Slide 45

Slide 45 text

©2023 CyberAgent Inc. Distribution prohibited Docker Architecture 45 Docker Host: 仮想環境の本体 DockerのContainerやImage, NetworkやVolumeを管理 Clientのコマンドを待受 https://docs.docker.com/get-started/overview/#docker-architecture

Slide 46

Slide 46 text

Slide 47

Slide 47 text

Slide 48

Slide 48 text

Slide 49

Slide 49 text

Slide 50

Slide 50 text

©2023 CyberAgent Inc. Distribution prohibited Docker Compose Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application’s services. Then, with a single command, you create and start all the services from your configuration. 50 https://docs.docker.com/compose/

Slide 51

Slide 51 text

©2023 CyberAgent Inc. Distribution prohibited Docker vs Docker Compose Docker: Docker Compose: 51 ➔ docker build . -t container4ml ➔ docker run --rm -p 8888:8888 -v ${PWD}:/app container4ml (compose.ymlがある状態で) ➔ docker compose up ↓概念の説明なので実行しなくて OK

Slide 52

Slide 52 text

©2023 CyberAgent Inc. Distribution prohibited 1. Dockerﬁleを書く ➔ cd 2-jupyter ➔ ls .dockerignore compose.yml Makefile pyproject.toml .gitignore Dockerfile poetry.lock train.ipynb ➔ $EDITOR Dockerfile -------------------------- 52

Slide 53

Slide 53 text

©2023 CyberAgent Inc. Distribution prohibited 1. Dockerﬁleを書く 53 ➔ $EDITOR Dockerfile -------------------------- FROM python:3.9-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends build-essential \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY *.bin . EXPOSE 8888 CMD ["jupyter", "lab", "--allow-root", "--ip=0.0.0.0", "--no-browser", "--ServerApp.allow_origin=*", "--ServerApp.token=", "--ServerApp.password="]

Slide 54

Slide 54 text

©2023 CyberAgent Inc. Distribution prohibited 1-simpleのDockerﬁleと比較 ➔ $EDITOR Dockerfile -------------------------- FROM python:3.9-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends build-essential \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY main.py . CMD ["python", "main.py"] 54

Slide 55

Slide 55 text

©2023 CyberAgent Inc. Distribution prohibited 1. Dockerﬁleを書く 55 ➔ $EDITOR Dockerfile -------------------------- FROM python:3.9-slim ENV APP_HOME /app RUN apt update && apt install -y --no-install-recommends build-essential \ && apt clean && rm -rf /var/lib/apt/lists/* \ && pip install -U pip && pip install --no-cache-dir poetry COPY pyproject.toml poetry.lock ./ RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \ && pip install -r requirements.txt --no-cache-dir WORKDIR ${APP_HOME} COPY *.bin . EXPOSE 8888 CMD ["jupyter", "lab", "--allow-root", "--ip=0.0.0.0", "--no-browser", "--ServerApp.allow_origin=*", "--ServerApp.token=", "--ServerApp.password="]

Slide 56

Slide 56 text

Slide 57

Slide 57 text

©2023 CyberAgent Inc. Distribution prohibited 2. compose.ymlを書く ➔ $EDITOR compose.yml -------------------------- services: jupyter: image: …/container4ml-jupyter:${USER} build: . ports: - "8888:8888" volumes: - ${PWD}:/app 57 任意のService名 Dockerﬁleを参照しながら指定Image名でdocker build portやvolume optionを付与してdocker runされ jupyter containerが起動

Slide 58

Slide 58 text

©2023 CyberAgent Inc. Distribution prohibited 3. Docker Containerの起動 (via docker compose) ➔ docker compose up Jupyter Server 2.5.0 is running at:... ➔ docker images ➔ docker ps 58 ←docker compose upとは別窓で実行 ←docker buildとdocker runが両方行われる

Slide 59

Slide 59 text

Slide 60

Slide 60 text

©2023 CyberAgent Inc. Distribution prohibited User Agent (UA) https://towardsdatascience.com/still-parsing-user-agent-strings-for-your-machine-learning-models-use-this-instead-8928c0e7e74f Client (e.g. ブラウザ) がServerアクセス時に付与するOS等の情報をまとめた文字列 Web広告の文脈では擬似的な個人情報として使えたりするため昨今規制が厳しい 60

Slide 61

Slide 61 text

Slide 62

Slide 62 text

©2023 CyberAgent Inc. Distribution prohibited Tips: ModelをCloud Storageに保存しておく例 ➔ ts = !TZ=Asia/Tokyo date +"%Y%m%d%H%M" ➔ aws s3 cp ua_classifier.bin s3://${AWS_BUCKET}/models/ua_classifier/{ts[0]}.bin ➔ aws s3 cp ua_classifier.bin s3://${AWS_BUCKET}/models/ua_classifier/latest.bin ➔ aws s3 ls --human-readable s3://${AWS_BUCKET}/models/ua_classifier/ 2023-05-02 08:19:44 281.9 KiB 202305020819.bin 2023-05-02 08:19:50 281.9 KiB latest.bin ↓Tipsなので実行しなくてOK

Slide 63

Slide 63 text

©2023 CyberAgent Inc. Distribution prohibited 3. Docker Containerの起動 (via docker compose) ➔ Ctrl-C └ Gracefully stopping... ➔ docker compose build ➔ docker run …/container4ml-jupyter:${USER} ls └ ua_classifier.bin ➔ aws ecr get-login-password --region ${AWS_REGION} | docker login -u AWS --password-stdin ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com ➔ docker push …/container4ml-jupyter:${USER} └ Amazon ECRのUIを確認 63

Slide 64

Slide 64 text

Slide 65

Slide 65 text

©2023 CyberAgent Inc. Distribution prohibited 1. Dockerﬁleを書く ➔ cd 3-fastapi ➔ ls bin Dockerfile main.py pyproject.toml compose.yml k8s poetry.lock setup.cfg ➔ $EDITOR Dockerfile -------------------------- 65

Slide 66

Slide 66 text

©2023 CyberAgent Inc. Distribution prohibited 1. Dockerﬁleを書く 66 ➔ $EDITOR Dockerfile -------------------------- ARG AWS_ACCOUNT_ID ARG AWS_REGION=${AWS_REGION:-"ap-northeast-1"} ARG TRAINER_VERSION=${TRAINER_VERSION:-"latest"} FROM ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/container4ml-jupyter:${TRAINER_VERSION} AS trainer FROM python:3.9-slim ...(中略)... WORKDIR /models COPY --from=trainer /app/ua_classifier.bin . WORKDIR ${APP_HOME} COPY main.py . EXPOSE 80 CMD ["gunicorn", "main:app", "--bind=0.0.0.0:80", "--workers=1", "--threads=8", "--timeout=0", "--worker-class=uvicorn.workers.UvicornWorker"]

Slide 67

Slide 67 text

©2023 CyberAgent Inc. Distribution prohibited 1. Dockerﬁleを書く 67 ➔ $EDITOR Dockerfile -------------------------- ARG AWS_ACCOUNT_ID ARG AWS_REGION=${AWS_REGION:-"ap-northeast-1"} ARG TRAINER_VERSION=${TRAINER_VERSION:-"latest"} FROM ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/container4ml-jupyter:${TRAINER_VERSION} AS trainer FROM python:3.9-slim ...(中略)... WORKDIR /models COPY --from=trainer /app/ua_classifier.bin . WORKDIR ${APP_HOME} COPY main.py . EXPOSE 80 CMD ["gunicorn", "main:app", "--bind=0.0.0.0:80", "--workers=1", "--threads=8", "--timeout=0", "--worker-class=uvicorn.workers.UvicornWorker"]

Slide 68

Slide 68 text

©2023 CyberAgent Inc. Distribution prohibited 2. compose.ymlを書く ➔ $EDITOR compose.yml -------------------------- services: myapp: image: ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/container4ml-fastapi:latest build: args: - AWS_ACCOUNT_ID=${AWS_ACCOUNT_ID} ports: - "3333:80" environment: … - REDISHOST=myredis - MODEL_NAME=A volumes: - ${PWD}:/app depends_on: - myredis myredis: image: redis:alpine 68

Slide 69

Slide 69 text

©2023 CyberAgent Inc. Distribution prohibited 2. compose.ymlを書く ➔ $EDITOR compose.yml -------------------------- services: myapp: image: …/container4ml-fastapi:latest build: args: - AWS_ACCOUNT_ID=${AWS_ACCOUNT_ID} ports: - "3333:80" environment: … - REDISHOST=myredis - MODEL_NAME=A volumes: - ${PWD}:/app depends_on: - myredis myredis: image: redis:alpine 69 任意のService名 Dockerﬁleを参照しながら指定Image名でdocker build 指定Imageをdocker pullで用意 docker runのoption相当 myredis containerの起動を待ってから myappを起動

Slide 70

Slide 70 text

©2023 CyberAgent Inc. Distribution prohibited 2. compose.ymlを書く ➔ $EDITOR compose.yml -------------------------- services: myapp: image: …/container4ml-fastapi:latest build: args: - AWS_ACCOUNT_ID=${AWS_ACCOUNT_ID} ports: - "3333:80" environment: … - REDISHOST=myredis - MODEL_NAME=A volumes: - ${PWD}:/app depends_on: - myredis myredis: image: redis:alpine 70 w/o docker compose ➔ docker network create mynwk ➔ docker run -p 3333:80 -e PYTHONUNBUFFERED=1 -e DEBUG=true -e REDISHOST=myredis -e MODEL_NAME=A -v ${PWD}:/app --net mynwk …/container4ml-fastapi:latest ➔ docker run --net mynwk redis:alpine w/ docker compose ➔ docker compose up Container間通信(mynwk) myapp myredis

Slide 71

Slide 71 text

Slide 72

Slide 72 text

©2023 CyberAgent Inc. Distribution prohibited 3. Docker Containerの起動 (via docker compose) ➔ aws ecr get-login-password --region ${AWS_REGION} | docker login -u AWS --password-stdin ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com ➔ docker compose up -d ➔ docker ps CONTAINER ID IMAGE ... PORTS ... NAMES ... .../container4ml-fastapi:1.0 ... 0.0.0.0:3333->80/tcp 3-fastapi-myapp-1 ... redis:alpine ... 6379/tcp 3-fastapi-myredis-1 ➔ open http://localhost:3333 ➔ open http://localhost:3333/stats ➔ docker compose down ➔ compose.ymlのMODEL_NAMEをAからBに変更後、 docker compose up --build -d 72

Slide 73

Slide 73 text

©2023 CyberAgent Inc. Distribution prohibited 時間が余った時用 - docker system prune -a - M1/M2 MacのPython開発事情 - 1 Image : 1 Appの理由 - Docker化が嬉しい場面 - Dataなどの重たいファイルはvolumeでsyncしておく話 - ModelをStorageに持つかImageに持つか - Docker container のDebug方法 - 変数によってbuildの振る舞いを変えたい時 - Training/Servingは同じImageか分けるか - Securityの話 73

Slide 74

Slide 74 text

Slide 75

Slide 75 text

Slide 76

Slide 76 text

Slide 77

Slide 77 text

Slide 78

Slide 78 text

Slide 79

Slide 79 text

Slide 80

Slide 80 text

©2023 CyberAgent Inc. Distribution prohibited Container Orchestration Containerの展開や状態の管理を担うService Docker ImageをどうServing するかの部分 Kubernetesを基とする Managed Serviceが複数展開 https://www.datadoghq.com/container-report-2020/ 80

Slide 81

Slide 81 text

Slide 82

Slide 82 text

Slide 83

Slide 83 text

©2023 CyberAgent Inc. Distribution prohibited Docker Compose vs Kubernetes ➔ docker compose up -d ➔ docker compose ps ➔ docker compose down ➔ cd 3-fastapi/k8s ➔ kind create cluster --config=kind-config.yml ➔ kubectl cluster-info --context kind-kind ➔ kubectl apply -f deployment.yml ➔ kubectl apply -f service.yml ➔ kubectl get po,svc,deploy ➔ kubectl delete -f service.yml ➔ kubectl delete -f deployment.yml ➔ kind delete cluster

Slide 84

Slide 84 text

Slide 85

Slide 85 text

Slide 86

Slide 86 text

Slide 87

Slide 87 text

©2023 CyberAgent Inc. Distribution prohibited Story 3-2 LocalのDocker Composeで動作確認のできたUA Classiﬁerを App RunnerにDeployしたい DeployのProcess 1. Docker ImageをRegistryにUpload 2. UploadしたImageを指定してApp Runnerを構築・起動 87

Slide 88

Slide 88 text

Slide 89

Slide 89 text

©2023 CyberAgent Inc. Distribution prohibited 1. RegistryにImageをUpload ➔ docker tag ${HUB_ID}/container4ml-fastapi:1.0 ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/container4ml-fastapi:1.0 ➔ docker tag ${HUB_ID}/container4ml-fastapi:latest ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/container4ml-fastapi:latest ➔ aws ecr get-login-password --region ap-northeast-1 | docker login --username AWS --password-stdin ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com ➔ docker push ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/container4ml-fastapi:1.0 ➔ docker push ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/container4ml-fastapi:latest ➔ open https://ap-northeast-1.console.aws.amazon.com/ecr/repositories 89

Slide 90

Slide 90 text

Slide 91

Slide 91 text

Slide 92

Slide 92 text

Slide 93

Slide 93 text

Slide 94

Slide 94 text

Slide 95

Slide 95 text

Slide 96

Slide 96 text

©2023 CyberAgent Inc. Distribution prohibited Autoscaling https://github.com/rakyll/hey WebAppの負荷試験ができるheyでbenchmarkを取ってみる ➔ sh ./test-scale.sh ➔ sh ./test-scale.sh https://${APPRUNNER_DOMAIN} App Runnerのmetricsやheyのbenchmarkを観察 96

Slide 97

Slide 97 text

Slide 98

Slide 98 text

©2023 CyberAgent Inc. Distribution prohibited GitHub ActionsによるDeployの自動化 ➔ cat .github/workflows/deploy-container4ml.yml -------------------------- ... on: push: branches: [main] ... jobs: infra: runs-on: ubuntu-latest steps: - run: echo "container4ml deploy job triggered." changes: runs-on: ubuntu-latest needs: infra ... トリガ条件 jobs ∋ steps needsで実行順を制御できる実際に実行されるのは stepsのコマンド

Slide 99

Slide 99 text

©2023 CyberAgent Inc. Distribution prohibited ➔ cat .github/workflows/deploy-container4ml.yml -------------------------- ... on: push: branches: [main] ... jobs: infra: runs-on: ubuntu-latest steps: - run: echo "container4ml deploy job triggered." changes: runs-on: ubuntu-latest needs: infra ... GitHub ActionsによるDeployの自動化 GitHub repo -> Actionsから実行履歴が見れる

Slide 100

Slide 100 text

©2023 CyberAgent Inc. Distribution prohibited GitHub ActionsによるDeployの自動化トリガ条件: main branchにpushされたら実行内容: paths-ﬁlterで更に分岐 └ trainerルート... '2-jupyter/**' の差分 └ S3からua_classiﬁer.binをdownloadし、それを含めて　　　　　 container4ml-jupyterをdocker build、ECRにpush └ predictorルート... '3-fastapi/**' の差分 └ container4ml-fastapiをdocker build、ECRにpush

Slide 101

Slide 101 text

©2023 CyberAgent Inc. Distribution prohibited Blue-green deployment 既存のApp (Blue) 稼働中の裏に新版のApp (Green) をTraﬃcが来ない状態でDeployし、StandbyになったタイミングでRouter内部の向き先を Greenに変えることで無停止Deployを行う仕組み App Runnerもこれに対応 ➔ sh ./test-switch.sh https://${APPRUNNER_DOMAIN} ➔ # Deploy New Version in App Runner 101 https://candost.blog/the-blue-green-deployment-strategy/

Slide 102

Slide 102 text

Slide 103

Slide 103 text

Slide 104

Slide 104 text

Slide 105

Slide 105 text

©2023 CyberAgent Inc. Distribution prohibited A/B Testingの実装方法 Load balancerでSplitting Container A, Bの前段にLoad balancerを立ててm:nに分散 - Pros 👍 ??? - Cons 👎 ??? 105 Application内でSplitting Python Code内でrandomやuuidのhash値、 DB参照などのルールに基づいてif else等で分割 - Pros 👍 ??? - Cons 👎 ??? ざっくり2種類の方法がある

Slide 106

Slide 106 text

©2023 CyberAgent Inc. Distribution prohibited A/B Testingの実装方法 Load balancerでSplitting Container A, Bの前段にLoad balancerを立ててm:nに分散 - Pros 👍 ModelとContainerが1:1で紐づくので Appの実装がシンプル - Cons 👎 Cache戦略によっては同じuuidでも Requestの度にA/Bを横断してしまう 106 Application内でSplitting Python Code内でrandomやuuidのhash値、 DB参照などのルールに基づいてif else等で分割 - Pros 👍 QueryのUUIDに応じて分割先のルール設計が簡単 - Cons 👎 App内で複数VersionのModelをLoadしておくResourceが必要ざっくり2種類の方法がある

Slide 107

Slide 107 text

Slide 108

Slide 108 text