機械学習開発のためのコンテナ入門 / Container for ML

CA 1Day Youth Boot Campでの発表資料です

Transcript

1. Container
   for ML
   YouthBootCamp20211203
   Yuki Iwazaki@chck / CyberAgent
   

   View Slide

2. Today’s Goal
   2
   

   View Slide

3. 今日から使える
   Portableな実験環境
   Scalableな推論App
   3
   

   View Slide

4. Prerequisites
   4
   

   View Slide

5. Yuki Iwazaki
   2014...Backend Engineer in DSP
   └2017...ML/DS in Inhouse
   └2018-...Research Engineer in AI Lab
   Multimedia (Vision & Language)
   5
   chck
   

   View Slide

6. 難解な環境構築
   通らないpip install
   競合するlibrary version
   引き継ぎできない実験コード
   6
   

   View Slide

7. コンテナ技術の世界へ
   7
   

   View Slide

8. Docker for ML
   MLのためのDockerについて
   8
   

   View Slide

9. Docker
   Docker is an open platform for developing, shipping, and running
   applications. Docker enables you to separate your applications from your
   infrastructure so you can deliver software quickly. With Docker, you can
   manage your infrastructure in the same ways you manage your applications.
   By taking advantage of Docker’s methodologies for shipping, testing, and
   deploying code quickly, you can significantly reduce the delay between
   writing code and running it in production.
   https://docs.docker.com/get-started/overview/
   9
   

   View Slide

10. Why Docker?
    • コードを動かすための依存をDockerさえ入っていれば動く状態にまでPackagingできる
    ○ CondaやPoetry, RStudioではだめ?
    ■ 自分のPCでだけ動かすならOK 。移行や共有時どうする?
    ■ OSへの直Installや別途Downloadが必要な依存がある場合は?
    • 実験コードを動かすための長く複雑な手順を
    Dockerで隠蔽できる
    ○ READMEにはDocker commandを書くだけ
    • Kubernetesを始めとする強力なContainer ServiceにML AppをDeployできる
    ○ 2021年現在、一定規模以上のAppの運用を考えると業界標準に
    10
    

    View Slide

11. https://www.datadoghq.com/docker-adoption/
    11
    

    View Slide

12. LiveDemo
    ➔ git clone https://github.com/chck/container4ml.git
    ➔ cd container4ml
    ➔ ls
    .git .gitignore 1-simple 2-jupyter 3-fastapi
    README.md
    12
    

    View Slide

13. Story 1
    • 既にあるPython ScriptをDocker化したい
    • Docker化のProcess
    ○ Python Scriptを動かすための依存を含んだDocker Imageを作成
    ○ Docker Containerとして実行し、挙動を確認
    ○ DockerfileのCommitやDocker RegistryにImageをUploadして完了
    13
    

    View Slide

14. Docker Image
    An image is a read-only template
    with instructions for creating a
    Docker container. Often, an image is
    based on another image, with some
    additional customization.
    https://docs.docker.com/get-started/overview/
    14
    

    View Slide

15. Docker Container
    A container is a runnable instance
    of an image. You can create, start,
    stop, move, or delete a container
    using the Docker CLI.
    It is a standard unit of software that
    packages up code and all its
    dependencies.
    https://www.docker.com/resources/what-container
    15
    

    View Slide

16. Docker Container
    A container is a runnable instance
    of an image. You can create, start,
    stop, move, or delete a container
    using the Docker CLI.
    It is a standard unit of software that
    packages up code and all its
    dependencies.
    https://www.docker.com/resources/what-container
    MacBook, ThinkPad,
    EC2, GCE
    macOS, Windows10,
    Ubuntu
    JupyterLab, Flask,
    MySQL, Original App
    16
    

    View Slide

17. Story 1: 再掲
    • 既にあるPython ScriptをDocker化したい
    • Docker化のProcess
    ○ Python Scriptを動かすための依存を含んだDocker Imageを作成
    ○ Docker Containerとして実行し、挙動を確認
    ○ DockerfileのCommitやDocker RegistryにImageをUploadして完了
    17
    

    View Slide

18. 1. BaseとなるImageを決める
    “$keyword docker hub”で検索
    18
    

    View Slide

19. 2. Dockerfileを書く
    ➔ cd 1-simple
    ➔ ls
    .dockerignore Dockerfile Makefile
    pyproject.toml
    docker-compose.yml main.py poetry.lock
    ➔ vi Dockerfile
    --------------------------
    19
    

    View Slide

20. 2. Dockerfileを書く
    ➔ vi Dockerfile
    --------------------------
    FROM python:3.8-slim
    ENV APP_HOME /app
    RUN apt update && apt install -y --no-install-recommends curl \
    && apt clean && rm -rf /var/lib/apt/lists/* \
    && pip install -U pip && pip install --no-cache-dir poetry
    COPY pyproject.toml poetry.lock ./
    RUN poetry export --without-hashes -f requirements.txt -o requirements.txt
    \
    && pip install -r requirements.txt --no-cache-dir
    WORKDIR ${APP_HOME}
    COPY main.py .
    CMD ["python", "main.py"]
    20
    

    View Slide

21. 2. Dockerfileを書く
    ➔ vi Dockerfile
    --------------------------
    FROM python:3.8-slim
    ENV APP_HOME /app
    RUN apt update && apt install -y --no-install-recommends curl \
    && apt clean && rm -rf /var/lib/apt/lists/* \
    && pip install -U pip && pip install --no-cache-dir poetry
    COPY pyproject.toml poetry.lock ./
    RUN poetry export --without-hashes -f requirements.txt -o requirements.txt
    \
    && pip install -r requirements.txt --no-cache-dir
    WORKDIR ${APP_HOME}
    COPY main.py .
    CMD ["python", "main.py"]
    21
    

    View Slide

22. 2. Dockerfileを書く
    ➔ vi Dockerfile
    --------------------------
    FROM python:3.8-slim
    ENV APP_HOME /app
    RUN apt update && apt install -y --no-install-recommends curl \
    && apt clean && rm -rf /var/lib/apt/lists/* \
    && pip install -U pip && pip install --no-cache-dir poetry
    COPY pyproject.toml poetry.lock ./
    RUN poetry export --without-hashes -f requirements.txt -o requirements.txt
    \
    && pip install -r requirements.txt --no-cache-dir
    WORKDIR ${APP_HOME}
    COPY main.py .
    CMD ["python", "main.py"]
    Base Imageを指定
    22
    

    View Slide

23. 2. Dockerfileを書く
    ➔ vi Dockerfile
    --------------------------
    FROM python:3.8-slim
    ENV APP_HOME /app
    RUN apt update && apt install -y --no-install-recommends curl \
    && apt clean && rm -rf /var/lib/apt/lists/* \
    && pip install -U pip && pip install --no-cache-dir poetry
    COPY pyproject.toml poetry.lock ./
    RUN poetry export --without-hashes -f requirements.txt -o requirements.txt
    \
    && pip install -r requirements.txt --no-cache-dir
    WORKDIR ${APP_HOME}
    COPY main.py .
    CMD ["python", "main.py"]
    環境変数の定義
    23
    

    View Slide

24. 2. Dockerfileを書く
    ➔ vi Dockerfile
    --------------------------
    FROM python:3.8-slim
    ENV APP_HOME /app
    RUN apt update && apt install -y --no-install-recommends curl \
    && apt clean && rm -rf /var/lib/apt/lists/* \
    && pip install -U pip && pip install --no-cache-dir poetry
    COPY pyproject.toml poetry.lock ./
    RUN poetry export --without-hashes -f requirements.txt -o requirements.txt
    \
    && pip install -r requirements.txt --no-cache-dir
    WORKDIR ${APP_HOME}
    COPY main.py .
    CMD ["python", "main.py"]
    環境構築の実行コマンドを記述
    24
    

    View Slide

25. 2. Dockerfileを書く
    ➔ vi Dockerfile
    --------------------------
    FROM python:3.8-slim
    ENV APP_HOME /app
    RUN apt update && apt install -y --no-install-recommends curl \
    && apt clean && rm -rf /var/lib/apt/lists/* \
    && pip install -U pip && pip install --no-cache-dir poetry
    COPY pyproject.toml poetry.lock ./
    RUN poetry export --without-hashes -f requirements.txt -o requirements.txt
    \
    && pip install -r requirements.txt --no-cache-dir
    WORKDIR ${APP_HOME}
    COPY main.py .
    CMD ["python", "main.py"]
    Image内でのDirectory移動
    25
    

    View Slide

26. 2. Dockerfileを書く
    ➔ vi Dockerfile
    --------------------------
    FROM python:3.8-slim
    ENV APP_HOME /app
    RUN apt update && apt install -y --no-install-recommends curl \
    && apt clean && rm -rf /var/lib/apt/lists/* \
    && pip install -U pip && pip install --no-cache-dir poetry
    COPY pyproject.toml poetry.lock ./
    RUN poetry export --without-hashes -f requirements.txt -o requirements.txt
    \
    && pip install -r requirements.txt --no-cache-dir
    WORKDIR ${APP_HOME}
    COPY main.py .
    CMD ["python", "main.py"]
    local://${pwd}/main.py
    docker://app/main.py
    COPY main.py .
    26
    

    View Slide

27. 2. Dockerfileを書く
    ➔ vi Dockerfile
    --------------------------
    FROM python:3.8-slim
    ENV APP_HOME /app
    RUN apt update && apt install -y --no-install-recommends curl \
    && apt clean && rm -rf /var/lib/apt/lists/* \
    && pip install -U pip && pip install --no-cache-dir poetry
    COPY pyproject.toml poetry.lock ./
    RUN poetry export --without-hashes -f requirements.txt -o requirements.txt
    \
    && pip install -r requirements.txt --no-cache-dir
    WORKDIR ${APP_HOME}
    COPY main.py .
    CMD ["python", "main.py"]
    起動時に実行されるコマンド
    27
    

    View Slide

28. 3. Docker Imageの作成
    ➔ docker build . -t container4ml-simple:1.0
    28
    

    View Slide

29. 3. Docker Imageの作成
    ➔ docker build . -t container4ml-simple:1.0
    コンテキストの指定. Dockerに見てほしいpath.
    COPY句でどこを起点にするか . 基本的にDockerfileのある場所でOK
    イメージ名. お作法的にはowner/image_name
    イメージタグ. イメージは更新されゆくのでいわゆる versioning
    29
    

    View Slide

30. 3. Docker Imageの作成
    ➔ docker build . -t container4ml-simple:1.0
    ➔ docker images
    REPOSITORY TAG IMAGE ID CREATED SIZE
    container4ml-simple 1.0 be5c53cd3ad6 7 seconds ago 437MB
    Image sizeはなるべく小さくする
    重いとImageのupload/downloadに時間がかかりML Lifecycleの効率も落ちる
    30
    

    View Slide

31. 4. Docker Containerの起動
    ➔ docker run container4ml-simple:1.0
    scikit-learn: 1.0.1
    ➔ docker ps -a
    CONTAINER ID IMAGE COMMAND ... NAMES
    30f773835fbc container4ml-simple:1.0 "python main.py" $(CONTAINER_NAME)
    ➔ cat main.py
    import sklearn
    print(f"scikit-learn: {sklearn.__version__}")
    31
    

    View Slide

32. 4. Docker Containerの実行上書き
    ➔ docker run --rm container4ml-simple:1.0 ls -lh
    total 4.0K
    -rw-r--r-- 1 root root 62 Dec 2 14:54 main.py
    実行コマンドは上書きできる
    同Imageで挙動だけ変えたい時等で活用
    例えばTraining/Servingを同Imageにするとか
    32
    

    View Slide

33. 5. Docker ImageのUpload/Download
    ➔ docker tag container4ml-simple:1.0 chck/container4ml-simple:1.0
    ➔ docker images
    REPOSITORY TAG IMAGE ID CREATED SIZE
    chck/container4ml-simple 1.0 c1575db34e5d 2 hours ago 437MB
    container4ml-simple 1.0 c1575db34e5d 2 hours ago 437MB
    ➔ docker push chck/container4ml-simple:1.0
    ➔ open https://hub.docker.com/repository/docker/chck/container4ml-simple
    ➔ docker rmi chck/container4ml-simple:1.0
    ➔ docker pull chck/container4ml-simple:1.0
    ➔ docker images 33
    

    View Slide

34. 5. Docker ImageのUpload/Download
    ➔ docker tag container4ml-simple:1.0 chck/container4ml-simple:1.0
    ➔ docker images
    REPOSITORY TAG IMAGE ID CREATED SIZE
    chck/container4ml-simple 1.0 c1575db34e5d 2 hours ago 437MB
    container4ml-simple 1.0 c1575db34e5d 2 hours ago 437MB
    ➔ docker push chck/container4ml-simple:1.0
    ➔ open https://hub.docker.com/repository/docker/chck/container4ml-simple
    ➔ docker rmi chck/container4ml-simple:1.0
    ➔ docker pull chck/container4ml-simple:1.0
    ➔ docker images
    tagはAliasとして働くので同じImage IDを持ち, Diskも重複消費しない
    指定registryにimageがuploadされる
    34
    

    View Slide

35. 5. Docker ImageのUpload/Download
    ➔ docker tag container4ml-simple:1.0 chck/container4ml-simple:1.0
    ➔ docker images
    REPOSITORY TAG IMAGE ID CREATED SIZE
    chck/container4ml-simple 1.0 c1575db34e5d 2 hours ago 437MB
    container4ml-simple 1.0 c1575db34e5d 2 hours ago 437MB
    ➔ docker push chck/container4ml-simple:1.0
    ➔ open https://hub.docker.com/repository/docker/chck/container4ml-simple
    ➔ docker rmi chck/container4ml-simple:1.0
    ➔ docker pull chck/container4ml-simple:1.0
    ➔ docker images
    指定したDocker Imageを削除. Registryに上げていれば安心
    指定RegistryからImageをDownload
    これはDocker Hubの参照例 35
    

    View Slide

36. Tips: Image Tagは複数付与できる
    ➔ docker tag container4ml-simple:1.0 chck/container4ml-simple:1.0
    ➔ docker tag container4ml-simple:1.0 chck/container4ml-simple:latest
    ➔ docker push chck/container4ml-simple:1.0
    ➔ docker push chck/container4ml-simple:latest
    36
    

    View Slide

37. Tips: slim? alpine?
    ➔ docker pull python:3.8-alpine
    ➔ docker pull python:3.8-slim
    ➔ docker pull python:3.8
    ➔ docker images
    REPOSITORY TAG IMAGE ID CREATED SIZE
    python 3.8-alpine d314e28e240c 2 days ago 43.2MB
    python 3.8-slim 64458f531a7e 2 weeks ago 122MB
    python 3.8 67ec76d9f73b 2 weeks ago 909MB
    Debian baseのfull image (無印)
    > 軽量版 (slim)
    > 最軽量版 (alpine)
    依存も少なくなっていくのでトレードオフ
    最初はslimがおすすめ
    37
    

    View Slide

38. Docker Architecture
    38
    Docker Client:
    CUIとしてコマンドで操作
    https://docs.docker.com/get-started/overview/#docker-architecture
    

    View Slide

39. Docker Architecture
    39
    Docker Host:
    仮想環境の本体
    DockerのContainerやImage,
    NetworkやVolumeを管理
    Clientのコマンドも待受
    https://docs.docker.com/get-started/overview/#docker-architecture
    

    View Slide

40. Docker Architecture
    40
    Docker Registry:
    Docker Imageの共有先
    PublicとPrivateを使い分け
    https://docs.docker.com/get-started/overview/#docker-architecture
    

    View Slide

41. Docker Architecture
    41
    今回は
    手元のPCがこの部分
    https://docs.docker.com/get-started/overview/#docker-architecture
    

    View Slide

42. Docker Architecture
    42
    こっちは
    Remote Server
    Docker Hubや
    GCR, ECRが相当
    https://docs.docker.com/get-started/overview/#docker-architecture
    

    View Slide

43. Story 2
    • 学習コードの乗ったJupyterLabをDocker上で動かしたい
    • Docker Compose化のProcess
    ○ Dockerfileを書くところまで同じ
    ○ docker-compose.ymlを作成
    ○ Docker Composeによる実行、挙動の確認
    43
    

    View Slide

44. Docker Compose
    Compose is a tool for defining and running multi-container Docker
    applications.
    With Compose, you use a YAML file to configure your application’s services.
    Then, with a single command, you create and start all the services from your
    configuration.
    44
    https://docs.docker.com/compose/
    

    View Slide

45. Docker Compose
    Docker:
    Docker Compose:
    45
    ➔ docker build . -t container4ml
    ➔ docker run --rm -p 8888:8888 -v ${PWD}:/app container4ml
    (docker-compose.ymlがある状態で)
    ➔ docker compose up
    

    View Slide

46. 1. Dockerfileを書く
    ➔ cd 2-jupyter
    ➔ ls
    .dockerignore docker-compose.yml Makefile pyproject.toml
    .gitignore Dockerfile poetry.lock train.ipynb
    ➔ vi Dockerfile
    --------------------------
    46
    

    View Slide

47. 1. Dockerfileを書く
    47
    ➔ vi Dockerfile
    --------------------------
    FROM python:3.8-slim
    ENV APP_HOME /app
    RUN apt update && apt install -y --no-install-recommends git build-essential \
    && apt clean && rm -rf /var/lib/apt/lists/* \
    && pip install -U pip && pip install --no-cache-dir poetry
    COPY pyproject.toml poetry.lock ./
    RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \
    && pip install -r requirements.txt --no-cache-dir
    WORKDIR ${APP_HOME}
    COPY *.bin .
    EXPOSE 8888
    CMD ["jupyter", "lab", "--allow-root", "--ip=0.0.0.0", "--no-browser",
    "--ServerApp.allow_origin=*", "--ServerApp.token=", "--ServerApp.password="]
    

    View Slide

48. 1-simpleのDockerfileと比較
    ➔ vi Dockerfile
    --------------------------
    FROM python:3.8-slim
    ENV APP_HOME /app
    RUN apt update && apt install -y --no-install-recommends curl \
    && apt clean && rm -rf /var/lib/apt/lists/* \
    && pip install -U pip && pip install --no-cache-dir poetry
    COPY pyproject.toml poetry.lock ./
    RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \
    && pip install -r requirements.txt --no-cache-dir
    WORKDIR ${APP_HOME}
    COPY main.py .
    CMD ["python", "main.py"]
    48
    

    View Slide

49. 1. Dockerfileを書く
    49
    ➔ vi Dockerfile
    --------------------------
    FROM python:3.8-slim
    ENV APP_HOME /app
    RUN apt update && apt install -y --no-install-recommends git build-essential \
    && apt clean && rm -rf /var/lib/apt/lists/* \
    && pip install -U pip && pip install --no-cache-dir poetry
    COPY pyproject.toml poetry.lock ./
    RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \
    && pip install -r requirements.txt --no-cache-dir
    WORKDIR ${APP_HOME}
    COPY *.bin .
    EXPOSE 8888
    CMD ["jupyter", "lab", "--allow-root", "--ip=0.0.0.0", "--no-browser",
    "--ServerApp.allow_origin=*", "--ServerApp.token=", "--ServerApp.password="]
    

    View Slide

50. 2. docker-compose.ymlを書く
    ➔ vi docker-compose.yml
    --------------------------
    services:
    jupyter:
    image: chck/container4ml-jupyter:1.0
    build: .
    ports:
    - "8888:8888"
    volumes:
    - ${PWD}:/app
    50
    

    View Slide

51. 2. docker-compose.ymlを書く
    ➔ vi docker-compose.yml
    --------------------------
    services:
    jupyter:
    image: chck/container4ml-jupyter:1.0
    build: .
    ports:
    - "8888:8888"
    volumes:
    - ${PWD}:/app
    51
    任意のService名
    Dockerfileを参照しながら
    指定Image名でdocker build
    portやvolume optionを付与してdocker runされ
    jupyter containerが起動
    

    View Slide

52. 3. Docker Containerの起動
    (via docker compose)
    ➔ docker compose up
    Jupyter Server 1.12.1 is running at:...
    ➔ docker images
    ➔ docker ps
    CONTAINER ID IMAGE ... PORTS ...
    47ac8f57c169 chck/container4ml-jupyter:1.0 ... 0.0.0.0:8888->8888/tcp …
    ➔ open http://localhost:8888
    # train.ipynbを実行、model.binが作られたことを確認後、 docker compose build (次ページ)
    ➔ Ctrl-C
    Gracefully stopping... 52
    

    View Slide

53. UA Classifier
    Simpleな学習モデルの例として、
    UserAgent StringからPC or Mobileを
    分類するタスクを考える
    53
    

    View Slide

54. UserAgent
    https://towardsdatascience.com/still-parsing-user-agent-strings-for-your-machine-learning-models-use-this-instead-8928c0e7e74f
    Client (e.g. ブラウザ) がServerアクセス時に付与するOS等の情報をまとめた文字列
    Web広告の文脈では擬似的な個人情報として使えたりするため昨今規制が厳しい
    54
    

    View Slide

55. Story 3
    • JupyterLab上で学習したModelを乗せたFastAPI製推論AppをDockerに乗せたい
    • Docker Compose化のProcess
    ○ 今までと同じ
    55
    

    View Slide

56. 1. Dockerfileを書く
    ➔ cd 3-fastapi
    ➔ ls
    .env Dockerfile Makefile test-scale.sh
    .envrc infra poetry.lock test-switch.sh
    docker-compose.yml main.py pyproject.toml
    ➔ vi Dockerfile
    --------------------------
    56
    

    View Slide

57. 1. Dockerfileを書く
    57
    ➔ vi Dockerfile
    --------------------------
    FROM chck/container4ml-jupyter:1.0 AS trainer
    FROM python:3.8-slim
    ENV APP_HOME /app
    RUN apt update && apt install -y --no-install-recommends git build-essential \
    && apt clean && rm -rf /var/lib/apt/lists/* \
    && pip install -U pip && pip install --no-cache-dir poetry
    COPY pyproject.toml poetry.lock ./
    RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \
    && pip install -r requirements.txt --no-cache-dir
    WORKDIR /models
    COPY --from=trainer /app/*.bin .
    WORKDIR ${APP_HOME}
    COPY main.py .
    EXPOSE 80
    CMD ["gunicorn", "main:app", "--bind=0.0.0.0:80", "--workers=1", "--threads=8", "--timeout=0",
    "--worker-class=uvicorn.workers.UvicornWorker"]
    

    View Slide

58. 1. Dockerfileを書く
    58
    ➔ vi Dockerfile
    --------------------------
    FROM chck/container4ml-jupyter:1.0 AS trainer
    FROM python:3.8-slim
    ENV APP_HOME /app
    RUN apt update && apt install -y --no-install-recommends git build-essential \
    && apt clean && rm -rf /var/lib/apt/lists/* \
    && pip install -U pip && pip install --no-cache-dir poetry
    COPY pyproject.toml poetry.lock ./
    RUN poetry export --without-hashes -f requirements.txt -o requirements.txt \
    && pip install -r requirements.txt --no-cache-dir
    WORKDIR /models
    COPY --from=trainer /app/*.bin .
    WORKDIR ${APP_HOME}
    COPY main.py .
    EXPOSE 80
    CMD ["gunicorn", "main:app", "--bind=0.0.0.0:80", "--workers=1", "--threads=8", "--timeout=0",
    "--worker-class=uvicorn.workers.UvicornWorker"]
    

    View Slide

59. 2. docker-compose.ymlを書く
    ➔ vi docker-compose.yml
    --------------------------
    services:
    myapp:
    image: chck/container4ml-fastapi:1.0
    build: .
    ports:
    - "2222:80"
    environment:
    - PYTHONUNBUFFERED=1
    - DEBUG=true
    - REDISHOST=myredis
    - MODEL_PATH=/models/ua_classifier.bin
    volumes:
    - ${PWD}:/app
    depends_on:
    - myredis
    myredis:
    image: redis:alpine 59
    

    View Slide

60. 2. docker-compose.ymlを書く
    ➔ vi docker-compose.yml
    --------------------------
    services:
    myapp:
    image: chck/container4ml-fastapi:1.0
    build: .
    ports:
    - "2222:80"
    environment:
    - PYTHONUNBUFFERED=1
    - DEBUG=true
    - REDISHOST=myredis
    - MODEL_PATH=/models/ua_classifier.bin
    volumes:
    - ${PWD}:/app
    depends_on:
    - myredis
    myredis:
    image: redis:alpine 60
    任意のService名
    任意のService名
    Dockerfileを参照しながら
    指定Image名でdocker build
    指定Imageをdocker pullで用意
    portやvolume, env optionを付与してdocker runされ
    jupyter containerが起動
    myredis containerの起動を待ってから myappを起動
    

    View Slide

61. 2. docker-compose.ymlを書く
    ➔ vi docker-compose.yml
    --------------------------
    services:
    myapp:
    image: chck/container4ml-fastapi:1.0
    build: .
    ports:
    - "2222:80"
    environment:
    - PYTHONUNBUFFERED=1
    - DEBUG=true
    - REDISHOST=myredis
    - MODEL_PATH=/models/ua_classifier.bin
    volumes:
    - ${PWD}:/app
    depends_on:
    - myredis
    myredis:
    image: redis:alpine 61
    w/o docker compose
    ➔ docker network create mynwk
    ➔ docker run -p 2222:80 -e
    PYTHONUNBUFFERED=1 -e DEBUG=true -e
    REDISHOST=myredis -e
    MODEL_PATH=/models/ua_classifier.bin
    -v ${PWD}:/app --net mynwk
    chck/container4ml-fastapi:1.0
    ➔ docker run --net mynwk redis:alpine
    w/ docker compose
    ➔ docker compose up
    Container間通信(mynwk)
    myapp
    myredis
    

    View Slide

62. 3. Docker Containerの起動
    (via docker compose)
    ➔ docker compose up -d
    ➔ docker compose logs -f
    ➔ docker ps
    CONTAINER ID IMAGE ... PORTS ... NAMES
    b36811e9f286 chck/container4ml-fastapi:1.0 ... 0.0.0.0:2222->80/tcp 3-fastapi-myapp-1
    7304fab5c081 redis:alpine ... 6379/tcp 3-fastapi-myredis-1
    ➔ open http://localhost:2222
    ➔ open http://localhost:2222/stats
    ➔ cat main.py
    ➔ docker compose down
    ➔ # docker-compose.ymlのenvironmentsにMODEL_NAME=Bを追加して
    docker compose up --build -d
    62
    

    View Slide

63. 時間が余った時用
    • docker system prune -a
    • 1 Image: 1 Appの理由
    • Docker化が嬉しい場面
    • Dataなどの重たいファイルはvolumeでsyncしておく話
    • ModelをStorageに持つかImageに持つか
    • Docker container のDebug方法
    • 変数によってbuildの振る舞いを変えたい時
    • Training/Servingは同じImageか分けるか
    • Securityの話
    63
    

    View Slide

64. 質問 & 休憩
    hh:mm - hh:mm
    64
    

    View Slide

65. Container Deployment
    ML AppのServingについて
    65
    

    View Slide

66. Why Container Deployment?
    https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/
    66
    

    View Slide

67. https://www.datadoghq.com/docker-adoption/
    67
    

    View Slide

68. https://www.datadoghq.com/container-report-2020/
    68
    

    View Slide

69. Container Orchestration
    • Containerの展開や状態の管理を
    担うService
    • Docker ImageをどうServingする
    かの部分
    • Kubernetesを基とする
    Managed Serviceが複数展開
    https://www.datadoghq.com/container-report-2020/ 69
    

    View Slide

70. DockerとKubernetesの関係
    https://docs.bytemark.co.uk/article/kubernetes-terminology-glossary/
    70
    

    View Slide

71. https://www.docker.com/blog/top-questions-docker-kubernetes-competitors-or-together/docker-kubernetes-together/
    71
    

    View Slide

72. Kubernetes Architecture
    72
    

    View Slide

73. Cloud Run Archirecture
    73
    

    View Slide

74. Cloud Run
    • GCPが提供する
    Managed Container Service
    • k8sの細かい所をwrapしてApp
    のCodingに集中できるようにし
    たもの
    • Managing Level
    ○ GCE < GKE < Cloud Run
    https://www.idkrtm.com/5-kinds-of-compute-on-google-cloud/
    74
    

    View Slide

75. Kubernetes vs Cloud Run
    https://medium.com/google-cloud-jp/cloudrun-buildpacks-110438eccb35
    75
    

    View Slide

76. Story 3-2
    • LocalのDocker Composeで動作確認のできたUA Estimatorを
    Cloud RunにDeployしたい
    • DeployのProcess
    ○ Docker ImageをRegistryにUpload
    ○ Cloud Run Serviceを構築・起動
    ○ Cloud Run ServiceにUploadしたImageをApply
    76
    

    View Slide

77. 1. RegistryにImageをUpload
    ➔ docker tag chck/container4ml-fastapi:1.0 gcr.io/$(GCP_PROJECT)/container4ml/fastapi:1.0
    ➔ docker tag gcr.io/$(GCP_PROJECT)/container4ml/fastapi:1.0
    gcr.io/$(GCP_PROJECT)/container4ml/fastapi:latest
    ➔ docker push gcr.io/$(GCP_PROJECT)/container4ml/fastapi:1.0
    ➔ docker push gcr.io/$(GCP_PROJECT)/container4ml/fastapi:latest
    ➔ open https://console.cloud.google.com/gcr/images
    77
    

    View Slide

78. 2. Cloud Run Serviceを構築・起動
    https://console.cloud.google.com/run
    78
    

    View Slide

79. 3. UploadしたImageを
    Cloud RunにApply
    https://console.cloud.google.com/run/deploy
    79
    

    View Slide

80. Local to Cloud Run
    80
    docker push
    

    View Slide

81. Autoscaling
    https://github.com/rakyll/hey
    WebAppの負荷試験ができるheyでbenchmarkを取ってみる
    ➔ brew install hey
    ➔ sh ./test-scale.sh
    ➔ sh ./test-scale.sh https://${CLOUDRUN_DOMAIN}
    Cloud Runのmetricsやheyのbenchmarkを観察
    81
    

    View Slide

82. Blue-green deployment
    既存のApp (blue) 稼働中の裏に
    新版のApp (green) をTrafficが来ない状態で
    Deployし、Standbyになったタイミングで
    Router内部の向き先をgreenに変えることで
    無停止Deployを行う仕組み
    Cloud RunではRevisionによってこれを達成
    ➔ sh ./test-switch.sh https://${CLOUDRUN_DOMAIN}
    ➔ # Deploy New Revision in Cloud Run
    82
    https://candost.blog/the-blue-green-deployment-strategy/
    

    View Slide

83. Traffic Splitting
    Manage trafficで50%(A):50%(B)に
    Requestを分配してみる
    この仕組みでSimpleなA/B Testも可
    能
    83
    

    View Slide

84. Simple A/A Testing
    同じImageでTrafficを分け、Metricsを計測
    有意差がないことを確認する
    84
    Version:1.0 Version:1.0
    Routing
    50% 50%
    

    View Slide

85. Simple A/B Testing
    Image A とImage BでTrafficを分け、Metricsを計測
    有意差を検証
    85
    Version:1.0 Version:1.1
    Routing
    50% 50%
    

    View Slide

86. Today’s Goal
    • Portableな実験環境
    ○ -> Docker, Docker-Compose
    • Scalableな推論App
    ○ -> Cloud Run
    • まずは研究コードのDocker化から始めてみる
    86
    

    View Slide

87. Thank you!
    質問などあれば 🙌
    or slack: @chck
    88
    

    View Slide

88. 質問 & 休憩
    hh:mm - 17:00
    89
    

    View Slide