Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ML/DSプロダクトにはServerless(Framework) / should use serverless framework for ml-ds product

ikedaosushi
September 19, 2019

ML/DSプロダクトにはServerless(Framework) / should use serverless framework for ml-ds product

ikedaosushi

September 19, 2019
Tweet

More Decks by ikedaosushi

Other Decks in Technology

Transcript

  1. ML/DS
    Serverless (Framework)⚡
    Ikeda Yutaro @ikedaosushi
    Kaizen Platform

    View Slide

  2. Ikeda Yutaro @ikedaosushi
    #
    #
    Kaizen Platform
    /
    #
    #vue #nuxt #serverless #aws
    #python #pyconjp #tsukuba.py
    #bq # rebase

    View Slide

  3. https://blog.ikedaosushi.com

    View Slide

  4. ML/DS Serverless
    Architecture
    ?

    View Slide

  5. ML/DS Serverless
    Architecture

    View Slide

  6. ML/DS

    View Slide

  7. ML/DS
    Serverless (kube ow )
    ML/DS "
    Serverless
    AWS
    Public Cloud

    View Slide

  8. ML/DS
    2

    View Slide

  9. ML/DS 2

    View Slide

  10. ML/DS /
    :
    1 1
    50GB

    View Slide

  11. :
    MVP

    View Slide

  12. /
    SRE

    View Slide

  13. Serverless⚡

    View Slide

  14. Serverless
    AWS Lambda - Hub
    https://employment.en-japan.com/engineerhub/entry/2018/07/03/110000
    ( )

    View Slide

  15. Serverless
    cncf/wg-serverless
    https://github.com/cncf/wg-serverless/tree/master/whitepapers/serverless-overview

    View Slide

  16. Serverless
    Asynchronous, concurrent, easy to parallelize into independent units
    of work
    Infrequent or has sporadic demand, with large, unpredictable
    variance in scaling requirements
    Stateless, ephemeral, without a major need for instantaneous cold
    start time
    Highly dynamic in terms of changing business requirements that
    drive a need for accelerated developer velocity
    cncf/wg-serverless
    https://github.com/cncf/wg-serverless/tree/master/whitepapers/serverless-overview

    View Slide

  17. Serverless
    Asynchronous, concurrent, easy to parallelize into independent units
    of work
    Infrequent or has sporadic demand, with large, unpredictable
    variance in scaling requirements
    Stateless, ephemeral, without a major need for instantaneous cold
    start time
    Highly dynamic in terms of changing business requirements that
    drive a need for accelerated developer velocity
    cncf/wg-serverless
    https://github.com/cncf/wg-serverless/tree/master/whitepapers/serverless-overview

    View Slide

  18. ML/DS Serverless
    Architecture

    View Slide

  19. Serverless

    View Slide

  20. Serverless
    API

    View Slide

  21. Serverless
    API

    View Slide

  22. 3
    Serverless Architectures
    https://martinfowler.com/articles/serverless.html#InherentDrawbacks

    View Slide

  23. ML/DS
    BaaS
    DevOps

    View Slide

  24. BaaS
    Fargate
    SagaMaker Batch
    Personalize
    Forecast
    Rekognition DynamoDB Aurora
    S3
    (ARN)

    View Slide

  25. Jupyter Notebook
    DevOps
    DevOps

    View Slide


  26. Serverless Framework

    View Slide

  27. Serverless Framework
    Serverless
    AWS GCP Azure Public Cloud
    (AWS ) CloudFormation

    View Slide

  28. Serverless Framework
    CloudFormation
    Lambda
    DevOps

    View Slide

  29. 'BSHBUF
    4
    4BHB.BLFS
    ϩʔσʔλ
    લॲཧͯ͠Ξοϓϩʔυ
    Ճ޻ࡁσʔλ
    ֶशˍσϓϩΠ
    'BSHBUFݺͼग़͠ $MPVE8BUDI
    ఆظ࣮ߦ
    ਪ࿦
    : Sagemaker
    -BNCEB

    View Slide

  30. Serverless Framework
    CloudFormation
    Lambda
    DevOps

    View Slide

  31. custom

    View Slide

  32. resources(Cfn)

    View Slide

  33. Lambda

    View Slide

  34. 'BSHBUF
    4
    4BHB.BLFS
    ϩʔσʔλ
    લॲཧͯ͠Ξοϓϩʔυ
    Ճ޻ࡁσʔλ
    ֶशˍσϓϩΠ
    'BSHBUFݺͼग़͠ $MPVE8BUDI
    ఆظ࣮ߦ
    ਪ࿦
    Lambda
    -BNCEB
    custom Readabilityほ

    View Slide

  35. SageMaker Serverless scikit-learn API -
    https://blog.ikedaosushi.com/entry/2019/05/03/024510

    View Slide

  36. Serverless Framework
    CloudFormation
    Lambda
    DevOps

    View Slide

  37. stage

    View Slide

  38. Serverless Framework - Qiita
    https://qiita.com/horike37/items/b295a91908fcfd4033a2

    View Slide

  39. Serverless Framework
    BaaS
    DevOps

    View Slide

  40. ML/DS

    View Slide

  41. ML/DS /
    Serverless Architecture ML/DS
    ML/DS on Serverless Architecture
    Serverless Framework
    horike

    View Slide

  42. ML/DS
    Serverless (Framework)⚡

    View Slide