ML/DSプロダクトにはServerless(Framework) / should use serverless framework for ml-ds product

Transcript

1. ML/DS
   Serverless (Framework)⚡
   Ikeda Yutaro @ikedaosushi
   Kaizen Platform
   

   View Slide

2. Ikeda Yutaro @ikedaosushi
   #
   #
   Kaizen Platform
   /
   #
   #vue #nuxt #serverless #aws
   #python #pyconjp #tsukuba.py
   #bq # rebase
   

   View Slide

3. https://blog.ikedaosushi.com
   

   View Slide

4. ML/DS Serverless
   Architecture
   ?
   ⚡
   

   View Slide

5. ML/DS Serverless
   Architecture
   ⚡
   

   View Slide

6. ML/DS
   

   View Slide

7. ML/DS
   Serverless (kube ow )
   ML/DS "
   Serverless
   AWS
   Public Cloud
   

   View Slide

8. ML/DS
   2
   

   View Slide

9. ML/DS 2
   

   View Slide

10. ML/DS /
    :
    1 1
    50GB
    

    View Slide

11. :
    MVP
    

    View Slide

12. /
    SRE
    

    View Slide

13. Serverless⚡
    

    View Slide

14. Serverless
    AWS Lambda - Hub
    https://employment.en-japan.com/engineerhub/entry/2018/07/03/110000
    ( )
    

    View Slide

15. Serverless
    cncf/wg-serverless
    https://github.com/cncf/wg-serverless/tree/master/whitepapers/serverless-overview
    

    View Slide

16. Serverless
    Asynchronous, concurrent, easy to parallelize into independent units
    of work
    Infrequent or has sporadic demand, with large, unpredictable
    variance in scaling requirements
    Stateless, ephemeral, without a major need for instantaneous cold
    start time
    Highly dynamic in terms of changing business requirements that
    drive a need for accelerated developer velocity
    cncf/wg-serverless
    https://github.com/cncf/wg-serverless/tree/master/whitepapers/serverless-overview
    

    View Slide

17. Serverless
    Asynchronous, concurrent, easy to parallelize into independent units
    of work
    Infrequent or has sporadic demand, with large, unpredictable
    variance in scaling requirements
    Stateless, ephemeral, without a major need for instantaneous cold
    start time
    Highly dynamic in terms of changing business requirements that
    drive a need for accelerated developer velocity
    cncf/wg-serverless
    https://github.com/cncf/wg-serverless/tree/master/whitepapers/serverless-overview
    

    View Slide

18. ML/DS Serverless
    Architecture
    ⚡
    

    View Slide

19. Serverless
    

    View Slide

20. Serverless
    API
    

    View Slide

21. Serverless
    API
    

    View Slide

22. 3
    Serverless Architectures
    https://martinfowler.com/articles/serverless.html#InherentDrawbacks
    

    View Slide

23. ML/DS
    BaaS
    DevOps
    

    View Slide

24. BaaS
    Fargate
    SagaMaker Batch
    Personalize
    Forecast
    Rekognition DynamoDB Aurora
    S3
    (ARN)
    

    View Slide

25. Jupyter Notebook
    DevOps
    DevOps
    

    View Slide

26. ⚡
    Serverless Framework
    

    View Slide

27. Serverless Framework
    Serverless
    AWS GCP Azure Public Cloud
    (AWS ) CloudFormation
    

    View Slide

28. Serverless Framework
    CloudFormation
    Lambda
    DevOps
    

    View Slide

29. 'BSHBUF
    4
    4BHB.BLFS
    ϩʔσʔλ
    લॲཧͯ͠Ξοϓϩʔυ
    Ճ޻ࡁσʔλ
    ֶशˍσϓϩΠ
    'BSHBUFݺͼग़͠ $MPVE8BUDI
    ఆظ࣮ߦ
    ਪ࿦
    : Sagemaker
    -BNCEB
    

    View Slide

30. Serverless Framework
    CloudFormation
    Lambda
    DevOps
    

    View Slide

31. custom
    

    View Slide

32. resources(Cfn)
    

    View Slide

33. Lambda
    

    View Slide

34. 'BSHBUF
    4
    4BHB.BLFS
    ϩʔσʔλ
    લॲཧͯ͠Ξοϓϩʔυ
    Ճ޻ࡁσʔλ
    ֶशˍσϓϩΠ
    'BSHBUFݺͼग़͠ $MPVE8BUDI
    ఆظ࣮ߦ
    ਪ࿦
    Lambda
    -BNCEB
    custom Readabilityほ
    

    View Slide

35. SageMaker Serverless scikit-learn API -
    https://blog.ikedaosushi.com/entry/2019/05/03/024510
    

    View Slide

36. Serverless Framework
    CloudFormation
    Lambda
    DevOps
    

    View Slide

37. stage
    

    View Slide

38. Serverless Framework - Qiita
    https://qiita.com/horike37/items/b295a91908fcfd4033a2
    

    View Slide

39. Serverless Framework
    BaaS
    DevOps
    

    View Slide

40. ML/DS
    

    View Slide

41. ML/DS /
    Serverless Architecture ML/DS
    ML/DS on Serverless Architecture
    Serverless Framework
    horike
    

    View Slide

42. ML/DS
    Serverless (Framework)⚡
    
    

    View Slide