紙面ビューアーを支える サーバーレスアーキテクチャ / serverless architecture supports Nikkei's paper viewer

Transcript

1. ࢴ໘ϏϡʔΞʔΛࢧ͑Δ
   αʔόʔϨεΞʔΩςΫνϟ
   ழࣂ େࢤ ೔ຊܦࡁ৽ฉࣾ
   Serverless Conf Tokyo 2016
   

   View Slide

2. 2
   About me
   ழࣂ େࢤ (Ikai Taishi)
   ೔ຊܦࡁ৽ฉࣾͷΤϯδχΞ
   ࢴ໘ϏϡʔΞʔͷ։ൃΛ୲౰
   Swift, Python, JavaScript Github: @ikait
   

   View Slide

3. 3
   ༗ྉձһ ໿48ສਓ
   ແྉձһ ໿300ສਓ
   ඇొ࿥ ໿2500ສUB
   ೔ܦిࢠ൛ͱ͸
   2010 ೥ 3 ݄ʹ૑ץ
   ݄ؒΞΫηε 3 ԯ݅
   ຖ೔໿ 900 ຊͷهࣄΛ഑৴
   PCɾϞόΠϧαΠτɺ

   iOS, Android ΞϓϦͳͲʹରԠ
   

   View Slide

4. 4
   ࢴ໘ϏϡʔΞʔʁ
   

   View Slide

5. 5
   ೔ܦిࢠ൛ ࢴ໘ϏϡʔΞʔ
   

   View Slide

6. 6
   ೔ຊܦࡁ৽ฉΛɺ

   εϚϗŋλϒϨοτͰಡΉ
   iOSŋAndroid ରԠ
   2016.3.15 ϦχϡʔΞϧ
   

   View Slide

7. 7
   ேŋ༦ץ ͕ಡΊΔ
   ຖேɺຖ༦഑৴͞Ε·͢
   (ٳץ೔͸আ͘)
   աڈ1िؒ෼ಡΊ·͢
   ΦϓγϣϯͰɺ೔ܦ࢈ۀ৽ฉɺ
   ೔ܦ MJ ͕ಡΊ·͢
   

   View Slide

8. 8
   ৽ฉͱಉ͡ϨΠΞ΢τ
   ొ࿥ͨ͠ΩʔϫʔυΛ

   ϋΠϥΠτͰ͖·͢
   ݕࡧͰ͖·͢
   μ΢ϯϩʔυ͓ͯ͘͠ͱ

   ΦϑϥΠϯͰಡΊ·͢
   

   View Slide

9. 9
   ࢴ໘ϏϡʔΞʔ
   ಺੡͍ͯ͠·͢
   iOSɺAPIɺը૾ੜ੒ͳͲ
   Ϟμϯͳ։ൃϑϩʔʢ❗ʣ
   ϚωʔδυɾαʔϏεΛ
   ੵۃతʹར༻͍ͯ͘͠
   

   View Slide

10. 10
    https://twitter.com/naoya_ito/status/779241231029186561
    

    View Slide

11. 11
    https://github.com/serverless/serverless
    

    View Slide

12. 12
    API (ೝূܥ) Ͱ͸ɺ͜ͷ Microservice ࣮ߦ؀ڥΛར༻͍ͯ͠·͢
    https://speakerdeck.com/yosukesuzuki/nikkei-microservice
    

    View Slide

13. ࢴ໘ϏϡʔΞʔΛࢧ͑Δ
    αʔόʔϨεΞʔΩςΫνϟ
    ழࣂ େࢤ ೔ຊܦࡁ৽ฉࣾ
    Serverless Conf Tokyo 2016
    ͱϚΠΫϩαʔϏεΞʔΩςΫνϟ
    

    View Slide

14. 14
    ࢴ໘ը૾͕ΞϓϦʹಧ͘·Ͱ
    

    View Slide

15. 15
    ޻৔
    S3
    ฤू
    લఏ
    هࣄ

    ϨΠΞ΢τ
    ιʔε:
    ࢴ໘ը૾
    ϝλσʔλ XML
    ࠲ඪͳͲ
    

    View Slide

16. 16
    S3
    SQS EC2
    CloudFront Client
    ιʔεऔಘ

    ϑΝΠϧ഑ஔ
    ϦαΠζ
    ը૾ϑΟϧλʔ
    هࣄը૾ۣܗ੾ΓऔΓ
    ࢴ໘ը૾෼ׂ
    ϝλσʔλऔಘ
    WebP ม׵
    JPEG ม׵
    ιʔε
    enqueue
    polling / dequeue
    ΞΫηε੍ݶ
    όονॲཧͩͱ͢Δͱ..
    

    View Slide

17. 17
    ओʹேץɺ༦ץͷ࣌ؒͷՔಇ͕ͩɺࢴ໘ߋ৽͸ෆఆظʹ͋Δ
    → ਺෼ִؒͰ SQS ΛϙʔϦϯά͢Δ
    ɾ Πϯελϯε͸ৗʹىಈ͓ͯ͘͠?
    ɾ εέʔϧΞ΢τ͸एׯ͕͔͔࣌ؒΔ
    ॲཧ͕ଟ͍ɺը૾ม׵(ϦαΠζɺϑΟϧλɺWebPม׵) ͸ॏΊ
    → ͍ܰॲཧ͸ϚϧνεϨουͰฒྻԽ
    → Πϯελϯε͸ෳ਺୆ىಈ͢ΔͳͲ
    ՝୊
    

    View Slide

18. 18
    Lambda ͕
    ʢ౦ژϦʔδϣϯʹʣ
    ͖ͨ
    2015/06/29
    

    View Slide

19. 19
    S3
    SQS EC2
    CMS
    CloudFront Client
    ιʔεऔಘ

    ϑΝΠϧ഑ஔ
    ϦαΠζ
    ը૾ϑΟϧλʔ
    هࣄը૾ۣܗ੾ΓऔΓ
    ࢴ໘ը૾෼ׂ
    ϝλσʔλऔಘ
    WebP ม׵
    JPEG ม׵
    ιʔε
    enqueue
    polling / dequeue
    

    View Slide

20. 20
    S3
    SQS
    CMS
    CloudFront Client
    ιʔεऔಘ

    ϑΝΠϧ഑ஔ
    ϦαΠζ
    ը૾ϑΟϧλʔ
    هࣄը૾ۣܗ੾ΓऔΓ
    ࢴ໘ը૾෼ׂ
    ϝλσʔλऔಘ
    WebP ม׵
    JPEG ม׵
    ιʔε
    enqueue
    polling / dequeue
    Lambda
    

    View Slide

21. 21
    S3
    CMS
    CloudFront Client
    ιʔεऔಘ

    ϑΝΠϧ഑ஔ
    ϦαΠζ
    ը૾ϑΟϧλʔ
    هࣄը૾ۣܗ੾ΓऔΓ
    ࢴ໘ը૾෼ׂ
    ϝλσʔλऔಘ
    WebP ม׵
    JPEG ม׵
    ιʔε
    Lambda
    S3 ͔ΒͷΠϕϯτͰ
    Lambda ͕࣮ߦ͞ΕΔ
    ͜Μͳײ͡ʁ
    

    View Slide

22. 22
    ࡞Γ௚ͨ͠
    

    View Slide

23. 23
    Lambda ؔ਺ͷ։ൃ
    

    View Slide

24. 24
    ϦαΠζ
    γϟʔϓɺίϯτϥετ
    هࣄը૾ۣܗ੾ΓऔΓ
    ࢴ໘ը૾෼ׂ
    WebP ม׵
    JPEG ม׵
    ϝλσʔλม׵ / औಘ
    Lambda ্Ͱߦ͏͜ͱ
    v0.10.36
    

    View Slide

25. 25
    4000px
    5296px
    

    View Slide

26. 26
    ϦαΠζ
    γϟʔϓɺίϯτϥετ
    هࣄը૾ۣܗ੾ΓऔΓ
    ࢴ໘ը૾෼ׂ
    WebP ม׵
    JPEG ม׵
    Lambda ্ͷ ImageMagick Λ
    Ͱར༻
    

    View Slide

27. 27
    ϦαΠζ
    γϟʔϓɺίϯτϥετ
    هࣄը૾ۣܗ੾ΓऔΓ
    ࢴ໘ը૾෼ׂ
    WebP ม׵
    JPEG ม׵
    Linux Ͱ੩తϏϧυͨ͠ cwebp όΠφϦ
    ΛɺΞοϓϩʔυ͢Δ zip ʹؚΊΔ
    cwebp Λ node ͔Βѻ͏ͨΊͷ

    ϥούʔϥΠϒϥϦ
    install cwebp
    

    View Slide

28. 28
    όΠφϦͷύεΛίʔυ಺Ͱ؀ڥม਺ʹ௥Ճ
    JG GTFYJTUT4ZOD VTSMPDBMCJODXFCQ
    
    \
    DPOTU-*#@%*3QBUIKPJO @@EJSOBNF MJC
    
    QSPDFTTFOW1"5)A\-*#@%*3^A
    QSPDFTTFOW-%@-*#3"3:@1"5)A\-*#@%*3^A
    ^
    

    View Slide

29. 29
    ϩʔΧϧͰϢχοτςετ
    

    View Slide

30. 30
    Ϗϧυ / σϓϩΠ
    Lambda
    ςετ
    Babel
    ʢes6 → es5ʣ
    ɺιʔεϚοϓ
    npm Πϯετʔϧ
    σϓϩΠ (node-aws-lambda)
    npm: source-map-support
    Increment
    Version
    

    View Slide

31. 31
    ಈ࡞ϑϩʔ (ུ֓)
    

    View Slide

32. 32
    Source S3 Dest S3
    Order 

    Regenerate
    Dispatch
    Scheduled 

    events
    Invoke
    Tasks
    Login,
    Get
    Signed
    URL
    Request

    Signed
    URL
    API
    

    View Slide

33. 33
    \
    OBNFEJWJEF
    TPVSDFYZ[QOH
    ˑEFTUJOBUJPOYZ[@QOH
    EBUB\
    SFDUBOHMF\
    Y Z X I
    ^
    ^
    ^
    S3 ΁ PUT Λ͏͚ͯɺॲཧ಺༰Λॻ͍ͨϑΝΠϧΛ഑ஔ
    Dest S3
    Dispatch
    Tasks
    Source S3
    

    View Slide

34. 34
    ֤ॲཧ୲౰ͷ Lambda ͕ S3 ͔Β
    ඞཁͳϑΝΠϧΛऔಘͯ͠ॲཧ
    Dest S3
    Dispatch
    Tasks
    S3 ΁ PUT Λ͏͚ͯɺॲཧ಺༰Λॻ͍ͨϑΝΠϧΛ഑ஔ
    Source S3
    

    View Slide

35. 35
    Dest S3
    Dispatch
    Tasks
    S3 ΁ PUT Λ͏͚ͯɺॲཧ಺༰Λॻ͍ͨϑΝΠϧΛ഑ஔ
    ̍ΠϕϯτͰෳ਺ͷLambda 

    ΛىಈͰ͖ͳ͍ͨΊɺͻͱͭͷ

    Lambda ؔ਺ͰҰ୴ΠϕϯτΛड͚औΓɺ
    ϑΝΠϧ໊Λϕʔεʹͯ͠ɺଞͷ Lambda Λىಈ (S3ܦ༝)
    

    View Slide

36. 36
    Lambda ͔Βىಈ͢Δ৔߹ɺεϩοτϧ͢Δͱ
    ࣦഊ͢Δ͔΋͠Εͳ͍ͷͰ S3 Λܦ༝͢Δ
    Dest S3
    Dispatch
    Scheduled 

    events
    Tasks
    

    View Slide

37. 37
    1 ෼͓͖ʹىಈ
    

    ϑΝΠϧͷଘࡏ֬ೝΛߦ͍

    ഔମϦετΛߋ৽͢Δ
    ഔମϦετͷଘࡏΛ΋ͬͯɺ
    ഔମͷ४උ׬ྃΛࣔ͢
    Dest S3
    Scheduled 

    events
    

    View Slide

38. 38
    API Ͱೝূ
    API ͸ϢʔβʔΛೝՄͯ͠ɺ
    CloudFront ॺ໊෇͖URL Λൃߦ
    Private Cache Distribution
    Dest S3
    Login,
    Get
    Signed
    URL
    Request

    Signed
    URL
    API
    

    View Slide

39. 39
    Private Cache Distribution ύλʔϯ
    ① ೝূࡁΈϦΫΤετ GET
    https://api/sign
    ② ॺ໊
    ③ CF ΁ॺ໊Λ͚ͭͯ

    ϦΫΤετ
    

    View Slide

40. 40
    Private Cache Distribution ύλʔϯ
    CloudFront Λҙࣝ͢Δ͜ͱͳ͘ɺ
    CloudFront ্ͷίϯςϯπͷऔಘ͕Մೳ
    ① ೝূࡁΈϦΫΤετ
    ② 303 (ॺ໊෇͖URL΁ͷϦμΠϨΫτ)
    ②’ ϦΫΤετ
    ॺ໊෇͖URL
    ॺ໊෇͖URLͷੜ੒
    

    View Slide

41. 41
    ӡ༻
    

    View Slide

42. 42
    ேץલ͸ 18000 invocations/m
    

    View Slide

43. 43
    Lambda ༝དྷͷΤϥʔ͸Ίͬͨʹى͖ͳ͍
    

    View Slide

44. 44
    Slack ࿈ܞ
    Քಇঢ়گΛ௨஌
    Τϥʔͷᮢ஋Λ௒͑ͨͱ͖Λ௨஌
    Notify
    Log
    Log
    Alarm
    (when exceeded threshold)
    Notify
    

    View Slide

45. 45
    Source S3
    Order 

    Regenerate
    Dispatch
    Invoke
    Slash commands Ͱ API Gateway
    ͔Β Lambda ΛىಈɻΤϥʔ࣌ɺ

    QA ࣌ͳͲ͸ɺखಈͰഔମΛੜ੒
    Slack ࿈ܞ
    

    View Slide

46. 46
    Lambda ͔Β Lambda ΛݺΜͩޙʹ෮ؼͰ͖ͳ͍Τϥʔ͸ݫې
    ΤϥʔΛى͜͢ͱ࠷௿ 3 ճϦτϥΠ͞ΕΔͷͰ…
    < 429: You are sending too 

    ɹmany requests. Please relax.
    time
    dying 

    lambda
    

    View Slide

47. 47
    API ଆͷ E2E ςετ͕Քಇ
    ෳ਺ճ࿈ଓࣦͯ͠ഊ͢Δͱ PagerDuty ͔Β API νʔϜʹి࿩
    Lambda ͕ࢮ๢͍ͯ͠Δ৔߹͸ɺखॱʹͦͬͯϩʔΧϧͰॲཧ
    

    View Slide

48. 48
    CloudFront ͕ࢮ๢ͨ͠ࡍ͸υϝΠϯͷ޲͖ઌΛ S3 ΁
    

    View Slide

49. 49
    S3 ʹॻ͖ࠐ·ΕΔ CloudFront ͷϩάΛ
    Lambda Ͱύʔεͯ͠ Elasticsearch ΁౤ೖɺ
    kibana ͰՄࢹԽ
    Log S3
    Parse
    

    View Slide

50. 50
    Ϣʔβʔͷ੠Λड͚औͬͯɺSlack ͱ Backlog ΁౤ߘ
    API Gateway ܦ༝ͷ Lambda ىಈͷ

    ϨΠςϯγʔ͸ؾʹͳΔ
    

    View Slide

51. 51
    ৼΓฦΓ
    

    View Slide

52. 52
    ։ൃ଎౓͸ɺ޲্ʁ
    Lambda ͷͨΊͷ։ൃ͸ඞཁͩͬͨ
    ΞϓϦʹԠͨ͡σʔλଆͷॊೈͳมߋ͕Մೳͩͬͨ
    αʔόʔෛՙΛؾʹ͠ͳ͍ͰΑ͘ͳͬͨʢը૾ੜ੒ɺ഑৴ʣ
    ։ൃମ੍ͷԸܙʢݱ৔Ͱٕज़తͳҙࢥܾఆ͕͋Δ͜ͱʣ
    ӡ༻ίετ͸ݮͬͨʢ΄΅ͳ͍ʣ
    

    View Slide

53. 53
    αʔϏεɾঢ়گʹΑͬͯ͸ɺ୅ସΛҙࣝ͢Δඞཁ͸͋Γͦ͏
    Lambda ؔ਺͸ɺLambda Ҏ֎ͷͲ͜Ͱಈ͔ͤͦ͏ʁ
    S3, CloudFront ͕མͪͨ࣌͸ʁ
    πʔϧ͕ग़ἧͬͯɺݴޠͷબ୒ࢶ΋૿͑ͨ
    Serverless, Apex, Lamvery ͳͲ
    ͲͷαʔϏε΋҆ఆ͍ͯ͠Δ
    ࠓ͸҆৺ͯ͠৸ΒΕΔ
    

    View Slide

54. 54
    ҙ֎ʹࣗ༝౓͸͋Δ
    OS ͸ Amazon Linux ͳͷͰɺ͜͜Ͱಈ͘΋ͷ͸େମಈ͘
    Linux ༻ͷ cwebp όΠφϦͰ WebP ม׵΋Ͱ͖ͨ
    Lambda ͷͨΊͷ։ൃ͕ඞཁʹͳΔ
    ಛʹɺS3 ͳͲͱ࿈ܞͯ͠ΠϕϯτϕʔεͰॲཧ͢Δ৔߹
    ΠϕϯτͷϧʔςΟϯάɺσΟεύονϟ͕ඞཁʹͳΔ৔߹΋
    Ϣχοτςετ͸Ͱ͖Δ͕ɺࣗಈͰ݁߹ςετ͕೉͍͠
    Lambda ʹ͍ͭͯ
    

    View Slide

55. 55
    ࡉ͔͍πʔϧΛ࡞Δʹ͸͔ͳΓศར
    Slack ͷ Slash commands ͳͲ (API Gateway ͱͷ૊Έ߹Θͤ)
    ͪΌΜͱͰ͖͍ͯΔ
    ΠϕϯτΛ࿙Βͨ͠Γ͠ͳ͍
    ͔ͳΓ͍҆
    FaaS ͷະདྷΛײ͡Δ
    Lambda ʹ͍ͭͯ
    

    View Slide

56. 56
    ؆୯ʹ࢖͑Δ PaaS ج൫ͷ४උ͕͋ͬͨ
    Elastic Beanstalk + Docker + Python + Django
    ೝূඞਢɺ༗ྉձһݶఆͷαʔϏεͳͷͰɺݎ࿚Ͱ։ൃମ੍͕
    ཱ͍֬ͯ͠Δํ͕Α͔ͬͨ
    API Gateway + Lambda ͸؅ཧͷίετ͕͔͔Γͦ͏ɺϨΠςϯ
    γʔ͕େ͖͍
    PaaS?
    

    View Slide

57. 57
    εςʔτϨεΛҙࣝͤ͟ΔΛಘͳ͍
    ෭࡞༻΍ঢ়ଶอ࣋ʹཔΕͳ͍࣮૷্ͷ੍໿
    ີ݁߹ͳίʔυ͕ࣗવʹݮΓɺݟ௨͕͠෇͖΍͘͢ͳΔ
    ϑϩϯτΤϯυͷ։ൃʹ΋ੜ͖ͨ
    αʔόʔϨεͰͷ։ൃΛ௨ͯ͡
    

    View Slide

58. 58
    ·ͱΊ
    

    View Slide

59. 59
    Lambda + S3 + CloudWatch Ͱࢴ໘ը૾ͷม׵
    S3 + CloudFront Ͱը૾ͷ഑৴: Private cache distribution
    Lambda Ͱ WebP ͷม׵΍ը૾ॲཧΛߦͬͨ
    ݁Ռͷ଴ͪ߹ΘͤΛ͢ΔͨΊʹ ɺCloudWatch Ͱ cron తʹ
    Lambda Λ࣮ߦ
    ҰͭͷΠϕϯτͰෳ਺ͷ Lambda Λىಈ͢Δ Workaround
    Slack, Backlog, PagerDuty ͳͲͷ SaaS ΋׆༻
    αʔόʔϨεͰखͷ͔͔Βͳ͍ΠϯϑϥΛߴ଎ʹߏஙͰ͖ͨ
    

    View Slide

60. ࢴ໘ϏϡʔΞʔΛࢧ͑Δ
    αʔόʔϨεΞʔΩςΫνϟ
    ழࣂ େࢤ ೔ຊܦࡁ৽ฉࣾ
    Serverless Conf Tokyo 2016
    

    View Slide