紙面ビューアーを支える サーバーレスアーキテクチャ / serverless architecture supports Nikkei's paper viewer

Transcript

1. ࢴ໘ϏϡʔΞʔΛࢧ͑Δ αʔόʔϨεΞʔΩςΫνϟ ழࣂ େࢤ ೔ຊܦࡁ৽ฉࣾ Serverless Conf Tokyo 2016

2. 2 About me ழࣂ େࢤ (Ikai Taishi) ೔ຊܦࡁ৽ฉࣾͷΤϯδχΞ ࢴ໘ϏϡʔΞʔͷ։ൃΛ୲౰ Swift,

   Python, JavaScript Github: @ikait

3. 3 ༗ྉձһ ໿48ສਓ ແྉձһ ໿300ສਓ ඇొ࿥ ໿2500ສUB ೔ܦిࢠ൛ͱ͸ 2010 ೥

   3 ݄ʹ૑ץ ݄ؒΞΫηε 3 ԯ݅ ຖ೔໿ 900 ຊͷهࣄΛ഑৴ PCɾϞόΠϧαΠτɺ
 iOS, Android ΞϓϦͳͲʹରԠ

4. 4 ࢴ໘ϏϡʔΞʔʁ

5. 5 ೔ܦిࢠ൛ ࢴ໘ϏϡʔΞʔ

6. 6 ೔ຊܦࡁ৽ฉΛɺ
 εϚϗŋλϒϨοτͰಡΉ iOSŋAndroid ରԠ 2016.3.15 ϦχϡʔΞϧ

7. 7 ேŋ༦ץ ͕ಡΊΔ ຖேɺຖ༦഑৴͞Ε·͢ (ٳץ೔͸আ͘) աڈ1िؒ෼ಡΊ·͢ ΦϓγϣϯͰɺ೔ܦ࢈ۀ৽ฉɺ ೔ܦ MJ ͕ಡΊ·͢

8. 8 ৽ฉͱಉ͡ϨΠΞ΢τ ొ࿥ͨ͠ΩʔϫʔυΛ
 ϋΠϥΠτͰ͖·͢ ݕࡧͰ͖·͢ μ΢ϯϩʔυ͓ͯ͘͠ͱ
 ΦϑϥΠϯͰಡΊ·͢

9. 9 ࢴ໘ϏϡʔΞʔ ಺੡͍ͯ͠·͢ iOSɺAPIɺը૾ੜ੒ͳͲ Ϟμϯͳ։ൃϑϩʔʢ❗ʣ ϚωʔδυɾαʔϏεΛ ੵۃతʹར༻͍ͯ͘͠

10. 10 https://twitter.com/naoya_ito/status/779241231029186561

11. 11 https://github.com/serverless/serverless

12. 12 API (ೝূܥ) Ͱ͸ɺ͜ͷ Microservice ࣮ߦ؀ڥΛར༻͍ͯ͠·͢ https://speakerdeck.com/yosukesuzuki/nikkei-microservice

13. ࢴ໘ϏϡʔΞʔΛࢧ͑Δ αʔόʔϨεΞʔΩςΫνϟ ழࣂ େࢤ ೔ຊܦࡁ৽ฉࣾ Serverless Conf Tokyo 2016 ͱϚΠΫϩαʔϏεΞʔΩςΫνϟ

14. 14 ࢴ໘ը૾͕ΞϓϦʹಧ͘·Ͱ

15. 15 ޻৔ S3 ฤू લఏ هࣄ
 ϨΠΞ΢τ ιʔε: ࢴ໘ը૾ ϝλσʔλ

    XML ࠲ඪͳͲ

16. 16 S3 SQS EC2 CloudFront Client ιʔεऔಘ
 ϑΝΠϧ഑ஔ ϦαΠζ ը૾ϑΟϧλʔ

    هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ ϝλσʔλऔಘ WebP ม׵ JPEG ม׵ ιʔε enqueue polling / dequeue ΞΫηε੍ݶ όονॲཧͩͱ͢Δͱ..

17. 17 ओʹேץɺ༦ץͷ࣌ؒͷՔಇ͕ͩɺࢴ໘ߋ৽͸ෆఆظʹ͋Δ → ਺෼ִؒͰ SQS ΛϙʔϦϯά͢Δ ɾ Πϯελϯε͸ৗʹىಈ͓ͯ͘͠? ɾ εέʔϧΞ΢τ͸एׯ͕͔͔࣌ؒΔ

    ॲཧ͕ଟ͍ɺը૾ม׵(ϦαΠζɺϑΟϧλɺWebPม׵) ͸ॏΊ → ͍ܰॲཧ͸ϚϧνεϨουͰฒྻԽ → Πϯελϯε͸ෳ਺୆ىಈ͢ΔͳͲ ՝୊

18. 18 Lambda ͕ ʢ౦ژϦʔδϣϯʹʣ ͖ͨ 2015/06/29

19. 19 S3 SQS EC2 CMS CloudFront Client ιʔεऔಘ
 ϑΝΠϧ഑ஔ ϦαΠζ

    ը૾ϑΟϧλʔ هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ ϝλσʔλऔಘ WebP ม׵ JPEG ม׵ ιʔε enqueue polling / dequeue

20. 20 S3 SQS CMS CloudFront Client ιʔεऔಘ
 ϑΝΠϧ഑ஔ ϦαΠζ ը૾ϑΟϧλʔ

    هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ ϝλσʔλऔಘ WebP ม׵ JPEG ม׵ ιʔε enqueue polling / dequeue Lambda

21. 21 S3 CMS CloudFront Client ιʔεऔಘ
 ϑΝΠϧ഑ஔ ϦαΠζ ը૾ϑΟϧλʔ هࣄը૾ۣܗ੾ΓऔΓ

    ࢴ໘ը૾෼ׂ ϝλσʔλऔಘ WebP ม׵ JPEG ม׵ ιʔε Lambda S3 ͔ΒͷΠϕϯτͰ Lambda ͕࣮ߦ͞ΕΔ ͜Μͳײ͡ʁ

22. 22 ࡞Γ௚ͨ͠

23. 23 Lambda ؔ਺ͷ։ൃ

24. 24 ϦαΠζ γϟʔϓɺίϯτϥετ هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ WebP ม׵ JPEG ม׵ ϝλσʔλม׵

    / औಘ Lambda ্Ͱߦ͏͜ͱ v0.10.36

25. 25 4000px 5296px

26. 26 ϦαΠζ γϟʔϓɺίϯτϥετ هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ WebP ม׵ JPEG ม׵ Lambda

    ্ͷ ImageMagick Λ Ͱར༻

27. 27 ϦαΠζ γϟʔϓɺίϯτϥετ هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ WebP ม׵ JPEG ม׵ Linux

    Ͱ੩తϏϧυͨ͠ cwebp όΠφϦ ΛɺΞοϓϩʔυ͢Δ zip ʹؚΊΔ cwebp Λ node ͔Βѻ͏ͨΊͷ
 ϥούʔϥΠϒϥϦ install cwebp

28. 28 όΠφϦͷύεΛίʔυ಺Ͱ؀ڥม਺ʹ௥Ճ JG
    GTFYJTUT4ZOD VTSMPDBMCJODXFCQ
    \
    
    
    
    
    DPOTU
    -*#@%*3
    
    QBUIKPJO @@EJSOBNF
    MJC 
    

    
    
    
    
    QSPDFTTFOW1"5)
    
    A\-*#@%*3^A
    
    
    
    
    QSPDFTTFOW-%@-*#3"3:@1"5)
    
    A\-*#@%*3^A
    ^

29. 29 ϩʔΧϧͰϢχοτςετ

30. 30 Ϗϧυ / σϓϩΠ Lambda ςετ Babel ʢes6 → es5ʣ

    ɺιʔεϚοϓ npm Πϯετʔϧ σϓϩΠ (node-aws-lambda) npm: source-map-support Increment Version

31. 31 ಈ࡞ϑϩʔ (ུ֓)

32. 32 Source S3 Dest S3 Order 
 Regenerate Dispatch Scheduled

    
 events Invoke Tasks Login, Get Signed URL Request
 Signed URL API

33. 33 \
    
    
    OBNF
    EJWJEF
    
    
    TPVSDF
    YZ[QOH
    
    
    ˑEFTUJOBUJPO
    YZ[@QOH
    
    
    EBUB
    \
    
    
    
    
    SFDUBOHMF
    \
    
    
    
    
    
    
    Y

    
    Z
    X
    I
    
    
    
    
    ^
    
    
    ^
    ^ S3 ΁ PUT Λ͏͚ͯɺॲཧ಺༰Λॻ͍ͨϑΝΠϧΛ഑ஔ Dest S3 Dispatch Tasks Source S3

34. 34 ֤ॲཧ୲౰ͷ Lambda ͕ S3 ͔Β ඞཁͳϑΝΠϧΛऔಘͯ͠ॲཧ Dest S3 Dispatch

    Tasks S3 ΁ PUT Λ͏͚ͯɺॲཧ಺༰Λॻ͍ͨϑΝΠϧΛ഑ஔ Source S3

35. 35 Dest S3 Dispatch Tasks S3 ΁ PUT Λ͏͚ͯɺॲཧ಺༰Λॻ͍ͨϑΝΠϧΛ഑ஔ ̍ΠϕϯτͰෳ਺ͷLambda

    
 ΛىಈͰ͖ͳ͍ͨΊɺͻͱͭͷ
 Lambda ؔ਺ͰҰ୴ΠϕϯτΛड͚औΓɺ ϑΝΠϧ໊Λϕʔεʹͯ͠ɺଞͷ Lambda Λىಈ (S3ܦ༝)

36. 36 Lambda ͔Βىಈ͢Δ৔߹ɺεϩοτϧ͢Δͱ ࣦഊ͢Δ͔΋͠Εͳ͍ͷͰ S3 Λܦ༝͢Δ Dest S3 Dispatch Scheduled

    
 events Tasks

37. 37 1 ෼͓͖ʹىಈ 
 ϑΝΠϧͷଘࡏ֬ೝΛߦ͍
 ഔମϦετΛߋ৽͢Δ ഔମϦετͷଘࡏΛ΋ͬͯɺ ഔମͷ४උ׬ྃΛࣔ͢ Dest S3

    Scheduled 
 events

38. 38 API Ͱೝূ API ͸ϢʔβʔΛೝՄͯ͠ɺ CloudFront ॺ໊෇͖URL Λൃߦ Private Cache

    Distribution Dest S3 Login, Get Signed URL Request
 Signed URL API

39. 39 Private Cache Distribution ύλʔϯ ① ೝূࡁΈϦΫΤετ GET https://api/sign ②

    ॺ໊ ③ CF ΁ॺ໊Λ͚ͭͯ
 ϦΫΤετ

40. 40 Private Cache Distribution ύλʔϯ CloudFront Λҙࣝ͢Δ͜ͱͳ͘ɺ CloudFront ্ͷίϯςϯπͷऔಘ͕Մೳ ①

    ೝূࡁΈϦΫΤετ ② 303 (ॺ໊෇͖URL΁ͷϦμΠϨΫτ) ②’ ϦΫΤετ ॺ໊෇͖URL ॺ໊෇͖URLͷੜ੒

41. 41 ӡ༻

42. 42 ேץલ͸ 18000 invocations/m

43. 43 Lambda ༝དྷͷΤϥʔ͸Ίͬͨʹى͖ͳ͍

44. 44 Slack ࿈ܞ Քಇঢ়گΛ௨஌ Τϥʔͷᮢ஋Λ௒͑ͨͱ͖Λ௨஌ Notify Log Log Alarm (when

    exceeded threshold) Notify

45. 45 Source S3 Order 
 Regenerate Dispatch Invoke Slash commands

    Ͱ API Gateway ͔Β Lambda ΛىಈɻΤϥʔ࣌ɺ
 QA ࣌ͳͲ͸ɺखಈͰഔମΛੜ੒ Slack ࿈ܞ

46. 46 Lambda ͔Β Lambda ΛݺΜͩޙʹ෮ؼͰ͖ͳ͍Τϥʔ͸ݫې ΤϥʔΛى͜͢ͱ࠷௿ 3 ճϦτϥΠ͞ΕΔͷͰ… < 429:

    You are sending too 
 ɹmany requests. Please relax. time dying 
 lambda

47. 47 API ଆͷ E2E ςετ͕Քಇ ෳ਺ճ࿈ଓࣦͯ͠ഊ͢Δͱ PagerDuty ͔Β API νʔϜʹి࿩

    Lambda ͕ࢮ๢͍ͯ͠Δ৔߹͸ɺखॱʹͦͬͯϩʔΧϧͰॲཧ

48. 48 CloudFront ͕ࢮ๢ͨ͠ࡍ͸υϝΠϯͷ޲͖ઌΛ S3 ΁

49. 49 S3 ʹॻ͖ࠐ·ΕΔ CloudFront ͷϩάΛ Lambda Ͱύʔεͯ͠ Elasticsearch ΁౤ೖɺ kibana

    ͰՄࢹԽ Log S3 Parse

50. 50 Ϣʔβʔͷ੠Λड͚औͬͯɺSlack ͱ Backlog ΁౤ߘ API Gateway ܦ༝ͷ Lambda ىಈͷ


    ϨΠςϯγʔ͸ؾʹͳΔ

51. 51 ৼΓฦΓ

52. 52 ։ൃ଎౓͸ɺ޲্ʁ Lambda ͷͨΊͷ։ൃ͸ඞཁͩͬͨ ΞϓϦʹԠͨ͡σʔλଆͷॊೈͳมߋ͕Մೳͩͬͨ αʔόʔෛՙΛؾʹ͠ͳ͍ͰΑ͘ͳͬͨʢը૾ੜ੒ɺ഑৴ʣ ։ൃମ੍ͷԸܙʢݱ৔Ͱٕज़తͳҙࢥܾఆ͕͋Δ͜ͱʣ ӡ༻ίετ͸ݮͬͨʢ΄΅ͳ͍ʣ

53. 53 αʔϏεɾঢ়گʹΑͬͯ͸ɺ୅ସΛҙࣝ͢Δඞཁ͸͋Γͦ͏ Lambda ؔ਺͸ɺLambda Ҏ֎ͷͲ͜Ͱಈ͔ͤͦ͏ʁ S3, CloudFront ͕མͪͨ࣌͸ʁ πʔϧ͕ग़ἧͬͯɺݴޠͷબ୒ࢶ΋૿͑ͨ Serverless,

    Apex, Lamvery ͳͲ ͲͷαʔϏε΋҆ఆ͍ͯ͠Δ ࠓ͸҆৺ͯ͠৸ΒΕΔ

54. 54 ҙ֎ʹࣗ༝౓͸͋Δ OS ͸ Amazon Linux ͳͷͰɺ͜͜Ͱಈ͘΋ͷ͸େମಈ͘ Linux ༻ͷ cwebp

    όΠφϦͰ WebP ม׵΋Ͱ͖ͨ Lambda ͷͨΊͷ։ൃ͕ඞཁʹͳΔ ಛʹɺS3 ͳͲͱ࿈ܞͯ͠ΠϕϯτϕʔεͰॲཧ͢Δ৔߹ ΠϕϯτͷϧʔςΟϯάɺσΟεύονϟ͕ඞཁʹͳΔ৔߹΋ Ϣχοτςετ͸Ͱ͖Δ͕ɺࣗಈͰ݁߹ςετ͕೉͍͠ Lambda ʹ͍ͭͯ

55. 55 ࡉ͔͍πʔϧΛ࡞Δʹ͸͔ͳΓศར Slack ͷ Slash commands ͳͲ (API Gateway ͱͷ૊Έ߹Θͤ)

    ͪΌΜͱͰ͖͍ͯΔ ΠϕϯτΛ࿙Βͨ͠Γ͠ͳ͍ ͔ͳΓ͍҆ FaaS ͷະདྷΛײ͡Δ Lambda ʹ͍ͭͯ

56. 56 ؆୯ʹ࢖͑Δ PaaS ج൫ͷ४උ͕͋ͬͨ Elastic Beanstalk + Docker + Python

    + Django ೝূඞਢɺ༗ྉձһݶఆͷαʔϏεͳͷͰɺݎ࿚Ͱ։ൃମ੍͕ ཱ͍֬ͯ͠Δํ͕Α͔ͬͨ API Gateway + Lambda ͸؅ཧͷίετ͕͔͔Γͦ͏ɺϨΠςϯ γʔ͕େ͖͍ PaaS?

57. 57 εςʔτϨεΛҙࣝͤ͟ΔΛಘͳ͍ ෭࡞༻΍ঢ়ଶอ࣋ʹཔΕͳ͍࣮૷্ͷ੍໿ ີ݁߹ͳίʔυ͕ࣗવʹݮΓɺݟ௨͕͠෇͖΍͘͢ͳΔ ϑϩϯτΤϯυͷ։ൃʹ΋ੜ͖ͨ αʔόʔϨεͰͷ։ൃΛ௨ͯ͡

58. 58 ·ͱΊ

59. 59 Lambda + S3 + CloudWatch Ͱࢴ໘ը૾ͷม׵ S3 + CloudFront

    Ͱը૾ͷ഑৴: Private cache distribution Lambda Ͱ WebP ͷม׵΍ը૾ॲཧΛߦͬͨ ݁Ռͷ଴ͪ߹ΘͤΛ͢ΔͨΊʹ ɺCloudWatch Ͱ cron తʹ Lambda Λ࣮ߦ ҰͭͷΠϕϯτͰෳ਺ͷ Lambda Λىಈ͢Δ Workaround Slack, Backlog, PagerDuty ͳͲͷ SaaS ΋׆༻ αʔόʔϨεͰखͷ͔͔Βͳ͍ΠϯϑϥΛߴ଎ʹߏஙͰ͖ͨ

60. ࢴ໘ϏϡʔΞʔΛࢧ͑Δ αʔόʔϨεΞʔΩςΫνϟ ழࣂ େࢤ ೔ຊܦࡁ৽ฉࣾ Serverless Conf Tokyo 2016