紙面ビューアーを支える サーバーレスアーキテクチャ / serverless architecture supports Nikkei's paper viewer

紙面ビューアーを支える サーバーレスアーキテクチャ / serverless architecture supports Nikkei's paper viewer

86b45faad28dfccd89b4ec04ef00c62e?s=128

Taishi Ikai

October 01, 2016
Tweet

Transcript

  1. ࢴ໘ϏϡʔΞʔΛࢧ͑Δ αʔόʔϨεΞʔΩςΫνϟ ழࣂ େࢤ ೔ຊܦࡁ৽ฉࣾ Serverless Conf Tokyo 2016

  2. 2 About me ழࣂ େࢤ (Ikai Taishi) ೔ຊܦࡁ৽ฉࣾͷΤϯδχΞ ࢴ໘ϏϡʔΞʔͷ։ൃΛ୲౰ Swift,

    Python, JavaScript Github: @ikait
  3. 3 ༗ྉձһ ໿48ສਓ ແྉձһ ໿300ສਓ ඇొ࿥ ໿2500ສUB ೔ܦిࢠ൛ͱ͸ 2010 ೥

    3 ݄ʹ૑ץ ݄ؒΞΫηε 3 ԯ݅ ຖ೔໿ 900 ຊͷهࣄΛ഑৴ PCɾϞόΠϧαΠτɺ
 iOS, Android ΞϓϦͳͲʹରԠ
  4. 4 ࢴ໘ϏϡʔΞʔʁ

  5. 5 ೔ܦిࢠ൛ ࢴ໘ϏϡʔΞʔ

  6. 6 ೔ຊܦࡁ৽ฉΛɺ
 εϚϗŋλϒϨοτͰಡΉ iOSŋAndroid ରԠ 2016.3.15 ϦχϡʔΞϧ

  7. 7 ேŋ༦ץ ͕ಡΊΔ ຖேɺຖ༦഑৴͞Ε·͢ (ٳץ೔͸আ͘) աڈ1िؒ෼ಡΊ·͢ ΦϓγϣϯͰɺ೔ܦ࢈ۀ৽ฉɺ ೔ܦ MJ ͕ಡΊ·͢

  8. 8 ৽ฉͱಉ͡ϨΠΞ΢τ ొ࿥ͨ͠ΩʔϫʔυΛ
 ϋΠϥΠτͰ͖·͢ ݕࡧͰ͖·͢ μ΢ϯϩʔυ͓ͯ͘͠ͱ
 ΦϑϥΠϯͰಡΊ·͢

  9. 9 ࢴ໘ϏϡʔΞʔ ಺੡͍ͯ͠·͢ iOSɺAPIɺը૾ੜ੒ͳͲ Ϟμϯͳ։ൃϑϩʔʢ❗ʣ ϚωʔδυɾαʔϏεΛ ੵۃతʹར༻͍ͯ͘͠

  10. 10 https://twitter.com/naoya_ito/status/779241231029186561

  11. 11 https://github.com/serverless/serverless

  12. 12 API (ೝূܥ) Ͱ͸ɺ͜ͷ Microservice ࣮ߦ؀ڥΛར༻͍ͯ͠·͢ https://speakerdeck.com/yosukesuzuki/nikkei-microservice

  13. ࢴ໘ϏϡʔΞʔΛࢧ͑Δ αʔόʔϨεΞʔΩςΫνϟ ழࣂ େࢤ ೔ຊܦࡁ৽ฉࣾ Serverless Conf Tokyo 2016 ͱϚΠΫϩαʔϏεΞʔΩςΫνϟ

  14. 14 ࢴ໘ը૾͕ΞϓϦʹಧ͘·Ͱ

  15. 15 ޻৔ S3 ฤू લఏ هࣄ
 ϨΠΞ΢τ ιʔε: ࢴ໘ը૾ ϝλσʔλ

    XML ࠲ඪͳͲ
  16. 16 S3 SQS EC2 CloudFront Client ιʔεऔಘ
 ϑΝΠϧ഑ஔ ϦαΠζ ը૾ϑΟϧλʔ

    هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ ϝλσʔλऔಘ WebP ม׵ JPEG ม׵ ιʔε enqueue polling / dequeue ΞΫηε੍ݶ όονॲཧͩͱ͢Δͱ..
  17. 17 ओʹேץɺ༦ץͷ࣌ؒͷՔಇ͕ͩɺࢴ໘ߋ৽͸ෆఆظʹ͋Δ → ਺෼ִؒͰ SQS ΛϙʔϦϯά͢Δ ɾ Πϯελϯε͸ৗʹىಈ͓ͯ͘͠? ɾ εέʔϧΞ΢τ͸एׯ͕͔͔࣌ؒΔ

    ॲཧ͕ଟ͍ɺը૾ม׵(ϦαΠζɺϑΟϧλɺWebPม׵) ͸ॏΊ → ͍ܰॲཧ͸ϚϧνεϨουͰฒྻԽ → Πϯελϯε͸ෳ਺୆ىಈ͢ΔͳͲ ՝୊
  18. 18 Lambda ͕ ʢ౦ژϦʔδϣϯʹʣ ͖ͨ 2015/06/29

  19. 19 S3 SQS EC2 CMS CloudFront Client ιʔεऔಘ
 ϑΝΠϧ഑ஔ ϦαΠζ

    ը૾ϑΟϧλʔ هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ ϝλσʔλऔಘ WebP ม׵ JPEG ม׵ ιʔε enqueue polling / dequeue
  20. 20 S3 SQS CMS CloudFront Client ιʔεऔಘ
 ϑΝΠϧ഑ஔ ϦαΠζ ը૾ϑΟϧλʔ

    هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ ϝλσʔλऔಘ WebP ม׵ JPEG ม׵ ιʔε enqueue polling / dequeue Lambda
  21. 21 S3 CMS CloudFront Client ιʔεऔಘ
 ϑΝΠϧ഑ஔ ϦαΠζ ը૾ϑΟϧλʔ هࣄը૾ۣܗ੾ΓऔΓ

    ࢴ໘ը૾෼ׂ ϝλσʔλऔಘ WebP ม׵ JPEG ม׵ ιʔε Lambda S3 ͔ΒͷΠϕϯτͰ Lambda ͕࣮ߦ͞ΕΔ ͜Μͳײ͡ʁ
  22. 22 ࡞Γ௚ͨ͠

  23. 23 Lambda ؔ਺ͷ։ൃ

  24. 24 ϦαΠζ γϟʔϓɺίϯτϥετ هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ WebP ม׵ JPEG ม׵ ϝλσʔλม׵

    / औಘ Lambda ্Ͱߦ͏͜ͱ v0.10.36
  25. 25 4000px 5296px

  26. 26 ϦαΠζ γϟʔϓɺίϯτϥετ هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ WebP ม׵ JPEG ม׵ Lambda

    ্ͷ ImageMagick Λ Ͱར༻
  27. 27 ϦαΠζ γϟʔϓɺίϯτϥετ هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ WebP ม׵ JPEG ม׵ Linux

    Ͱ੩తϏϧυͨ͠ cwebp όΠφϦ ΛɺΞοϓϩʔυ͢Δ zip ʹؚΊΔ cwebp Λ node ͔Βѻ͏ͨΊͷ
 ϥούʔϥΠϒϥϦ install cwebp
  28. 28 όΠφϦͷύεΛίʔυ಺Ͱ؀ڥม਺ʹ௥Ճ JG GTFYJTUT4ZOD VTSMPDBMCJODXFCQ \ DPOTU-*#@%*3QBUIKPJO @@EJSOBNF MJC 

    QSPDFTTFOW1"5) A\-*#@%*3^A QSPDFTTFOW-%@-*#3"3:@1"5) A\-*#@%*3^A ^
  29. 29 ϩʔΧϧͰϢχοτςετ

  30. 30 Ϗϧυ / σϓϩΠ Lambda ςετ Babel ʢes6 → es5ʣ

    ɺιʔεϚοϓ npm Πϯετʔϧ σϓϩΠ (node-aws-lambda) npm: source-map-support Increment Version
  31. 31 ಈ࡞ϑϩʔ (ུ֓)

  32. 32 Source S3 Dest S3 Order 
 Regenerate Dispatch Scheduled

    
 events Invoke Tasks Login, Get Signed URL Request
 Signed URL API
  33. 33 \ OBNFEJWJEF  TPVSDFYZ[QOH  ˑEFTUJOBUJPOYZ[@QOH EBUB\ SFDUBOHMF\ Y

    Z X I ^ ^ ^ S3 ΁ PUT Λ͏͚ͯɺॲཧ಺༰Λॻ͍ͨϑΝΠϧΛ഑ஔ Dest S3 Dispatch Tasks Source S3
  34. 34 ֤ॲཧ୲౰ͷ Lambda ͕ S3 ͔Β ඞཁͳϑΝΠϧΛऔಘͯ͠ॲཧ Dest S3 Dispatch

    Tasks S3 ΁ PUT Λ͏͚ͯɺॲཧ಺༰Λॻ͍ͨϑΝΠϧΛ഑ஔ Source S3
  35. 35 Dest S3 Dispatch Tasks S3 ΁ PUT Λ͏͚ͯɺॲཧ಺༰Λॻ͍ͨϑΝΠϧΛ഑ஔ ̍ΠϕϯτͰෳ਺ͷLambda

    
 ΛىಈͰ͖ͳ͍ͨΊɺͻͱͭͷ
 Lambda ؔ਺ͰҰ୴ΠϕϯτΛड͚औΓɺ ϑΝΠϧ໊Λϕʔεʹͯ͠ɺଞͷ Lambda Λىಈ (S3ܦ༝)
  36. 36 Lambda ͔Βىಈ͢Δ৔߹ɺεϩοτϧ͢Δͱ ࣦഊ͢Δ͔΋͠Εͳ͍ͷͰ S3 Λܦ༝͢Δ Dest S3 Dispatch Scheduled

    
 events Tasks
  37. 37 1 ෼͓͖ʹىಈ 
 ϑΝΠϧͷଘࡏ֬ೝΛߦ͍
 ഔମϦετΛߋ৽͢Δ ഔମϦετͷଘࡏΛ΋ͬͯɺ ഔମͷ४උ׬ྃΛࣔ͢ Dest S3

    Scheduled 
 events
  38. 38 API Ͱೝূ API ͸ϢʔβʔΛೝՄͯ͠ɺ CloudFront ॺ໊෇͖URL Λൃߦ Private Cache

    Distribution Dest S3 Login, Get Signed URL Request
 Signed URL API
  39. 39 Private Cache Distribution ύλʔϯ ① ೝূࡁΈϦΫΤετ GET https://api/sign ②

    ॺ໊ ③ CF ΁ॺ໊Λ͚ͭͯ
 ϦΫΤετ
  40. 40 Private Cache Distribution ύλʔϯ CloudFront Λҙࣝ͢Δ͜ͱͳ͘ɺ CloudFront ্ͷίϯςϯπͷऔಘ͕Մೳ ①

    ೝূࡁΈϦΫΤετ ② 303 (ॺ໊෇͖URL΁ͷϦμΠϨΫτ) ②’ ϦΫΤετ ॺ໊෇͖URL ॺ໊෇͖URLͷੜ੒
  41. 41 ӡ༻

  42. 42 ேץલ͸ 18000 invocations/m

  43. 43 Lambda ༝དྷͷΤϥʔ͸Ίͬͨʹى͖ͳ͍

  44. 44 Slack ࿈ܞ Քಇঢ়گΛ௨஌ Τϥʔͷᮢ஋Λ௒͑ͨͱ͖Λ௨஌ Notify Log Log Alarm (when

    exceeded threshold) Notify
  45. 45 Source S3 Order 
 Regenerate Dispatch Invoke Slash commands

    Ͱ API Gateway ͔Β Lambda ΛىಈɻΤϥʔ࣌ɺ
 QA ࣌ͳͲ͸ɺखಈͰഔମΛੜ੒ Slack ࿈ܞ
  46. 46 Lambda ͔Β Lambda ΛݺΜͩޙʹ෮ؼͰ͖ͳ͍Τϥʔ͸ݫې ΤϥʔΛى͜͢ͱ࠷௿ 3 ճϦτϥΠ͞ΕΔͷͰ… < 429:

    You are sending too 
 ɹmany requests. Please relax. time dying 
 lambda
  47. 47 API ଆͷ E2E ςετ͕Քಇ ෳ਺ճ࿈ଓࣦͯ͠ഊ͢Δͱ PagerDuty ͔Β API νʔϜʹి࿩

    Lambda ͕ࢮ๢͍ͯ͠Δ৔߹͸ɺखॱʹͦͬͯϩʔΧϧͰॲཧ
  48. 48 CloudFront ͕ࢮ๢ͨ͠ࡍ͸υϝΠϯͷ޲͖ઌΛ S3 ΁

  49. 49 S3 ʹॻ͖ࠐ·ΕΔ CloudFront ͷϩάΛ Lambda Ͱύʔεͯ͠ Elasticsearch ΁౤ೖɺ kibana

    ͰՄࢹԽ Log S3 Parse
  50. 50 Ϣʔβʔͷ੠Λड͚औͬͯɺSlack ͱ Backlog ΁౤ߘ API Gateway ܦ༝ͷ Lambda ىಈͷ


    ϨΠςϯγʔ͸ؾʹͳΔ
  51. 51 ৼΓฦΓ

  52. 52 ։ൃ଎౓͸ɺ޲্ʁ Lambda ͷͨΊͷ։ൃ͸ඞཁͩͬͨ ΞϓϦʹԠͨ͡σʔλଆͷॊೈͳมߋ͕Մೳͩͬͨ αʔόʔෛՙΛؾʹ͠ͳ͍ͰΑ͘ͳͬͨʢը૾ੜ੒ɺ഑৴ʣ ։ൃମ੍ͷԸܙʢݱ৔Ͱٕज़తͳҙࢥܾఆ͕͋Δ͜ͱʣ ӡ༻ίετ͸ݮͬͨʢ΄΅ͳ͍ʣ

  53. 53 αʔϏεɾঢ়گʹΑͬͯ͸ɺ୅ସΛҙࣝ͢Δඞཁ͸͋Γͦ͏ Lambda ؔ਺͸ɺLambda Ҏ֎ͷͲ͜Ͱಈ͔ͤͦ͏ʁ S3, CloudFront ͕མͪͨ࣌͸ʁ πʔϧ͕ग़ἧͬͯɺݴޠͷબ୒ࢶ΋૿͑ͨ Serverless,

    Apex, Lamvery ͳͲ ͲͷαʔϏε΋҆ఆ͍ͯ͠Δ ࠓ͸҆৺ͯ͠৸ΒΕΔ
  54. 54 ҙ֎ʹࣗ༝౓͸͋Δ OS ͸ Amazon Linux ͳͷͰɺ͜͜Ͱಈ͘΋ͷ͸େମಈ͘ Linux ༻ͷ cwebp

    όΠφϦͰ WebP ม׵΋Ͱ͖ͨ Lambda ͷͨΊͷ։ൃ͕ඞཁʹͳΔ ಛʹɺS3 ͳͲͱ࿈ܞͯ͠ΠϕϯτϕʔεͰॲཧ͢Δ৔߹ ΠϕϯτͷϧʔςΟϯάɺσΟεύονϟ͕ඞཁʹͳΔ৔߹΋ Ϣχοτςετ͸Ͱ͖Δ͕ɺࣗಈͰ݁߹ςετ͕೉͍͠ Lambda ʹ͍ͭͯ
  55. 55 ࡉ͔͍πʔϧΛ࡞Δʹ͸͔ͳΓศར Slack ͷ Slash commands ͳͲ (API Gateway ͱͷ૊Έ߹Θͤ)

    ͪΌΜͱͰ͖͍ͯΔ ΠϕϯτΛ࿙Βͨ͠Γ͠ͳ͍ ͔ͳΓ͍҆ FaaS ͷະདྷΛײ͡Δ Lambda ʹ͍ͭͯ
  56. 56 ؆୯ʹ࢖͑Δ PaaS ج൫ͷ४උ͕͋ͬͨ Elastic Beanstalk + Docker + Python

    + Django ೝূඞਢɺ༗ྉձһݶఆͷαʔϏεͳͷͰɺݎ࿚Ͱ։ൃମ੍͕ ཱ͍֬ͯ͠Δํ͕Α͔ͬͨ API Gateway + Lambda ͸؅ཧͷίετ͕͔͔Γͦ͏ɺϨΠςϯ γʔ͕େ͖͍ PaaS?
  57. 57 εςʔτϨεΛҙࣝͤ͟ΔΛಘͳ͍ ෭࡞༻΍ঢ়ଶอ࣋ʹཔΕͳ͍࣮૷্ͷ੍໿ ີ݁߹ͳίʔυ͕ࣗવʹݮΓɺݟ௨͕͠෇͖΍͘͢ͳΔ ϑϩϯτΤϯυͷ։ൃʹ΋ੜ͖ͨ αʔόʔϨεͰͷ։ൃΛ௨ͯ͡

  58. 58 ·ͱΊ

  59. 59 Lambda + S3 + CloudWatch Ͱࢴ໘ը૾ͷม׵ S3 + CloudFront

    Ͱը૾ͷ഑৴: Private cache distribution Lambda Ͱ WebP ͷม׵΍ը૾ॲཧΛߦͬͨ ݁Ռͷ଴ͪ߹ΘͤΛ͢ΔͨΊʹ ɺCloudWatch Ͱ cron తʹ Lambda Λ࣮ߦ ҰͭͷΠϕϯτͰෳ਺ͷ Lambda Λىಈ͢Δ Workaround Slack, Backlog, PagerDuty ͳͲͷ SaaS ΋׆༻ αʔόʔϨεͰखͷ͔͔Βͳ͍ΠϯϑϥΛߴ଎ʹߏஙͰ͖ͨ
  60. ࢴ໘ϏϡʔΞʔΛࢧ͑Δ αʔόʔϨεΞʔΩςΫνϟ ழࣂ େࢤ ೔ຊܦࡁ৽ฉࣾ Serverless Conf Tokyo 2016