Upgrade to Pro — share decks privately, control downloads, hide ads and more …

紙面ビューアーを支える サーバーレスアーキテクチャ / serverless architec...

紙面ビューアーを支える サーバーレスアーキテクチャ / serverless architecture supports Nikkei's paper viewer

Taishi Ikai

October 01, 2016
Tweet

More Decks by Taishi Ikai

Other Decks in Technology

Transcript

  1. 3 ༗ྉձһ ໿48ສਓ ແྉձһ ໿300ສਓ ඇొ࿥ ໿2500ສUB ೔ܦిࢠ൛ͱ͸ 2010 ೥

    3 ݄ʹ૑ץ ݄ؒΞΫηε 3 ԯ݅ ຖ೔໿ 900 ຊͷهࣄΛ഑৴ PCɾϞόΠϧαΠτɺ
 iOS, Android ΞϓϦͳͲʹରԠ
  2. 16 S3 SQS EC2 CloudFront Client ιʔεऔಘ
 ϑΝΠϧ഑ஔ ϦαΠζ ը૾ϑΟϧλʔ

    هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ ϝλσʔλऔಘ WebP ม׵ JPEG ม׵ ιʔε enqueue polling / dequeue ΞΫηε੍ݶ όονॲཧͩͱ͢Δͱ..
  3. 17 ओʹேץɺ༦ץͷ࣌ؒͷՔಇ͕ͩɺࢴ໘ߋ৽͸ෆఆظʹ͋Δ → ਺෼ִؒͰ SQS ΛϙʔϦϯά͢Δ ɾ Πϯελϯε͸ৗʹىಈ͓ͯ͘͠? ɾ εέʔϧΞ΢τ͸एׯ͕͔͔࣌ؒΔ

    ॲཧ͕ଟ͍ɺը૾ม׵(ϦαΠζɺϑΟϧλɺWebPม׵) ͸ॏΊ → ͍ܰॲཧ͸ϚϧνεϨουͰฒྻԽ → Πϯελϯε͸ෳ਺୆ىಈ͢ΔͳͲ ՝୊
  4. 19 S3 SQS EC2 CMS CloudFront Client ιʔεऔಘ
 ϑΝΠϧ഑ஔ ϦαΠζ

    ը૾ϑΟϧλʔ هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ ϝλσʔλऔಘ WebP ม׵ JPEG ม׵ ιʔε enqueue polling / dequeue
  5. 20 S3 SQS CMS CloudFront Client ιʔεऔಘ
 ϑΝΠϧ഑ஔ ϦαΠζ ը૾ϑΟϧλʔ

    هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ ϝλσʔλऔಘ WebP ม׵ JPEG ม׵ ιʔε enqueue polling / dequeue Lambda
  6. 21 S3 CMS CloudFront Client ιʔεऔಘ
 ϑΝΠϧ഑ஔ ϦαΠζ ը૾ϑΟϧλʔ هࣄը૾ۣܗ੾ΓऔΓ

    ࢴ໘ը૾෼ׂ ϝλσʔλऔಘ WebP ม׵ JPEG ม׵ ιʔε Lambda S3 ͔ΒͷΠϕϯτͰ Lambda ͕࣮ߦ͞ΕΔ ͜Μͳײ͡ʁ
  7. 27 ϦαΠζ γϟʔϓɺίϯτϥετ هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ WebP ม׵ JPEG ม׵ Linux

    Ͱ੩తϏϧυͨ͠ cwebp όΠφϦ ΛɺΞοϓϩʔυ͢Δ zip ʹؚΊΔ cwebp Λ node ͔Βѻ͏ͨΊͷ
 ϥούʔϥΠϒϥϦ install cwebp
  8. 30 Ϗϧυ / σϓϩΠ Lambda ςετ Babel ʢes6 → es5ʣ

    ɺιʔεϚοϓ npm Πϯετʔϧ σϓϩΠ (node-aws-lambda) npm: source-map-support Increment Version
  9. 32 Source S3 Dest S3 Order 
 Regenerate Dispatch Scheduled

    
 events Invoke Tasks Login, Get Signed URL Request
 Signed URL API
  10. 33 \ OBNFEJWJEF  TPVSDFYZ[QOH  ˑEFTUJOBUJPOYZ[@QOH EBUB\ SFDUBOHMF\ Y

    Z X I ^ ^ ^ S3 ΁ PUT Λ͏͚ͯɺॲཧ಺༰Λॻ͍ͨϑΝΠϧΛ഑ஔ Dest S3 Dispatch Tasks Source S3
  11. 34 ֤ॲཧ୲౰ͷ Lambda ͕ S3 ͔Β ඞཁͳϑΝΠϧΛऔಘͯ͠ॲཧ Dest S3 Dispatch

    Tasks S3 ΁ PUT Λ͏͚ͯɺॲཧ಺༰Λॻ͍ͨϑΝΠϧΛ഑ஔ Source S3
  12. 35 Dest S3 Dispatch Tasks S3 ΁ PUT Λ͏͚ͯɺॲཧ಺༰Λॻ͍ͨϑΝΠϧΛ഑ஔ ̍ΠϕϯτͰෳ਺ͷLambda

    
 ΛىಈͰ͖ͳ͍ͨΊɺͻͱͭͷ
 Lambda ؔ਺ͰҰ୴ΠϕϯτΛड͚औΓɺ ϑΝΠϧ໊Λϕʔεʹͯ͠ɺଞͷ Lambda Λىಈ (S3ܦ༝)
  13. 38 API Ͱೝূ API ͸ϢʔβʔΛೝՄͯ͠ɺ CloudFront ॺ໊෇͖URL Λൃߦ Private Cache

    Distribution Dest S3 Login, Get Signed URL Request
 Signed URL API
  14. 40 Private Cache Distribution ύλʔϯ CloudFront Λҙࣝ͢Δ͜ͱͳ͘ɺ CloudFront ্ͷίϯςϯπͷऔಘ͕Մೳ ①

    ೝূࡁΈϦΫΤετ ② 303 (ॺ໊෇͖URL΁ͷϦμΠϨΫτ) ②’ ϦΫΤετ ॺ໊෇͖URL ॺ໊෇͖URLͷੜ੒
  15. 45 Source S3 Order 
 Regenerate Dispatch Invoke Slash commands

    Ͱ API Gateway ͔Β Lambda ΛىಈɻΤϥʔ࣌ɺ
 QA ࣌ͳͲ͸ɺखಈͰഔମΛੜ੒ Slack ࿈ܞ
  16. 46 Lambda ͔Β Lambda ΛݺΜͩޙʹ෮ؼͰ͖ͳ͍Τϥʔ͸ݫې ΤϥʔΛى͜͢ͱ࠷௿ 3 ճϦτϥΠ͞ΕΔͷͰ… < 429:

    You are sending too 
 ɹmany requests. Please relax. time dying 
 lambda
  17. 47 API ଆͷ E2E ςετ͕Քಇ ෳ਺ճ࿈ଓࣦͯ͠ഊ͢Δͱ PagerDuty ͔Β API νʔϜʹి࿩

    Lambda ͕ࢮ๢͍ͯ͠Δ৔߹͸ɺखॱʹͦͬͯϩʔΧϧͰॲཧ
  18. 54 ҙ֎ʹࣗ༝౓͸͋Δ OS ͸ Amazon Linux ͳͷͰɺ͜͜Ͱಈ͘΋ͷ͸େମಈ͘ Linux ༻ͷ cwebp

    όΠφϦͰ WebP ม׵΋Ͱ͖ͨ Lambda ͷͨΊͷ։ൃ͕ඞཁʹͳΔ ಛʹɺS3 ͳͲͱ࿈ܞͯ͠ΠϕϯτϕʔεͰॲཧ͢Δ৔߹ ΠϕϯτͷϧʔςΟϯάɺσΟεύονϟ͕ඞཁʹͳΔ৔߹΋ Ϣχοτςετ͸Ͱ͖Δ͕ɺࣗಈͰ݁߹ςετ͕೉͍͠ Lambda ʹ͍ͭͯ
  19. 55 ࡉ͔͍πʔϧΛ࡞Δʹ͸͔ͳΓศར Slack ͷ Slash commands ͳͲ (API Gateway ͱͷ૊Έ߹Θͤ)

    ͪΌΜͱͰ͖͍ͯΔ ΠϕϯτΛ࿙Βͨ͠Γ͠ͳ͍ ͔ͳΓ͍҆ FaaS ͷະདྷΛײ͡Δ Lambda ʹ͍ͭͯ
  20. 56 ؆୯ʹ࢖͑Δ PaaS ج൫ͷ४උ͕͋ͬͨ Elastic Beanstalk + Docker + Python

    + Django ೝূඞਢɺ༗ྉձһݶఆͷαʔϏεͳͷͰɺݎ࿚Ͱ։ൃମ੍͕ ཱ͍֬ͯ͠Δํ͕Α͔ͬͨ API Gateway + Lambda ͸؅ཧͷίετ͕͔͔Γͦ͏ɺϨΠςϯ γʔ͕େ͖͍ PaaS?
  21. 59 Lambda + S3 + CloudWatch Ͱࢴ໘ը૾ͷม׵ S3 + CloudFront

    Ͱը૾ͷ഑৴: Private cache distribution Lambda Ͱ WebP ͷม׵΍ը૾ॲཧΛߦͬͨ ݁Ռͷ଴ͪ߹ΘͤΛ͢ΔͨΊʹ ɺCloudWatch Ͱ cron తʹ Lambda Λ࣮ߦ ҰͭͷΠϕϯτͰෳ਺ͷ Lambda Λىಈ͢Δ Workaround Slack, Backlog, PagerDuty ͳͲͷ SaaS ΋׆༻ αʔόʔϨεͰखͷ͔͔Βͳ͍ΠϯϑϥΛߴ଎ʹߏஙͰ͖ͨ