紙面ビューアーを支える サーバーレスアーキテクチャ / serverless architecture supports Nikkei's paper viewer

Transcript

1. ࢴ໘ϏϡʔΞʔΛࢧ͑Δ αʔόʔϨεΞʔΩςΫνϟ ழࣂ େࢤ ೔ຊܦࡁ৽ฉࣾ Serverless Conf Tokyo 2016
2. 2 About me ழࣂ େࢤ (Ikai Taishi) ೔ຊܦࡁ৽ฉࣾͷΤϯδχΞ ࢴ໘ϏϡʔΞʔͷ։ൃΛ୲౰ Swift, Python, JavaScript Github: @ikait
3. 3 ༗ྉձһ ໿48ສਓ ແྉձһ ໿300ສਓ ඇొ࿥ ໿2500ສUB ೔ܦిࢠ൛ͱ͸ 2010 ೥ 3 ݄ʹ૑ץ ݄ؒΞΫηε 3 ԯ݅ ຖ೔໿ 900 ຊͷهࣄΛ഑৴ PCɾϞόΠϧαΠτɺ
 iOS, Android ΞϓϦͳͲʹରԠ
4. 4 ࢴ໘ϏϡʔΞʔʁ
5. 5 ೔ܦిࢠ൛ ࢴ໘ϏϡʔΞʔ
6. 6 ೔ຊܦࡁ৽ฉΛɺ
 εϚϗŋλϒϨοτͰಡΉ iOSŋAndroid ରԠ 2016.3.15 ϦχϡʔΞϧ
7. 7 ேŋ༦ץ ͕ಡΊΔ ຖேɺຖ༦഑৴͞Ε·͢ (ٳץ೔͸আ͘) աڈ1िؒ෼ಡΊ·͢ ΦϓγϣϯͰɺ೔ܦ࢈ۀ৽ฉɺ ೔ܦ MJ ͕ಡΊ·͢
8. 8 ৽ฉͱಉ͡ϨΠΞ΢τ ొ࿥ͨ͠ΩʔϫʔυΛ
 ϋΠϥΠτͰ͖·͢ ݕࡧͰ͖·͢ μ΢ϯϩʔυ͓ͯ͘͠ͱ
 ΦϑϥΠϯͰಡΊ·͢
9. 9 ࢴ໘ϏϡʔΞʔ ಺੡͍ͯ͠·͢ iOSɺAPIɺը૾ੜ੒ͳͲ Ϟμϯͳ։ൃϑϩʔʢ❗ʣ ϚωʔδυɾαʔϏεΛ ੵۃతʹར༻͍ͯ͘͠
10. 10 https://twitter.com/naoya_ito/status/779241231029186561
11. 11 https://github.com/serverless/serverless
12. 12 API (ೝূܥ) Ͱ͸ɺ͜ͷ Microservice ࣮ߦ؀ڥΛར༻͍ͯ͠·͢ https://speakerdeck.com/yosukesuzuki/nikkei-microservice
13. ࢴ໘ϏϡʔΞʔΛࢧ͑Δ αʔόʔϨεΞʔΩςΫνϟ ழࣂ େࢤ ೔ຊܦࡁ৽ฉࣾ Serverless Conf Tokyo 2016 ͱϚΠΫϩαʔϏεΞʔΩςΫνϟ
14. 14 ࢴ໘ը૾͕ΞϓϦʹಧ͘·Ͱ
15. 15 ޻৔ S3 ฤू લఏ هࣄ
 ϨΠΞ΢τ ιʔε: ࢴ໘ը૾ ϝλσʔλ XML ࠲ඪͳͲ
16. 16 S3 SQS EC2 CloudFront Client ιʔεऔಘ
 ϑΝΠϧ഑ஔ ϦαΠζ ը૾ϑΟϧλʔ هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ ϝλσʔλऔಘ WebP ม׵ JPEG ม׵ ιʔε enqueue polling / dequeue ΞΫηε੍ݶ όονॲཧͩͱ͢Δͱ..
17. 17 ओʹேץɺ༦ץͷ࣌ؒͷՔಇ͕ͩɺࢴ໘ߋ৽͸ෆఆظʹ͋Δ → ਺෼ִؒͰ SQS ΛϙʔϦϯά͢Δ ɾ Πϯελϯε͸ৗʹىಈ͓ͯ͘͠? ɾ εέʔϧΞ΢τ͸एׯ͕͔͔࣌ؒΔ ॲཧ͕ଟ͍ɺը૾ม׵(ϦαΠζɺϑΟϧλɺWebPม׵) ͸ॏΊ → ͍ܰॲཧ͸ϚϧνεϨουͰฒྻԽ → Πϯελϯε͸ෳ਺୆ىಈ͢ΔͳͲ ՝୊
18. 18 Lambda ͕ ʢ౦ژϦʔδϣϯʹʣ ͖ͨ 2015/06/29
19. 19 S3 SQS EC2 CMS CloudFront Client ιʔεऔಘ
 ϑΝΠϧ഑ஔ ϦαΠζ ը૾ϑΟϧλʔ هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ ϝλσʔλऔಘ WebP ม׵ JPEG ม׵ ιʔε enqueue polling / dequeue
20. 20 S3 SQS CMS CloudFront Client ιʔεऔಘ
 ϑΝΠϧ഑ஔ ϦαΠζ ը૾ϑΟϧλʔ هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ ϝλσʔλऔಘ WebP ม׵ JPEG ม׵ ιʔε enqueue polling / dequeue Lambda
21. 21 S3 CMS CloudFront Client ιʔεऔಘ
 ϑΝΠϧ഑ஔ ϦαΠζ ը૾ϑΟϧλʔ هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ ϝλσʔλऔಘ WebP ม׵ JPEG ม׵ ιʔε Lambda S3 ͔ΒͷΠϕϯτͰ Lambda ͕࣮ߦ͞ΕΔ ͜Μͳײ͡ʁ
22. 22 ࡞Γ௚ͨ͠
23. 23 Lambda ؔ਺ͷ։ൃ
24. 24 ϦαΠζ γϟʔϓɺίϯτϥετ هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ WebP ม׵ JPEG ม׵ ϝλσʔλม׵ / औಘ Lambda ্Ͱߦ͏͜ͱ v0.10.36
25. 25 4000px 5296px
26. 26 ϦαΠζ γϟʔϓɺίϯτϥετ هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ WebP ม׵ JPEG ม׵ Lambda ্ͷ ImageMagick Λ Ͱར༻
27. 27 ϦαΠζ γϟʔϓɺίϯτϥετ هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ WebP ม׵ JPEG ม׵ Linux Ͱ੩తϏϧυͨ͠ cwebp όΠφϦ ΛɺΞοϓϩʔυ͢Δ zip ʹؚΊΔ cwebp Λ node ͔Βѻ͏ͨΊͷ
 ϥούʔϥΠϒϥϦ install cwebp
28. 28 όΠφϦͷύεΛίʔυ಺Ͱ؀ڥม਺ʹ௥Ճ JG
    GTFYJTUT4ZOD VTSMPDBMCJODXFCQ
    \
    
    
    
    
    DPOTU
    -*#@%*3
    
    QBUIKPJO @@EJSOBNF
    MJC 
    
    
    
    
    QSPDFTTFOW1"5)
    
    A\-*#@%*3^A
    
    
    
    
    QSPDFTTFOW-%@-*#3"3:@1"5)
    
    A\-*#@%*3^A
    ^
29. 29 ϩʔΧϧͰϢχοτςετ
30. 30 Ϗϧυ / σϓϩΠ Lambda ςετ Babel ʢes6 → es5ʣ ɺιʔεϚοϓ npm Πϯετʔϧ σϓϩΠ (node-aws-lambda) npm: source-map-support Increment Version
31. 31 ಈ࡞ϑϩʔ (ུ֓)
32. 32 Source S3 Dest S3 Order 
 Regenerate Dispatch Scheduled 
 events Invoke Tasks Login, Get Signed URL Request
 Signed URL API
33. 33 \
    
    
    OBNF
    EJWJEF
    
    
    TPVSDF
    YZ[QOH
    
    
    ˑEFTUJOBUJPO
    YZ[@QOH
    
    
    EBUB
    \
    
    
    
    
    SFDUBOHMF
    \
    
    
    
    
    
    
    Y
    Z
    X
    I
    
    
    
    
    ^
    
    
    ^
    ^ S3 ΁ PUT Λ͏͚ͯɺॲཧ಺༰Λॻ͍ͨϑΝΠϧΛ഑ஔ Dest S3 Dispatch Tasks Source S3
34. 34 ֤ॲཧ୲౰ͷ Lambda ͕ S3 ͔Β ඞཁͳϑΝΠϧΛऔಘͯ͠ॲཧ Dest S3 Dispatch Tasks S3 ΁ PUT Λ͏͚ͯɺॲཧ಺༰Λॻ͍ͨϑΝΠϧΛ഑ஔ Source S3
35. 35 Dest S3 Dispatch Tasks S3 ΁ PUT Λ͏͚ͯɺॲཧ಺༰Λॻ͍ͨϑΝΠϧΛ഑ஔ ̍ΠϕϯτͰෳ਺ͷLambda 
 ΛىಈͰ͖ͳ͍ͨΊɺͻͱͭͷ
 Lambda ؔ਺ͰҰ୴ΠϕϯτΛड͚औΓɺ ϑΝΠϧ໊Λϕʔεʹͯ͠ɺଞͷ Lambda Λىಈ (S3ܦ༝)
36. 36 Lambda ͔Βىಈ͢Δ৔߹ɺεϩοτϧ͢Δͱ ࣦഊ͢Δ͔΋͠Εͳ͍ͷͰ S3 Λܦ༝͢Δ Dest S3 Dispatch Scheduled 
 events Tasks
37. 37 1 ෼͓͖ʹىಈ 
 ϑΝΠϧͷଘࡏ֬ೝΛߦ͍
 ഔମϦετΛߋ৽͢Δ ഔମϦετͷଘࡏΛ΋ͬͯɺ ഔମͷ४උ׬ྃΛࣔ͢ Dest S3 Scheduled 
 events
38. 38 API Ͱೝূ API ͸ϢʔβʔΛೝՄͯ͠ɺ CloudFront ॺ໊෇͖URL Λൃߦ Private Cache Distribution Dest S3 Login, Get Signed URL Request
 Signed URL API
39. 39 Private Cache Distribution ύλʔϯ ① ೝূࡁΈϦΫΤετ GET https://api/sign ② ॺ໊ ③ CF ΁ॺ໊Λ͚ͭͯ
 ϦΫΤετ
40. 40 Private Cache Distribution ύλʔϯ CloudFront Λҙࣝ͢Δ͜ͱͳ͘ɺ CloudFront ্ͷίϯςϯπͷऔಘ͕Մೳ ① ೝূࡁΈϦΫΤετ ② 303 (ॺ໊෇͖URL΁ͷϦμΠϨΫτ) ②’ ϦΫΤετ ॺ໊෇͖URL ॺ໊෇͖URLͷੜ੒
41. 41 ӡ༻
42. 42 ேץલ͸ 18000 invocations/m
43. 43 Lambda ༝དྷͷΤϥʔ͸Ίͬͨʹى͖ͳ͍
44. 44 Slack ࿈ܞ Քಇঢ়گΛ௨஌ Τϥʔͷᮢ஋Λ௒͑ͨͱ͖Λ௨஌ Notify Log Log Alarm (when exceeded threshold) Notify
45. 45 Source S3 Order 
 Regenerate Dispatch Invoke Slash commands Ͱ API Gateway ͔Β Lambda ΛىಈɻΤϥʔ࣌ɺ
 QA ࣌ͳͲ͸ɺखಈͰഔମΛੜ੒ Slack ࿈ܞ
46. 46 Lambda ͔Β Lambda ΛݺΜͩޙʹ෮ؼͰ͖ͳ͍Τϥʔ͸ݫې ΤϥʔΛى͜͢ͱ࠷௿ 3 ճϦτϥΠ͞ΕΔͷͰ… < 429: You are sending too 
 ɹmany requests. Please relax. time dying 
 lambda
47. 47 API ଆͷ E2E ςετ͕Քಇ ෳ਺ճ࿈ଓࣦͯ͠ഊ͢Δͱ PagerDuty ͔Β API νʔϜʹి࿩ Lambda ͕ࢮ๢͍ͯ͠Δ৔߹͸ɺखॱʹͦͬͯϩʔΧϧͰॲཧ
48. 48 CloudFront ͕ࢮ๢ͨ͠ࡍ͸υϝΠϯͷ޲͖ઌΛ S3 ΁
49. 49 S3 ʹॻ͖ࠐ·ΕΔ CloudFront ͷϩάΛ Lambda Ͱύʔεͯ͠ Elasticsearch ΁౤ೖɺ kibana ͰՄࢹԽ Log S3 Parse
50. 50 Ϣʔβʔͷ੠Λड͚औͬͯɺSlack ͱ Backlog ΁౤ߘ API Gateway ܦ༝ͷ Lambda ىಈͷ
 ϨΠςϯγʔ͸ؾʹͳΔ
51. 51 ৼΓฦΓ
52. 52 ։ൃ଎౓͸ɺ޲্ʁ Lambda ͷͨΊͷ։ൃ͸ඞཁͩͬͨ ΞϓϦʹԠͨ͡σʔλଆͷॊೈͳมߋ͕Մೳͩͬͨ αʔόʔෛՙΛؾʹ͠ͳ͍ͰΑ͘ͳͬͨʢը૾ੜ੒ɺ഑৴ʣ ։ൃମ੍ͷԸܙʢݱ৔Ͱٕज़తͳҙࢥܾఆ͕͋Δ͜ͱʣ ӡ༻ίετ͸ݮͬͨʢ΄΅ͳ͍ʣ
53. 53 αʔϏεɾঢ়گʹΑͬͯ͸ɺ୅ସΛҙࣝ͢Δඞཁ͸͋Γͦ͏ Lambda ؔ਺͸ɺLambda Ҏ֎ͷͲ͜Ͱಈ͔ͤͦ͏ʁ S3, CloudFront ͕མͪͨ࣌͸ʁ πʔϧ͕ग़ἧͬͯɺݴޠͷબ୒ࢶ΋૿͑ͨ Serverless, Apex, Lamvery ͳͲ ͲͷαʔϏε΋҆ఆ͍ͯ͠Δ ࠓ͸҆৺ͯ͠৸ΒΕΔ
54. 54 ҙ֎ʹࣗ༝౓͸͋Δ OS ͸ Amazon Linux ͳͷͰɺ͜͜Ͱಈ͘΋ͷ͸େମಈ͘ Linux ༻ͷ cwebp όΠφϦͰ WebP ม׵΋Ͱ͖ͨ Lambda ͷͨΊͷ։ൃ͕ඞཁʹͳΔ ಛʹɺS3 ͳͲͱ࿈ܞͯ͠ΠϕϯτϕʔεͰॲཧ͢Δ৔߹ ΠϕϯτͷϧʔςΟϯάɺσΟεύονϟ͕ඞཁʹͳΔ৔߹΋ Ϣχοτςετ͸Ͱ͖Δ͕ɺࣗಈͰ݁߹ςετ͕೉͍͠ Lambda ʹ͍ͭͯ
55. 55 ࡉ͔͍πʔϧΛ࡞Δʹ͸͔ͳΓศར Slack ͷ Slash commands ͳͲ (API Gateway ͱͷ૊Έ߹Θͤ) ͪΌΜͱͰ͖͍ͯΔ ΠϕϯτΛ࿙Βͨ͠Γ͠ͳ͍ ͔ͳΓ͍҆ FaaS ͷະདྷΛײ͡Δ Lambda ʹ͍ͭͯ
56. 56 ؆୯ʹ࢖͑Δ PaaS ج൫ͷ४උ͕͋ͬͨ Elastic Beanstalk + Docker + Python + Django ೝূඞਢɺ༗ྉձһݶఆͷαʔϏεͳͷͰɺݎ࿚Ͱ։ൃମ੍͕ ཱ͍֬ͯ͠Δํ͕Α͔ͬͨ API Gateway + Lambda ͸؅ཧͷίετ͕͔͔Γͦ͏ɺϨΠςϯ γʔ͕େ͖͍ PaaS?
57. 57 εςʔτϨεΛҙࣝͤ͟ΔΛಘͳ͍ ෭࡞༻΍ঢ়ଶอ࣋ʹཔΕͳ͍࣮૷্ͷ੍໿ ີ݁߹ͳίʔυ͕ࣗવʹݮΓɺݟ௨͕͠෇͖΍͘͢ͳΔ ϑϩϯτΤϯυͷ։ൃʹ΋ੜ͖ͨ αʔόʔϨεͰͷ։ൃΛ௨ͯ͡
58. 58 ·ͱΊ
59. 59 Lambda + S3 + CloudWatch Ͱࢴ໘ը૾ͷม׵ S3 + CloudFront Ͱը૾ͷ഑৴: Private cache distribution Lambda Ͱ WebP ͷม׵΍ը૾ॲཧΛߦͬͨ ݁Ռͷ଴ͪ߹ΘͤΛ͢ΔͨΊʹ ɺCloudWatch Ͱ cron తʹ Lambda Λ࣮ߦ ҰͭͷΠϕϯτͰෳ਺ͷ Lambda Λىಈ͢Δ Workaround Slack, Backlog, PagerDuty ͳͲͷ SaaS ΋׆༻ αʔόʔϨεͰखͷ͔͔Βͳ͍ΠϯϑϥΛߴ଎ʹߏஙͰ͖ͨ
60. ࢴ໘ϏϡʔΞʔΛࢧ͑Δ αʔόʔϨεΞʔΩςΫνϟ ழࣂ େࢤ ೔ຊܦࡁ৽ฉࣾ Serverless Conf Tokyo 2016