紙面ビューアーを支える サーバーレスアーキテクチャ / serverless architecture supports Nikkei's paper viewer

86b45faad28dfccd89b4ec04ef00c62e?s=47 Taishi Ikai
October 01, 2016
Technology
43
37k

紙面ビューアーを支える サーバーレスアーキテクチャ / serverless architecture supports Nikkei's paper viewer

86b45faad28dfccd89b4ec04ef00c62e?s=128

Taishi Ikai

October 01, 2016
Tweet

More Decks by Taishi Ikai

See All by Taishi Ikai
86b45faad28dfccd89b4ec04ef00c62e?s=48 ikait
1
1.9k
86b45faad28dfccd89b4ec04ef00c62e?s=48 ikait
19
4.4k
86b45faad28dfccd89b4ec04ef00c62e?s=48 ikait
1
460
86b45faad28dfccd89b4ec04ef00c62e?s=48 ikait
2
2.7k

Other Decks in Technology

See All in Technology
9de7a10d760bcc0a690f8b23b408fb6c?s=48 jakalada
1
270
03858bb7b6debdd69ea0531b0f57c612?s=48 con_mame
4
2k
Dd7fa413be25d8672c03a487db6a8fe6?s=48 fujiihda
4
520
Abea0dcdfdd2fccad64461d1e2798864?s=48 hacker2202
0
750
A4d5b3aae2e67f31b513c88c726514c2?s=48 smzksts
0
190
9e7a653ed313b3da4b263a772bfc5026?s=48 clustervr
0
190
A64ac825509279a770c13c6fc517f11a?s=48 kawaguti
0
110
D7f27502ce4dfef29afae545df767b27?s=48 hsano
0
120
0d29d155ce5c5a93ed46363626da3ea7?s=48 ocise
1
1.3k
842515eaf8fbb2dfcc75197e7797dc15?s=48 sat
1
970
0d29d155ce5c5a93ed46363626da3ea7?s=48 ocise
1
910
Cd891a89dfec6ca7bd278b5f5bd87c90?s=48 tzkoba
0
380

Featured

See All Featured
A9704266587836f7e784235e5073b93e?s=48 jmmastey
8
530
F383c6a4dc55e331bbe2987b622cee6b?s=48 stephaniewalter
260
11k
61857dafbd287b3027c4dcea9008ad3c?s=48 carmenhchung
26
1.3k
8ec1383b240b5ba15ffb9743fceb3c0e?s=48 philnash
8
480
A415db3ac9e9668acbc1fb36eead84ac?s=48 mthomps
39
2.3k
64827b31aef81498662e8af4bd6d5157?s=48 jonrohan
1021
380k
Ae14cc4491ac334f9cd23f9f93b4305e?s=48 orderedlist
PRO
329
35k
1f74b13f1e5c6c69cb5d7fbaabb1e2cb?s=48 sferik
609
54k
1519587b1a0febb658c36c94f6272b0d?s=48 roundedbygravity
241
21k
4394ceb8b277f35ee3da7030384efb5d?s=48 davidbonilla
69
3.5k
Fde6c3a50ca518a909ef35c22c0ee0e4?s=48 destraynor
223
47k
C4de88ea47538e4594750e3861a341c8?s=48 brettharned
93
3k

Transcript

  1. ࢴ໘ϏϡʔΞʔΛࢧ͑Δ αʔόʔϨεΞʔΩςΫνϟ ழࣂ େࢤ ೔ຊܦࡁ৽ฉࣾ Serverless Conf Tokyo 2016

  2. 2 About me ழࣂ େࢤ (Ikai Taishi) ೔ຊܦࡁ৽ฉࣾͷΤϯδχΞ ࢴ໘ϏϡʔΞʔͷ։ൃΛ୲౰ Swift,

    Python, JavaScript Github: @ikait

  3. 3 ༗ྉձһ ໿48ສਓ ແྉձһ ໿300ສਓ ඇొ࿥ ໿2500ສUB ೔ܦిࢠ൛ͱ͸ 2010 ೥

    3 ݄ʹ૑ץ ݄ؒΞΫηε 3 ԯ݅ ຖ೔໿ 900 ຊͷهࣄΛ഑৴ PCɾϞόΠϧαΠτɺ
 iOS, Android ΞϓϦͳͲʹରԠ

  4. 4 ࢴ໘ϏϡʔΞʔʁ

  5. 5 ೔ܦిࢠ൛ ࢴ໘ϏϡʔΞʔ

  6. 6 ೔ຊܦࡁ৽ฉΛɺ
 εϚϗŋλϒϨοτͰಡΉ iOSŋAndroid ରԠ 2016.3.15 ϦχϡʔΞϧ

  7. 7 ேŋ༦ץ ͕ಡΊΔ ຖேɺຖ༦഑৴͞Ε·͢ (ٳץ೔͸আ͘) աڈ1िؒ෼ಡΊ·͢ ΦϓγϣϯͰɺ೔ܦ࢈ۀ৽ฉɺ ೔ܦ MJ ͕ಡΊ·͢

  8. 8 ৽ฉͱಉ͡ϨΠΞ΢τ ొ࿥ͨ͠ΩʔϫʔυΛ
 ϋΠϥΠτͰ͖·͢ ݕࡧͰ͖·͢ μ΢ϯϩʔυ͓ͯ͘͠ͱ
 ΦϑϥΠϯͰಡΊ·͢

  9. 9 ࢴ໘ϏϡʔΞʔ ಺੡͍ͯ͠·͢ iOSɺAPIɺը૾ੜ੒ͳͲ Ϟμϯͳ։ൃϑϩʔʢ❗ʣ ϚωʔδυɾαʔϏεΛ ੵۃతʹར༻͍ͯ͘͠

  10. 10 https://twitter.com/naoya_ito/status/779241231029186561

  11. 11 https://github.com/serverless/serverless

  12. 12 API (ೝূܥ) Ͱ͸ɺ͜ͷ Microservice ࣮ߦ؀ڥΛར༻͍ͯ͠·͢ https://speakerdeck.com/yosukesuzuki/nikkei-microservice

  13. ࢴ໘ϏϡʔΞʔΛࢧ͑Δ αʔόʔϨεΞʔΩςΫνϟ ழࣂ େࢤ ೔ຊܦࡁ৽ฉࣾ Serverless Conf Tokyo 2016 ͱϚΠΫϩαʔϏεΞʔΩςΫνϟ

  14. 14 ࢴ໘ը૾͕ΞϓϦʹಧ͘·Ͱ

  15. 15 ޻৔ S3 ฤू લఏ هࣄ
 ϨΠΞ΢τ ιʔε: ࢴ໘ը૾ ϝλσʔλ

    XML ࠲ඪͳͲ

  16. 16 S3 SQS EC2 CloudFront Client ιʔεऔಘ
 ϑΝΠϧ഑ஔ ϦαΠζ ը૾ϑΟϧλʔ

    هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ ϝλσʔλऔಘ WebP ม׵ JPEG ม׵ ιʔε enqueue polling / dequeue ΞΫηε੍ݶ όονॲཧͩͱ͢Δͱ..

  17. 17 ओʹேץɺ༦ץͷ࣌ؒͷՔಇ͕ͩɺࢴ໘ߋ৽͸ෆఆظʹ͋Δ → ਺෼ִؒͰ SQS ΛϙʔϦϯά͢Δ ɾ Πϯελϯε͸ৗʹىಈ͓ͯ͘͠? ɾ εέʔϧΞ΢τ͸एׯ͕͔͔࣌ؒΔ

    ॲཧ͕ଟ͍ɺը૾ม׵(ϦαΠζɺϑΟϧλɺWebPม׵) ͸ॏΊ → ͍ܰॲཧ͸ϚϧνεϨουͰฒྻԽ → Πϯελϯε͸ෳ਺୆ىಈ͢ΔͳͲ ՝୊

  18. 18 Lambda ͕ ʢ౦ژϦʔδϣϯʹʣ ͖ͨ 2015/06/29

  19. 19 S3 SQS EC2 CMS CloudFront Client ιʔεऔಘ
 ϑΝΠϧ഑ஔ ϦαΠζ

    ը૾ϑΟϧλʔ هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ ϝλσʔλऔಘ WebP ม׵ JPEG ม׵ ιʔε enqueue polling / dequeue

  20. 20 S3 SQS CMS CloudFront Client ιʔεऔಘ
 ϑΝΠϧ഑ஔ ϦαΠζ ը૾ϑΟϧλʔ

    هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ ϝλσʔλऔಘ WebP ม׵ JPEG ม׵ ιʔε enqueue polling / dequeue Lambda

  21. 21 S3 CMS CloudFront Client ιʔεऔಘ
 ϑΝΠϧ഑ஔ ϦαΠζ ը૾ϑΟϧλʔ هࣄը૾ۣܗ੾ΓऔΓ

    ࢴ໘ը૾෼ׂ ϝλσʔλऔಘ WebP ม׵ JPEG ม׵ ιʔε Lambda S3 ͔ΒͷΠϕϯτͰ Lambda ͕࣮ߦ͞ΕΔ ͜Μͳײ͡ʁ

  22. 22 ࡞Γ௚ͨ͠

  23. 23 Lambda ؔ਺ͷ։ൃ

  24. 24 ϦαΠζ γϟʔϓɺίϯτϥετ هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ WebP ม׵ JPEG ม׵ ϝλσʔλม׵

    / औಘ Lambda ্Ͱߦ͏͜ͱ v0.10.36

  25. 25 4000px 5296px

  26. 26 ϦαΠζ γϟʔϓɺίϯτϥετ هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ WebP ม׵ JPEG ม׵ Lambda

    ্ͷ ImageMagick Λ Ͱར༻

  27. 27 ϦαΠζ γϟʔϓɺίϯτϥετ هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ WebP ม׵ JPEG ม׵ Linux

    Ͱ੩తϏϧυͨ͠ cwebp όΠφϦ ΛɺΞοϓϩʔυ͢Δ zip ʹؚΊΔ cwebp Λ node ͔Βѻ͏ͨΊͷ
 ϥούʔϥΠϒϥϦ install cwebp

  28. 28 όΠφϦͷύεΛίʔυ಺Ͱ؀ڥม਺ʹ௥Ճ JG GTFYJTUT4ZOD VTSMPDBMCJODXFCQ \ DPOTU-*#@%*3QBUIKPJO @@EJSOBNF MJC 

    QSPDFTTFOW1"5) A\-*#@%*3^A QSPDFTTFOW-%@-*#3"3:@1"5) A\-*#@%*3^A ^

  29. 29 ϩʔΧϧͰϢχοτςετ

  30. 30 Ϗϧυ / σϓϩΠ Lambda ςετ Babel ʢes6 → es5ʣ

    ɺιʔεϚοϓ npm Πϯετʔϧ σϓϩΠ (node-aws-lambda) npm: source-map-support Increment Version

  31. 31 ಈ࡞ϑϩʔ (ུ֓)

  32. 32 Source S3 Dest S3 Order 
 Regenerate Dispatch Scheduled

    
 events Invoke Tasks Login, Get Signed URL Request
 Signed URL API

  33. 33 \ OBNFEJWJEF  TPVSDFYZ[QOH  ˑEFTUJOBUJPOYZ[@QOH EBUB\ SFDUBOHMF\ Y

    Z X I ^ ^ ^ S3 ΁ PUT Λ͏͚ͯɺॲཧ಺༰Λॻ͍ͨϑΝΠϧΛ഑ஔ Dest S3 Dispatch Tasks Source S3

  34. 34 ֤ॲཧ୲౰ͷ Lambda ͕ S3 ͔Β ඞཁͳϑΝΠϧΛऔಘͯ͠ॲཧ Dest S3 Dispatch

    Tasks S3 ΁ PUT Λ͏͚ͯɺॲཧ಺༰Λॻ͍ͨϑΝΠϧΛ഑ஔ Source S3

  35. 35 Dest S3 Dispatch Tasks S3 ΁ PUT Λ͏͚ͯɺॲཧ಺༰Λॻ͍ͨϑΝΠϧΛ഑ஔ ̍ΠϕϯτͰෳ਺ͷLambda

    
 ΛىಈͰ͖ͳ͍ͨΊɺͻͱͭͷ
 Lambda ؔ਺ͰҰ୴ΠϕϯτΛड͚औΓɺ ϑΝΠϧ໊Λϕʔεʹͯ͠ɺଞͷ Lambda Λىಈ (S3ܦ༝)

  36. 36 Lambda ͔Βىಈ͢Δ৔߹ɺεϩοτϧ͢Δͱ ࣦഊ͢Δ͔΋͠Εͳ͍ͷͰ S3 Λܦ༝͢Δ Dest S3 Dispatch Scheduled

    
 events Tasks

  37. 37 1 ෼͓͖ʹىಈ 
 ϑΝΠϧͷଘࡏ֬ೝΛߦ͍
 ഔମϦετΛߋ৽͢Δ ഔମϦετͷଘࡏΛ΋ͬͯɺ ഔମͷ४උ׬ྃΛࣔ͢ Dest S3

    Scheduled 
 events

  38. 38 API Ͱೝূ API ͸ϢʔβʔΛೝՄͯ͠ɺ CloudFront ॺ໊෇͖URL Λൃߦ Private Cache

    Distribution Dest S3 Login, Get Signed URL Request
 Signed URL API

  39. 39 Private Cache Distribution ύλʔϯ ① ೝূࡁΈϦΫΤετ GET https://api/sign ②

    ॺ໊ ③ CF ΁ॺ໊Λ͚ͭͯ
 ϦΫΤετ

  40. 40 Private Cache Distribution ύλʔϯ CloudFront Λҙࣝ͢Δ͜ͱͳ͘ɺ CloudFront ্ͷίϯςϯπͷऔಘ͕Մೳ ①

    ೝূࡁΈϦΫΤετ ② 303 (ॺ໊෇͖URL΁ͷϦμΠϨΫτ) ②’ ϦΫΤετ ॺ໊෇͖URL ॺ໊෇͖URLͷੜ੒

  41. 41 ӡ༻

  42. 42 ேץલ͸ 18000 invocations/m

  43. 43 Lambda ༝དྷͷΤϥʔ͸Ίͬͨʹى͖ͳ͍

  44. 44 Slack ࿈ܞ Քಇঢ়گΛ௨஌ Τϥʔͷᮢ஋Λ௒͑ͨͱ͖Λ௨஌ Notify Log Log Alarm (when

    exceeded threshold) Notify

  45. 45 Source S3 Order 
 Regenerate Dispatch Invoke Slash commands

    Ͱ API Gateway ͔Β Lambda ΛىಈɻΤϥʔ࣌ɺ
 QA ࣌ͳͲ͸ɺखಈͰഔମΛੜ੒ Slack ࿈ܞ

  46. 46 Lambda ͔Β Lambda ΛݺΜͩޙʹ෮ؼͰ͖ͳ͍Τϥʔ͸ݫې ΤϥʔΛى͜͢ͱ࠷௿ 3 ճϦτϥΠ͞ΕΔͷͰ… < 429:

    You are sending too 
 ɹmany requests. Please relax. time dying 
 lambda

  47. 47 API ଆͷ E2E ςετ͕Քಇ ෳ਺ճ࿈ଓࣦͯ͠ഊ͢Δͱ PagerDuty ͔Β API νʔϜʹి࿩

    Lambda ͕ࢮ๢͍ͯ͠Δ৔߹͸ɺखॱʹͦͬͯϩʔΧϧͰॲཧ

  48. 48 CloudFront ͕ࢮ๢ͨ͠ࡍ͸υϝΠϯͷ޲͖ઌΛ S3 ΁

  49. 49 S3 ʹॻ͖ࠐ·ΕΔ CloudFront ͷϩάΛ Lambda Ͱύʔεͯ͠ Elasticsearch ΁౤ೖɺ kibana

    ͰՄࢹԽ Log S3 Parse

  50. 50 Ϣʔβʔͷ੠Λड͚औͬͯɺSlack ͱ Backlog ΁౤ߘ API Gateway ܦ༝ͷ Lambda ىಈͷ


    ϨΠςϯγʔ͸ؾʹͳΔ

  51. 51 ৼΓฦΓ

  52. 52 ։ൃ଎౓͸ɺ޲্ʁ Lambda ͷͨΊͷ։ൃ͸ඞཁͩͬͨ ΞϓϦʹԠͨ͡σʔλଆͷॊೈͳมߋ͕Մೳͩͬͨ αʔόʔෛՙΛؾʹ͠ͳ͍ͰΑ͘ͳͬͨʢը૾ੜ੒ɺ഑৴ʣ ։ൃମ੍ͷԸܙʢݱ৔Ͱٕज़తͳҙࢥܾఆ͕͋Δ͜ͱʣ ӡ༻ίετ͸ݮͬͨʢ΄΅ͳ͍ʣ

  53. 53 αʔϏεɾঢ়گʹΑͬͯ͸ɺ୅ସΛҙࣝ͢Δඞཁ͸͋Γͦ͏ Lambda ؔ਺͸ɺLambda Ҏ֎ͷͲ͜Ͱಈ͔ͤͦ͏ʁ S3, CloudFront ͕མͪͨ࣌͸ʁ πʔϧ͕ग़ἧͬͯɺݴޠͷબ୒ࢶ΋૿͑ͨ Serverless,

    Apex, Lamvery ͳͲ ͲͷαʔϏε΋҆ఆ͍ͯ͠Δ ࠓ͸҆৺ͯ͠৸ΒΕΔ

  54. 54 ҙ֎ʹࣗ༝౓͸͋Δ OS ͸ Amazon Linux ͳͷͰɺ͜͜Ͱಈ͘΋ͷ͸େମಈ͘ Linux ༻ͷ cwebp

    όΠφϦͰ WebP ม׵΋Ͱ͖ͨ Lambda ͷͨΊͷ։ൃ͕ඞཁʹͳΔ ಛʹɺS3 ͳͲͱ࿈ܞͯ͠ΠϕϯτϕʔεͰॲཧ͢Δ৔߹ ΠϕϯτͷϧʔςΟϯάɺσΟεύονϟ͕ඞཁʹͳΔ৔߹΋ Ϣχοτςετ͸Ͱ͖Δ͕ɺࣗಈͰ݁߹ςετ͕೉͍͠ Lambda ʹ͍ͭͯ

  55. 55 ࡉ͔͍πʔϧΛ࡞Δʹ͸͔ͳΓศར Slack ͷ Slash commands ͳͲ (API Gateway ͱͷ૊Έ߹Θͤ)

    ͪΌΜͱͰ͖͍ͯΔ ΠϕϯτΛ࿙Βͨ͠Γ͠ͳ͍ ͔ͳΓ͍҆ FaaS ͷະདྷΛײ͡Δ Lambda ʹ͍ͭͯ

  56. 56 ؆୯ʹ࢖͑Δ PaaS ج൫ͷ४උ͕͋ͬͨ Elastic Beanstalk + Docker + Python

    + Django ೝূඞਢɺ༗ྉձһݶఆͷαʔϏεͳͷͰɺݎ࿚Ͱ։ൃମ੍͕ ཱ͍֬ͯ͠Δํ͕Α͔ͬͨ API Gateway + Lambda ͸؅ཧͷίετ͕͔͔Γͦ͏ɺϨΠςϯ γʔ͕େ͖͍ PaaS?

  57. 57 εςʔτϨεΛҙࣝͤ͟ΔΛಘͳ͍ ෭࡞༻΍ঢ়ଶอ࣋ʹཔΕͳ͍࣮૷্ͷ੍໿ ີ݁߹ͳίʔυ͕ࣗવʹݮΓɺݟ௨͕͠෇͖΍͘͢ͳΔ ϑϩϯτΤϯυͷ։ൃʹ΋ੜ͖ͨ αʔόʔϨεͰͷ։ൃΛ௨ͯ͡

  58. 58 ·ͱΊ

  59. 59 Lambda + S3 + CloudWatch Ͱࢴ໘ը૾ͷม׵ S3 + CloudFront

    Ͱը૾ͷ഑৴: Private cache distribution Lambda Ͱ WebP ͷม׵΍ը૾ॲཧΛߦͬͨ ݁Ռͷ଴ͪ߹ΘͤΛ͢ΔͨΊʹ ɺCloudWatch Ͱ cron తʹ Lambda Λ࣮ߦ ҰͭͷΠϕϯτͰෳ਺ͷ Lambda Λىಈ͢Δ Workaround Slack, Backlog, PagerDuty ͳͲͷ SaaS ΋׆༻ αʔόʔϨεͰखͷ͔͔Βͳ͍ΠϯϑϥΛߴ଎ʹߏஙͰ͖ͨ

  60. ࢴ໘ϏϡʔΞʔΛࢧ͑Δ αʔόʔϨεΞʔΩςΫνϟ ழࣂ େࢤ ೔ຊܦࡁ৽ฉࣾ Serverless Conf Tokyo 2016