紙面ビューアーを支える サーバーレスアーキテクチャ / serverless architecture supports Nikkei's paper viewer

86b45faad28dfccd89b4ec04ef00c62e?s=47 Taishi Ikai
October 01, 2016
Technology
43
36k

紙面ビューアーを支える サーバーレスアーキテクチャ / serverless architecture supports Nikkei's paper viewer

86b45faad28dfccd89b4ec04ef00c62e?s=128

Taishi Ikai

October 01, 2016
Tweet

More Decks by Taishi Ikai

See All by Taishi Ikai
86b45faad28dfccd89b4ec04ef00c62e?s=48 ikait
1
1.8k
86b45faad28dfccd89b4ec04ef00c62e?s=48 ikait
19
4.3k
86b45faad28dfccd89b4ec04ef00c62e?s=48 ikait
1
460
86b45faad28dfccd89b4ec04ef00c62e?s=48 ikait
2
2.7k

Other Decks in Technology

See All in Technology
93fc2d713bc4d0e9c594bf8c9e6bcc62?s=48 oztick139
0
130
D98ee74ffe0fafbdc83b23907dda3665?s=48 doublemarket
2
220
2102a6b8760bd6f57f672805723dd83a?s=48 line_developers_tw
0
870
8aba6de431668fc8d09977c0e33c63c1?s=48 jaguar_imo
0
160
D3764b33b44dcefd6a453ade0b8fec75?s=48 heriet
0
170
4b2f3a64637b51e81813accbe8a98083?s=48 miura55
0
100
14493d3489b1441918bfddfe298415d9?s=48 yanaga
0
340
961126d9892ae4df45249529a0721571?s=48 sugamasao
2
920
27a17f6dcd75a913de398d1ed4d75a9e?s=48 hinatades
0
450
13d936e697fe0f4fa96f926d0a712f6c?s=48 sansanbuildersbox
0
240
Dc3f88b2c2c4210e3068d94ce7c25bf2?s=48 terkel
17
4.4k
9b600b6d2f592d0944507e989770b9b3?s=48 issei126
1
760

Featured

See All Featured
6804f1775cb4babfcc3851298566fbce?s=48 tanoku
85
8.4k
F716e5f737fcfb03f2cf8d1a184f1dbe?s=48 nonsquared
79
3.2k
C1daf20e5c49ff910745198ef9869ac2?s=48 hatefulcrawdad
257
16k
761be20b5ebd271008bcee1244fc5b52?s=48 matthewcrist
71
7k
Aff5641764408271f7bc398f2097edd0?s=48 denniskardys
219
110k
A60068bce2e73de3a37ca9d2dbe36092?s=48 bryan
100
11k
027d1ebf66cc039a0bd3b55eeadbe75d?s=48 sachag
446
36k
Eb8975af8e49e19e3dd6b6b84a542e26?s=48 qrush
283
18k
A9704266587836f7e784235e5073b93e?s=48 jmmastey
5
380
016edace78ffe826f2348d1e0c504afd?s=48 maggiecrowley
5
310
8081b26e05bb4354f7d65ffc34cbbd67?s=48 chriscoyier
682
180k
812e1705ff0f8bc1bcf18587bde687d5?s=48 62gerente
583
200k

Transcript

  1. ࢴ໘ϏϡʔΞʔΛࢧ͑Δ αʔόʔϨεΞʔΩςΫνϟ ழࣂ େࢤ ೔ຊܦࡁ৽ฉࣾ Serverless Conf Tokyo 2016

  2. 2 About me ழࣂ େࢤ (Ikai Taishi) ೔ຊܦࡁ৽ฉࣾͷΤϯδχΞ ࢴ໘ϏϡʔΞʔͷ։ൃΛ୲౰ Swift,

    Python, JavaScript Github: @ikait

  3. 3 ༗ྉձһ ໿48ສਓ ແྉձһ ໿300ສਓ ඇొ࿥ ໿2500ສUB ೔ܦిࢠ൛ͱ͸ 2010 ೥

    3 ݄ʹ૑ץ ݄ؒΞΫηε 3 ԯ݅ ຖ೔໿ 900 ຊͷهࣄΛ഑৴ PCɾϞόΠϧαΠτɺ
 iOS, Android ΞϓϦͳͲʹରԠ

  4. 4 ࢴ໘ϏϡʔΞʔʁ

  5. 5 ೔ܦిࢠ൛ ࢴ໘ϏϡʔΞʔ

  6. 6 ೔ຊܦࡁ৽ฉΛɺ
 εϚϗŋλϒϨοτͰಡΉ iOSŋAndroid ରԠ 2016.3.15 ϦχϡʔΞϧ

  7. 7 ேŋ༦ץ ͕ಡΊΔ ຖேɺຖ༦഑৴͞Ε·͢ (ٳץ೔͸আ͘) աڈ1िؒ෼ಡΊ·͢ ΦϓγϣϯͰɺ೔ܦ࢈ۀ৽ฉɺ ೔ܦ MJ ͕ಡΊ·͢

  8. 8 ৽ฉͱಉ͡ϨΠΞ΢τ ొ࿥ͨ͠ΩʔϫʔυΛ
 ϋΠϥΠτͰ͖·͢ ݕࡧͰ͖·͢ μ΢ϯϩʔυ͓ͯ͘͠ͱ
 ΦϑϥΠϯͰಡΊ·͢

  9. 9 ࢴ໘ϏϡʔΞʔ ಺੡͍ͯ͠·͢ iOSɺAPIɺը૾ੜ੒ͳͲ Ϟμϯͳ։ൃϑϩʔʢ❗ʣ ϚωʔδυɾαʔϏεΛ ੵۃతʹར༻͍ͯ͘͠

  10. 10 https://twitter.com/naoya_ito/status/779241231029186561

  11. 11 https://github.com/serverless/serverless

  12. 12 API (ೝূܥ) Ͱ͸ɺ͜ͷ Microservice ࣮ߦ؀ڥΛར༻͍ͯ͠·͢ https://speakerdeck.com/yosukesuzuki/nikkei-microservice

  13. ࢴ໘ϏϡʔΞʔΛࢧ͑Δ αʔόʔϨεΞʔΩςΫνϟ ழࣂ େࢤ ೔ຊܦࡁ৽ฉࣾ Serverless Conf Tokyo 2016 ͱϚΠΫϩαʔϏεΞʔΩςΫνϟ

  14. 14 ࢴ໘ը૾͕ΞϓϦʹಧ͘·Ͱ

  15. 15 ޻৔ S3 ฤू લఏ هࣄ
 ϨΠΞ΢τ ιʔε: ࢴ໘ը૾ ϝλσʔλ

    XML ࠲ඪͳͲ

  16. 16 S3 SQS EC2 CloudFront Client ιʔεऔಘ
 ϑΝΠϧ഑ஔ ϦαΠζ ը૾ϑΟϧλʔ

    هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ ϝλσʔλऔಘ WebP ม׵ JPEG ม׵ ιʔε enqueue polling / dequeue ΞΫηε੍ݶ όονॲཧͩͱ͢Δͱ..

  17. 17 ओʹேץɺ༦ץͷ࣌ؒͷՔಇ͕ͩɺࢴ໘ߋ৽͸ෆఆظʹ͋Δ → ਺෼ִؒͰ SQS ΛϙʔϦϯά͢Δ ɾ Πϯελϯε͸ৗʹىಈ͓ͯ͘͠? ɾ εέʔϧΞ΢τ͸एׯ͕͔͔࣌ؒΔ

    ॲཧ͕ଟ͍ɺը૾ม׵(ϦαΠζɺϑΟϧλɺWebPม׵) ͸ॏΊ → ͍ܰॲཧ͸ϚϧνεϨουͰฒྻԽ → Πϯελϯε͸ෳ਺୆ىಈ͢ΔͳͲ ՝୊

  18. 18 Lambda ͕ ʢ౦ژϦʔδϣϯʹʣ ͖ͨ 2015/06/29

  19. 19 S3 SQS EC2 CMS CloudFront Client ιʔεऔಘ
 ϑΝΠϧ഑ஔ ϦαΠζ

    ը૾ϑΟϧλʔ هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ ϝλσʔλऔಘ WebP ม׵ JPEG ม׵ ιʔε enqueue polling / dequeue

  20. 20 S3 SQS CMS CloudFront Client ιʔεऔಘ
 ϑΝΠϧ഑ஔ ϦαΠζ ը૾ϑΟϧλʔ

    هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ ϝλσʔλऔಘ WebP ม׵ JPEG ม׵ ιʔε enqueue polling / dequeue Lambda

  21. 21 S3 CMS CloudFront Client ιʔεऔಘ
 ϑΝΠϧ഑ஔ ϦαΠζ ը૾ϑΟϧλʔ هࣄը૾ۣܗ੾ΓऔΓ

    ࢴ໘ը૾෼ׂ ϝλσʔλऔಘ WebP ม׵ JPEG ม׵ ιʔε Lambda S3 ͔ΒͷΠϕϯτͰ Lambda ͕࣮ߦ͞ΕΔ ͜Μͳײ͡ʁ

  22. 22 ࡞Γ௚ͨ͠

  23. 23 Lambda ؔ਺ͷ։ൃ

  24. 24 ϦαΠζ γϟʔϓɺίϯτϥετ هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ WebP ม׵ JPEG ม׵ ϝλσʔλม׵

    / औಘ Lambda ্Ͱߦ͏͜ͱ v0.10.36

  25. 25 4000px 5296px

  26. 26 ϦαΠζ γϟʔϓɺίϯτϥετ هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ WebP ม׵ JPEG ม׵ Lambda

    ্ͷ ImageMagick Λ Ͱར༻

  27. 27 ϦαΠζ γϟʔϓɺίϯτϥετ هࣄը૾ۣܗ੾ΓऔΓ ࢴ໘ը૾෼ׂ WebP ม׵ JPEG ม׵ Linux

    Ͱ੩తϏϧυͨ͠ cwebp όΠφϦ ΛɺΞοϓϩʔυ͢Δ zip ʹؚΊΔ cwebp Λ node ͔Βѻ͏ͨΊͷ
 ϥούʔϥΠϒϥϦ install cwebp

  28. 28 όΠφϦͷύεΛίʔυ಺Ͱ؀ڥม਺ʹ௥Ճ JG GTFYJTUT4ZOD VTSMPDBMCJODXFCQ \ DPOTU-*#@%*3QBUIKPJO @@EJSOBNF MJC 

    QSPDFTTFOW1"5) A\-*#@%*3^A QSPDFTTFOW-%@-*#3"3:@1"5) A\-*#@%*3^A ^

  29. 29 ϩʔΧϧͰϢχοτςετ

  30. 30 Ϗϧυ / σϓϩΠ Lambda ςετ Babel ʢes6 → es5ʣ

    ɺιʔεϚοϓ npm Πϯετʔϧ σϓϩΠ (node-aws-lambda) npm: source-map-support Increment Version

  31. 31 ಈ࡞ϑϩʔ (ུ֓)

  32. 32 Source S3 Dest S3 Order 
 Regenerate Dispatch Scheduled

    
 events Invoke Tasks Login, Get Signed URL Request
 Signed URL API

  33. 33 \ OBNFEJWJEF  TPVSDFYZ[QOH  ˑEFTUJOBUJPOYZ[@QOH EBUB\ SFDUBOHMF\ Y

    Z X I ^ ^ ^ S3 ΁ PUT Λ͏͚ͯɺॲཧ಺༰Λॻ͍ͨϑΝΠϧΛ഑ஔ Dest S3 Dispatch Tasks Source S3

  34. 34 ֤ॲཧ୲౰ͷ Lambda ͕ S3 ͔Β ඞཁͳϑΝΠϧΛऔಘͯ͠ॲཧ Dest S3 Dispatch

    Tasks S3 ΁ PUT Λ͏͚ͯɺॲཧ಺༰Λॻ͍ͨϑΝΠϧΛ഑ஔ Source S3

  35. 35 Dest S3 Dispatch Tasks S3 ΁ PUT Λ͏͚ͯɺॲཧ಺༰Λॻ͍ͨϑΝΠϧΛ഑ஔ ̍ΠϕϯτͰෳ਺ͷLambda

    
 ΛىಈͰ͖ͳ͍ͨΊɺͻͱͭͷ
 Lambda ؔ਺ͰҰ୴ΠϕϯτΛड͚औΓɺ ϑΝΠϧ໊Λϕʔεʹͯ͠ɺଞͷ Lambda Λىಈ (S3ܦ༝)

  36. 36 Lambda ͔Βىಈ͢Δ৔߹ɺεϩοτϧ͢Δͱ ࣦഊ͢Δ͔΋͠Εͳ͍ͷͰ S3 Λܦ༝͢Δ Dest S3 Dispatch Scheduled

    
 events Tasks

  37. 37 1 ෼͓͖ʹىಈ 
 ϑΝΠϧͷଘࡏ֬ೝΛߦ͍
 ഔମϦετΛߋ৽͢Δ ഔମϦετͷଘࡏΛ΋ͬͯɺ ഔମͷ४උ׬ྃΛࣔ͢ Dest S3

    Scheduled 
 events

  38. 38 API Ͱೝূ API ͸ϢʔβʔΛೝՄͯ͠ɺ CloudFront ॺ໊෇͖URL Λൃߦ Private Cache

    Distribution Dest S3 Login, Get Signed URL Request
 Signed URL API

  39. 39 Private Cache Distribution ύλʔϯ ① ೝূࡁΈϦΫΤετ GET https://api/sign ②

    ॺ໊ ③ CF ΁ॺ໊Λ͚ͭͯ
 ϦΫΤετ

  40. 40 Private Cache Distribution ύλʔϯ CloudFront Λҙࣝ͢Δ͜ͱͳ͘ɺ CloudFront ্ͷίϯςϯπͷऔಘ͕Մೳ ①

    ೝূࡁΈϦΫΤετ ② 303 (ॺ໊෇͖URL΁ͷϦμΠϨΫτ) ②’ ϦΫΤετ ॺ໊෇͖URL ॺ໊෇͖URLͷੜ੒

  41. 41 ӡ༻

  42. 42 ேץલ͸ 18000 invocations/m

  43. 43 Lambda ༝དྷͷΤϥʔ͸Ίͬͨʹى͖ͳ͍

  44. 44 Slack ࿈ܞ Քಇঢ়گΛ௨஌ Τϥʔͷᮢ஋Λ௒͑ͨͱ͖Λ௨஌ Notify Log Log Alarm (when

    exceeded threshold) Notify

  45. 45 Source S3 Order 
 Regenerate Dispatch Invoke Slash commands

    Ͱ API Gateway ͔Β Lambda ΛىಈɻΤϥʔ࣌ɺ
 QA ࣌ͳͲ͸ɺखಈͰഔମΛੜ੒ Slack ࿈ܞ

  46. 46 Lambda ͔Β Lambda ΛݺΜͩޙʹ෮ؼͰ͖ͳ͍Τϥʔ͸ݫې ΤϥʔΛى͜͢ͱ࠷௿ 3 ճϦτϥΠ͞ΕΔͷͰ… < 429:

    You are sending too 
 ɹmany requests. Please relax. time dying 
 lambda

  47. 47 API ଆͷ E2E ςετ͕Քಇ ෳ਺ճ࿈ଓࣦͯ͠ഊ͢Δͱ PagerDuty ͔Β API νʔϜʹి࿩

    Lambda ͕ࢮ๢͍ͯ͠Δ৔߹͸ɺखॱʹͦͬͯϩʔΧϧͰॲཧ

  48. 48 CloudFront ͕ࢮ๢ͨ͠ࡍ͸υϝΠϯͷ޲͖ઌΛ S3 ΁

  49. 49 S3 ʹॻ͖ࠐ·ΕΔ CloudFront ͷϩάΛ Lambda Ͱύʔεͯ͠ Elasticsearch ΁౤ೖɺ kibana

    ͰՄࢹԽ Log S3 Parse

  50. 50 Ϣʔβʔͷ੠Λड͚औͬͯɺSlack ͱ Backlog ΁౤ߘ API Gateway ܦ༝ͷ Lambda ىಈͷ


    ϨΠςϯγʔ͸ؾʹͳΔ

  51. 51 ৼΓฦΓ

  52. 52 ։ൃ଎౓͸ɺ޲্ʁ Lambda ͷͨΊͷ։ൃ͸ඞཁͩͬͨ ΞϓϦʹԠͨ͡σʔλଆͷॊೈͳมߋ͕Մೳͩͬͨ αʔόʔෛՙΛؾʹ͠ͳ͍ͰΑ͘ͳͬͨʢը૾ੜ੒ɺ഑৴ʣ ։ൃମ੍ͷԸܙʢݱ৔Ͱٕज़తͳҙࢥܾఆ͕͋Δ͜ͱʣ ӡ༻ίετ͸ݮͬͨʢ΄΅ͳ͍ʣ

  53. 53 αʔϏεɾঢ়گʹΑͬͯ͸ɺ୅ସΛҙࣝ͢Δඞཁ͸͋Γͦ͏ Lambda ؔ਺͸ɺLambda Ҏ֎ͷͲ͜Ͱಈ͔ͤͦ͏ʁ S3, CloudFront ͕མͪͨ࣌͸ʁ πʔϧ͕ग़ἧͬͯɺݴޠͷબ୒ࢶ΋૿͑ͨ Serverless,

    Apex, Lamvery ͳͲ ͲͷαʔϏε΋҆ఆ͍ͯ͠Δ ࠓ͸҆৺ͯ͠৸ΒΕΔ

  54. 54 ҙ֎ʹࣗ༝౓͸͋Δ OS ͸ Amazon Linux ͳͷͰɺ͜͜Ͱಈ͘΋ͷ͸େମಈ͘ Linux ༻ͷ cwebp

    όΠφϦͰ WebP ม׵΋Ͱ͖ͨ Lambda ͷͨΊͷ։ൃ͕ඞཁʹͳΔ ಛʹɺS3 ͳͲͱ࿈ܞͯ͠ΠϕϯτϕʔεͰॲཧ͢Δ৔߹ ΠϕϯτͷϧʔςΟϯάɺσΟεύονϟ͕ඞཁʹͳΔ৔߹΋ Ϣχοτςετ͸Ͱ͖Δ͕ɺࣗಈͰ݁߹ςετ͕೉͍͠ Lambda ʹ͍ͭͯ

  55. 55 ࡉ͔͍πʔϧΛ࡞Δʹ͸͔ͳΓศར Slack ͷ Slash commands ͳͲ (API Gateway ͱͷ૊Έ߹Θͤ)

    ͪΌΜͱͰ͖͍ͯΔ ΠϕϯτΛ࿙Βͨ͠Γ͠ͳ͍ ͔ͳΓ͍҆ FaaS ͷະདྷΛײ͡Δ Lambda ʹ͍ͭͯ

  56. 56 ؆୯ʹ࢖͑Δ PaaS ج൫ͷ४උ͕͋ͬͨ Elastic Beanstalk + Docker + Python

    + Django ೝূඞਢɺ༗ྉձһݶఆͷαʔϏεͳͷͰɺݎ࿚Ͱ։ൃମ੍͕ ཱ͍֬ͯ͠Δํ͕Α͔ͬͨ API Gateway + Lambda ͸؅ཧͷίετ͕͔͔Γͦ͏ɺϨΠςϯ γʔ͕େ͖͍ PaaS?

  57. 57 εςʔτϨεΛҙࣝͤ͟ΔΛಘͳ͍ ෭࡞༻΍ঢ়ଶอ࣋ʹཔΕͳ͍࣮૷্ͷ੍໿ ີ݁߹ͳίʔυ͕ࣗવʹݮΓɺݟ௨͕͠෇͖΍͘͢ͳΔ ϑϩϯτΤϯυͷ։ൃʹ΋ੜ͖ͨ αʔόʔϨεͰͷ։ൃΛ௨ͯ͡

  58. 58 ·ͱΊ

  59. 59 Lambda + S3 + CloudWatch Ͱࢴ໘ը૾ͷม׵ S3 + CloudFront

    Ͱը૾ͷ഑৴: Private cache distribution Lambda Ͱ WebP ͷม׵΍ը૾ॲཧΛߦͬͨ ݁Ռͷ଴ͪ߹ΘͤΛ͢ΔͨΊʹ ɺCloudWatch Ͱ cron తʹ Lambda Λ࣮ߦ ҰͭͷΠϕϯτͰෳ਺ͷ Lambda Λىಈ͢Δ Workaround Slack, Backlog, PagerDuty ͳͲͷ SaaS ΋׆༻ αʔόʔϨεͰखͷ͔͔Βͳ͍ΠϯϑϥΛߴ଎ʹߏஙͰ͖ͨ

  60. ࢴ໘ϏϡʔΞʔΛࢧ͑Δ αʔόʔϨεΞʔΩςΫνϟ ழࣂ େࢤ ೔ຊܦࡁ৽ฉࣾ Serverless Conf Tokyo 2016