創業3年のSaaSスタートアップが 規模急拡大に対してアーキテクチャを変革していった話 / Scalebase's architectural transformation

Transcript

1. J A P A N | S E P T E M B E R 2 8 , 2 0 2 1
2. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. ૑ۀ3೥ͷSaaSελʔτΞοϓ͕ ن໛ٸ֦େʹରͯ͠ΞʔΩςΫνϟΛมֵ͍ͯͬͨ͠࿩ Speaker Name Shoma Takeo | Alp, Inc. B - 3
3. Ξϧϓגࣜձࣾ ڞಉ૑ۀऀ औక໾ (tw: showmant_ ) 2014೥ʹגࣜձࣾαΠόʔΤʔδΣϯτʹೖࣾɻಈը ޿ࠂ഑৴ࣄۀʹ͓͚Δ։ൃ੹೚ऀΛ୲౰ɻͦͷޙήʔ Ϝࣄۀऀ޲͚ͷϦλʔήςΟϯά޿ࠂ഑৴ࣄۀͷถࠃ ࣄۀ্ཱͪ͛ʹ։ൃऀͱͯ͠ैࣄɻ ΞϧϓͰ͸όοΫΤϯυΤϯδχΞΛओͱͯ͠ɺݱࡏ͸ ։ൃ૊৫શମͷϚωʔδϝϯτɾ࠾༻Λ୲౰ɻ झຯ͸ϥʔϝϯ(࠷ۙ͸ࣽׯ͠ܥɾ୶ྷܥɾೋ࿠)Λ৯΂ Δ͜ͱɾεϙʔπ؍ઓΛ͢Δ͜ͱ ࣗݾ঺հ - ஛ඌ ਖ਼അ -
4. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. ձࣾ঺հ
7. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. ϓϩμΫτ঺հ
8. SaaS/
9. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. SaaS/αϒεΫϦϓγϣϯϏδωεʹ͓͚Δ
15. 1 30 5 10 1 MRR/ 120
16. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. ࠾༻ٕज़
17. ࠾༻ٕज़
18. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. ૑ۀॳظͷΞʔΩςΫνϟͷ3ͭͷ޻෉
19. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. 1. AWS FinTech ϦϑΝϨϯεɾΞʔΩςΫνϟʔΛࢀߟʹͨ͠
20. • ͳͥϑΟϯςοΫϦϑΝϨϯεΞʔΩςΫνϟʁ • ސ٬ͷࣄۀσʔλ΍੥ٻσʔλΛѻ͏ͨΊηΩϡϦςΟΛҙࣝͨ͠ΞʔΩςΫνϟ͢Δඞཁੑ • BtoBࣄۀͨΊߴ͍ηΩϡϦςΟཁ͕݅ސ٬͔ΒٻΊΒΕΔ৔߹͕ଟ͍ AWS FinTech ϦϑΝϨϯεɾΞʔΩςΫνϟʔ
21. • ޙ͔Β࣮ࢪ͢ΔͱɺΞΧ΢ϯτΛ෼཭͢Δͷ͸େม • ։ൃ؀ڥͱຊ൪؀ڥͰΞΫηεݖݶΛ໌֬ʹ෼͚ΔϝϦοτ • ۀ຿ʹඞཁͳݖݶΛϦεΫ཈͑ͯ౉ͤΔͷͰޮ཰Α͘࢓ࣄͰ͖Δ • IAMઃఆʹޡΓ͕͋ͬͨͱ͖ʹΞΫηεൣғ֦େͷඃ֐Λ཈͑ΒΕΔ ಛʹ஫ҙͨ͠ϙΠϯτ ຊ ൪ ؀ ڥ ͱ ։ ൃ ؀ ڥ Ͱ AW S Ξ Χ ΢ ϯ τ ͷ ෼ ཭
22. ※ݱঢ়ߏ੒ͱ͸ଟগҟͳΓ·͢
23. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. 2. Amazon Elastic Kubernetes Service (Amazon EKS)Λ࠾༻ͨ͠
24. • k8sͷӡ༻ͷ஌ࣝΛ࣋ͬͨϝϯόʔ͕͍͔ͨΒ࠾༻Ͱ͖ͨ • ͢΂ͯͷέʔεʹ͓͍ͯAmazon EKS͕༗ޮͰ͋Δͱ͍͏͜ͱ͸ͳ͍ • ίϯτʔϥϏϦςΟͷߴ͞ • ࣗࣾͷϢʔεέʔεʹซͤͯࣗ༝౓ͷߴ͍֦ுՄೳ • KubernetesͷڊେͳίϛϡχςΟͷࢿݯΛ׆༻Մೳ • ίϛϡχςΟͷ੒௕ੑ΁ͷظ଴ • OSSͰߏ੒͞Ε͍ͯΔ෼τϥϒϧγϡʔςΟϯά͕͠΍͍͢ • ࠔͬͨͱ͖ʹཔΕΔΦʔϓϯͳ৘ใ͕๛෋ Why Amazon EKS?
25. • δϣϒΛ୯ൃͰಈ͔͠ɺඪ४ग़ྗͰϩάΛड͚औΓ͍ͨέʔεΛ؆୯ʹ ରԠͰ͖ΔΑ͏ʹͨ͠ • goccy/kubejobΛ࢖ͬͯɺMain ClassΛࢦఆͯ͠೚ҙͷScalaόονΛk8sͰىಈͰ͖ΔΑ͏ʹͨ͠ Amazon EKSͷ޻෉ ख ಈ ࣮ ߦ δ ϣ ϒ
26. • CronJobͷ੍໿ • ࣮ߦ؂ࢹΛ͢Δඞཁ͕͋Δ • ಉҰεέδϡʔϧͰ2ճ࣮ߦ͞ΕΔ͜ͱ΋͋ΔͷͰႈ౳ʹॲཧΛ૊Ήඞཁ͕͋Δ • default namespaceΛ࢖͏ࡍ͸஫ҙ • applyͷద༻Λޡͬͨ࣌ʹdefault namespaceʹ͋ͨͬͯ͠·͏ ؾΛ͚ͭΔ͜ͱɾΑ͘ͳ͔ͬͨ͜ͱ
27. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. ૑ۀ͔Β3೥ؒͷΞʔΩςΫνϟͷมֵ
28. • ϞϊϦε => ϞδϡϥϞϊϦεԽ • ϚΠΫϩαʔϏε΁ͷҠߦ΋Մೳ • ϦʔυϞσϧԽରԠ • σʔλϞσϧͷංେԽʹΑΓࢀরύϑΥʔϚϯε͕Լ͕ͬͨͨΊɺࢀর༻ͷϏϡʔϞσϧΛ όοΫΤϯυΞϓϦέʔγϣϯ಺Ͱ࡞੒ • ݁Ռ੔߹ੑʹଇͬͨඇಉظΞϓϦέʔγϣϯઃܭ • Spark on Kubernetes ͷಋೖ ΞʔΩςΫνϟมֵ֓ཁ
29. • ϢʔβʔຖͰσʔλྔʹ෯͕͋Δ৔߹ͷઃܭ • σʔλྔʹىҼͯ͠ॲཧ͕௕͘ͳΔ՝୊ʹର͢Δઃܭ ࠓճ࿩͍ͨ͠ςʔϚ
30. • ސ٬਺ͷن໛͕ҧ͏Ϣʔβ ʔ͕ར༻͢ΔͱɺٸܹʹϨ ίʔυ਺ͷΦʔμʔ͕มԽ ͢Δ • σʔλن໛ͷେখʹؔΘΒ ͣɺ͢΂ͯͷϢʔβʔʹૉ ੖Β͍͠ϓϩμΫτମݧΛ ఏڙ͢Δඞཁ͕͋Δ ςʔϚ1: ϢʔβʔຖͰσʔλྔʹ෯͕͋Δ
31. • ϓϩμΫτͷੑ্࣭ɺ1ܖ໿ͷΦϒδΣΫταΠζ͕େ͖͍ • ॳظͷScalebaseΛར༻͍ͯͨ͠Ϣʔβʔͷσʔλྔ͸1ΦϒδΣΫτ͋ ͨΓଟͯ͘΋ ~ 1000Ϩίʔυ • ୯ҰͷॲཧͰѻ͏ΦϒδΣΫτ΋ ~ 1000 ఔ౓ͩͬͨ ςʔϚ2: σʔλྔʹىҼͯ͠ॲཧ͕௕͘ͳΔ՝୊΁ͷରॲ લ ఏ
32. • େن໛σʔλΛҰ౓ʹૢ࡞͢Δঢ়گʹͳΔ·ͰɺΦϒδΣΫτ਺Λؾʹ ͤͣɺ͢΂ͯಉҰτϥϯβΫγϣϯͰॲཧ͍ͯͨ͠ • 10000~ ͷΦϒδΣΫτΛಉ࣌ʹૢ࡞͢Δ৔߹ʹಉظతॲཧͰ͸Ϣʔβ ʔମݧ͕ஶ͘͠ଛͳΘΕΔঢ়گʹ • ސ٬ମݧΛ௥ٻ͢Δ্Ͱ݁Ռ੔߹ੑΛՃຯͨ͠ඇಉظΞϓϦέʔγϣϯઃܭʹτϥΠ ςʔϚ2: σʔλྔʹىҼͯ͠ॲཧ͕௕͘ͳΔ՝୊΁ͷରॲ ՝ ୊
33. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. CSVϑΝΠϧΛ࢖ͬͨ࢖༻ྔΠϯϙʔτ ػೳͷઃܭ
34. • αϒεΫϦϓγϣϯͷதʹ͸AWSͷΑ͏ʹ࢖༻ͨ͠ྔʹԠ͡੥ٻֹ͕ܾఆ͢ Δϓϥϯମܥ͕͋Δ • ࢖༻ྔΛCSVग़ྗͯ͠ɺμογϡϘʔυ͔ΒΠϯϙʔτ • σʔλྫ) ސ٬A͸αϒεΫϦϓγϣϯ1ʹରͯ͠ 2021-09-28 ʹ50GBར༻ͨ͠ • ݄຤ʹ࢖༻ྔΛूܭͯ͠ɺͦͷ߹ܭͰ੥ٻֹΛܾఆ ※ API࿈ܞʹΑΔ࢖༻ྔͷΠϯϙʔτػೳ͸ૣظఏڙ༧ఆ ࢖༻ྔCSVΠϯϙʔτػೳઆ໌
36. • CSVϑΝΠϧ͕ΞοϓϩʔυͰ͖Δ • μογϡϘʔυܦ༝Ͱ͸ͳ͘खಈ (curlίϚϯυ) ͰΞοϓϩʔυͰ͖Δ • 10ߦ ~ 100ສߦͷCSVϑΝΠϧʹ଱͑ΒΕΔ Version 1ͷػೳཁ݅
37. ॳظͷઃܭɾ࣮૷
38. • CSVΞοϓϩʔυͷͨͼʹAmazon EMRΫϥελʔͷ Πϯελϯε֬อ ͱ ηοτΞοϓ͢ΔͨΊʹΦʔόʔϔου͕େ͖͍ • େ༰ྔσʔλͰ͋Ε͹ڐ༰Ͱ͖Δ • ࣮ࡍʹӡ༻Λ։࢝͢Δͱ100ߦఔ౓ͷϑΝΠϧ͕ଟ͘ɺඇػೳཁ݅ͱͯ͠Ξοϓϩʔυ׬ྃ·Ͱͷ ͕͔͔࣌ؒΓ͗͢Δ • γεςϜ/Τϥʔϩά͕Amazon EMR্ʹ͔͠ͳ͍ • Πϯϙʔτͷਐḿ͕Amazon EMRͷεςʔλεΛݟΔҎ֎ํ๏͕ͳ͍ ු͔ͼ্͕ͬͨ՝୊
39. • CSVϑΝΠϧ͕ΞοϓϩʔυͰ͖Δ͜ͱ • μογϡϘʔυͰϢʔβʔ͕Ξοϓϩʔυ͕Ͱ͖Δ͜ͱ • 10ߦ ~100ສߦΫϥεͷCSVϑΝΠϧʹ଱͑ΒΕΔ͜ͱ • αΠζͷখ͍͞CSVϑΝΠϧͰ͸Ξοϓϩʔυ଎౓Λ޲্ͤ͞Δ͜ͱ • ΠϯϙʔτεςʔλεΛμογϡϘʔυ্͔Β֬ೝͰ͖Δ͜ͱ • Πϯϙʔτ࣌ʹΤϥʔΛμογϡϘʔυͰ֬ೝͰ͖Δ͜ͱ Version 2 Ͱຬͨ͢΂͖ཁ݅
41. • αʔόʔϨεͷϫʔΫϑϩʔΛ؆୯ʹߏஙͰ͖Δ • S3ʹϑΝΠϧ͕ஔ͔Εͨ͜ͱΛىҼʹॲཧΛൃՐͤ͞Δ ▪ S3Πϕϯτͷ࢓্༷ At Least Once Ͱݺͼग़͞ΕΔͨΊɺLambdaͱͷ௚઀ͷ࿈ܞͷ৔߹ɺॏෳͰॲཧ ͕૸ͬͯ͠·͏Մೳੑ͕͋Δ – CloudTrailͷར༻ ઃܭͷϙΠϯτ AW S S T E P F U N C T I O N S Λ ར ༻ ͯ͠ ͍ Δ
42. • CSVϑΝΠϧͷαΠζʹΑͬͯॲཧΛ෼ذ • ॲཧ಺༰͸ಉ͕ͩ͡ɺํ๏͸Amazon EMR·ͨ͸Spark on KubernetesΛ࢖͏͔Ͱ෼ذ • σʔλ༰ྔ͕খ => Spark on Kubernetes • 1෼ఔ౓Ͱॲཧ͕׬ྃ͢Δ • σʔλ༰ྔ͕େ => Amazon EMR Λར༻͢Δ • ࣮ࡍͪ͜ΒΛ࢖͏͜ͱ͸Ίͬͨʹͳ͍͕ىಈ·Ͱʹ10෼Ҏ্࢖͏ • Spot instance ͳͷͰɺΫϥελʔ֬อʹ࣌ؒΛཁ͢Δ͜ͱ΋͋Δ ઃܭͷϙΠϯτ σ ʔ λ α Π ζ ʹ Α ͬͯ σ ʔ λ ॲ ཧ ํ ๏ Λ ม ߋ Ͱ ͖ Δ Α ͏ ʹ ͠ ͨ
43. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. ੥ٻੜ੒ػೳʹ͓͚Δ ඇಉظδϣϒ؅ཧͷઃܭ
44. • αϒεΫϦϓγϣϯ͸جຊతʹ͸ɺຖ݄ҰఆͷλΠϛϯάͰ੥ٻ͕ൃੜ͢Δ • ScalebaseͰ͸΋ͱ΋ͱɺαϒεΫϦϓγϣϯ͝ͱʹܾ·ͬͨλΠϛϯάͰ੥ٻΛग़͢ػೳΛఏڙͯ͠ ͍Δ • ͔͠͠ɺಛʹBtoBͷαʔϏεͰ͸ॴఆͷλΠϛϯάΛ଴ͨͣʹ੥ٻΛ͢Δ৔ ߹͕ൃੜ͢Δ • ྫ) ૣΊʹ੥ٻΛग़͢͜ͱΛސ٬ʹٻΊΒΕΔ • ྫ) ΞοϓηϧͳͲʹΑΓɺΠϨΪϡϥʔͳλΠϛϯάͰ੥ٻ͕ൃੜ͢Δ • ೚ҙͷλΠϛϯάͰͷαϒεΫϦϓγϣϯ͔Β੥ٻੜ੒Λ࣮ߦͰ͖Δ࢓૊Έ ͷߏங͕ඞཁ ೚ҙλΠϛϯάͷ੥ٻੜ੒ػೳ
48. • ෳ਺ͷαϒεΫϦϓγϣϯΛબ୒ͯ͠ಉ࣌ʹ੥ٻੜ੒͕Ͱ͖Δ͜ͱ • ඇಉظʹॲཧΛ࣮ߦͰ͖Δ͜ͱ • ੥ٻੜ੒ͷਐߦঢ়گ͕μογϡϘʔυͰ֬ೝͰ͖Δ͜ͱ • ࣦഊͨ͠৔߹ͷΤϥʔΛμογϡϘʔυͰ֬ೝͰ͖Δ͜ͱ • ෳ਺ͷδϣϒΛੵΉ͜ͱ͕Ͱ͖Δ͜ͱ ػೳཁ݅
50. • Why not Amazon SQS? • େྔͷΠϕϯτ͕ྲྀΕͯ͘ΔՄೳੑ͕ߴ͘ɺγϟʔυΛ௥Ճ͢Δ͜ͱͰεέʔϧ͕༰қͰ͋Δ఺ • 1ͭͷΠϕϯτͰෳ਺ͷॲཧΛ૸Β͍ͤͨχʔζ͕ࠓޙ͋Γ͏ΔͨΊ • ྫ) ؂ࠪϩά΋ग़͍ͨ͠ => ؂ࠪϩάΛग़ྗ͢ΔConsumerΛ௥Ճ͢Δ͜ͱͰରԠͰ͖Δ ͳͥKinesisΛ࠾༻͔ͨ͠ʁ ઃ ܭ ͷ ϙ Π ϯ τ
51. • νʔϜͰ࢖͍΍͘͢͢Δ޻෉ • ϨίʔυͷॲཧʹίϯγϡʔϚʔͷॲཧ͕ࣦഊͨ͠৔߹ɺνΣοΫϙΠϯτΛଧͨͳ͍͜ͱͰ ࣦഊ࣌ͷΤϥʔϋϯυϦϯάΛ͠΍͍ͯ͘͢͠Δ • ϦτϥΠ͠ଓ͚ΔϨίʔυ͕ൃੜͨ͠ࡍ͸ɺҎԼͷ3ͭͷ͍ͣΕ͔ͷରԠΛߦ͍ͬͯΔ • ର৅ͷϨίʔυΛਖ਼͘͠ॲཧͰ͖ΔΑ͏ʹίʔυΛमਖ਼ͯ͠ΞϓϦΛσϓϩΠ͠௚͢ • ର৅ͷϨίʔυΛਖ਼͘͠ॲཧͰ͖ΔΑ͏DBͷσʔλΛमਖ਼͢Δ • ର৅ͷϨίʔυࣗମʹ໰୊͕͋Δ৔߹͸ϨίʔυΛεΩοϓ͢Δӡ༻खॱΛ࣮ࢪ͢Δ ॳΊͯͷ Kinesis ࠾༻ ઃ ܭ ͷ ϙ Π ϯ τ
52. • ಉ͡ϨίʔυΛॏෳͯ͠ίϯγϡʔϚʔ͕ॲཧ͢ΔՄೳੑ͕͋Δ • ຊདྷ͸ႈ౳ੑΛอͭ͜ͱͰॏෳͷՄೳੑΛߟྀ͢΂͖ • δϣϒͷঢ়ଶ؅ཧͷඞཁੑͱ࣮૷ͷ౎߹্ͰɺॏෳΛ๷͙ख๏ΛऔΓೖΕͨ • Elasticacheʹॲཧͷঢ়گΛӬଓԽ͓ͯ͘͜͠ͱͰॏෳ͍ͯͨ͠Β஄͘࢓૊ΈΛऔ͍ͬͯΔ ॏෳνΣοΫʹAmazon ElastiCache Λ࠾༻ ઃ ܭ ͷ ϙ Π ϯ τ
53. • KinesisͱElasticacheΛ૊Έ߹ΘͤΔ͜ͱͰɺඇಉظʹδϣϒΛ࣮ߦ͠ͳ ͕ΒɺਐḿΛ؅ཧ͢Δ͜ͱ͕Մೳ • Τϥʔ͕ൃੜͨ͠Πϕϯτʹରͯ͠ɺνΣοΫϙΠϯτΛ͚ͭͳ͍͜ͱ ͰΤϥʔରԠ͕༰қʹͳͬͨ • ҰํͰΤϥʔʹͳͬͨϨίʔυ͕ετϦʔϜ಺ʹ଺ཹͯ͠͠·͏໰୊΋͋Δ • Τϥʔ༻ͷετϦʔϜʹྲྀͯ͠ɺޙଓΛ٧·Βͤͳ͍վળ͕ඞཁ ྑ͔ͬͨ఺/վળ఺ ݁ Ռ ੔ ߹ ੑ ʹ ଇ ͬ ͨ Ξ ϓ Ϧ έ ʔ γ ϣ ϯ Λ ॳ Ί ͯ ν ʔϜ Ͱ ։ ൃ Ͱ ͖ ͨ
54. • ΞϓϦέʔγϣϯ͕ྫ֎Λεϩʔͨ͠ͱ͖ʹɺKCLͰϋϯυϦϯά͠ͳ ͍ͱɺνΣοΫϙΠϯτΛ͏ͬͯ͠·͍ɺਖ਼͘͠ॲཧ͞Εͳ͍··ͷΠ ϕϯτ͕ൃੜͯ͠͠·͏ • ॲཧਖ਼ৗॲཧ׬ྃޙνΣοΫϙΠϯτ͕ͭ͘Α͏ʹྫ֎࣌͸νΣοΫϙΠϯτΛଧͨͣʹϦτϥΠ͢Δ ࢓૊ΈΛಋೖͨ͠ • Ϩίʔυͷ͋ͨΓͷαΠζʹ੍ݶ͕͋ΔͷͰαΠζͷେ͖͍ϨίʔυΛ Kinesis ʹ౤͛Δ৔߹͸஫ҙ • ੍ݶΛߟྀͯ͠ϦετΛௐ੔͢Δඞཁ͕͋ͬͨ • ຊདྷ͸1ཁૉ1Πϕϯτʹ͍͕ͨ͠ɺδϣϒͷதʹෳ਺ͷΠϕϯτ͕͋Δ৔߹ɺ࢝఺ͱऴ఺Λҙ࣮ࣝͨ͠ ૷͢Δඞཁ͕͋ΓɺϦετܕʹͨ͠ ͭ·͍ͮͨͱ͜Ζɺۤ࿑ͨ͠ͱ͜Ζ
55. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. ͍͞͝ʹ
56. • Amazon EMR on Amazon Elastic Kubernetes Service (EKS) ͷಋೖ • EMR Λ࢖༻ͯ͠ɺϑϨʔϜϫʔΫͷΠϯετʔϧͱ؅ཧͳͲͷ޻਺Λ࡟ݮͰ͖ͦ͏ • Kinesis ͷԠ༻ • Τϥʔ࣌ͷϋϯυϦϯάͱͯ͠ผͷStreamʹΠϕϯτΛૹΔ • ิঈτϥϯβΫγϣϯͷ࣮૷ ࠓޙɺݕ౼/௅ઓ͍͖͍ͯͨ͜͠ͱ
57. © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved. WE ARE HIRING.
58. Thank you! © 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved.