創業3年のSaaSスタートアップが 規模急拡大に対してアーキテクチャを変革していった話 / Scalebase's architectural transformation

Transcript

1. J A P A N | S E P T

   E M B E R 2 8 , 2 0 2 1

2. © 2021, Amazon Web Services, Inc. or its affiliates. All

   rights reserved. ૑ۀ3೥ͷSaaSελʔτΞοϓ͕ ن໛ٸ֦େʹରͯ͠ΞʔΩςΫνϟΛมֵ͍ͯͬͨ͠࿩ Speaker Name Shoma Takeo | Alp, Inc. B - 3

3. Ξϧϓגࣜձࣾ ڞಉ૑ۀऀ औక໾ (tw: showmant_ ) 2014೥ʹגࣜձࣾαΠόʔΤʔδΣϯτʹೖࣾɻಈը ޿ࠂ഑৴ࣄۀʹ͓͚Δ։ൃ੹೚ऀΛ୲౰ɻͦͷޙήʔ Ϝࣄۀऀ޲͚ͷϦλʔήςΟϯά޿ࠂ഑৴ࣄۀͷถࠃ ࣄۀ্ཱͪ͛ʹ։ൃऀͱͯ͠ैࣄɻ

   ΞϧϓͰ͸όοΫΤϯυΤϯδχΞΛओͱͯ͠ɺݱࡏ͸ ։ൃ૊৫શମͷϚωʔδϝϯτɾ࠾༻Λ୲౰ɻ झຯ͸ϥʔϝϯ(࠷ۙ͸ࣽׯ͠ܥɾ୶ྷܥɾೋ࿠)Λ৯΂ Δ͜ͱɾεϙʔπ؍ઓΛ͢Δ͜ͱ ࣗݾ঺հ - ஛ඌ ਖ਼അ -

4. © 2021, Amazon Web Services, Inc. or its affiliates. All

   rights reserved. ձࣾ঺հ
5. None
6. None

7. © 2021, Amazon Web Services, Inc. or its affiliates. All

   rights reserved. ϓϩμΫτ঺հ

8. SaaS/

9. © 2021, Amazon Web Services, Inc. or its affiliates. All

   rights reserved. SaaS/αϒεΫϦϓγϣϯϏδωεʹ͓͚Δ
10. None
11. None
12. None
13. None
14. None

15. 1 30 5 10 1 MRR/ 120

16. © 2021, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. ࠾༻ٕज़

17. ࠾༻ٕज़

18. © 2021, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. ૑ۀॳظͷΞʔΩςΫνϟͷ3ͭͷ޻෉

19. © 2021, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. 1. AWS FinTech ϦϑΝϨϯεɾΞʔΩςΫνϟʔΛࢀߟʹͨ͠

20. • ͳͥϑΟϯςοΫϦϑΝϨϯεΞʔΩςΫνϟʁ • ސ٬ͷࣄۀσʔλ΍੥ٻσʔλΛѻ͏ͨΊηΩϡϦςΟΛҙࣝͨ͠ΞʔΩςΫνϟ͢Δඞཁੑ • BtoBࣄۀͨΊߴ͍ηΩϡϦςΟཁ͕݅ސ٬͔ΒٻΊΒΕΔ৔߹͕ଟ͍ AWS FinTech ϦϑΝϨϯεɾΞʔΩςΫνϟʔ

21. • ޙ͔Β࣮ࢪ͢ΔͱɺΞΧ΢ϯτΛ෼཭͢Δͷ͸େม • ։ൃ؀ڥͱຊ൪؀ڥͰΞΫηεݖݶΛ໌֬ʹ෼͚ΔϝϦοτ • ۀ຿ʹඞཁͳݖݶΛϦεΫ཈͑ͯ౉ͤΔͷͰޮ཰Α͘࢓ࣄͰ͖Δ • IAMઃఆʹޡΓ͕͋ͬͨͱ͖ʹΞΫηεൣғ֦େͷඃ֐Λ཈͑ΒΕΔ ಛʹ஫ҙͨ͠ϙΠϯτ ຊ

    ൪ ؀ ڥ ͱ ։ ൃ ؀ ڥ Ͱ AW S Ξ Χ ΢ ϯ τ ͷ ෼ ཭

22. ※ݱঢ়ߏ੒ͱ͸ଟগҟͳΓ·͢

23. © 2021, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. 2. Amazon Elastic Kubernetes Service (Amazon EKS)Λ࠾༻ͨ͠

24. • k8sͷӡ༻ͷ஌ࣝΛ࣋ͬͨϝϯόʔ͕͍͔ͨΒ࠾༻Ͱ͖ͨ • ͢΂ͯͷέʔεʹ͓͍ͯAmazon EKS͕༗ޮͰ͋Δͱ͍͏͜ͱ͸ͳ͍ • ίϯτʔϥϏϦςΟͷߴ͞ • ࣗࣾͷϢʔεέʔεʹซͤͯࣗ༝౓ͷߴ͍֦ுՄೳ •

    KubernetesͷڊେͳίϛϡχςΟͷࢿݯΛ׆༻Մೳ • ίϛϡχςΟͷ੒௕ੑ΁ͷظ଴ • OSSͰߏ੒͞Ε͍ͯΔ෼τϥϒϧγϡʔςΟϯά͕͠΍͍͢ • ࠔͬͨͱ͖ʹཔΕΔΦʔϓϯͳ৘ใ͕๛෋ Why Amazon EKS?

25. • δϣϒΛ୯ൃͰಈ͔͠ɺඪ४ग़ྗͰϩάΛड͚औΓ͍ͨέʔεΛ؆୯ʹ ରԠͰ͖ΔΑ͏ʹͨ͠ • goccy/kubejobΛ࢖ͬͯɺMain ClassΛࢦఆͯ͠೚ҙͷScalaόονΛk8sͰىಈͰ͖ΔΑ͏ʹͨ͠ Amazon EKSͷ޻෉ ख ಈ

    ࣮ ߦ δ ϣ ϒ

26. • CronJobͷ੍໿ • ࣮ߦ؂ࢹΛ͢Δඞཁ͕͋Δ • ಉҰεέδϡʔϧͰ2ճ࣮ߦ͞ΕΔ͜ͱ΋͋ΔͷͰႈ౳ʹॲཧΛ૊Ήඞཁ͕͋Δ • default namespaceΛ࢖͏ࡍ͸஫ҙ •

    applyͷద༻Λޡͬͨ࣌ʹdefault namespaceʹ͋ͨͬͯ͠·͏ ؾΛ͚ͭΔ͜ͱɾΑ͘ͳ͔ͬͨ͜ͱ

27. © 2021, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. ૑ۀ͔Β3೥ؒͷΞʔΩςΫνϟͷมֵ

28. • ϞϊϦε => ϞδϡϥϞϊϦεԽ • ϚΠΫϩαʔϏε΁ͷҠߦ΋Մೳ • ϦʔυϞσϧԽରԠ • σʔλϞσϧͷංେԽʹΑΓࢀরύϑΥʔϚϯε͕Լ͕ͬͨͨΊɺࢀর༻ͷϏϡʔϞσϧΛ

    όοΫΤϯυΞϓϦέʔγϣϯ಺Ͱ࡞੒ • ݁Ռ੔߹ੑʹଇͬͨඇಉظΞϓϦέʔγϣϯઃܭ • Spark on Kubernetes ͷಋೖ ΞʔΩςΫνϟมֵ֓ཁ

29. • ϢʔβʔຖͰσʔλྔʹ෯͕͋Δ৔߹ͷઃܭ • σʔλྔʹىҼͯ͠ॲཧ͕௕͘ͳΔ՝୊ʹର͢Δઃܭ ࠓճ࿩͍ͨ͠ςʔϚ

30. • ސ٬਺ͷن໛͕ҧ͏Ϣʔβ ʔ͕ར༻͢ΔͱɺٸܹʹϨ ίʔυ਺ͷΦʔμʔ͕มԽ ͢Δ • σʔλن໛ͷେখʹؔΘΒ ͣɺ͢΂ͯͷϢʔβʔʹૉ ੖Β͍͠ϓϩμΫτମݧΛ ఏڙ͢Δඞཁ͕͋Δ

    ςʔϚ1: ϢʔβʔຖͰσʔλྔʹ෯͕͋Δ

31. • ϓϩμΫτͷੑ্࣭ɺ1ܖ໿ͷΦϒδΣΫταΠζ͕େ͖͍ • ॳظͷScalebaseΛར༻͍ͯͨ͠Ϣʔβʔͷσʔλྔ͸1ΦϒδΣΫτ͋ ͨΓଟͯ͘΋ ~ 1000Ϩίʔυ • ୯ҰͷॲཧͰѻ͏ΦϒδΣΫτ΋ ~

    1000 ఔ౓ͩͬͨ ςʔϚ2: σʔλྔʹىҼͯ͠ॲཧ͕௕͘ͳΔ՝୊΁ͷରॲ લ ఏ

32. • େن໛σʔλΛҰ౓ʹૢ࡞͢Δঢ়گʹͳΔ·ͰɺΦϒδΣΫτ਺Λؾʹ ͤͣɺ͢΂ͯಉҰτϥϯβΫγϣϯͰॲཧ͍ͯͨ͠ • 10000~ ͷΦϒδΣΫτΛಉ࣌ʹૢ࡞͢Δ৔߹ʹಉظతॲཧͰ͸Ϣʔβ ʔମݧ͕ஶ͘͠ଛͳΘΕΔঢ়گʹ • ސ٬ମݧΛ௥ٻ͢Δ্Ͱ݁Ռ੔߹ੑΛՃຯͨ͠ඇಉظΞϓϦέʔγϣϯઃܭʹτϥΠ ςʔϚ2:

    σʔλྔʹىҼͯ͠ॲཧ͕௕͘ͳΔ՝୊΁ͷରॲ ՝ ୊

33. © 2021, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. CSVϑΝΠϧΛ࢖ͬͨ࢖༻ྔΠϯϙʔτ ػೳͷઃܭ

34. • αϒεΫϦϓγϣϯͷதʹ͸AWSͷΑ͏ʹ࢖༻ͨ͠ྔʹԠ͡੥ٻֹ͕ܾఆ͢ Δϓϥϯମܥ͕͋Δ • ࢖༻ྔΛCSVग़ྗͯ͠ɺμογϡϘʔυ͔ΒΠϯϙʔτ • σʔλྫ) ސ٬A͸αϒεΫϦϓγϣϯ1ʹରͯ͠ 2021-09-28 ʹ50GBར༻ͨ͠

    • ݄຤ʹ࢖༻ྔΛूܭͯ͠ɺͦͷ߹ܭͰ੥ٻֹΛܾఆ ※ API࿈ܞʹΑΔ࢖༻ྔͷΠϯϙʔτػೳ͸ૣظఏڙ༧ఆ ࢖༻ྔCSVΠϯϙʔτػೳઆ໌
35. None

36. • CSVϑΝΠϧ͕ΞοϓϩʔυͰ͖Δ • μογϡϘʔυܦ༝Ͱ͸ͳ͘खಈ (curlίϚϯυ) ͰΞοϓϩʔυͰ͖Δ • 10ߦ ~ 100ສߦͷCSVϑΝΠϧʹ଱͑ΒΕΔ

    Version 1ͷػೳཁ݅

37. ॳظͷઃܭɾ࣮૷

38. • CSVΞοϓϩʔυͷͨͼʹAmazon EMRΫϥελʔͷ Πϯελϯε֬อ ͱ ηοτΞοϓ͢ΔͨΊʹΦʔόʔϔου͕େ͖͍ • େ༰ྔσʔλͰ͋Ε͹ڐ༰Ͱ͖Δ • ࣮ࡍʹӡ༻Λ։࢝͢Δͱ100ߦఔ౓ͷϑΝΠϧ͕ଟ͘ɺඇػೳཁ݅ͱͯ͠Ξοϓϩʔυ׬ྃ·Ͱͷ

    ͕͔͔࣌ؒΓ͗͢Δ • γεςϜ/Τϥʔϩά͕Amazon EMR্ʹ͔͠ͳ͍ • Πϯϙʔτͷਐḿ͕Amazon EMRͷεςʔλεΛݟΔҎ֎ํ๏͕ͳ͍ ු͔ͼ্͕ͬͨ՝୊

39. • CSVϑΝΠϧ͕ΞοϓϩʔυͰ͖Δ͜ͱ • μογϡϘʔυͰϢʔβʔ͕Ξοϓϩʔυ͕Ͱ͖Δ͜ͱ • 10ߦ ~100ສߦΫϥεͷCSVϑΝΠϧʹ଱͑ΒΕΔ͜ͱ • αΠζͷখ͍͞CSVϑΝΠϧͰ͸Ξοϓϩʔυ଎౓Λ޲্ͤ͞Δ͜ͱ •

    ΠϯϙʔτεςʔλεΛμογϡϘʔυ্͔Β֬ೝͰ͖Δ͜ͱ • Πϯϙʔτ࣌ʹΤϥʔΛμογϡϘʔυͰ֬ೝͰ͖Δ͜ͱ Version 2 Ͱຬͨ͢΂͖ཁ݅
40. None

41. • αʔόʔϨεͷϫʔΫϑϩʔΛ؆୯ʹߏஙͰ͖Δ • S3ʹϑΝΠϧ͕ஔ͔Εͨ͜ͱΛىҼʹॲཧΛൃՐͤ͞Δ ▪ S3Πϕϯτͷ࢓্༷ At Least Once Ͱݺͼग़͞ΕΔͨΊɺLambdaͱͷ௚઀ͷ࿈ܞͷ৔߹ɺॏෳͰॲཧ

    ͕૸ͬͯ͠·͏Մೳੑ͕͋Δ – CloudTrailͷར༻ ઃܭͷϙΠϯτ AW S S T E P F U N C T I O N S Λ ར ༻ ͯ͠ ͍ Δ

42. • CSVϑΝΠϧͷαΠζʹΑͬͯॲཧΛ෼ذ • ॲཧ಺༰͸ಉ͕ͩ͡ɺํ๏͸Amazon EMR·ͨ͸Spark on KubernetesΛ࢖͏͔Ͱ෼ذ • σʔλ༰ྔ͕খ =>

    Spark on Kubernetes • 1෼ఔ౓Ͱॲཧ͕׬ྃ͢Δ • σʔλ༰ྔ͕େ => Amazon EMR Λར༻͢Δ • ࣮ࡍͪ͜ΒΛ࢖͏͜ͱ͸Ίͬͨʹͳ͍͕ىಈ·Ͱʹ10෼Ҏ্࢖͏ • Spot instance ͳͷͰɺΫϥελʔ֬อʹ࣌ؒΛཁ͢Δ͜ͱ΋͋Δ ઃܭͷϙΠϯτ σ ʔ λ α Π ζ ʹ Α ͬͯ σ ʔ λ ॲ ཧ ํ ๏ Λ ม ߋ Ͱ ͖ Δ Α ͏ ʹ ͠ ͨ

43. © 2021, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. ੥ٻੜ੒ػೳʹ͓͚Δ ඇಉظδϣϒ؅ཧͷઃܭ

44. • αϒεΫϦϓγϣϯ͸جຊతʹ͸ɺຖ݄ҰఆͷλΠϛϯάͰ੥ٻ͕ൃੜ͢Δ • ScalebaseͰ͸΋ͱ΋ͱɺαϒεΫϦϓγϣϯ͝ͱʹܾ·ͬͨλΠϛϯάͰ੥ٻΛग़͢ػೳΛఏڙͯ͠ ͍Δ • ͔͠͠ɺಛʹBtoBͷαʔϏεͰ͸ॴఆͷλΠϛϯάΛ଴ͨͣʹ੥ٻΛ͢Δ৔ ߹͕ൃੜ͢Δ • ྫ)

    ૣΊʹ੥ٻΛग़͢͜ͱΛސ٬ʹٻΊΒΕΔ • ྫ) ΞοϓηϧͳͲʹΑΓɺΠϨΪϡϥʔͳλΠϛϯάͰ੥ٻ͕ൃੜ͢Δ • ೚ҙͷλΠϛϯάͰͷαϒεΫϦϓγϣϯ͔Β੥ٻੜ੒Λ࣮ߦͰ͖Δ࢓૊Έ ͷߏங͕ඞཁ ೚ҙλΠϛϯάͷ੥ٻੜ੒ػೳ
45. None
46. None
47. None

48. • ෳ਺ͷαϒεΫϦϓγϣϯΛબ୒ͯ͠ಉ࣌ʹ੥ٻੜ੒͕Ͱ͖Δ͜ͱ • ඇಉظʹॲཧΛ࣮ߦͰ͖Δ͜ͱ • ੥ٻੜ੒ͷਐߦঢ়گ͕μογϡϘʔυͰ֬ೝͰ͖Δ͜ͱ • ࣦഊͨ͠৔߹ͷΤϥʔΛμογϡϘʔυͰ֬ೝͰ͖Δ͜ͱ • ෳ਺ͷδϣϒΛੵΉ͜ͱ͕Ͱ͖Δ͜ͱ

    ػೳཁ݅
49. None

50. • Why not Amazon SQS? • େྔͷΠϕϯτ͕ྲྀΕͯ͘ΔՄೳੑ͕ߴ͘ɺγϟʔυΛ௥Ճ͢Δ͜ͱͰεέʔϧ͕༰қͰ͋Δ఺ • 1ͭͷΠϕϯτͰෳ਺ͷॲཧΛ૸Β͍ͤͨχʔζ͕ࠓޙ͋Γ͏ΔͨΊ •

    ྫ) ؂ࠪϩά΋ग़͍ͨ͠ => ؂ࠪϩάΛग़ྗ͢ΔConsumerΛ௥Ճ͢Δ͜ͱͰରԠͰ͖Δ ͳͥKinesisΛ࠾༻͔ͨ͠ʁ ઃ ܭ ͷ ϙ Π ϯ τ

51. • νʔϜͰ࢖͍΍͘͢͢Δ޻෉ • ϨίʔυͷॲཧʹίϯγϡʔϚʔͷॲཧ͕ࣦഊͨ͠৔߹ɺνΣοΫϙΠϯτΛଧͨͳ͍͜ͱͰ ࣦഊ࣌ͷΤϥʔϋϯυϦϯάΛ͠΍͍ͯ͘͢͠Δ • ϦτϥΠ͠ଓ͚ΔϨίʔυ͕ൃੜͨ͠ࡍ͸ɺҎԼͷ3ͭͷ͍ͣΕ͔ͷରԠΛߦ͍ͬͯΔ • ର৅ͷϨίʔυΛਖ਼͘͠ॲཧͰ͖ΔΑ͏ʹίʔυΛमਖ਼ͯ͠ΞϓϦΛσϓϩΠ͠௚͢ •

    ର৅ͷϨίʔυΛਖ਼͘͠ॲཧͰ͖ΔΑ͏DBͷσʔλΛमਖ਼͢Δ • ର৅ͷϨίʔυࣗମʹ໰୊͕͋Δ৔߹͸ϨίʔυΛεΩοϓ͢Δӡ༻खॱΛ࣮ࢪ͢Δ ॳΊͯͷ Kinesis ࠾༻ ઃ ܭ ͷ ϙ Π ϯ τ

52. • ಉ͡ϨίʔυΛॏෳͯ͠ίϯγϡʔϚʔ͕ॲཧ͢ΔՄೳੑ͕͋Δ • ຊདྷ͸ႈ౳ੑΛอͭ͜ͱͰॏෳͷՄೳੑΛߟྀ͢΂͖ • δϣϒͷঢ়ଶ؅ཧͷඞཁੑͱ࣮૷ͷ౎߹্ͰɺॏෳΛ๷͙ख๏ΛऔΓೖΕͨ • Elasticacheʹॲཧͷঢ়گΛӬଓԽ͓ͯ͘͜͠ͱͰॏෳ͍ͯͨ͠Β஄͘࢓૊ΈΛऔ͍ͬͯΔ ॏෳνΣοΫʹAmazon ElastiCache

    Λ࠾༻ ઃ ܭ ͷ ϙ Π ϯ τ

53. • KinesisͱElasticacheΛ૊Έ߹ΘͤΔ͜ͱͰɺඇಉظʹδϣϒΛ࣮ߦ͠ͳ ͕ΒɺਐḿΛ؅ཧ͢Δ͜ͱ͕Մೳ • Τϥʔ͕ൃੜͨ͠Πϕϯτʹରͯ͠ɺνΣοΫϙΠϯτΛ͚ͭͳ͍͜ͱ ͰΤϥʔରԠ͕༰қʹͳͬͨ • ҰํͰΤϥʔʹͳͬͨϨίʔυ͕ετϦʔϜ಺ʹ଺ཹͯ͠͠·͏໰୊΋͋Δ • Τϥʔ༻ͷετϦʔϜʹྲྀͯ͠ɺޙଓΛ٧·Βͤͳ͍վળ͕ඞཁ

    ྑ͔ͬͨ఺/վળ఺ ݁ Ռ ੔ ߹ ੑ ʹ ଇ ͬ ͨ Ξ ϓ Ϧ έ ʔ γ ϣ ϯ Λ ॳ Ί ͯ ν ʔϜ Ͱ ։ ൃ Ͱ ͖ ͨ

54. • ΞϓϦέʔγϣϯ͕ྫ֎Λεϩʔͨ͠ͱ͖ʹɺKCLͰϋϯυϦϯά͠ͳ ͍ͱɺνΣοΫϙΠϯτΛ͏ͬͯ͠·͍ɺਖ਼͘͠ॲཧ͞Εͳ͍··ͷΠ ϕϯτ͕ൃੜͯ͠͠·͏ • ॲཧਖ਼ৗॲཧ׬ྃޙνΣοΫϙΠϯτ͕ͭ͘Α͏ʹྫ֎࣌͸νΣοΫϙΠϯτΛଧͨͣʹϦτϥΠ͢Δ ࢓૊ΈΛಋೖͨ͠ • Ϩίʔυͷ͋ͨΓͷαΠζʹ੍ݶ͕͋ΔͷͰαΠζͷେ͖͍ϨίʔυΛ Kinesis

    ʹ౤͛Δ৔߹͸஫ҙ • ੍ݶΛߟྀͯ͠ϦετΛௐ੔͢Δඞཁ͕͋ͬͨ • ຊདྷ͸1ཁૉ1Πϕϯτʹ͍͕ͨ͠ɺδϣϒͷதʹෳ਺ͷΠϕϯτ͕͋Δ৔߹ɺ࢝఺ͱऴ఺Λҙ࣮ࣝͨ͠ ૷͢Δඞཁ͕͋ΓɺϦετܕʹͨ͠ ͭ·͍ͮͨͱ͜Ζɺۤ࿑ͨ͠ͱ͜Ζ

55. © 2021, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. ͍͞͝ʹ

56. • Amazon EMR on Amazon Elastic Kubernetes Service (EKS) ͷಋೖ

    • EMR Λ࢖༻ͯ͠ɺϑϨʔϜϫʔΫͷΠϯετʔϧͱ؅ཧͳͲͷ޻਺Λ࡟ݮͰ͖ͦ͏ • Kinesis ͷԠ༻ • Τϥʔ࣌ͷϋϯυϦϯάͱͯ͠ผͷStreamʹΠϕϯτΛૹΔ • ิঈτϥϯβΫγϣϯͷ࣮૷ ࠓޙɺݕ౼/௅ઓ͍͖͍ͯͨ͜͠ͱ

57. © 2021, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. WE ARE HIRING.

58. Thank you! © 2021, Amazon Web Services, Inc. or its

    affiliates. All rights reserved.