SERVERLESS ARCHITECTING AND NOSQL DATA MODELING FOR SUCCESSFUL MICROSERVICES MASASHI TERUI @ SERVERLESSCONF TOKYO 2018

SERVERWORKS CO.,LTD. + FREELANCER • Serverless Oji-san • Serverless Framework Plugin Developer • Serverlessconf Tokyo 2016,2017,2018 speaker • Remote worker (in Sapporo-shi, Hokkaido) • The best Serverless Architect in Japan !!(Ͱ͋Γ͍ͨʣ MASASHI TERUI ARCHITECT / DEVELOPER

SERVERLESS × MICROSERVICES ͦͷؔ܎͸ʁ

SERVERLESS ARCHITECTURES (MARTINFOWLER.COM) “In the original version, all flow, control, and security was managed by the central server application. In the Serverless version there is no central arbiter of these concerns. Instead we see a preference for choreography over orchestration, with each component playing a more architecturally aware role—an idea also common in a microservices approach.” https://martinfowler.com/articles/serverless.html

• Microservice(s)ͬͯ୯ޠ͕12ճ΋ग़ͯ͘Δ(͏ͪ3ճ͘Β͍͸ίϯςφͷจ຺) • “The orchestration of microservice workloads that execute a series of steps in a business process” • “Strongly promotes the microservices model, as most serverless runtimes enforce limits on the size or execution time of each individual function” • “Is the latency between the microservices, vs co-located features within a single deployment, an issue?” • “Once an application is split into multiple components, or microservices, you then have the freedom to deploy each one separately on completely different infrastructures, if that’s what’s best for your needs.” • “Likewise, each microservice can also be developed with the best technology (i.e. language) for its particular purpose. The freedom that comes with "breaking up of the monolith" brings new challenges though, and the following sections highlight some of the aspects that should be considered when choosing a platform and developing your microservices.” CNCF SERVERLESS WHITEPAPER

• ෼ׂɾ౷࣏ • Functionͷཻ౓ • Serviceͷ੾Γํ • Observability • Monitoring • Log management • Traceability • ͭ·Δͱ͜ΖMicroservicesͱಉ͡՝୊Λ๊͍͑ͯΔ • ͜͜ʹڧ͍αʔϏεґଘͱ͍͏໰୊͕ՃΘΔ Ͱɺ࣮ࡍͷॴͲ͏ͳΜʁ

SERVERLESS × NOSQL ͦͷؔ܎͸ʁ

ࡢ೥ɺେ޷ධ(?)ͩͬͨ

• ίωΫγϣϯϞσϧ • εςʔτϑϧ • ޮ཰తͳϓʔϦϯά͕Ͱ͖ͳ͍ • ηΩϡϦςΟϞσϧ • ऑ͍ೝূڧ౓ • VPC LambdaͷENIੜ੒Φʔόϔου • εέʔϦϯά • ·ͣਨ௚εέʔϧ • ݶఆతͳਫฏεέʔϧ FaaSͱRDBͷ૬ੑ͕ѱ͍ʁ

• ίωΫγϣϯϞσϧ • εςʔτϨε • HTTP(S) • ηΩϡϦςΟ • ڧ͍ೝূڧ౓ʢIAMʣ • εέʔϦϯά • ਫฏεέʔϧ͕ϝΠϯ • ENIੜ੒Φʔόϔου͕ͱʹ͔͘ݫ͗ͯ͢͠ଞʹબ୒ࢶ͕ͳʢ͈́ FaaSͱNoSQLͷ૬ੑ͕ྑ͍ʁ

SERVERLESSͬͯԿ͕خ͍͠Μ͚ͩͬ…ʁ ͋Εʁ

• Πϯϑϥɾϛυϧ΢ΣΞ؅ཧ͔Βͷ։์ • ੍໿͕΋ͨΒ͢Ըܙ • ॊೈͳεέʔϧ • ֎෦αʔϏεͷ׆༻ ͪΌΜͱ͋Γ·͢Α

τϨʔυΦϑͳΘ͚Ͱ͢Α ݁ہͷͱ͜Ζ

͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠

ͱ͜͜ͰऴΘͬͯ͠·ͬͯ͸ ੈք͸มΘΒͳ͍ ͜͜ͰऴΘͬͯ͠·͏ͳΜͪΌͬͯΞʔΩςΫτ/ίϯαϧλϯτ͕ଟ͗͢Ͱ͸ʁ

SERVERLESS͕΋ͨΒͨ͠ຊ౰ͷՁ஋ Ή͠Ζ͔͜͜Β͕ຊ൪

EVENT DRIVEN & SOURCING

ࢲୡ͕࡞Γ͍ͨγεςϜ͸ͳΜͩͬͨͷ͔ʁ

͜ΕͰྑ͍ͷ͔ʁ

͜Ε΋ͪΐͬͱҧ͏ͷͰ͸ʁ

͜ΕͰ͸ʁ

• ϝοηʔδϯά • ಉظ/ඇಉظ • Interactive/PubSub • σʔλετΞ • εέʔϦϯά • σʔλΞΫηε • ࣮ߦํ๏ • ฒྻ/௚ྻ • ܾఆత/ඇܾఆత ໨తΛՌͨͨ͢Ίͷ࠷దղ͸ҟͳΔ

• Ϋϥ΢υϕϯμ • SaaSϓϩόΠμ • BaaSϓϩόΠμ ઐ໳Ոʹ೚ͤΒΕΔͱ͜Ζ͸೚ͤΔ

͜ΕΒΛ্ख࣮͘ݱ͢Δඞཁ͕͋Δ

BE EVENT DRIVEN

͜Ε͕

͜͏ͳΔ

͜͏෼͚Δ

͢΂ͯ͸EVENTͱACTIONͷ૊Έ߹ΘͤͰ͋Δ

• ಉظ • API Gateway • ඇಉظ • ௚ྻ • Kinesis Streams • ฒྻ • SNS • SQS • ͦͷଞ • Step Functions EVENTΛͲͷΑ͏ʹॲཧ͢΂͖͔ʁ

ඇಉظΛϕʔεʹߟ͑Δ ࢦࣔͨ͠ΒऴΘΔ·ͰԿ΋ͤͣʹ଴ͬͯΔͳΒࣗ෼Ͱ΍Ε͹ྑ͍ͷͰ͸ʁ

• ͭ·ΔॴΤϥʔϋϯυϦϯά • ϦτϥΠՄೳͳΤϥʔ͸ϦτϥΠ͢Δ • Ή͠ΖಉظͩͱϦτϥΠͮ͠Β͍ • ϦτϥΠͰ͖ͳ͍Τϥʔ΋ҰͭͷΠϕϯτͰ͋Δ • ผͷܦ࿏ͰવΔ΂͖ॴʹ఻͑Δͱ͍͏͜ͱΛߟ͑Δ • ͦͯ͠ϞχλϦϯάɾτϨʔγϯά • ඞཁͳ৘ใ͸ू໿͢Δ • ଞͷਓ͕͍ͬͺ͍࿩ͯ͠ΔͷͰׂѪ ඇಉظɾ෼ࢄʹ͓͚Δෆ҆

ͭ·Γ͜͏͍͏͜ͱ

• CloudFormation • SAM • Serverless Framework • ґଘੑͷ؅ཧɺEventSourceͱAction(Lambda)ͷඥ͚ͮ • એݴత • Ref • ImportValue/ExportValue ͲͷΑ͏ʹ؅ཧ͢΂͖͔ʁ Environment Variables

for SAM (Stream -> Action -> Publish) Parameters: AppEnv: Type: String Default: develop Globals: Function: Environment: Variables: SELF_STREAM: !Ref FooStream BAR_TOPIC Fn::ImportValue: !Sub "BarTopic-${AppEnv}" Outputs: SelfStream: Value: !Ref FooStream Export: Name: !Sub "FooStream-${AppEnv}" StageΛParameter ͱͯ͠ड͚औΔ ࣗ਎ʹඥͮ͘Event SourceΛ Export͢Δ ൃߦઌEvent SourceΛ Importͯ͠؀ڥม਺΁

CloudFormationɺ࣮ʹྑ͘ग़དྷͯΔ ཉΛݴ͑͹΋ͬͱॻ͖΍͘͢ͳͬͯ΄͍͠ɾɾɾʂ

• EventSourceͱActionͷ୯ҐͰStackΛ෼͚Δ • ໌֬ͳ෼཭ج४ • એݴతͳඥ͚ͮ • EventSourceΛ઀ଓ఺ͱͯ͠ґଘؔ܎Λू໿͢Δ • ඇಉظͳؔ܎ੑΛॏࢹ͢Δ͜ͱͰґଘؔ܎ͷ޲͖͕ἧ͏ • ॥؀ґଘ͕ͳ͘ͳΔʢσϓϩΠ͕ഁ୼͢Δʣ POINTS

Like CREAN ARCHITECTURE ʮґଘؔ܎ͷ޲͖Λἧ͑Δʯͷ͕ྑ͍ઃܭΛอͭॏཁͳཁૉ ʢจ຺͸શવҧ͏͚Ͳɾɾɾʣ

• εςʔδ໊ΛύϥϝʔλԽͯ͠εςʔδຖʹEventSourceΛೖΕସ͑ΒΕΔ • ఆٛΛશ͘ม͑ͣʹ࣮αʔϏεΛMockԽͨ͠Integration Test͕Ͱ͖Δ • Outputs͔ΒEventSourceΛऔಘͯ͠ςετ༻ͷEventΛൃߦ • ͦͷAction͔Βൃߦ͞ΕΔEventͷର৅΋ಉ༷ʹݕূՄೳͳ΋ͷʹஔ͖׵͑Δ • σʔλετΞ΋ஔ͖׵͑Մೳ • શͯΛ઀ଓͨ͠ςετ΋ཧ࿦্Մೳ͕ͩͦ΋ͦ΋ςετࣗମ͕ࠔ೉Ͱ͋Δ • EventͱͦΕʹର͢ΔActionΛอূ͢Δ͜ͱΛॏࢹ͢Δ • ↑͜ΕΛCDCతʹ୲อ͢Δྑ͍ํ๏͕͋Δͱخ͍͠ Integration Test

IN PRODUCTION

IN INTEGRATION TEST ࣌ؒ͸͔͔ΔͷͰجຊCIԽ

σʔλετΞ͸ɾɾɾʁ

A. Ͳ͜Ͱ΋ྑ͍

• “As emphasized earlier, most well designed applications require only one table, unless there is a specific reason for using multiple tables” • “Schema-less” ͷҙຯΛߟ͑Δ • List, Map͕࣋ͯΔ͜ͱʁ • શͯͷAttribute͕ඞਢͰͳ͍͜ͱʁ • A. ͳΜͰ΋ಥͬࠐΊΔ͜ͱ • ద੾ʹ෼ࢄͤ͞ΒΕΕ͹ςʔϒϧΛ෼͚Δඞཁ͕ͳ͍ https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-general-nosql-design.html DynamoDB Best Practices

• Event Sourcing • CQRS(Command and Query Responsibility Segregation Pattern) • Materialized View • Eventual Consistency ॏཁͳཁૉ

ARCHITECTURE PATTERN AppSync͕͜Ε͔ΒॏཁͳҐஔΛ઎ΊΔ͸ͣ ͜͜͸ࢲࣗ਎·ͩ༧ײͰ͔͠ͳ͍

• ΞϓϦέʔγϣϯͱσʔλϞσϧ͸ಉ࣌ʹઃܭ͢Δ • ॻ͖ࠐΈͱಡΈࠐΈͰಉ͡σʔλΛѻ͏ඞཁ͸ͳ͍ • ॻ͖ࠐΈͷݪ୯ҐΛߟ͑ɺΞΠςϜʹ·ͱΊΔ͜ͱͰ੔߹ੑΛकΔ • ಡΈࠐΈ͸݁Ռ੔߹Λड͚ೖΕͯޮ཰ͷྑ͍σʔλΛੜ੒͍ͯ͘͠ • ͜ͷݪଇ͸DynamoDB͚ͩʹద༻͞ΕΔ΋ͷͰ͸ͳ͍ POINTS

௚ྻԽͯ͠RDBʹॻ͖ࠐΊ͹໰୊ͳ͍

• ֤σʔλετΞͷಛੑ • DynamoDBͳΒPartition Key, Sort Keyͷ෼ࢄͷ࢓૊Έͱ͔ • σʔλͷ੔߹ੑΛकΔ࢓૊Έ • ACIDτϥϯβΫγϣϯ • σʔλΞΫηε • B+Tree Index • σʔλϕʔεͷجຊతͳ஌ࣝ • ͜Ε͕ͳ͍ͱͦ΋ͦ΋ద੾ͳબ୒͕Ͱ͖ͳ͍ ஌Δ΂͖͜ͱ

DATA PLACEMENT Distributed by Partition Key
 and Indexed(B+tree) by Sort Key, LSI GSI is a projection of sorted(indexed) data

B+TREE INDEX B-treeͳͷ͔B+treeͳͷ͔͸ެද͞Ε͍ͯͳ͍͚Ͳ Range Keyͱ͔ݺͿ͘Β͍͔ͩΒ…Ͷʁ

TARGETING QUERY B+treeͷߏ଄Λ஌͍ͬͯΕ͹ ૊Έ߹ΘͤΩʔͷΠϯσοΫε͕ແͯ͘΋ ͜͏΍Ε͹ྑ͍͜ͱ͕෼͔Δ

DATA ATOMICITY ͜ͷσʔλͷݪ୯Ґ͕αʔϏεΛލ͕ͳ͍͜ͱ͕ॏཁ σʔλετΞɾςʔϒϧͷ෼͚ํͰ͸ͳ͍ ލ͕͍ͤͨͳΒMaterialized Viewͱͯ͠ίϐʔΛ࡞Δ User Profile User Status Name Email Read/Write Search Read/Write

ৄ͘͠͸ “DynamoDB ދͷר” [ݕࡧ] http://marcy.hatenablog.com/entry/2018/07/31/213705

“BE EVENT DRIVEN” “BECOME THE FRIEND OF NOSQL” THANKS!! “LET'S EXPAND SERVERLESS WORLD”