Upgrade to Pro — share decks privately, control downloads, hide ads and more …

マイクロサービスを成功させるためのサーバーレスアーキテクチャ設計とNoSQLデータモデリング / Serverless Architecting and NoSQL Data Modeling for Successful Microservices

Masashi Terui
September 29, 2018

マイクロサービスを成功させるためのサーバーレスアーキテクチャ設計とNoSQLデータモデリング / Serverless Architecting and NoSQL Data Modeling for Successful Microservices

Serverlessconf Tokyo 2018

Masashi Terui

September 29, 2018
Tweet

More Decks by Masashi Terui

Other Decks in Technology

Transcript

  1. SERVERLESS ARCHITECTING
    AND NOSQL DATA MODELING
    FOR SUCCESSFUL MICROSERVICES
    MASASHI TERUI @ SERVERLESSCONF TOKYO 2018

    View full-size slide

  2. SERVERWORKS CO.,LTD.
    + FREELANCER
    • Serverless Oji-san
    • Serverless Framework Plugin Developer
    • Serverlessconf Tokyo 2016,2017,2018 speaker
    • Remote worker (in Sapporo-shi, Hokkaido)
    • The best Serverless Architect in Japan !!(Ͱ͋Γ͍ͨʣ
    MASASHI TERUI
    ARCHITECT / DEVELOPER

    View full-size slide

  3. SERVERLESS × MICROSERVICES
    ͦͷؔ܎͸ʁ

    View full-size slide

  4. SERVERLESS ARCHITECTURES (MARTINFOWLER.COM)
    “In the original version, all flow, control, and security was managed by the
    central server application. In the Serverless version there is no central
    arbiter of these concerns. Instead we see a preference for choreography
    over orchestration, with each component playing a more architecturally
    aware role—an idea also common in a microservices approach.”
    https://martinfowler.com/articles/serverless.html

    View full-size slide

  5. • Microservice(s)ͬͯ୯ޠ͕12ճ΋ग़ͯ͘Δ(͏ͪ3ճ͘Β͍͸ίϯςφͷจ຺)
    • “The orchestration of microservice workloads that execute a series of steps in a business
    process”
    • “Strongly promotes the microservices model, as most serverless runtimes enforce limits on
    the size or execution time of each individual function”
    • “Is the latency between the microservices, vs co-located features within a single deployment,
    an issue?”
    • “Once an application is split into multiple components, or microservices, you then have the
    freedom to deploy each one separately on completely different infrastructures, if that’s
    what’s best for your needs.”
    • “Likewise, each microservice can also be developed with the best technology (i.e. language)
    for its particular purpose. The freedom that comes with "breaking up of the monolith" brings
    new challenges though, and the following sections highlight some of the aspects that should
    be considered when choosing a platform and developing your microservices.”
    CNCF SERVERLESS WHITEPAPER

    View full-size slide

  6. • ෼ׂɾ౷࣏
    • Functionͷཻ౓
    • Serviceͷ੾Γํ
    • Observability
    • Monitoring
    • Log management
    • Traceability
    • ͭ·Δͱ͜ΖMicroservicesͱಉ͡՝୊Λ๊͍͑ͯΔ
    • ͜͜ʹڧ͍αʔϏεґଘͱ͍͏໰୊͕ՃΘΔ
    Ͱɺ࣮ࡍͷॴͲ͏ͳΜʁ

    View full-size slide

  7. SERVERLESS × NOSQL
    ͦͷؔ܎͸ʁ

    View full-size slide

  8. ࡢ೥ɺେ޷ධ(?)ͩͬͨ

    View full-size slide

  9. • ίωΫγϣϯϞσϧ
    • εςʔτϑϧ
    • ޮ཰తͳϓʔϦϯά͕Ͱ͖ͳ͍
    • ηΩϡϦςΟϞσϧ
    • ऑ͍ೝূڧ౓
    • VPC LambdaͷENIੜ੒Φʔόϔου
    • εέʔϦϯά
    • ·ͣਨ௚εέʔϧ
    • ݶఆతͳਫฏεέʔϧ
    FaaSͱRDBͷ૬ੑ͕ѱ͍ʁ

    View full-size slide

  10. • ίωΫγϣϯϞσϧ
    • εςʔτϨε
    • HTTP(S)
    • ηΩϡϦςΟ
    • ڧ͍ೝূڧ౓ʢIAMʣ
    • εέʔϦϯά
    • ਫฏεέʔϧ͕ϝΠϯ
    • ENIੜ੒Φʔόϔου͕ͱʹ͔͘ݫ͗ͯ͢͠ଞʹબ୒ࢶ͕ͳʢ͈́
    FaaSͱNoSQLͷ૬ੑ͕ྑ͍ʁ

    View full-size slide

  11. SERVERLESSͬͯԿ͕خ͍͠Μ͚ͩͬ…ʁ
    ͋Εʁ

    View full-size slide

  12. • Πϯϑϥɾϛυϧ΢ΣΞ؅ཧ͔Βͷ։์
    • ੍໿͕΋ͨΒ͢Ըܙ
    • ॊೈͳεέʔϧ
    • ֎෦αʔϏεͷ׆༻
    ͪΌΜͱ͋Γ·͢Α

    View full-size slide

  13. τϨʔυΦϑͳΘ͚Ͱ͢Α
    ݁ہͷͱ͜Ζ

    View full-size slide

  14. ͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠

    View full-size slide

  15. ͱ͜͜ͰऴΘͬͯ͠·ͬͯ͸
    ੈք͸มΘΒͳ͍
    ͜͜ͰऴΘͬͯ͠·͏ͳΜͪΌͬͯΞʔΩςΫτ/ίϯαϧλϯτ͕ଟ͗͢Ͱ͸ʁ

    View full-size slide

  16. SERVERLESS͕΋ͨΒͨ͠ຊ౰ͷՁ஋
    Ή͠Ζ͔͜͜Β͕ຊ൪

    View full-size slide

  17. EVENT DRIVEN & SOURCING

    View full-size slide

  18. ࢲୡ͕࡞Γ͍ͨγεςϜ͸ͳΜͩͬͨͷ͔ʁ

    View full-size slide

  19. ͜ΕͰྑ͍ͷ͔ʁ

    View full-size slide

  20. ͜Ε΋ͪΐͬͱҧ͏ͷͰ͸ʁ

    View full-size slide

  21. • ϝοηʔδϯά
    • ಉظ/ඇಉظ
    • Interactive/PubSub
    • σʔλετΞ
    • εέʔϦϯά
    • σʔλΞΫηε
    • ࣮ߦํ๏
    • ฒྻ/௚ྻ
    • ܾఆత/ඇܾఆత
    ໨తΛՌͨͨ͢Ίͷ࠷దղ͸ҟͳΔ

    View full-size slide

  22. • Ϋϥ΢υϕϯμ
    • SaaSϓϩόΠμ
    • BaaSϓϩόΠμ
    ઐ໳Ոʹ೚ͤΒΕΔͱ͜Ζ͸೚ͤΔ

    View full-size slide

  23. ͜ΕΒΛ্ख࣮͘ݱ͢Δඞཁ͕͋Δ

    View full-size slide

  24. BE EVENT DRIVEN

    View full-size slide

  25. ͢΂ͯ͸EVENTͱACTIONͷ૊Έ߹ΘͤͰ͋Δ

    View full-size slide

  26. • ಉظ
    • API Gateway
    • ඇಉظ
    • ௚ྻ
    • Kinesis Streams
    • ฒྻ
    • SNS
    • SQS
    • ͦͷଞ
    • Step Functions
    EVENTΛͲͷΑ͏ʹॲཧ͢΂͖͔ʁ

    View full-size slide

  27. ඇಉظΛϕʔεʹߟ͑Δ
    ࢦࣔͨ͠ΒऴΘΔ·ͰԿ΋ͤͣʹ଴ͬͯΔͳΒࣗ෼Ͱ΍Ε͹ྑ͍ͷͰ͸ʁ

    View full-size slide

  28. • ͭ·ΔॴΤϥʔϋϯυϦϯά
    • ϦτϥΠՄೳͳΤϥʔ͸ϦτϥΠ͢Δ
    • Ή͠ΖಉظͩͱϦτϥΠͮ͠Β͍
    • ϦτϥΠͰ͖ͳ͍Τϥʔ΋ҰͭͷΠϕϯτͰ͋Δ
    • ผͷܦ࿏ͰવΔ΂͖ॴʹ఻͑Δͱ͍͏͜ͱΛߟ͑Δ
    • ͦͯ͠ϞχλϦϯάɾτϨʔγϯά
    • ඞཁͳ৘ใ͸ू໿͢Δ
    • ଞͷਓ͕͍ͬͺ͍࿩ͯ͠ΔͷͰׂѪ
    ඇಉظɾ෼ࢄʹ͓͚Δෆ҆

    View full-size slide

  29. ͭ·Γ͜͏͍͏͜ͱ

    View full-size slide

  30. • CloudFormation
    • SAM
    • Serverless Framework
    • ґଘੑͷ؅ཧɺEventSourceͱAction(Lambda)ͷඥ͚ͮ
    • એݴత
    • Ref
    • ImportValue/ExportValue
    ͲͷΑ͏ʹ؅ཧ͢΂͖͔ʁ
    Environment Variables

    View full-size slide

  31. for SAM (Stream -> Action -> Publish)
    Parameters:
    AppEnv:
    Type: String
    Default: develop
    Globals:
    Function:
    Environment:
    Variables:
    SELF_STREAM: !Ref FooStream
    BAR_TOPIC
    Fn::ImportValue: !Sub "BarTopic-${AppEnv}"
    Outputs:
    SelfStream:
    Value: !Ref FooStream
    Export:
    Name: !Sub "FooStream-${AppEnv}"
    StageΛParameter
    ͱͯ͠ड͚औΔ
    ࣗ਎ʹඥͮ͘Event SourceΛ
    Export͢Δ
    ൃߦઌEvent SourceΛ
    Importͯ͠؀ڥม਺΁

    View full-size slide

  32. CloudFormationɺ࣮ʹྑ͘ग़དྷͯΔ
    ཉΛݴ͑͹΋ͬͱॻ͖΍͘͢ͳͬͯ΄͍͠ɾɾɾʂ

    View full-size slide

  33. • EventSourceͱActionͷ୯ҐͰStackΛ෼͚Δ
    • ໌֬ͳ෼཭ج४
    • એݴతͳඥ͚ͮ
    • EventSourceΛ઀ଓ఺ͱͯ͠ґଘؔ܎Λू໿͢Δ
    • ඇಉظͳؔ܎ੑΛॏࢹ͢Δ͜ͱͰґଘؔ܎ͷ޲͖͕ἧ͏
    • ॥؀ґଘ͕ͳ͘ͳΔʢσϓϩΠ͕ഁ୼͢Δʣ
    POINTS

    View full-size slide

  34. Like CREAN ARCHITECTURE
    ʮґଘؔ܎ͷ޲͖Λἧ͑Δʯͷ͕ྑ͍ઃܭΛอͭॏཁͳཁૉ
    ʢจ຺͸શવҧ͏͚Ͳɾɾɾʣ

    View full-size slide

  35. • εςʔδ໊ΛύϥϝʔλԽͯ͠εςʔδຖʹEventSourceΛೖΕସ͑ΒΕΔ
    • ఆٛΛશ͘ม͑ͣʹ࣮αʔϏεΛMockԽͨ͠Integration Test͕Ͱ͖Δ
    • Outputs͔ΒEventSourceΛऔಘͯ͠ςετ༻ͷEventΛൃߦ
    • ͦͷAction͔Βൃߦ͞ΕΔEventͷର৅΋ಉ༷ʹݕূՄೳͳ΋ͷʹஔ͖׵͑Δ
    • σʔλετΞ΋ஔ͖׵͑Մೳ
    • શͯΛ઀ଓͨ͠ςετ΋ཧ࿦্Մೳ͕ͩͦ΋ͦ΋ςετࣗମ͕ࠔ೉Ͱ͋Δ
    • EventͱͦΕʹର͢ΔActionΛอূ͢Δ͜ͱΛॏࢹ͢Δ
    • ↑͜ΕΛCDCతʹ୲อ͢Δྑ͍ํ๏͕͋Δͱخ͍͠
    Integration Test

    View full-size slide

  36. IN PRODUCTION

    View full-size slide

  37. IN INTEGRATION TEST ࣌ؒ͸͔͔ΔͷͰجຊCIԽ

    View full-size slide

  38. σʔλετΞ͸ɾɾɾʁ

    View full-size slide

  39. A. Ͳ͜Ͱ΋ྑ͍

    View full-size slide

  40. • “As emphasized earlier, most well designed applications
    require only one table, unless there is a specific reason
    for using multiple tables”
    • “Schema-less” ͷҙຯΛߟ͑Δ
    • List, Map͕࣋ͯΔ͜ͱʁ
    • શͯͷAttribute͕ඞਢͰͳ͍͜ͱʁ
    • A. ͳΜͰ΋ಥͬࠐΊΔ͜ͱ
    • ద੾ʹ෼ࢄͤ͞ΒΕΕ͹ςʔϒϧΛ෼͚Δඞཁ͕ͳ͍
    https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-general-nosql-design.html
    DynamoDB Best Practices

    View full-size slide

  41. • Event Sourcing
    • CQRS(Command and Query Responsibility Segregation Pattern)
    • Materialized View
    • Eventual Consistency
    ॏཁͳཁૉ

    View full-size slide

  42. ARCHITECTURE PATTERN AppSync͕͜Ε͔ΒॏཁͳҐஔΛ઎ΊΔ͸ͣ
    ͜͜͸ࢲࣗ਎·ͩ༧ײͰ͔͠ͳ͍

    View full-size slide

  43. • ΞϓϦέʔγϣϯͱσʔλϞσϧ͸ಉ࣌ʹઃܭ͢Δ
    • ॻ͖ࠐΈͱಡΈࠐΈͰಉ͡σʔλΛѻ͏ඞཁ͸ͳ͍
    • ॻ͖ࠐΈͷݪ୯ҐΛߟ͑ɺΞΠςϜʹ·ͱΊΔ͜ͱͰ੔߹ੑΛकΔ
    • ಡΈࠐΈ͸݁Ռ੔߹Λड͚ೖΕͯޮ཰ͷྑ͍σʔλΛੜ੒͍ͯ͘͠
    • ͜ͷݪଇ͸DynamoDB͚ͩʹద༻͞ΕΔ΋ͷͰ͸ͳ͍
    POINTS

    View full-size slide

  44. ௚ྻԽͯ͠RDBʹॻ͖ࠐΊ͹໰୊ͳ͍

    View full-size slide

  45. • ֤σʔλετΞͷಛੑ
    • DynamoDBͳΒPartition Key, Sort Keyͷ෼ࢄͷ࢓૊Έͱ͔
    • σʔλͷ੔߹ੑΛकΔ࢓૊Έ
    • ACIDτϥϯβΫγϣϯ
    • σʔλΞΫηε
    • B+Tree Index
    • σʔλϕʔεͷجຊతͳ஌ࣝ
    • ͜Ε͕ͳ͍ͱͦ΋ͦ΋ద੾ͳબ୒͕Ͱ͖ͳ͍
    ஌Δ΂͖͜ͱ

    View full-size slide

  46. DATA PLACEMENT Distributed by Partition Key

    and Indexed(B+tree) by Sort Key, LSI
    GSI is a projection of sorted(indexed) data

    View full-size slide

  47. B+TREE INDEX B-treeͳͷ͔B+treeͳͷ͔͸ެද͞Ε͍ͯͳ͍͚Ͳ
    Range Keyͱ͔ݺͿ͘Β͍͔ͩΒ…Ͷʁ

    View full-size slide

  48. TARGETING QUERY B+treeͷߏ଄Λ஌͍ͬͯΕ͹
    ૊Έ߹ΘͤΩʔͷΠϯσοΫε͕ແͯ͘΋
    ͜͏΍Ε͹ྑ͍͜ͱ͕෼͔Δ

    View full-size slide

  49. DATA ATOMICITY ͜ͷσʔλͷݪ୯Ґ͕αʔϏεΛލ͕ͳ͍͜ͱ͕ॏཁ
    σʔλετΞɾςʔϒϧͷ෼͚ํͰ͸ͳ͍
    ލ͕͍ͤͨͳΒMaterialized Viewͱͯ͠ίϐʔΛ࡞Δ
    User
    Profile
    User
    Status
    Name Email
    Read/Write
    Search
    Read/Write

    View full-size slide

  50. ৄ͘͠͸ “DynamoDB ދͷר” [ݕࡧ]
    http://marcy.hatenablog.com/entry/2018/07/31/213705

    View full-size slide

  51. “BE EVENT DRIVEN”
    “BECOME THE FRIEND OF NOSQL”
    THANKS!!
    “LET'S EXPAND SERVERLESS WORLD”

    View full-size slide