Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Designing testable serverless apps using hexagonal architecture - NineUp

Designing testable serverless apps using hexagonal architecture - NineUp

2663a9ac90cc6ed420e0e1560db57782?s=128

Slobodan Stojanović

June 11, 2019
Tweet

Transcript

  1. What's the scariest thing about serverless?

  2. @slobodan_ Long-running tasks?

  3. @slobodan_ Compliance?

  4. @slobodan_ Using binaries and large dependencies?

  5. @slobodan_ Coldstart?

  6. @slobodan_ Cold start with VPN?

  7. @slobodan_ Node.js?

  8. @slobodan_ Node.js?

  9. @slobodan_ But, what about…

  10. @slobodan_ BIG

  11. @slobodan_ BAD

  12. @slobodan_ VENDOR LOCK-IN

  13. @slobodan_ What is vendor lock-in?

  14. @slobodan_ "In economics, vendor lock-in, makes a customer dependent on

    a vendor for products and services, unable to use another vendor without substantial switching costs."
  15. @slobodan_

  16. @slobodan_

  17. @slobodan_ A guy with a lot of servers.
 Let's call

    him Jeff.
  18. @slobodan_ A guy with a lot of servers.
 Let's call

    him Jeff.
  19. @slobodan_ Jeff is smart, and he knows how do you

    use his servers.
  20. @slobodan_ Jeff is smart, and he knows how do you

    use his servers.
  21. @slobodan_ Jeff is smart, and he knows how do you

    use his servers.
  22. @slobodan_ Jeff is smart, and he knows how do you

    use his servers.
  23. @slobodan_ Jeff is smart, and he knows how do you

    use his servers.
  24. @slobodan_ Jeff is smart, and he knows how do you

    use his servers.
  25. @slobodan_ Jeff is smart, and he knows how do you

    use his servers.
  26. @slobodan_ But what if Jeff is actually a villain?

  27. @slobodan_ Your wallet would not be happy…

  28. @slobodan_ Another guy with a lot of servers.
 Let's call

    him Bill.
  29. @slobodan_ Another guy with a lot of servers.
 Let's call

    him Bill.
  30. @slobodan_ Another guy with a lot of servers.
 Let's call

    him Bill.
  31. @slobodan_ Another guy with a lot of servers.
 Let's call

    him Bill.
  32. @slobodan_ Another guy with a lot of servers.
 Let's call

    him Bill.
  33. @slobodan_ That's vendor lock-in in the cloud

  34. @slobodan_ "My train of thought went like this: the term

    “lock-in” is misleading. We are really talking about switching costs." Mark Schwartz
 Enterprise Strategist at AWS
  35. @slobodan_ "As soon as you commit yourself to a platform

    or a vendor you will have switching costs if you later decide to change." Mark Schwartz
 Enterprise Strategist at AWS
  36. @slobodan_ Howtofight vendorlock-in?

  37. @slobodan_ Or, how to keep your switching costs reasonable?

  38. @slobodan_ • Planning and analysis • Good architecture • Deployment

    procedures How likely wi! I n"d to switch? What would be the cost?
  39. @slobodan_ That leads us to our topic…

  40. Designing testable serverless apps using hexagonal architecture

  41. @slobodan_ But, before we continue…

  42. Slobodan Stojanovic CTO @ Cloud Horizon & CTO @ Vacation

    Tracker co-author of Serverless Applications with Node.js book AWS Serverless Hero @slobodan_
  43. @slobodan_ Designing testable serverless apps using hexagonal architecture Designing testable

    serverless apps using hexagonal architecture
  44. @slobodan_ Designing testable serverless apps using hexagonal architecture

  45. @slobodan_ Why is testing important for serverless apps?

  46. @slobodan_ Most of the time serverless apps are not fully

    isolated monoliths without integrations
  47. @slobodan_ Instead, they contain many services interacting with each other

    and with external dependencies
  48. @slobodan_ An Example: Vacation Tracker

  49. @slobodan_ VacationTracker.io

  50. @slobodan_

  51. @slobodan_ Integrations can change every moment!

  52. @slobodan_ Tests don't prevent changes. They make sure your changes

    are not accidental.
  53. @slobodan_ But how do we prevent changes?

  54. @slobodan_ We can't. Our app needs to adopt fast! But

    we'! discuss that later today.
  55. @slobodan_ But how do you know what should you test

    in a serverless app?
  56. @slobodan_ Testing pyramid

  57. @slobodan_

  58. @slobodan_ Testing pyramid vs "Serverless testing pyramid"

  59. @slobodan_

  60. @slobodan_ Integration tests are cheaper, but also more important, because

    the common serverless app is split into many small pieces
  61. @slobodan_ Designing testable serverless apps using hexagonal architecture Designing testable

    serverless apps using hexagonal architecture
  62. @slobodan_ Designing testable serverless apps using hexagonal architecture

  63. @slobodan_ Ok, so which architecture is the best for serverless

    apps?
  64. @slobodan_ Any architecture that will let you test your serverless

    app easily and keep switching costs low.
  65. @slobodan_ Because sooner or later you'll need to switch/migrate pieces

    of your app.
  66. @slobodan_ Not to another cloud vendor, but to your new

    service, new or changed integration…
  67. @slobodan_ Risks to consider when building a serverless app

  68. @slobodan_ • configuration risks • technical workflow risks • business

    logic risks • integration risks
  69. @slobodan_ One of the architectures that fits these needs is


    Hexagonal Architecture or Ports and Adapters
  70. @slobodan_ Designing testable serverless apps using hexagonal architecture Designing testable

    serverless apps using hexagonal architecture
  71. @slobodan_ Designing testable serverless apps using hexagonal architecture

  72. @slobodan_ "Allow an application to equally be driven by users,

    programs, automated test or batch scripts, and to be developed and tested in isolation from its eventual run-time devices and databases." Alistair Cockburn
 Creator of Hexagonal architecture
  73. @slobodan_

  74. @slobodan_

  75. @slobodan_ Let's go back to Vacation Tracker for an example

  76. @slobodan_

  77. @slobodan_

  78. @slobodan_ Code, please!

  79. @slobodan_ const {
 httpResponse, parseApiEvent,
 SnsRepository
 } = require('../common')
 const

    main = require('./main')
 async function handler(event) { const { body, headers } = parseApiEvent(event) // Create instance of SNS notification repository
 const notification = new SnsRepository(
 process.env.topic ) // Invoke main function with all dependencies
 await main(body, headers, notification)
 return httpResponse()
 }
  80. @slobodan_ await main(body, headers, notification)

  81. @slobodan_ Unit tests

  82. @slobodan_ await main(body, headers, mockNotification) Mock notification repository instance Some

    static values
  83. @slobodan_ Integration tests

  84. @slobodan_ await main(body, headers, localNotification) Local notification adapter, using JS

    events for example Some static values
  85. @slobodan_ Simple and nice

  86. @slobodan_ What about End-to-End and UI tests?

  87. @slobodan_ Serverless is mainly a back end thing, but it

    can help with UI tests!
  88. @slobodan_ UI tests are slow and expensive

  89. @slobodan_ Benefits of serverless are cheap infrastructure and easy/fast parallelization

  90. @slobodan_

  91. @slobodan_ Or, you can use tools like cypress.io!

  92. @slobodan_ But, do you remember…

  93. @slobodan_ BIG

  94. @slobodan_ BAD

  95. @slobodan_ VENDOR LOCK-IN

  96. @slobodan_ How does hexagonal architecture help you fightingvendorlock-in?

  97. @slobodan_ How does hexagonal architecture help you to keep switching

    costs reasonable?
  98. @slobodan_ Story time

  99. @slobodan_ Vacation Tracker VacationTracker.io

  100. @slobodan_ • Serverless prototype • Small team (1 fulltime developer)

    • Initial product was Serverless chatbot + Express.js and MongoDB • Growing fast (200+ teams using it)
  101. @slobodan_ + A few bad decisions as a bonus :)

  102. @slobodan_ We did some migrations in past few months.

  103. @slobodan_ • Express API -> Serverless API migration • MongoDB

    -> DynamoDB migration For example:
  104. @slobodan_ Let's talk about MongoDB -> DynamoDB switch

  105. @slobodan_ Our MongoDB Repository has a certain interface.

  106. @slobodan_ For example, this: Returns a single user with its

    properties. const db = new MongoDbRepository(something) const user = db.getUser(userId)
  107. @slobodan_ We created DynamoDB Repository with the same interface.

  108. @slobodan_ Returns a single user with the same properties. const

    mdb = new MongoDbRepository(something) const ddb = new DynamoDbRepository(somethingElse) const user1 = mdb.getUser(userId) const user2 = ddb.getUser(userId) expect(user1).toEqual(user2) // They are equal! For example, this:
  109. @slobodan_ So we simply did the following:

  110. @slobodan_

  111. @slobodan_

  112. @slobodan_

  113. @slobodan_ But, how does this look like?

  114. @slobodan_ describe('DynamoDB repository', () => { describe('unit', () => {

    ... }) describe('integration', () => { beforeAll(() => { // Create test DB }) afterAll(() => { // Destroy test DB }) // Tests }) })
  115. @slobodan_ beforeAll(async () => { const params = { ...

    } await dynamoDb.createTable(params).promise() await dynamoDb.waitFor('tableExists', { TableName: tableName }).promise() })
  116. @slobodan_ afterAll(async () => { await dynamoDb.deleteTable({ TableName: tableName }).promise()

    await dynamoDb.waitFor('tableNotExists', { TableName: tableName }).promise() })
  117. @slobodan_ And we lived happily ever after…

  118. @slobodan_ Designing testable serverless apps using hexagonal architecture Designing testable

    serverless apps using hexagonal architecture
  119. @slobodan_ Designing testable serverless apps using hexagonal architecture Beyond testing

  120. @slobodan_ What should we do with things that can't be

    tested? For example, Slack changes an API while your app is in production
  121. @slobodan_ Make sure you are monitoring your app and tracking

    errors
  122. @slobodan_ Monitoring/error-tracking tools

  123. @slobodan_ • Built-in tools (CloudWatch, X-Ray) • Epsagon • IOpipe

    • Thundra • Lumigo • and many others
  124. @slobodan_ Serverless apps often heavily relies on front end, make

    sure you track front end errors too!
  125. @slobodan_ Also, services are smaller and smaller, but integrations require

    fine grained permissions.
  126. @slobodan_ There are tools that can help you to improve

    your permissions and keep your app secure.* * Protego, Puresec, and others.
  127. @slobodan_ But sometimes even monitoring can't help you!

  128. @slobodan_ You'll need a direct communication with your end users

    all the time!
  129. @slobodan_

  130. @slobodan_

  131. @slobodan_ Summary

  132. @slobodan_ • Good architecture helps you to maintain your switching

    costs low (or at least reasonable) • Hexagonal architecture is a nice fit for serverless apps • Test your integrations (and app in general) • Testing is not enough, you'll need monitoring and error tracking for your serverless apps
  133. @slobodan_ Links to the things I mentioned

  134. @slobodan_ • Mark Schwartz - Switching Costs and Lock-In: amzn.to/2S4iT3G

    • Hexagonal Architecture: bit.ly/hex-arch • Vacation Tracker: vacationtracker.io • Hexagonal Architecture at Vacation Tracker: bit.ly/vt-hex-arch • This presentation as an article: bit.ly/testable-serverless-apps
  135. @slobodan_ One more thing!

  136. @slobodan_

  137. @slobodan_ Serverless Days Belgrade Friday, September 13, 2019 serverlessbelgrade.com

  138. @slobodan_ And one more…

  139. @slobodan_ Serverless Applications with Node.js Use claudia40 promo code for

    40% off serverless.pub/book @slobodan_