Designing testable serverless apps using hexagonal architecture

Designing testable serverless apps using hexagonal architecture

Presentation from Serverless Days Hamburg 2019 (https://hamburg.serverlessdays.io).

Description:

The main idea of hexagonal architecture is to allow an application to equally be driven by users, programs, automated test or batch scripts, and to be developed and tested in isolation from its eventual run-time devices and databases. On the other side, it is not trivial to run and test serverless applications locally. But what if we apply hexagonal architecture to serverless functions?

It seems they are natural fit!

With hexagonal architecture, your can design beautiful serverless functions, that can be run and tested locally or in the serverless environment. It also allows you to write unit tests without complex mocks, and as many integration tests as you need. As a bonus, your code looks clean and it’s easy to maintain, and hexagonal architecture helps you fighting vendor lock-in.

The main goal of this talk is to show you how to design testable serverless functions. It starts with explanation of hexagonal architecture and it’s importance for serverless apps, and it ends with real-world examples using Node.js.

2663a9ac90cc6ed420e0e1560db57782?s=128

Slobodan Stojanović

February 15, 2019
Tweet

Transcript

  1. What's the scariest thing about serverless?

  2. @slobodan_ Long-running tasks?

  3. @slobodan_ Compliance?

  4. @slobodan_ Using binaries and large dependencies?

  5. @slobodan_ Cold start?

  6. @slobodan_ Cold start with VPN?

  7. @slobodan_ Node.js?

  8. @slobodan_ But, what about…

  9. @slobodan_ BIG

  10. @slobodan_ BAD

  11. @slobodan_ VENDOR LOCK-IN

  12. @slobodan_

  13. @slobodan_ What is vendor lock-in?

  14. @slobodan_ "In economics, vendor lock-in, makes a customer dependent on

    a vendor for products and services, unable to use another vendor without substantial switching costs."
  15. @slobodan_

  16. @slobodan_

  17. @slobodan_

  18. @slobodan_ A guy with a lot of servers.
 Let's call

    him Jeff.
  19. @slobodan_ A guy with a lot of servers.
 Let's call

    him Jeff.
  20. @slobodan_ Jeff is smart, and he knows how do you

    use his servers.
  21. @slobodan_ Jeff is smart, and he knows how do you

    use his servers.
  22. @slobodan_ Jeff is smart, and he knows how do you

    use his servers.
  23. @slobodan_ Jeff is smart, and he knows how do you

    use his servers.
  24. @slobodan_ Jeff is smart, and he knows how do you

    use his servers.
  25. @slobodan_ Jeff is smart, and he knows how do you

    use his servers.
  26. @slobodan_ Jeff is smart, and he knows how do you

    use his servers.
  27. @slobodan_ But what if Jeff is actually a villain?

  28. @slobodan_ But what if Jeff is actually a villain?

  29. @slobodan_ But what if Jeff is actually a villain?

  30. @slobodan_ Your wallet would not be happy…

  31. @slobodan_

  32. @slobodan_ Another guy with a lot of servers.
 Let's call

    him Bill.
  33. @slobodan_ Another guy with a lot of servers.
 Let's call

    him Bill.
  34. @slobodan_ Another guy with a lot of servers.
 Let's call

    him Bill.
  35. @slobodan_ Another guy with a lot of servers.
 Let's call

    him Bill.
  36. @slobodan_ Another guy with a lot of servers.
 Let's call

    him Bill.
  37. @slobodan_ That's vendor lock-in in the cloud

  38. @slobodan_ "My train of thought went like this: the term

    “lock-in” is misleading. We are really talking about switching costs." Mark Schwartz
 Enterprise Strategist at AWS
  39. @slobodan_ "As soon as you commit yourself to a platform

    or a vendor you will have switching costs if you later decide to change." Mark Schwartz
 Enterprise Strategist at AWS
  40. @slobodan_ How to fight vendor lock-in?

  41. @slobodan_ Or how to keep your switching costs reasonable?

  42. @slobodan_ • Planning and analysis • Good architecture • Deployment

    procedures How likely will I need to switch? What would be the cost?
  43. @slobodan_ That leads us to our topic…

  44. Designing testable serverless apps using hexagonal architecture

  45. @slobodan_ But, before we continue…

  46. Slobodan Stojanovic CTO @ Cloud Horizon & CTO @ Vacation

    Tracker co-author of Serverless Applications with Node.js book AWS Serverless Hero @slobodan_ serverless.pub
  47. @slobodan_ Designing testable serverless apps using hexagonal architecture

  48. @slobodan_ Why is testing important for serverless apps?

  49. @slobodan_ Most of the time serverless apps are not fully

    isolated monoliths without integrations
  50. @slobodan_ Instead, they contain many services interacting with each other

    and with external dependencies
  51. @slobodan_

  52. @slobodan_ Integrations can change every moment!

  53. @slobodan_ Tests don't prevent changes. They make sure your changes

    are not accidental.
  54. @slobodan_ But how do you know what should you test

    in a serverless app?
  55. @slobodan_ Testing pyramid

  56. @slobodan_

  57. @slobodan_ Testing pyramid vs "Serverless testing pyramid"

  58. @slobodan_

  59. @slobodan_ Integration tests are cheaper, but also more important, because

    the common serverless app is split into many small pieces
  60. @slobodan_ Designing testable serverless apps using hexagonal architecture

  61. @slobodan_ Ok, so which architecture is the best for serverless

    apps?
  62. @slobodan_ Any architecture that will let you test your serverless

    app easily and keep switching costs low.
  63. @slobodan_ Because sooner or later you'll need to switch/migrate pieces

    of your app.
  64. @slobodan_ Not to another cloud vendor, but to your new

    service, new or changed integration…
  65. @slobodan_ Risks to consider when building a serverless app

  66. @slobodan_ • configuration risks • technical workflow risks • business

    logic risks • integration risks
  67. @slobodan_ One of the app architectures that fits these needs

    is
 hexagonal architecture
  68. @slobodan_ Designing testable serverless apps using hexagonal architecture

  69. @slobodan_ "Allow an application to equally be driven by users,

    programs, automated test or batch scripts, and to be developed and tested in isolation from its eventual run-time devices and databases." Alistair Cockburn
 Creator of Hexagonal architecture
  70. @slobodan_

  71. @slobodan_

  72. @slobodan_ An example

  73. @slobodan_

  74. @slobodan_

  75. @slobodan_

  76. @slobodan_ Code, please

  77. @slobodan_ const {
 httpResponse, parseApiEvent,
 SnsRepository
 } = require('../common')
 const

    main = require('./main')
 async function handler(event) { const { body, headers } = parseApiEvent(event) // Create instance of SNS notification repository
 const notification = new SnsRepository(
 process.env.topic ) // Invoke main function with all dependencies
 await main(body, headers, notification)
 return httpResponse()
 }
  78. @slobodan_ await main(body, headers, notification)

  79. @slobodan_ Unit tests

  80. @slobodan_ await main(body, headers, mockNotification) Mock notification repository instance Some

    static values
  81. @slobodan_ Integration tests

  82. @slobodan_ await main(body, headers, localNotification) Local notification adapter, using JS

    events for example Some static values
  83. @slobodan_ Simple and nice

  84. @slobodan_ But, do you remember…

  85. @slobodan_ BIG

  86. @slobodan_ BAD

  87. @slobodan_ VENDOR LOCK-IN

  88. @slobodan_

  89. @slobodan_ How does hexagonal architecture help you fighting vendor lock-in?

  90. @slobodan_ Story time

  91. @slobodan_ Vacation Tracker vacationtracker.io

  92. @slobodan_ vacationtracker.io

  93. @slobodan_ • Serverless prototype • Small team (1 fulltime developer)

    • Initial product was Serverless chatbot + Express.js and MongoDB • Growing fast (200+ teams using it)
  94. @slobodan_ • Express -> Serverless migration • MongoDB -> DynamoDB

    migration
  95. @slobodan_ Let's talk about MongoDB -> DynamoDB switch

  96. @slobodan_

  97. @slobodan_

  98. @slobodan_

  99. @slobodan_ How does this look like?

  100. @slobodan_ describe('DynamoDB repository', () => { describe('unit', () => {

    ... }) describe('integration', () => { beforeAll(() => { // Create test DB }) afterAll(() => { // Destroy test DB }) // Tests }) })
  101. @slobodan_ beforeAll(async () => { const params = { ...

    } await dynamoDb.createTable(params).promise() await dynamoDb.waitFor('tableExists', { TableName: tableName }).promise() })
  102. @slobodan_ afterAll(async () => { await dynamoDb.deleteTable({ TableName: tableName }).promise()

    await dynamoDb.waitFor('tableNotExists', { TableName: tableName }).promise() })
  103. @slobodan_ And they lived happily ever after…

  104. @slobodan_ Beyond testing

  105. @slobodan_ What should we do with things that can't be

    tested?
  106. @slobodan_ What should we do with things that can't be

    tested? For example, Google or someone else deprecated an API while your app is in production
  107. @slobodan_ Make sure you are monitoring your app and tracking

    errors
  108. @slobodan_ Monitoring/error-tracking tools

  109. @slobodan_ • Built-in tools (CloudWatch, X-Ray) • Epsagon • IOpipe

    • Thundra • Lumigo • and many others
  110. @slobodan_ Serverless apps often heavily relies on front end, in

    those cases you need to track front end errors as well
  111. @slobodan_ desole.io

  112. @slobodan_

  113. @slobodan_

  114. @slobodan_ Summary

  115. @slobodan_ • Good architecture helps you to maintain your switching

    costs low (or at least reasonable) • Hexagonal architecture is a nice fit for serverless apps • Test your integrations (and app in general) • Testing is not enough, you'll need monitoring and error tracking for your serverless apps
  116. @slobodan_ Links to the things I mentioned

  117. @slobodan_ • Mark Schwartz - Switching Costs and Lock-In: amzn.to/2S4iT3G

    • Hexagonal Architecture: bit.ly/hex-arch • Vacation Tracker: vacationtracker.io • Hexagonal Architecture at Vacation Tracker: bit.ly/vt-hex-arch • Desole (error-tracking tool): desole.io
  118. @slobodan_ One more thing!

  119. @slobodan_

  120. @slobodan_ Serverless Applications with Node.js Use claudia40 promo code for

    40% off serverless.pub/book @slobodan_
  121. @slobodan_ Thank you!

  122. @slobodan_ Thank you! Danke