Exploring the Testing Hyperpyramid with Kotlin & http4k

We all know that testing is an important factor in software development.

But.

What if your tests could also work for you in dimensions that you didn’t even know existed? What if they could give you superpowers? Superpowers like easily running an entire bank on a single developer workstation, generating visual documentation of all the interactions across your fleet of microservices with a simple test plugin, or building reusable infrastructure which allow you to test your entire codebase at the unit, integration or end-to-end level.

In this talk, you'll learn about why your design and technology choices matter for testing and some Kotlin and http4k techniques that allow you to do much more by actually writing less test code.

Transcript

1. Exploring
   the
   Testing
   Hyperpyramid
   with Kotlin
   & http4k
   David Denton | Ivan Sanchez
   KotlinConf 2023
   

   View Slide

2. View Slide

3. What
   we’re
   told
   about
   the
   Testing
   Pyramid
   Hard,
   slow,
   flakey
   

   View Slide

4. We
   think
   it’s
   more
   like
   *this!
   *Probably not what a real hyperpyramid looks like
   

   View Slide

5. What makes a good test?
   

   View Slide

6. Need to master:
   
   
   - Time
   
   
   - Randomness
   
   
   - Concurrency
   
   
   - Networking
   
   
   - Known state/data
   Barriers
   for
   good
   testing
   

   View Slide

7. Our example
   system:
   eCommerce
   Website
   ACTOR
   SERVICE CLOUD
   EXTERNAL
   

   View Slide

8. Technique:
   Layering
   with
   hexagons
   

   View Slide

9. Technique:
   Server as a
   function
   (Req) -> Rsp
   (Req) -> Rsp
   

   View Slide

10. How http4k
    implements
    Server as a
    Function
    typealias HttpHandler = (Request)
    ->
    Response
    
    
    

    View Slide

11. Technique:
    Events
    aka
    Observability
    Port interface Event
    
    
    typealias Events = (Event)
    ->
    Unit
    
    
    data class HttpCall(val uri: Uri) : Event
    

    View Slide

12. Technique:
    Screenplay
    actors
    interface Customer {
    
    
    fun listItems(): List
    
    
    fun order(id: ItemId, email: String): OrderId
    
    
    }
    

    View Slide

13. class HttpCustomer(baseUri: Uri, clock: Clock, events: Events, http: HttpHandler) : Customer {
    
    
    val http = ResetRequestTracing()
    
    
    .then(SetHostFrom(baseUri))
    
    
    .then(AppOutgoingHttp(AppEvents("Customer", clock, events), http))
    
    
    override fun listItems(): List {
    
    
    val response = http(Request(GET, "/list"))
    
    
    return Body.auto>>
    ().toLens()(response)
    
    
    }
    
    
    override fun order(id: ItemId, email: String): OrderId {
    
    
    val response = http(Request(POST, "/order/$id").query("email", email))
    
    
    return Body.auto().toLens()(response)
    
    
    }
    
    
    }
    Implementing the Customer
    

    View Slide

14. Technique:
    Fake
    Adapters
    fun FakeWarehouse(): HttpHandler {
    
    
    val stockLevels = mutableMapOf(ItemId.of("1") to 5)
    
    
    return routes(
    
    
    "/v1/items" bind GET to {
    
    
    Response(OK)
    
    
    .with(
    
    
    Body.auto> >
    ().toLens() of
    
    
    stockLevels.map { (item, stock)
    - >
    
    InventoryItem(item, "bar", stock)
    
    
    }
    
    
    )
    
    
    },
    
    
    "/v1/dispatch" bind POST to {
    
    
    val order = Body.auto().toLens()(it)
    
    
    stockLevels[order.id] = stockLevels[order.id]
    !!
    - order.amount
    
    
    Response(ACCEPTED)
    
    
    }
    
    
    )
    
    
    }
    
    
    fun main() {
    
    
    FakeWarehouse().asServer(SunHttp(8000)).start()
    
    
    }
    

    View Slide

15. Constructing
    our service
    fun ShopApi(env: Environment, clock: Clock, events: Events, http: HttpHandler): HttpHandler {
    
    
    val appEvents = AppEvents("shop", clock, events)
    
    
    val outgoingHttp = AppOutgoingHttp(appEvents, http)
    
    
    val shop = Shop(appEvents,
    
    
    Warehouse.Http(env[WAREHOUSE_URL], outgoingHttp),
    
    
    Notifications.SES(env, outgoingHttp)
    
    
    )
    
    
    return AppIncomingHttp(appEvents, routes(PlaceOrder(shop), ListAllItems(shop)))
    
    
    }
    

    View Slide

16. Service
    testing
    abstract class RecordTraces {
    
    
    @RegisterExtension
    
    
    val events = TracerBulletEvents(
    
    
    listOf(
    ::
    HttpTracer,
    ::
    DbTracer).map { it(ActorByService) },
    
    
    listOf(PumlSequenceDiagram, PumlInteractionDiagram),
    
    
    TraceRenderPersistence.FileSystem(File(".generated"))
    
    
    )
    
    
    }
    
    
    interface ListItemsScenario {
    
    
    val customer: Customer
    
    
    @Test
    
    
    fun `can list items`() {
    
    
    expectThat(customer.listItems()).isEqualTo(listOf(ItemId.of(“1")))
    
    
    }
    
    
    }
    
    
    class ShopApiTests : RecordTraces(), ListItemsScenario {
    
    
    val http: HttpHandler = ShopApi(ShopTestEnv, TestClock, events, FakeWarehouse())
    
    
    override val customer = HttpCustomer(Uri.of("http:
    //
    shop"), TestClock, events, http)
    
    
    }
    

    View Slide

17. Tests as
    diagrams as
    code
    

    View Slide

18. Service
    composition
    

    View Slide

19. Server System as a function
    class EcommerceSystem(env: Environment, clock: Clock, events: Events, theInternet: HttpHandler) : HttpHandler {
    
    
    private val networkAccess = NetworkAccess()
    
    
    private val apiGateway = ApiGateway(env, clock, events, networkAccess)
    
    
    private val shop = ShopApi(env, clock, events, networkAccess)
    
    
    private val warehouse = WarehouseApi(env, clock, events, networkAccess, Inventory.InMemory(events, clock))
    
    
    init {
    
    
    networkAccess.http = routes(
    
    
    reverseProxyRouting(
    
    
    env[API_GATEWAY_URL].authority to apiGateway,
    
    
    env[SHOP_URL].authority to shop,
    
    
    env[WAREHOUSE_URL].authority to warehouse
    
    
    ),
    
    
    Router.orElse bind theInternet,
    
    
    )
    
    
    }
    
    
    override fun invoke(request: Request) = networkAccess(request)
    
    
    }
    http://api-gateway/order/
    ECOMMERCE
    
    
    SYSTEM
    

    View Slide

20. Server
    Internet
    as a
    function
    http://s3/item-image-1
    class TheInternet: HttpHandler {
    
    
    val emails = Storage.InMemory>
    >
    ()
    
    
    val departmentStore = FakeDepartmentStore()
    
    
    val ses = FakeSES(emails)
    
    
    val s3 = FakeS3()
    
    
    val http = reverseProxy(
    
    
    "dept-store" to departmentStore,
    
    
    "email" to ses,
    
    
    "s3" to s3
    
    
    )
    
    
    override fun invoke(request: Request) = http(request)
    
    
    }
    THE
    INTERNET
    RP
    

    View Slide

21. Server
    System
    Internet
    Universe
    as a
    function
    R
    THE
    
    
    INTERNET
    UNIVERSE
    ECOMMERCE
    
    
    SYSTEM
    

    View Slide

22. Universe
    testing
    interface CustomerBuysItemScenario {
    
    
    val storeManager: StoreManager
    
    
    val customer: WebsiteCustomer
    
    
    @Test
    
    
    fun `views item, orders, receives confirmation`() {
    
    
    val itemId = customer.listItems().first()
    
    
    expectThat(customer.canSeeImage(itemId)).isTrue()
    
    
    val orderId = customer.order(itemId, "joe@kotlin.org")
    
    
    expectThat(customer.hasEmailFor(orderId, "joe@kotlin.org")).isTrue()
    
    
    expectThat(storeManager.hasOrderItems(orderId)).isEqualTo(listOf(itemId))
    
    
    }
    
    
    }
    
    
    class UniverseTests : RecordTraces(), CustomerBuysItemScenario {
    
    
    val iNet = TheInternet()
    
    
    val env = UniverseTestEnv overrides iNet.createCloudResourcesAndEnv()
    
    
    val clock = TestClock
    
    
    val system = EcommerceSystem(env, clock, events, iNet)
    
    
    override val customer = HttpWebsiteCustomer(env[API_GATEWAY_URL], clock, events, system, iNet.emails)
    
    
    override val storeManager = InternetStoreManager(iNet)
    
    
    }
    

    View Slide

23. Universe level
    diagrams
    

    View Slide

24. After the
    Big Bang!
    

    View Slide

25. Practices
    that
    make
    this
    easier
    - Monorepo
    
    - TDD (outside in)
    
    - Contract Testing (3rd party)
    
    - ADRs
    
    

    View Slide

26. What
    do
    we
    gain?
    - Shift-left of testing & observability
    
    - Massive reuse of testing infra
    
    - Higher level tests with the properties of UTs
    
    - Test one or more services in the same style
    
    - Free documentation!
    

    View Slide

27. http4k
    v5*
    and
    beyond
    *coming April ’23. Contains Loom. +
    

    View Slide

28. Thank you!

    Don’t forget

    to vote.
    David & Ivan
    Talk materials @ https://http4k.org/hyperpyramid
    

    View Slide