Flawless testing for the functional folks

Flawless testing for the functional folks

The year is 2019 and there have never been more people writing functional Scala. We’ve grown tired of frameworks, magic (reflection) and omnipotent testing libraries with hundreds of testing styles. I want to fight back against tests that stop on the first failed assertion. Against beforeAll and afterEach. I want to seek vengeance for unexpected, globally configured test parallelism and flaky tests. I want to rethink how we test pure and effective functional code and show another approach to doing it in Scala. In this talk, I will share what I’ve learned trying to make that approach become a reality. We’ll talk about the characteristics of test frameworks, functional API design and making trade-offs.

08f642741fba006656cb86fb61c160b3?s=128

Jakub Kozłowski

July 05, 2019
Tweet

Transcript

  1. Flawless testing for the functional folks Jakub Kozłowski https://bit.ly/2RXH05N

  2. Expectation Testing Reality

  3. object HelloTests extends TestSuite { val tests = Tests {

    test("1 + 2 == 3"){ assert(1 + 2 == 3) } test("2 + 1 == 3"){ assert(2 + 1 == 3) } } } Expectation Testing Reality
  4. object HelloTests extends TestSuite { val tests = Tests {

    test("1 + 2 == 3"){ assert(1 + 2 == 3) } test("2 + 1 == 3"){ assert(2 + 1 == 3) } } } implicit val sourceConfig = SourceConfig.singleSession[Int](Destination.queue("Input")) brokerAccess(brokerUrl).use { broker => (Deferred[IO, Unit], Deferred[IO, Unit], Deferred[IO, Unit], Ref[IO].of(Chain.empty[Int])).mapN { (firstReceived, secondReceived, brokerDown, consumedMessages) => def handleMessage(i: Int): IO[Unit] = consumedMessages.modify { alreadyConsumed => val action: IO[Unit] = alreadyConsumed.size match { case 0 => firstReceived.complete(()) *> brokerDown.get case 1 => secondReceived.complete(()) case _ => IO.unit } (alreadyConsumed.append(i), action) }.flatten val runStream = Consumer .stream[IO, Int] { case (_, i) => handleMessage(i).as(Nil) } .compile .drain val manageBroker = firstReceived.get *> //kill broker broker.shutdown *> //the consumer waits for this before trying to commit brokerDown.complete(()) *> //wait a sec to make sure the stream has time to realize the problem IO.sleep(1.second) *> //bring broker back broker.start *> //send another message sendMessages("Input")(2) *> secondReceived.get broker.start *> sendMessages("Input")(1) *> Deferred[IO, Unit].flatMap { stopStream => //run stream until promise is finished, then stop //in the meantime, run manageBroker, then finish promise (runStream race stopStream.get) &> manageBroker.guarantee(stopStream.complete(())) } *> consumedMessages.get.map { _.toList shouldBe List(1, 2) } } }.flatten.unsafeRunSync() Expectation Testing Reality
  5. Roadmap Traditional test frameworks Functional testing and the future Functional

    API design a + b a * b
  6. Functional API design

  7. Functional API design Algebraic data types + pure functions

  8. Algebraic data types

  9. Algebraic data types type Book = (Id, String) type Person

    = (String, List[Person])
  10. Algebraic data types type Boolean = True | False type

    Option a = Some a | None type Either a b = Left a | Right b type List a = Nil | Cons a (List a) type Book = (Id, String) type Person = (String, List[Person])
  11. Algebraic data types type Boolean = True | False type

    Option a = Some a | None type Either a b = Left a | Right b type List a = Nil | Cons a (List a) Coproducts ("one of") type Book = (Id, String) type Person = (String, List[Person]) Products ("all of")
  12. Algebraic data types type Boolean = True | False type

    Option a = Some a | None type Either a b = Left a | Right b type List a = Nil | Cons a (List a) Coproducts ("one of") Also called sum types type Book = (Id, String) type Person = (String, List[Person]) Products ("all of")
  13. We can encode any immutable value as an ADT type

    Nothing type Unit = Unit type Bit = Off | On type Byte = (Bit, Bit, Bit, Bit, Bit, Bit, Bit, Bit) type Char = Byte type String = List[Char]
  14. Why products and sums? (A, Option[B]) = Either[(A, B), A]

  15. Why products and sums? (A, Option[B]) = Either[(A, B), A]

    A * (1 + B) = (A * B) + A
  16. We can prove type equivalence/isomorphism with basic math Either[A, Option[B]]

    = Either[Option[A], B] (A, Option[B]) = Either[(A, B), A] Either[A, Either[B, C]] = Either[Either[A, B], C] Either[Option[A], Option[B]] = Either[A, Option[Option[B]]] Home exercise: prove these (and look for them in your code)
  17. Case study: messages for message queue - TextMessage (UTF-8 text)

    - ObjectMessage (Java Serializable objects) - Ping (no payload) Each one has both: - id: Long - timestamp: Instant
  18. Object oriented style Some fields are shared - abstracted away

    as interface/trait trait Message { def id: Long def timestamp: Instant } final case class TextMessage (id: Long, timestamp: Instant, payload: String) extends Message final case class ObjectMessage(id: Long, timestamp: Instant, payload: Serializable) extends Message final case class Ping (id: Long, timestamp: Instant) extends Message This is not an ADT
  19. Algebraic data types - naive attempt sealed trait Message extends

    Product with Serializable { def ident: Long = this match { case TextMessage(i, _, _) => i case ObjectMessage(i, _, _) => i case Ping(i, _) => i } def ts: Instant = ... } final case class TextMessage (id: Long, timestamp: Instant, payload: String) extends Message final case class ObjectMessage(id: Long, timestamp: Instant, payload: Serializable) extends Message final case class Ping (id: Long, timestamp: Instant) extends Message
  20. Put common fields together in a case class final case

    class Message(id: Long, timestamp: Instant, ...) Algebraic data types - reimagined
  21. Put common fields together in a case class final case

    class Message(id: Long, timestamp: Instant, payload: Payload) sealed trait Payload extends Product with Serializable object Payload { final case class Text(value: String) extends Payload final case class Object(value: Serializable) extends Payload case object Zilch extends Payload } Algebraic data types - reimagined Make the rest a coproduct
  22. Put common fields together in a case class Algebraic data

    types - reimagined Make the rest a coproduct Add friendly smart constructors object Message { def text(id: Long, timestamp: Instant, value: String ): Message = Message(id, timestamp, Payload.Text(value)) def obj (id: Long, timestamp: Instant, value: Serializable): Message = Message(id, timestamp, Payload.Object(value)) def ping(id: Long, timestamp: Instant ): Message = Message(id, timestamp, Payload.Zilch) } final case class Message(id: Long, timestamp: Instant, payload: Payload) sealed trait Payload extends Product with Serializable object Payload { final case class Text(value: String) extends Payload final case class Object(value: Serializable) extends Payload case object Zilch extends Payload }
  23. Need to know the exact type of a message? def

    showText(message: TextMessage): IO[Unit] = IO { ... } Before:
  24. Need to know the exact type of a message? def

    showText(message: TextMessage): IO[Unit] = IO { ... } Before: def showText(message: Message): IO[Unit] = IO { } After:
  25. Need to know the exact type of a message? def

    showText(message: TextMessage): IO[Unit] = IO { ... } Before: def showText(message: Message): IO[Unit] = IO { } After:
  26. final case class Message[P](id: Long, timestamp: Instant, payload: Payload[P]) object

    Message { def text(id: Long, timestamp: Instant, value: String) : Message[String] = Message(id, timestamp, Payload.Text(value)) def obj (id: Long, timestamp: Instant, value: Serializable): Message[Serializable] = Message(id, timestamp, Payload.Object(value)) def ping(id: Long, timestamp: Instant) : Message[Unit] = Message(id, timestamp, Payload.Zilch) } sealed trait Payload[P] extends Product with Serializable object Payload { final case class Text (value: String) extends Payload[String] final case class Object(value: Serializable) extends Payload[Serializable] case object Zilch extends Payload[Unit] } Generalized algebraic data types (GADTs) Here: Payload
  27. final case class Message[P](id: Long, timestamp: Instant, payload: Payload[P]) object

    Message { def text(id: Long, timestamp: Instant, value: String) : Message[String] = Message(id, timestamp, Payload.Text(value)) def obj (id: Long, timestamp: Instant, value: Serializable): Message[Serializable] = Message(id, timestamp, Payload.Object(value)) def ping(id: Long, timestamp: Instant) : Message[Unit] = Message(id, timestamp, Payload.Zilch) } sealed trait Payload[P] extends Product with Serializable object Payload { final case class Text (value: String) extends Payload[String] final case class Object(value: Serializable) extends Payload[Serializable] case object Zilch extends Payload[Unit] } Generalized algebraic data types (GADTs) Here: Payload
  28. Generalized algebraic data types (GADTs) Here: Payload final case class

    Message[P](id: Long, timestamp: Instant, payload: Payload[P]) object Message { def text(id: Long, timestamp: Instant, value: String) : Message[String] = Message(id, timestamp, Payload.Text(value)) def obj (id: Long, timestamp: Instant, value: Serializable): Message[Serializable] = Message(id, timestamp, Payload.Object(value)) def ping(id: Long, timestamp: Instant) : Message[Unit] = Message(id, timestamp, Payload.Zilch) } sealed trait Payload[P] extends Product with Serializable object Payload { final case class Text (value: String) extends Payload[String] final case class Object(value: Serializable) extends Payload[Serializable] case object Zilch extends Payload[Unit] }
  29. Functions on GADTs

  30. Functions on GADTs def showText(message: Message[String]): IO[Unit] = {

  31. Functions on GADTs def showText(message: Message[String]): IO[Unit] = { message.payload

    match { case Text(value) => IO { } } }
  32. Functions on GADTs def showText(message: Message[String]): IO[Unit] = { message.payload

    match { case Text(value) => IO { } } }
  33. ADT vs GADT vs generic type vs generic type with

    upper bound final case class Message( id: Long, timestamp: Instant, payload: Payload ) final case class Message[P]( id: Long, timestamp: Instant, payload: P ) final case class Message[P]( id: Long, timestamp: Instant, payload: Payload[P] ) (Payload) final case class Message[P <: Payload]( id: Long, timestamp: Instant, payload: P ) A C B D
  34. ADT vs GADT vs generic type vs generic type with

    upper bound final case class Message( id: Long, timestamp: Instant, payload: Payload ) final case class Message[P]( id: Long, timestamp: Instant, payload: P ) final case class Message[P]( id: Long, timestamp: Instant, payload: Payload[P] ) (Payload) final case class Message[P <: Payload]( id: Long, timestamp: Instant, payload: P ) ADT (A) GADT (B) Generic (C) Generic with bound (D) A C B D
  35. ADT vs GADT vs generic type vs generic type with

    upper bound final case class Message( id: Long, timestamp: Instant, payload: Payload ) final case class Message[P]( id: Long, timestamp: Instant, payload: P ) final case class Message[P]( id: Long, timestamp: Instant, payload: Payload[P] ) (Payload) final case class Message[P <: Payload]( id: Long, timestamp: Instant, payload: P ) ADT (A) GADT (B) Generic (C) Generic with bound (D) Constrain payload type A C B D
  36. ADT vs GADT vs generic type vs generic type with

    upper bound final case class Message( id: Long, timestamp: Instant, payload: Payload ) final case class Message[P]( id: Long, timestamp: Instant, payload: P ) final case class Message[P]( id: Long, timestamp: Instant, payload: Payload[P] ) (Payload) final case class Message[P <: Payload]( id: Long, timestamp: Instant, payload: P ) ADT (A) GADT (B) Generic (C) Generic with bound (D) Constrain payload type extends Payload A C B D
  37. ADT vs GADT vs generic type vs generic type with

    upper bound final case class Message( id: Long, timestamp: Instant, payload: Payload ) final case class Message[P]( id: Long, timestamp: Instant, payload: P ) final case class Message[P]( id: Long, timestamp: Instant, payload: Payload[P] ) (Payload) final case class Message[P <: Payload]( id: Long, timestamp: Instant, payload: P ) ADT (A) GADT (B) Generic (C) Generic with bound (D) Constrain payload type extends Payload ∃ A . A extends Payload[P] A C B D
  38. ADT vs GADT vs generic type vs generic type with

    upper bound final case class Message( id: Long, timestamp: Instant, payload: Payload ) final case class Message[P]( id: Long, timestamp: Instant, payload: P ) final case class Message[P]( id: Long, timestamp: Instant, payload: Payload[P] ) (Payload) final case class Message[P <: Payload]( id: Long, timestamp: Instant, payload: P ) ADT (A) GADT (B) Generic (C) Generic with bound (D) Constrain payload type extends Payload ∃ A . A extends Payload[P] Arbitrary A C B D
  39. ADT vs GADT vs generic type vs generic type with

    upper bound final case class Message( id: Long, timestamp: Instant, payload: Payload ) final case class Message[P]( id: Long, timestamp: Instant, payload: P ) final case class Message[P]( id: Long, timestamp: Instant, payload: Payload[P] ) (Payload) final case class Message[P <: Payload]( id: Long, timestamp: Instant, payload: P ) ADT (A) GADT (B) Generic (C) Generic with bound (D) Constrain payload type extends Payload ∃ A . A extends Payload[P] Arbitrary extends Payload A C B D
  40. ADT vs GADT vs generic type vs generic type with

    upper bound final case class Message( id: Long, timestamp: Instant, payload: Payload ) final case class Message[P]( id: Long, timestamp: Instant, payload: P ) final case class Message[P]( id: Long, timestamp: Instant, payload: Payload[P] ) (Payload) final case class Message[P <: Payload]( id: Long, timestamp: Instant, payload: P ) ADT (A) GADT (B) Generic (C) Generic with bound (D) Constrain payload type extends Payload ∃ A . A extends Payload[P] Arbitrary extends Payload Abstract over payload A C B D
  41. ADT vs GADT vs generic type vs generic type with

    upper bound final case class Message( id: Long, timestamp: Instant, payload: Payload ) final case class Message[P]( id: Long, timestamp: Instant, payload: P ) final case class Message[P]( id: Long, timestamp: Instant, payload: Payload[P] ) (Payload) final case class Message[P <: Payload]( id: Long, timestamp: Instant, payload: P ) ADT (A) GADT (B) Generic (C) Generic with bound (D) Constrain payload type extends Payload ∃ A . A extends Payload[P] Arbitrary extends Payload Abstract over payload ❌ A C B D
  42. ADT vs GADT vs generic type vs generic type with

    upper bound final case class Message( id: Long, timestamp: Instant, payload: Payload ) final case class Message[P]( id: Long, timestamp: Instant, payload: P ) final case class Message[P]( id: Long, timestamp: Instant, payload: Payload[P] ) (Payload) final case class Message[P <: Payload]( id: Long, timestamp: Instant, payload: P ) ADT (A) GADT (B) Generic (C) Generic with bound (D) Constrain payload type extends Payload ∃ A . A extends Payload[P] Arbitrary extends Payload Abstract over payload ❌ ✅ ✅ ✅ A C B D
  43. ADT vs GADT vs generic type vs generic type with

    upper bound final case class Message( id: Long, timestamp: Instant, payload: Payload ) final case class Message[P]( id: Long, timestamp: Instant, payload: P ) final case class Message[P]( id: Long, timestamp: Instant, payload: Payload[P] ) (Payload) final case class Message[P <: Payload]( id: Long, timestamp: Instant, payload: P ) ADT (A) GADT (B) Generic (C) Generic with bound (D) Constrain payload type extends Payload ∃ A . A extends Payload[P] Arbitrary extends Payload Abstract over payload ❌ ✅ ✅ ✅ Singleton types in payload abstraction A C B D
  44. ADT vs GADT vs generic type vs generic type with

    upper bound final case class Message( id: Long, timestamp: Instant, payload: Payload ) final case class Message[P]( id: Long, timestamp: Instant, payload: P ) final case class Message[P]( id: Long, timestamp: Instant, payload: Payload[P] ) (Payload) final case class Message[P <: Payload]( id: Long, timestamp: Instant, payload: P ) ADT (A) GADT (B) Generic (C) Generic with bound (D) Constrain payload type extends Payload ∃ A . A extends Payload[P] Arbitrary extends Payload Abstract over payload ❌ ✅ ✅ ✅ Singleton types in payload abstraction N/A A C B D
  45. ADT vs GADT vs generic type vs generic type with

    upper bound final case class Message( id: Long, timestamp: Instant, payload: Payload ) final case class Message[P]( id: Long, timestamp: Instant, payload: P ) final case class Message[P]( id: Long, timestamp: Instant, payload: Payload[P] ) (Payload) final case class Message[P <: Payload]( id: Long, timestamp: Instant, payload: P ) ADT (A) GADT (B) Generic (C) Generic with bound (D) Constrain payload type extends Payload ∃ A . A extends Payload[P] Arbitrary extends Payload Abstract over payload ❌ ✅ ✅ ✅ Singleton types in payload abstraction N/A Only if children of Payload support them A C B D
  46. ADT vs GADT vs generic type vs generic type with

    upper bound final case class Message( id: Long, timestamp: Instant, payload: Payload ) final case class Message[P]( id: Long, timestamp: Instant, payload: P ) final case class Message[P]( id: Long, timestamp: Instant, payload: Payload[P] ) (Payload) final case class Message[P <: Payload]( id: Long, timestamp: Instant, payload: P ) ADT (A) GADT (B) Generic (C) Generic with bound (D) Constrain payload type extends Payload ∃ A . A extends Payload[P] Arbitrary extends Payload Abstract over payload ❌ ✅ ✅ ✅ Singleton types in payload abstraction N/A Only if children of Payload support them ✅ ✅ A C B D
  47. Functions class CreditCard(var credit: Int) def pay(amount: PositiveInt, card: CreditCard):

    Unit = card.credit -= amount.value def program() = { val card = new CreditCard(300) pay(100, card) pay(200, card) println(getBalance(card)) //0 }
  48. Functions? class CreditCard(var credit: Int) def pay(amount: PositiveInt, card: CreditCard):

    Unit = card.credit -= amount.value def program() = { val card = new CreditCard(300) pay(100, card) println(card.credit) //200 }
  49. Functions? class CreditCard(var credit: Int) def pay(amount: PositiveInt, card: CreditCard):

    Unit = card.credit -= amount.value def program() = { val card = new CreditCard(300) pay(100, card) println(card.credit) //200 }
  50. Not functions: referential transparency broken class CreditCard(var credit: Int) def

    pay(amount: PositiveInt, card: CreditCard): Unit = card.credit -= amount.value def program() = { pay(100, new CreditCard(300)) println(new CreditCard(300).credit) //300 }
  51. Pure functions case class CreditCard(credit: Int) def pay(amount: PositiveInt, card:

    CreditCard): CreditCard = card.copy(credit = card.credit - amount.value) def program() = { val card = CreditCard(300) val newCard = pay(100, card) println(newCard.credit) //200 println(card.credit) //300 }
  52. Pure functions case class CreditCard(credit: Int) def pay(amount: PositiveInt, card:

    CreditCard): CreditCard = card.copy(credit = card.credit - amount.value) def program() = { val card = CreditCard(300) val newCard = pay(100, card) println(newCard.credit) //200 println(card.credit) //300 }
  53. Pure functions case class CreditCard(credit: Int) def pay(amount: PositiveInt, card:

    CreditCard): CreditCard = card.copy(credit = card.credit - amount.value) def program() = { val card = CreditCard(300) val newCard = pay(100, card) println(newCard.credit) //200 println(card.credit) //300 } Q: What about chaining operations? A: State monad, beyond the scope of this talk (see https://www.youtube.com/watch?v=Pgo73GfHk0U)
  54. Pure functions case class CreditCard(credit: Int) def pay(amount: PositiveInt, card:

    CreditCard): CreditCard = card.copy(credit = card.credit - amount.value) def program() = { val card = CreditCard(300) val newCard = pay(100, card) println(newCard.credit) //200 println(card.credit) //300 } Q: What about chaining operations? A: State monad, beyond the scope of this talk (see https://www.youtube.com/watch?v=Pgo73GfHk0U)
  55. Pure functions case class CreditCard(credit: Int) def pay(amount: PositiveInt, card:

    CreditCard): CreditCard = card.copy(credit = card.credit - amount.value) def program() = { val newCard = pay(100, CreditCard(300)) println(newCard.credit) //200 println(CreditCard(300).credit) //300 }
  56. Alright, hold on dude, these functions are pure because they

    don't interact with the real world! - You, 2019
  57. def program() = newDatabase(initialCredit = 300).flatMap { db => val

    card = CardId(1) for { _ <- pay(100, card, db) _ <- pay(100, card, db) credit <- getCredit(card, db) _ <- putStrLn(credit) //200 } yield () } Pure functions in the real world
  58. def program() = newDatabase(initialCredit = 300).flatMap { db => val

    card = CardId(1) for { _ <- pay(100, card, db) _ <- pay(100, card, db) credit <- getCredit(card, db) _ <- putStrLn(credit) //200 } yield () } Pure functions in the real world def pay(amount: PositiveInt, card: CardId, db: Database): IO[Unit] = db.update(_ |+| Map(card -> CreditCardDB(amount.value))) def getCredit(card: CardId, db: Database): IO[Int] = db.get.map(_.get(card).map(_.credit).getOrElse(0))
  59. def program() = newDatabase(initialCredit = 300).flatMap { db => val

    card = CardId(1) for { _ <- pay(100, card, db) _ <- pay(100, card, db) credit <- getCredit(card, db) _ <- putStrLn(credit) //200 } yield () } Pure functions in the real world def pay(amount: PositiveInt, card: CardId, db: Database): IO[Unit] = db.update(_ |+| Map(card -> CreditCardDB(amount.value))) def getCredit(card: CardId, db: Database): IO[Int] = db.get.map(_.get(card).map(_.credit).getOrElse(0)) def program() = newDatabase(initialCredit = 300).flatMap { db => val card = CardId(1)
  60. def program() = newDatabase(initialCredit = 300).flatMap { db => val

    card = CardId(1) for { _ <- pay(100, card, db) _ <- pay(100, card, db) credit <- getCredit(card, db) _ <- putStrLn(credit) //200 } yield () } Pure functions in the real world def pay(amount: PositiveInt, card: CardId, db: Database): IO[Unit] = db.update(_ |+| Map(card -> CreditCardDB(amount.value))) def getCredit(card: CardId, db: Database): IO[Int] = db.get.map(_.get(card).map(_.credit).getOrElse(0)) def program() = newDatabase(initialCredit = 300).flatMap { db => val card = CardId(1) for { _ <- pay(100, card, db) _ <- pay(100, card, db)
  61. def program() = newDatabase(initialCredit = 300).flatMap { db => val

    card = CardId(1) for { _ <- pay(100, card, db) _ <- pay(100, card, db) credit <- getCredit(card, db) _ <- putStrLn(credit) //200 } yield () } Pure functions in the real world def pay(amount: PositiveInt, card: CardId, db: Database): IO[Unit] = db.update(_ |+| Map(card -> CreditCardDB(amount.value))) def getCredit(card: CardId, db: Database): IO[Int] = db.get.map(_.get(card).map(_.credit).getOrElse(0)) def program() = newDatabase(initialCredit = 300).flatMap { db => val card = CardId(1) for { _ <- pay(100, card, db) _ <- pay(100, card, db) credit <- getCredit(card, db)
  62. def program() = newDatabase(initialCredit = 300).flatMap { db => val

    card = CardId(1) for { _ <- pay(100, card, db) _ <- pay(100, card, db) credit <- getCredit(card, db) _ <- putStrLn(credit) //200 } yield () } Pure functions in the real world def pay(amount: PositiveInt, card: CardId, db: Database): IO[Unit] = db.update(_ |+| Map(card -> CreditCardDB(amount.value))) def getCredit(card: CardId, db: Database): IO[Int] = db.get.map(_.get(card).map(_.credit).getOrElse(0)) def program() = newDatabase(initialCredit = 300).flatMap { db => val card = CardId(1) for { _ <- pay(100, card, db) _ <- pay(100, card, db) credit <- getCredit(card, db) _ <- putStrLn(credit) //200 } yield () }
  63. Pure functions in the real world def program() = newDatabase(initialCredit

    = 300).flatMap { db => val card = CardId(1) for { _ <- pay(100, card, db) _ <- pay(100, card, db) credit <- getCredit(card, db) _ <- putStrLn(credit) //200 } yield () }
  64. Pure functions in the real world def program() = newDatabase(initialCredit

    = 300).flatMap { db => val card = CardId(1) for { _ <- pay(100, card, db) _ <- pay(100, card, db) credit <- getCredit(card, db) _ <- putStrLn(credit) //200 } yield () }
  65. Pure functions in the real world def program() = newDatabase(initialCredit

    = 300).flatMap { db => val card = CardId(1) val pay100 = pay(100, card, db) for { _ <- pay100 _ <- pay100 credit <- getCredit(card, db) _ <- putStrLn(credit) //still 200 } yield () } ✅
  66. Traditional testing frameworks DSL Registering suites Parallelism Lifecycle management

  67. Traditional testing frameworks DSL - exceptions Registering suites Parallelism Lifecycle

    management test("list") { val list = List("foo", "bar") //fails ✅ assert(list.size == 3) //never gets run ❌ assert(list.forall(_ == "moo")) }
  68. Traditional testing frameworks DSL - exceptions, side effects Registering suites

    Parallelism Lifecycle management val tests = Tests { test("hello world") { 1 shouldBe 1 } test("hello world 2") { 1 shouldBe 1 } } val tests = Tests { test("hello world") { 1 shouldBe 1 } test("hello world 2") { 1 shouldBe 1 } }
  69. Traditional testing frameworks DSL - exceptions, side effects Registering suites

    Parallelism Lifecycle management val tests = Tests { test("hello world") { 1 shouldBe 1 } test("hello world 2") { 1 shouldBe 1 } } val tests = Tests { test("hello world") { 1 shouldBe 1 } test("hello world 2") { 1 shouldBe 1 } }
  70. Traditional testing frameworks DSL - exceptions, side effects Registering suites

    Parallelism Lifecycle management val tests = Tests { test("hello world") { 1 shouldBe 1 } test("hello world 2") { 1 shouldBe 1 } } val test2 = test("hello world 2") { 1 shouldBe 1 } val tests = Tests { test("hello world") { 1 shouldBe 1 } test2 //nope }
  71. def test(body: => Assertion): Unit Functional "Thunk-ctional" programming

  72. Traditional testing frameworks DSL - exceptions, side effects Registering suites

    - automatic Parallelism Lifecycle management //scalatest class ContainerSpec extends WordSpec //specs2 class ContainerSpec extends Specification //minitest object ContainerSpec extends SimpleTestSuite //µtest class ContainerSpec extends TestSuite
  73. Traditional testing frameworks DSL - exceptions, side effects Registering suites

    - automatic Parallelism - globally configured Lifecycle management //per sbt module parallelExecution in Test := false //per build concurrentRestrictions in Global += Tags.exclusive(Tags.Test)
  74. Traditional testing frameworks DSL - exceptions, side effects Registering suites

    - automatic Parallelism - globally configured Lifecycle management - hooks object MyTestSuite extends TestSuite[Int] { private var system: ActorSystem = _ override def setupSuite(): Unit = { system = ActorSystem.create() } override def tearDownSuite(): Unit = { TestKit.shutdownActorSystem(system) system = null } }
  75. Traditional testing frameworks DSL - exceptions, side effects Registering suites

    - automatic Parallelism - globally configured Lifecycle management - hooks
  76. Functional testing libraries DSL Registering suites Parallelism Lifecycle management

  77. val bigTest = test("big test") { for { //given time

    <- IO(ZonedDateTime.now()) timeProvider = TimeProvider.const(time) log <- Ref[IO].of(List.empty[AppEvent]) eventSender = EventSender.instance(e => log.update(_ :+ e)) videoService = VideoService.make(eventSender, timeProvider) //when rentResult <- videoService.rent(VideoId(1)) logged <- log.get } yield { val failedToRent = rentResult.shouldBe(Failed(VideoId(1))) val sentEvent = logged.shouldBe(List(AppEvent.FailedRent(time))) //then failedToRent |+| sentEvent } } val suite = tests( pureTest("hello") { testedFunction(1) shouldBe List(1) }, pureTest("hello 2") { val result = testedFunction(2) result.size.shouldBe(2) |+| result.shouldBe(List(2, 2)) }, bigTest ) Functional testing libraries DSL - purely functional, no exceptions Registering suites Parallelism Lifecycle management
  78. val bigTest = test("big test") { for { //given time

    <- IO(ZonedDateTime.now()) timeProvider = TimeProvider.const(time) log <- Ref[IO].of(List.empty[AppEvent]) eventSender = EventSender.instance(e => log.update(_ :+ e)) videoService = VideoService.make(eventSender, timeProvider) //when rentResult <- videoService.rent(VideoId(1)) logged <- log.get } yield { val failedToRent = rentResult.shouldBe(Failed(VideoId(1))) val sentEvent = logged.shouldBe(List(AppEvent.FailedRent(time))) //then failedToRent |+| sentEvent } } val suite = tests( pureTest("hello") { testedFunction(1) shouldBe List(1) }, pureTest("hello 2") { val result = testedFunction(2) result.size.shouldBe(2) |+| result.shouldBe(List(2, 2)) }, bigTest ) Functional testing libraries DSL - purely functional, no exceptions Registering suites Parallelism Lifecycle management
  79. s val bigTest = test("big test") { for { //given

    time <- IO(ZonedDateTime.now()) timeProvider = TimeProvider.const(time) log <- Ref[IO].of(List.empty[AppEvent]) eventSender = EventSender.instance(e => log.update(_ :+ e)) videoService = VideoService.make(eventSender, timeProvider) //when rentResult <- videoService.rent(VideoId(1)) logged <- log.get } yield { val failedToRent = rentResult.shouldBe(Failed(VideoId(1))) val sentEvent = logged.shouldBe(List(AppEvent.FailedRent(time))) //then failedToRent |+| sentEvent } } val suite = tests( pureTest("hello") { testedFunction(1) shouldBe List(1) }, pureTest("hello 2") { val result = testedFunction(2) result.size.shouldBe(2) |+| result.shouldBe(List(2, 2)) }, bigTest ) Functional testing libraries DSL - purely functional, no exceptions Registering suites Parallelism Lifecycle management
  80. val bigTest = test("big test") { for { //given time

    <- IO(ZonedDateTime.now()) timeProvider = TimeProvider.const(time) log <- Ref[IO].of(List.empty[AppEvent]) eventSender = EventSender.instance(e => log.update(_ :+ e)) videoService = VideoService.make(eventSender, timeProvider) //when rentResult <- videoService.rent(VideoId(1)) logged <- log.get } yield { val failedToRent = rentResult.shouldBe(Failed(VideoId(1))) val sentEvent = logged.shouldBe(List(AppEvent.FailedRent(time))) //then failedToRent |+| sentEvent } } val suite = tests( pureTest("hello") { testedFunction(1) shouldBe List(1) }, pureTest("hello 2") { val result = testedFunction(2) result.size.shouldBe(2) |+| result.shouldBe(List(2, 2)) }, bigTest ) Functional testing libraries DSL - purely functional, no exceptions Registering suites Parallelism Lifecycle management
  81. val bigTest = test("big test") { for { //given time

    <- IO(ZonedDateTime.now()) timeProvider = TimeProvider.const(time) log <- Ref[IO].of(List.empty[AppEvent]) eventSender = EventSender.instance(e => log.update(_ :+ e)) videoService = VideoService.make(eventSender, timeProvider) //when rentResult <- videoService.rent(VideoId(1)) logged <- log.get } yield { val failedToRent = rentResult.shouldBe(Failed(VideoId(1))) val sentEvent = logged.shouldBe(List(AppEvent.FailedRent(time))) //then failedToRent |+| sentEvent } } val suite = tests( pureTest("hello") { testedFunction(1) shouldBe List(1) }, pureTest("hello 2") { val result = testedFunction(2) result.size.shouldBe(2) |+| result.shouldBe(List(2, 2)) }, bigTest ) Functional testing libraries DSL - purely functional, no exceptions Registering suites Parallelism Lifecycle management
  82. Functional testing libraries DSL - purely functional, no exceptions Registering

    suites Parallelism Lifecycle management pureTest("simple things") { ((2 + 2) shouldBe 3) |+| (2 shouldBe 3) }
  83. Functional testing libraries DSL - purely functional, no exceptions Registering

    suites Parallelism Lifecycle management pureTest("simple things") { ((2 + 2) shouldBe 3) |+| (2 shouldBe 3) } Never lose failures again
  84. Functional testing libraries DSL - purely functional, no exceptions Registering

    suites - manual Parallelism Lifecycle management object FlawlessTests extends IOApp { def run(args: List[String]): IO[ExitCode] = runTests(args) { val sequentialTests = NonEmptyList.of( GetStatsTest, VisitTests ) Tests.sequence(sequentialTests) } }
  85. Functional testing libraries DSL - purely functional, no exceptions Registering

    suites - manual Parallelism - explicit composition Lifecycle management object FlawlessTests extends IOApp { def run(args: List[String]): IO[ExitCode] = runTests(args) { val sequentialTests = NonEmptyList.of( GetStatsTest, VisitTests ) val parallelTests = NonEmptyList.of( RunnerTests, SyntaxTests ) Tests.sequence(sequentialTests) |+| Tests.parSequence(parallelTests) } }
  86. Functional testing libraries DSL - purely functional, no exceptions Registering

    suites - manual Parallelism - explicit composition Lifecycle management object FlawlessTests extends IOApp { def run(args: List[String]): IO[ExitCode] = runTests(args) { val sequentialTests = NonEmptyList.of( GetStatsTest, VisitTests ) val parallelTests = NonEmptyList.of( RunnerTests, SyntaxTests ) Tests.sequence(sequentialTests) |+| Tests.parSequence(parallelTests) } } These run sequentially
  87. Functional testing libraries DSL - purely functional, no exceptions Registering

    suites - manual Parallelism - explicit composition Lifecycle management object FlawlessTests extends IOApp { def run(args: List[String]): IO[ExitCode] = runTests(args) { val sequentialTests = NonEmptyList.of( GetStatsTest, VisitTests ) val parallelTests = NonEmptyList.of( RunnerTests, SyntaxTests ) Tests.sequence(sequentialTests) |+| Tests.parSequence(parallelTests) } } These run in parallel
  88. Functional testing libraries DSL - purely functional, no exceptions Registering

    suites - manual Parallelism - explicit composition Lifecycle management object FlawlessTests extends IOApp { def run(args: List[String]): IO[ExitCode] = runTests(args) { val sequentialTests = NonEmptyList.of( GetStatsTest, VisitTests ) val parallelTests = NonEmptyList.of( RunnerTests, SyntaxTests ) Tests.sequence(sequentialTests) |+| Tests.parSequence(parallelTests) } } Sequential composition by default
  89. Functional testing libraries DSL - purely functional, no exceptions Registering

    suites - manual Parallelism - explicit composition Lifecycle management - Resource monad val makeActorSystem: Resource[IO, ActorSystem] = Resource.make(IO(ActorSystem())) { as => IO.fromFuture(IO(as.shutdown)) } val suite = Tests.resource(makeActorSystem).use { as => tests( test("system") { ... } ) }
  90. The future

  91. The future import flawless._

  92. Your tests as a recursive data structure val suite: Tests[NonEmptyList[SuiteResult]]

    = { tests( test("flaky") { IO(Random.nextBoolean()).map(_ shouldBe true) }, pureTest("obvious") { 1 shouldBe 1 } ).lift[NonEmptyList] |+| Tests.parSequence { val results = List(..., ..., ...) NonEmptyList.of(1, 2, 3).map { i => lazyTest(s"fib(${1000*i})") { fib(i * 1000) shouldBe results(i) } } } }.via(findFlaky(maxAttempts = 1000))
  93. Traverse the test tree how you want .via(findFlaky(maxAttempts = 1000))

    .via(catchNonFatal) .via(retry(schedule)) .via(cached(...)) .via(timeoutEach(5.seconds)) .via(disabled) .via(fast)
  94. Traverse the test tree how you want .via(findFlaky(maxAttempts = 1000))

    .via(catchNonFatal) .via(retry(schedule)) .via(cached(...)) .via(timeoutEach(5.seconds)) .via(disabled) .via(fast) Would retry effectful tests
  95. Traverse the test tree how you want .via(findFlaky(maxAttempts = 1000))

    .via(catchNonFatal) .via(retry(schedule)) .via(cached(...)) .via(timeoutEach(5.seconds)) .via(disabled) .via(fast) Would cache pure tests until recompiled
  96. Traverse the test tree how you want .via(findFlaky(maxAttempts = 1000))

    .via(catchNonFatal) .via(retry(schedule)) .via(cached(...)) .via(timeoutEach(5.seconds)) .via(disabled) .via(fast) Would limit running time of each test in the tree
  97. Traverse the test tree how you want .via(findFlaky(maxAttempts = 1000))

    .via(catchNonFatal) .via(retry(schedule)) .via(cached(...)) .via(timeoutEach(5.seconds)) .via(disabled) .via(fast) Would skip tests with resources / tests with IO
  98. Traverse the test tree how you want .via(findFlaky(maxAttempts = 1000))

    .via(catchNonFatal) .via(retry(schedule)) .via(cached(...)) .via(timeoutEach(5.seconds)) .via(disabled) .via(fast) Would skip tests with resources / tests with IO (or all tests!)
  99. Experiment more with the API Find the right abstractions Constraints

    vs liberties See what's possible Gather feedback Find other ridiculous ideas that might make sense TODO
  100. Appendix: what changed since last time I gave this talk

    @jdegoes @adamgfraser
  101. ZIO Test vs flawless testM("parallel offers and sequential takes") {

    effect.map(l => assert(l.toSet, equalTo(values.toSet))) } test("parallel offers and sequential takes") { effect.map(l => l.toSet shouldBe values.toSet) }
  102. ZIO Test vs flawless testM("parallel offers and sequential takes") {

    effect.map(l => assert(l.toSet, equalTo(values.toSet))) } test("parallel offers and sequential takes") { effect.map(l => l.toSet shouldBe values.toSet) }
  103. ZIO Test vs flawless ZIO Test flawless Purely functional ✅

    ✅ Tests as a data structure ✅ ✅ Aspects as functions ✅ ✅
  104. ZIO Test vs flawless ZIO Test flawless Purely functional ✅

    ✅ Tests as a data structure ✅ ✅ Aspects as functions ✅ ✅ Minimal ❌ ✅ (so far!) Arbitrary nesting of tests ✅ ❌ (yet) Available now ✅ ⏳
  105. WIP Demo time?

  106. Thank you blog.kubukoz.com @kubukoz Slides: https://bit.ly/2RXH05N Code: https://git.io/fj6oc