Type Parameter Power-Up!

Variance, Type Bounds, and Inference. Given at ScalaDays New York City, June 21, 2018.

Transcript

1. Copyright 2018 Tendril, Inc. All rights reserved. Type Parameter Power-Up! Variance, Bounds, and Inference
2. Copyright 2018 Tendril, Inc. All rights reserved. Motivations INTRO 2
3. Copyright 2018 Tendril, Inc. All rights reserved. Motivations INTRO • Home Energy Report Generation System • Free Monad • Workflow stages extending common base class • Workflow actions parameterized with a Report type • Different Reports extend Report base class • Single reports, Optional reports, Lists of reports 3
4. Copyright 2018 Tendril, Inc. All rights reserved. Helpful Compiler is Helpful INTRO [info] Compiling 1 Scala source to /Users/cphelps/learning/type-parameter-power- up/target/scala-2.12/test-classes ... [error] /Users/cphelps/learning/type-parameter-power- up/src/test/scala/example/VarianceSpec.scala:55:14: covariant type A occurs in contravariant position in type A of value newwrapped [error] def swap(newwrapped: A): Wrapper[A] = new Wrapper(newwrapped) [error] ^ [error] one error found 4
5. Copyright 2018 Tendril, Inc. All rights reserved. Helpful Compiler is Helpful INTRO [info] Compiling 1 Scala source to /Users/cphelps/learning/type-parameter-power- up/target/scala-2.12/test-classes ... [error] /Users/cphelps/learning/type-parameter-power- up/src/test/scala/example/VarianceSpec.scala:55:14: covariant type A occurs in contravariant position in type A of value newwrapped [error] def swap(newwrapped: A): Wrapper[A] = new Wrapper(newwrapped) [error] ^ [error] one error found 5
6. Copyright 2018 Tendril, Inc. All rights reserved. Motivations INTRO 6
7. Copyright 2018 Tendril, Inc. All rights reserved. Motivations INTRO • Users o Signatures of APIs– what is it looking for? o What will the API accept? o How do I make the API interact with my domain class hierarchy? o How do I make sense of these weird error messages? • API Designers o What do I want to allow? o What sorts of flexibility do I want to offer? o How do I make sense of these weird error messages? 7
8. Copyright 2018 Tendril, Inc. All rights reserved. “How I stopped worrying about variance: Just add +/- until it compiles” – Adriaan Moors
9. Copyright 2018 Tendril, Inc. All rights reserved. Overview INTRO • Variance • Constraints and Typeclasses • Dotty / Scala 3 9
10. Copyright 2018 Tendril, Inc. All rights reserved. Variance 10
11. Copyright 2018 Tendril, Inc. All rights reserved. Inheritance and Substitution VARIANCE • All values have a type • Types have a supertype relationship up to Any • Types have a subtype relationship down to Nothing • References can store subclass instances • Functions can be passed subclass instances • Functions can return subclass instances 11
12. Copyright 2018 Tendril, Inc. All rights reserved. Liskov Substitution Principle VARIANCE Subtype Requirement: Let ϕ ( x ) be a property provable about objects x of type T. Then ϕ ( y ) should be true for objects y of type S where S is a subtype of T. [Liskov and Wing, 1994] If S is a subtype of T, then objects of type T in a program may be replaced with objects of type S without altering any of the desirable properties of that program. [Wikipedia] 12
13. Copyright 2018 Tendril, Inc. All rights reserved. Higher-Kinded Types VARIANCE • Generic Classes • Types with parameters • Contain or implemented in terms of another type o List[Int] o List[T] 13
14. Copyright 2018 Tendril, Inc. All rights reserved. Two Axes of Subclassing VARIANCE 14 Container Superclass Container Subclass Contents Superclass Contents Subclass
15. Copyright 2018 Tendril, Inc. All rights reserved. Variance Defined VARIANCE • Variance is the correlation of subtyping relationships of complex types and the subtyping relationships of their component types. [Tour of Scala] • A central question that comes up when mixing OO with polymorphism is: if U is a subclass of T, is Container[U] considered a subclass of Container[T]? [Twitter Scala School] 15
16. Copyright 2018 Tendril, Inc. All rights reserved. Apply the LSP VARIANCE • Variance is the correlation of when we can substitute complex types and the when we can substitute their component types. [Tour of Scala] • A central question that comes up when mixing OO with polymorphism is: if U is a subclass of T, can Container[U] be substituted for Container[T]? [Twitter Scala School] 16
17. Copyright 2018 Tendril, Inc. All rights reserved. Substituting Compatible Machines VARIANCE 17
18. Copyright 2018 Tendril, Inc. All rights reserved. Domain Data VARIANCE abstract class Animal { def name: String } abstract class Pet extends Animal case class Cat(name: String) extends Pet case class Dog(name: String) extends Pet 18 Animal Pet Dog Cat
19. Copyright 2018 Tendril, Inc. All rights reserved. Invariance The Default Case VARIANCE • No subclass relationship • Cannot substitute Container[T] for Container[U] unless T is U • Subclasses SubContainer[T] can be substituted for Container[T] • Exposed vars must be invariant • Influences type inference 19
20. Copyright 2018 Tendril, Inc. All rights reserved. Invariance Examples VARIANCE class InvariantWrapper[A](wrapped: A) { def unwrapped: A = wrapped } val w: InvariantWrapper[Cat] = new InvariantWrapper(Cat(”Morris")) class SubWrapper[A](wrapped: A) extends InvariantWrapper(wrapped) {} val sw: InvariantWrapper[Cat] = new SubWrapper(Cat("Milo")) val aw: InvariantWrapper[Animal] = new InvariantWrapper[Cat](Cat("Bill")) 20 Animal Pet Dog Cat
21. Copyright 2018 Tendril, Inc. All rights reserved. Invariance Examples VARIANCE class InvariantWrapper[A](wrapped: A) { def unwrapped: A = wrapped } val w: InvariantWrapper[Cat] = new InvariantWrapper(Cat(”Morris")) class SubWrapper[A](wrapped: A) extends InvariantWrapper(wrapped) {} val sw: InvariantWrapper[Cat] = new SubWrapper(Cat("Milo")) val aw: InvariantWrapper[Animal] = new InvariantWrapper[Cat](Cat("Bill")) 21 type mismatch; found: InvariantWrapper[Cat] required: InvariantWrapper[Animal] Note: example.Cat <: example.Animal, but class InvariantWrapper is invariant in type A. Animal Pet Dog Cat
22. Copyright 2018 Tendril, Inc. All rights reserved. Covariance Sources of Values VARIANCE • Subclass relationship when contents have a subclass relationship • Can substitute Container[U] for Container[T] if U is a subclass of T • Useful for extracting the contents of the container o Instances of U can be used where T instances are expected o Container[U] will produce T instances o Types are sound • Problematic when adding or changing values 22
23. Copyright 2018 Tendril, Inc. All rights reserved. Covariance Examples VARIANCE class CovariantWrapper[+A](wrapped: A) { def unwrapped: A = wrapped } def doIt(thing: CovariantWrapper[Animal]) = ??? doIt(new CovariantWrapper[Cat](Cat("Garfield"))) doIt(new CovariantWrapper[Dog](Dog(“Odie”))) doIt(new CovariantWrapper[Any](Cat("Nermal"))) 23 Animal Pet Dog Cat
24. Copyright 2018 Tendril, Inc. All rights reserved. Covariance Examples VARIANCE class CovariantWrapper[+A](wrapped: A) { def unwrapped: A = wrapped } def doIt(thing: CovariantWrapper[Animal]) = ??? doIt(new CovariantWrapper[Cat](Cat("Garfield"))) doIt(new CovariantWrapper[Dog](Dog(“Odie”))) doIt(new CovariantWrapper[Any](Cat("Nermal"))) 24 type mismatch; found: CovariantWrapper[Any] required: CovariantWrapper[Animal] Animal Pet Dog Cat
25. Copyright 2018 Tendril, Inc. All rights reserved. Contravariance Consumers of Values VARIANCE • Subclass relationship when contents have a superclass relationship • Can substitute Container[T] for Container[U] if T is a superclass of U • Useful for processors or consumers of values o Instances of U can be used where T instances are expected o Consumer[U] can consume T instances o Types are sound • Problematic when returning or producing values 25
26. Copyright 2018 Tendril, Inc. All rights reserved. Contravariance Examples VARIANCE abstract class Keeper[-A] { def tend(input: A): Unit } class DogSitter extends Keeper[Dog] { override def tend(input: Dog): Unit = ??? } class ZooKeeper extends Keeper[Animal] { ... } class PetSitter extends Keeper[Pet] { ... } val ds: Keeper[Dog] = new DogSitter ds.tend(Dog("Tintin")) val zoo: Keeper[Dog] = new ZooKeeper zoo.tend(Dog("Scooby")) val petco: Keeper[Pet] = new DogSitter petco.tend(Dog("Ceasar")) 26 Animal Pet Dog Cat
27. Copyright 2018 Tendril, Inc. All rights reserved. Contravariance Examples VARIANCE abstract class Keeper[-A] { def tend(input: A): Unit } class DogSitter extends Keeper[Dog] { override def tend(input: Dog): Unit = ??? } class ZooKeeper extends Keeper[Animal] { ... } class PetSitter extends Keeper[Pet] { ... } val ds: Keeper[Dog] = new DogSitter ds.tend(Dog("Tintin")) val zoo: Keeper[Dog] = new ZooKeeper zoo.tend(Dog("Scooby")) val petco: Keeper[Pet] = new DogSitter petco.tend(Dog("Ceasar")) 27 type mismatch; found : DogSitter required: Keeper[example.Pet] Animal Pet Dog Cat
28. Copyright 2018 Tendril, Inc. All rights reserved. Positions VARIANCE • “Covariant Position” – method returns • “Contravariant Position” – method arguments • “Invariant Position” - mutable vars • Covariant parameters cannot appear in contravariant position • Contravariant parameters cannot appear in covariant position • Covariant or contravariant parameters cannot appear in invariant position o Scala 2.12.6 and IntelliJ error messages misleading 28
29. Copyright 2018 Tendril, Inc. All rights reserved. The Problem with Contravariant Positions VARIANCE class Box[+Pet](p: Pet) { // won't compile – Covariant in contravariant pos def swap(newp: Pet): Box[Pet] = new Box[Pet](newp) } val mypet = new Box[Cat](Cat(“Morris”)) // Seems okay so far mypet.swap(Cat("Cassie")) // Oops, we can't store a Dog in a Box[Cat] mypet.swap(Dog("Ceasar")) 29
30. Copyright 2018 Tendril, Inc. All rights reserved. The Problem with Covariant Positions VARIANCE class Sitter[-Pet](p: Pet) { // won't compile - contravariant in covariant pos def walk(): Pet = ??? } val mySitter: Sitter[Cat] = new Sitter[Pet](Cat("Doraemon")) // seems okay val walkedPet: Pet = mySitter.walk() // if walk returns a Cat, everything is ok // if it returns a Dog, we have a problems // if it returns a Pet, we have a problem. val walked: Cat = mySitter.walk() 30
31. Copyright 2018 Tendril, Inc. All rights reserved. Function Inputs VARIANCE val p1: Pet val p2: Pet f(p1, p2) def f(a: Pet, b: Pet): Pet ✅ def f(a: Animal, b: Pet): Pet ✅ def f(a: Any, b: Pet): Pet ✅ def f(a: Any, b: Any): Pet ✅ def f(a: Cat, b: Dog): Pet ❎ Function2[-T1, -T2, R] 31 Animal Pet Dog Cat
32. Copyright 2018 Tendril, Inc. All rights reserved. Function Outputs VARIANCE val p1: Pet val p2: Pet val r: Animal = f(p1, p2) def f(a: Pet, b: Pet): Animal ✅ def f(a: Pet, b: Pet): Pet ✅ def f(a: Pet, b: Pet): Cat ✅ def f(a: Pet, b: Pet): Any ❎ Function2[-T1, -T2, +R] 32 Animal Pet Dog Cat
33. Copyright 2018 Tendril, Inc. All rights reserved. When to Use VARIANCE • Covariance o Containers o Producers o Representing inputs • Contravariance o Consumers o Processors or Visitors o Representing outputs • Invariance o Distinct types o Markers or Labels o Influence inference 33
34. Copyright 2018 Tendril, Inc. All rights reserved. Constraints and Typeclasses 34
35. Copyright 2018 Tendril, Inc. All rights reserved. Upper and Lower Bounds CONSTRAINTS AND TYPECLASSES • Constrain the valid types • Upper bound – parameter is same or subtype o def pet[P <: Pet](animal: P): Unit = ??? • Lower bound – parameter is same or supertype o def pre[B >: A](x: B, xs: List[A]): List[B] = ???
36. Copyright 2018 Tendril, Inc. All rights reserved. Bounds and Variance CONSTRAINTS AND TYPECLASSES abstract class List[+A] { ... } case object Nil extends List[Nothing] { ... } case class Cons[B] (elem: B, tail: List[B]) extends List[B] { ... } 36
37. Copyright 2018 Tendril, Inc. All rights reserved. Bounds and Variance CONSTRAINTS AND TYPECLASSES abstract class List[+A] { def prepend(elem: A): List[A] = ??? } case object Nil extends List[Nothing] { ... } case class Cons[B] (elem: B, tail: List[B]) extends List[B] { ... } 37
38. Copyright 2018 Tendril, Inc. All rights reserved. Bounds and Variance CONSTRAINTS AND TYPECLASSES abstract class List[+A] { def prepend(elem: A): List[A] = ??? } case object Nil extends List[Nothing] { ... } case class Cons[B] (elem: B, tail: List[B]) extends List[B] { ... } 38 covariant type A occurs in contravariant position in type A of value elem
39. Copyright 2018 Tendril, Inc. All rights reserved. Bounds and Variance CONSTRAINTS AND TYPECLASSES abstract class List[+A] { def prepend[B](elem: B): List[B] = new Cons(elem, this) } case object Nil extends List[Nothing] { ... } case class Cons[B] (elem: B, tail: List[B]) extends List[B] { ... } 39
40. Copyright 2018 Tendril, Inc. All rights reserved. Bounds and Variance CONSTRAINTS AND TYPECLASSES abstract class List[+A] { def prepend[B](elem: B): List[B] = new Cons(elem, this) } case object Nil extends List[Nothing] { ... } case class Cons[B] (elem: B, tail: List[B]) extends List[B] { ... } 40 type mismatch; found: List[A] required: List[B]
41. Copyright 2018 Tendril, Inc. All rights reserved. Bounds and Variance CONSTRAINTS AND TYPECLASSES abstract class List[+A] { def prepend[B >: A] (elem: B): List[B] = ??? } case object Nil extends List[Nothing] { ... } case class Cons[B] (elem: B, tail: List[B]) extends List[B] { ... } 41
42. Copyright 2018 Tendril, Inc. All rights reserved. Change to a Wider Type CONSTRAINTS AND TYPECLASSES scala> List(1, 2, 3, 4, 5) res0: List[Int] = List(1, 2, 3, 4, 5) scala> 0 :: res0 res1: List[Int] = List(0, 1, 2, 3, 4, 5) scala> 1.5 :: res0 res2: List[AnyVal] = List(1.5, 1, 2, 3, 4, 5) scala> res2(0) res3: AnyVal = 1.5 scala> res2(1) res4: AnyVal = 1 42
43. Copyright 2018 Tendril, Inc. All rights reserved. View Bounds CONSTRAINTS AND TYPECLASSES • Deprecated from 2.11 • [ A <% B ] • Indicates A is convertable to B by implicit conversion • Introduces evidence parameter o def view[AA <% A](x: AA): A = ??? o def view$[AA](x: AA)(implicit ev1$: AA => A) = ??? 43
44. Copyright 2018 Tendril, Inc. All rights reserved. Context Bounds CONSTRAINTS AND TYPECLASSES • [ A : Ctx ] • Indicates A has some context Ctx o There exists a Ctx[A] in implicit scope • Introduces an evidence parameter o def ctxBound[X: M](x: X) = ??? o def ctxBound[X](x: X)(implicit ev1$: M[X]) = ??? • Access the context with the implicitly keyword o val m = implicitly[M[X]] o m.someMethod() 44
45. Copyright 2018 Tendril, Inc. All rights reserved. Typeclass Pattern CONSTRAINTS AND TYPECLASSES • External implementation of an interface o No need to extend directly o Possible without access to class source • Introduce via a context bound • Implement in terms of of other instances o Adder[Pair[Int]] defined in terms of Adder[Int] 45
46. Copyright 2018 Tendril, Inc. All rights reserved. Typeclass Pattern CONSTRAINTS AND TYPECLASSES • Define an interface • Implement for your class • Introduce to implicit scope • Pass to function as implicit parameter OR as type constraint • Call methods from the interface 46
47. Copyright 2018 Tendril, Inc. All rights reserved. Typeclass Example CONSTRAINTS AND TYPECLASSES trait Adder[A] { def add(x: A, y: A): A } implicit val IntAdder = new Adder[Int] { override def add(x: Int, y: Int): Int = x + y } def addThings[T: Adder](x: T, y: T): T = { implicitly[Adder[T]].add(x, y) } addThings(5, 7) 47
48. Copyright 2018 Tendril, Inc. All rights reserved. Inductive Typeclass Definition case class Pair[T](fst: T, snd: T) implicit def pairAdder[T: Adder] = new Adder[Pair[T]] { override def add(x: Pair[T], y: Pair[T]): Pair[T] = { val ta = implicitly[Adder[T]] Pair(ta.add(x.fst, y.fst), ta.add(x.snd, y.snd)) } } 48
49. Copyright 2018 Tendril, Inc. All rights reserved. Dotty / Scala 3 DOTTY
50. Copyright 2018 Tendril, Inc. All rights reserved. Dotty Changes DOTTY • “Scala 3 is fundamentally the same language as Scala 2” • Variance and type bounds still exist – essentially the same o Current dotc slightly different error messages • No existential types (but other constructs instead) • Structural types – different implementation
51. Copyright 2018 Tendril, Inc. All rights reserved. Takeaways • Variance o Substitution o Arises naturally as a consequence of Liskov Substitution o Deeply entwined with subclassing in higher-kinded types • Bounds o Interact with variance -> widening o Interact with implicits -> typeclasses 51
52. Copyright 2018 Tendril, Inc. All rights reserved. @CJPhelps github.com/chrisphelps/type-parameter-power-up