TwoFace values: a bridge between terms and types

Scala 2.13 introduces literal types, and with great types comes great thirst for power to control them. In this talk we get acquainted with the singleton-ops library, a typelevel programming library that enables constraining and performing operations on literal types. We learn about the library’s TwoFace value feature, and how it can be used to bridge the gap between types and terms by converting a type expression to term expression and vice-versa.

Transcript

1. TwoFace values: a bridge between terms and types Oron Port, @soronpo =>
2. About me • Electronic Engineering Ph.D. student at Technion – Israel Institute of Technology (under supervision of Associate Professor Yoav Etsion) • Research topic: “DFiant: A Dataflow Hardware Description Language”, a Scala-based DSL. First release coming soon https://dfianthdl.github.io/ • Contributor to the Scala ecosystem and mainly to the singleton-ops library. https://github.com/fthomas/singleton-ops
3. Overview •Motivation •Introduction to Literal Types •Introduction to the singleton-ops library •TwoFace values •Checked values •Onwards to Scala 3 •Conclusion
4. Motivation example: A fixed-size vector //Fixed-Size Vector class (no need for elements) //size must be verified positive class FixedSizeVector(val size : Int) { //Concatenating two vectors. //The result size is sum of both vectors. def concat(that : FixedSizeVector) : FixedSizeVector = ??? //Selecting partial vector from index sIdx to index eIdx. //Index bounds must be checked. def sel(sIdx : Int, eIdx : Int) : FixedSizeVector = ??? }
5. Attempt 1: size as a run-time integer value class FixedSizeVector private (val size : Int) { def concat(that : FixedSizeVector) : FixedSizeVector = new FixedSizeVector(size + that.size) def sel(sIdx : Int, eIdx : Int) : FixedSizeVector = { require(sIdx < size && sIdx >= 0, "start index out of bounds") require(eIdx < size && eIdx >= 0, "end index out of bounds") require(sIdx <= eIdx, "null-size vector selection") new FixedSizeVector(eIdx-sIdx+1) } } object FixedSizeVector { def apply(size: Int): FixedSizeVector = { require(size > 0, s"size must be positive. found: $size") new FixedSizeVector(size) } } Simple Run-time only. Very unsafe.
6. Introduction to Literal Types • SIP23 adds support for expressing literal types. • Under-the-hood, little has changed (compiler had constant types). • Available @ Typelevel Scala / Scala 2.13.0 type One = 1 (type One = shapeless.Witness.`1`.T) • Supported types: Char, Int, Long, Float, Double, String, Boolean • A `Singleton` upper-bound prevents widening def narrow[T <: Singleton](t : T) : T = t def narrowInt[T <: Int with Singleton](t : T) : T = t • ValueOf[T], valueOf[T] to fetch value from type def foo[T](implicit t : ValueOf[T]) : T = valueOf[T]
7. Widening/Narrowing examples Comment REPL output Code Automatic widening one: Int = 1 val one = 1 Forced narrow type 1 one: 1 = 1 val one : 1 = 1 Int(1) is same type as 1 one: Int(1) = 1 final val one = 1 Forced widening one: Int = 1 final val one : Int = 1 Different constant types error: type mismatch; val one : 1 = 2 The `.type` for singleton literals is a constant type oneA: 1 = 1 oneB: oneA.type = 1 val oneA : 1 = 1 val oneB : oneA.type = 1 `1` is narrower than `Int` oneA: Int = 1 error: type mismatch; val oneA = 1 val oneB : 1 = oneA Basic constant operations are inlined and yield constant types one: 1 = 1 val one : 1 = 2 - 1 `2 - 1` is assuming a `type -[A, B]` in infix use error: not found: type - val one : 2 - 1 = 1
8. Widening/Narrowing examples – Continue Comment REPL output Code No actual conversion here. See next example one: 1L = 1 val one : 1L = 1 As expected, `1 =!:= 1L`, (same behavior in Dotty and Scala 2.12) oneA: 1 = 1 error: type mismatch val oneA : 1 = 1 val oneB : 1L = oneA Conversion operations do not inline one: Long = 1 final val one = 1.toLong More complex operations do not inline one: Int = 1 final val one = math.max(1, 1) def wide[T](t : T) : T = t def narrow[T <: Singleton](t : T) : T = t `Singleton` forces narrowing one: 1 = 1 final val one = narrow(1) Without `Singleton` there is widening one: Int = 1 final val one = wide(1) No error? Again, same surprise behavior one: 1 = 1 final val one : 1 = wide(1) `valueOf` replaces shapeless `Witness` one: 1 = 1 final val one = valueOf[1] `valueOf` replaces shapeless `Witness` oneA: Int = 1 oneB: oneA.type = 1 final val oneA : Int = 1 final val oneB = valueOf[oneA.type]
9. Attempt 2: size as a compile-time literal value Cannot work without “special” help. No way to “calculate” or “constrain” the size type which is known at compile-time. class FixedSizeVector[S <: Int with Singleton] private (val size : S) { def concat[S2 <: Int with Singleton](that : FixedSizeVector[S2]) : FixedSizeVector[?] = new FixedSizeVector[?](size + that.size) //This can't work!!!! def sel(sIdx : Int, eIdx : Int) : FixedSizeVector[?] = ??? } object FixedSizeVector { def apply[S <: Int with Singleton](size: S): FixedSizeVector[S] = { require(size > 0, s"size must be positive. found: $size") new FixedSizeVector[S](size) } }
11. Library Functionality Coverage singleton-ops refined shapeless
12. Features of the singleton-ops library • Expands the default compiler inlining in “non-trivial” cases • Works around scalac’s eager widening • Supports various “type operations” • Supports any “type expression” (composition of operations) • Supports all internal singleton types: Char, Int, Long, Float, Double, String, Boolean • Allows interaction with shapeless Nat and Symbols • Simulates type expression compile-time view-equivalency • Provides compile-type constraints or run-time checks as fallback
13. How to use • In SBT • The imports: import singleton.ops._ Imports the default “type operations” (access the rest as `math._`) import singleton.twoface._ Imports TwoFace and Checked name spaces libraryDependencies ++= Seq( "eu.timepit" %% "singleton-ops" % "0.4.0" )
14. The basic type “operation” • Out will contain the type result • value will contain the singleton equivalent value trait Op { type Out val value: Out }
15. The real mechanism is a bit more complicated trait Op extends Serializable { type OutWide type Out type OutNat <: Nat type OutChar <: Char with Singleton type OutInt <: Int with Singleton type OutLong <: Long with Singleton type OutFloat <: Float with Singleton type OutDouble <: Double with Singleton type OutString <: String with Singleton type OutBoolean <: Boolean with Singleton type OutSymbol <: Symbol val value: Out val isLiteral : Boolean val valueWide: OutWide } trait OpMacro[N, P1, P2, P3] extends Op Op Name Op Parameters type +[P1, P2] = OpMacro[OpId.+, P1, P2, NP] type RequireMsgSym[Cond,Msg,Sym] = OpMacro[OpId.Require, Cond, Msg, GetType[Sym]] Parameters can also be Ops themselves
16. A simple example: average of three args def avg3[A, B, C](implicit calc : (A + B + C) / 3) : calc.Out = calc.value REPL output Code res: Int(6) = 6 final val res = avg3[1, 5, 12] error: Unsupported trait +[Long(1), Int(5), Int(0)] final val res = avg3[1L, 5, 12] error: Calculation has returned a non-literal type/value. To accept non-literal values, use `AcceptNonLiteral[T]`. val one = 1 final val res = avg3[one.type, 5, 12] def avg3[A, B, C](implicit calc : AcceptNonLiteral[(A + B + C) / 3]) : calc.Out = calc.value
17. singleton-ops “type operations” • Basic arithmetic ops: +, -, *, /, % (all between the same basic types) • Basic comparison ops: <, <=, >, >=, ==, != (all between the same basic types) • Basic logic ops: !, ||, &&, ITE (If-Then-Else) • Math ops & constants: Pi, E, Abs, Min, Max, Pow, Floor, Ceil, Round, Sin, Cos, Tan, Sqrt, Log, Log10, Negate, NumberOfLeadingZeros • Conversion ops: ToNat, ToChar, ToInt, ToLong, ToFloat, ToDouble, ToString, ToSymbol • Type checks: IsNat, IsChar, IsInt, IsLong, IsFloat, IsDouble, IsString, IsSymbol • String ops: Substring, Length, CharAt, Reverse, + (concat)
18. “Special” type operations • Literal/Non-literal control and statue: • AcceptNonLiteral[T] – Allow non-literal (constant) computations, returning a wide type. • IsNonLiteral[T] – Returns `true` when `T` is not a constant • Constraints: • Require[Cond] – Cond must be true to compile (without AcceptNonLiteral[T]) • RequireMsg[Cond, Msg] – Cond must be true, or else Msg is the error • RequireMsgSym[Cond, Msg, Sym] – Cond must be true, or else Msg is directed at the trait Sym (modifies @implicitNotFound msg). When Msg is `Warn` a warning is printed instead. • And others…
19. Attempt 3: size as a compile-time literal value + singleton-ops import singleton.ops._ class FixedSizeVector[S <: XInt] private (val size : S) { def concat[S2 <: XInt](that : FixedSizeVector[S2])(implicit scc : SafeInt[S + S2]) : FixedSizeVector[scc.Out] = new FixedSizeVector[scc.Out](scc.value) def sel(sIdx : Int, eIdx : Int) : FixedSizeVector[S] = ??? //ignore for now } object FixedSizeVector { protected type Cond[S] = S > 0 protected type Msg[S] = "size must be positive. found: " + ToString[S] def apply[S <: XInt](size: S)(implicit req : RequireMsg[Cond[S], Msg[S]]): FixedSizeVector[S] = { new FixedSizeVector[S](size) } }
20. Attempt 3: size as a compile-time literal value + singleton-ops REPL output Code vec1: FixedSizeVector[1] = … vec2: FixedSizeVector[2] = … val vec1 = FixedSizeVector(1) val vec2 = FixedSizeVector(2) error: size must be positive. found: 0 FixedSizeVector(0) vec3: FixedSizeVector[this.Out] = … val vec3 = FixedSizeVector(1) concat FixedSizeVector(2) vec3: FixedSizeVector[3] = … val vec3 : FixedSizeVector[3] = FixedSizeVector(1) concat FixedSizeVector(2) vec3: FixedSizeVector[3] = … val vec3 : FixedSizeVector[3] = FixedSizeVector(1) concat FixedSizeVector(1) concat FixedSizeVector(1) error: Cannot extract value from … val one = 1; val vecOne = FixedSizeVector(one)
21. Attempt 3: size as a compile-time literal value + singleton-ops import singleton.ops._ class FixedSizeVector[S <: XInt] private (val size : S) { def concat[S2 <: XInt](that : FixedSizeVector[S2])(implicit scc : SafeInt[S + S2]) : FixedSizeVector[scc.Out] = new FixedSizeVector[scc.Out](scc.value) def sel(sIdx : Int, eIdx : Int) : FixedSizeVector[S] = ??? //ignore for now } object FixedSizeVector { protected type Cond[S] = S > 0 protected type Msg[S] = "size must be positive. found: " + ToString[S] def apply[S <: XInt](size: S)(implicit req : RequireMsg[Cond[S], Msg[S]]): FixedSizeVector[S] = { new FixedSizeVector[S](size) } } Complete compile-time protection and working functionality. Does not accept run-time values
22. TwoFace values • TwoFace values are value classes dedicated for best-effort inlining. • They are drop-in replacements for their equivalent run-time primitives. • All primitives supported: TwoFace.Char[T], TwoFace.Int[T], … • TwoFace classes have a single type argument T (for compile-time) and a runtime value. `T =:= [Primitive]` when compile-time info is not available. Types Terms https://www.flickr.com/photos/shaunwong/3147457988/in/photostream/
23. TwoFace values - Examples Comment REPL Output (slightly modified) Code TwoFace.toString is the runtime value tf1: TwoFace.Int[Int(1)] = 1 val tf1 = TwoFace.Int(1) tfOne: TwoFace.Int[Int] = 1 val one = 1 val tfOne = TwoFace.Int(one) Literal op literal => literal : TwoFace.Int[Int(2)] = 2 tf1 + tf1 Literal op non-literal => non-literal : TwoFace.Int[Int] = 2 tf1 + tfOne : TwoFace.Int[Int] = 2 tfOne + tf1 : TwoFace.Int[Int] = 2 tf1 + tf1 TwoFace[T] => T : 2 = 2 tf1 + 1
24. Underneath the TwoFace’s `Shell`
25. Attempt 4: size as a TwoFace.Int import singleton.ops._ import singleton.twoface._ class FixedSizeVector[S] private (val size : TwoFace.Int[S]) { def concat[S2](that : FixedSizeVector[S2])(implicit tfs : TwoFace.Int.Shell2[+, S, Int, S2, Int]) : FixedSizeVector[tfs.Out] = new FixedSizeVector[tfs.Out](tfs(size, that.size)) def sel(sIdx : Int, eIdx : Int) : FixedSizeVector[S] = ??? //ignore for now } TwoFace.Shell is not necessary here. We can just directly use `size + that.size`. The price we pay is a larger type signature at the call-site. vec1 + vec2 will get a type: FixedSizeVector[1 + 2] (but much uglier) TwoFace term expression leads to a “type expression”
26. Checked values • Checked values are TwoFace values with constraints. • Constraints are checked at compile-time, if possible. • If not, constraints are checked at run-time, if `unsafeCheck` is called.
27. Attempt 4: size as a Checked.Int object FixedSizeVector { object Size extends Checked0Param.Int { type Cond[S] = S > 0 type Msg[S] = "size must be positive. found: " + ToString[S] } def apply[S](checkedSize : Size.Checked[S]) : FixedSizeVector[S] = new FixedSizeVector[S](checkedSize.unsafeCheck()) }
28. object IdxSE extends Checked1Param.Int { type Cond[E, S] = E >= S type Msg[E, S] = "Start index " + ToString[S] + " is bigger than End index " + ToString[E] type ParamFace = Int } object IdxBounds extends Checked1Param.Int { type Cond[I, S] = (I < S) && (I >= 0) type Msg[I, S] = "Index " + ToString[I] + " is out of range of size " + ToString[S] type ParamFace = Int } object RelSize { type Calc[EI, SI] = EI - SI + 1 type TF[EI, SI] = TwoFace.Int.Shell2[Calc, EI, Int, SI, Int] } def sel[SI, EI](sIdx : IdxBounds.Checked[SI, S], eIdx : IdxBounds.Checked[EI, S])( implicit seCheck : IdxSE.CheckedShell[EI, SI], relSize: RelSize.TF[EI, SI] ) : FixedSizeVector[relSize.Out] = { seCheck.unsafeCheck(eIdx.unsafeCheck(size), sIdx.unsafeCheck(size)) new FixedSizeVector[relSize.Out](relSize(eIdx, sIdx))
29. (Known) Pitfalls • Infix type precedence misleading in Scala 2.xx (see SIP33) • Missing unary types (see SIP36) • ==, != will not work with TwoFace values that are on the RHS of the op when compared with a numeric (see proposal to implement universal in/equality via extension methods, instead of Any members).
30. Onwards to Scala 3 • Remove symbol support (deprecated in Scala 3) • Use opaque types + extension methods to implement TwoFace values • Generalize with AnyKind • Use inline + tasty reflection to replace macro paradise implementation (can every feature be supported?) • Use union types for better upper bounds (e.g., T <: Op | XInt) •Do we really need singleton-ops?! Maybe it should be a language feature.
31. Conclusion • singleton-ops is lets you treat types more like terms and infact derives the one from another and vise-versa • TwoFace is your friendly neighborhood inliner • The future with dotty looks bright ☺
32. Acknowledgements • Thanks @fthomas, @milessabin, @paulp • Thanks to all singleton-ops contributors • This work has been supported by EU H2020 ICT project LEGaTO, contract #780681.
33. Thank you for listening!