Tweet
Tweet

Slide 1

Slide 1 text

A Tour of Functional Structures via Scodec and Simulacrum August 2015

Slide 2

Slide 2 text

Agenda 2 Introduction to scodec Structures relevant to Decoders Structures relevant to Encoders 1 2 3 4 Structures relevant to Codecs

Slide 3

Slide 3 text

About Me… • Michael Pilquist (@mpilquist) • Using Scala professionally since 2008 • Primary author of scodec and simulacrum • Work at CCAD, LLC. (Combined Conditional Access Development) • Joint-venture between Comcast and ARRIS Group, Inc. • Build conditional access technology • Have never used Haskell in earnest • Primary FP experience is with Scala • Some limited ML, Lisp, and λ-calculus 3

Slide 4

Slide 4 text

scodec • Suite of Scala libraries for working with binary data • Pure functional • Compositional • http://scodec.org 4 scodec-bits scodec-core scodec-stream scodec-scalaz scodec-spire scodec- protocols scalaz-stream shapeless

Slide 5

Slide 5 text

scodec, approximately… 5 trait Encoder[-A] { def encode(value: A): Attempt[BitVector] def sizeBound: SizeBound } trait Decoder[+A] { def decode(bits: BitVector): Attempt[DecodeResult[A]] } case class DecodeResult[+A](value: A, remainder: BitVector) trait Codec[A] extends Encoder[A] with Decoder[A] Similar to Try, \/, Or, Xor, right-biased Either

Slide 6

Slide 6 text

Introduction to scodec 6 case class Point(x: Int, y: Int) case class Line(start: Point, end: Point) case class Arrangement(lines: Vector[Line]) val arr = Arrangement(Vector( Line(Point(0, 0), Point(10, 10)), Line(Point(0, 10), Point(10, 0))))

Slide 7

Slide 7 text

Introduction to scodec 7 import scodec.Codec import scodec.codecs.implicits._ // arr: Arrangement = Arrangement(Vector(Line(Point(… val arrBinary = Codec.encode(arr).require // arrBinary: BitVector = BitVector(288 bits, 0x0000000200000000000000000000000a0000000a00000000000… val decoded = Codec[Arrangement]. decode(arrBinary).require.value // decoded: Arrangement = Arrangement(Vector(
 Line(Point(0, 0),Point(10, 10)),
 Line(Point(0, 10),Point(10, 0)))) Attempt[A] => A (throws if failure)

Slide 8

Slide 8 text

Controlling codec derivation with implicits 8 import scodec.codecs.implicits.{ implicitIntCodec => _, _ } implicit val ci = scodec.codecs.uint8 // arr: Arrangement = Arrangement(Vector(Line(Point(… val arrBinary = Codec.encode(arr).require // arrBinary: BitVector = BitVector(72 bits, 0x0200000a0a000a0a00) val result = Codec.encode( Arrangement(Vector( Line(Point(0, 0), Point(1, 1)), Line(Point(0, 0), Point(1, -1)) )) ) // result: Attempt[BitVector] = Failure(lines/1/end/y: -1 
 is less than minimum value 0 for 8-bit unsigned integer) Error contains path to problematic field All ints are encoded in 8 bits

Slide 9

Slide 9 text

Explicit codecs 9 case class Pid(value: Int) { require(value >= Pid.MinValue && value <= Pid.MaxValue) } object Pid { val MinValue = 0 val MaxValue = 8191 implicit val codec: Codec[Pid] = uint(13).xmap(Pid.apply, _.value) // Alternatively… uint(13).as[Pid] } Design-wise, scodec is optimized for defining explicit codecs Transforms a Codec[Int] => Codec[Pid] Supported by ✨ Shapeless✨

Slide 10

Slide 10 text

Explicit codecs 10 case class TransportStreamHeader( transportErrorIndicator: Boolean, payloadUnitStartIndicator: Boolean, transportPriority: Boolean, pid: Pid, scramblingControl: Int, adaptationFieldControl: Int, continuityCounter: ContinuityCounter ) { def adaptationFieldIncluded: Boolean = adaptationFieldControl >= 2 def payloadIncluded: Boolean = adaptationFieldControl == 1 || adaptationFieldControl == 3 }

Slide 11

Slide 11 text

Explicit codecs 11 object TransportStreamHeader { implicit val codec: Codec[TransportStreamHeader] = "transport_stream_header" | fixedSizeBytes(4, ("syncByte" | constant(0x47) ) :: ("transportErrorIndicator" | bool ) :: ("payloadUnitStartIndicator" | bool ) :: ("transportPriority" | bool ) :: ("pid" | Codec[Pid] ) :: ("scramblingControl" | uint2 ) :: ("adaptationFieldControl" | uint2 ) :: ("continuityCounter" | Codec[ContinuityCounter]) ).as[TransportStreamHeader] } Case class bindings via codec transformations

Slide 12

Slide 12 text

Explicit codecs 12 case class MpegPacket( header: TransportStreamHeader, adaptationField: Option[AdaptationField], payload: Option[ByteVector] ) implicit val mpegPacket: Codec[MpegPacket] = { ("header" | transportStreamHeader) flatPrepend { hdr => ("adaptation_field" | conditional(hdr.adaptationFieldIncluded, adaptationField)) :: ("payload" | conditional(hdr.payloadIncluded, bytes(184))) }}.as[MpegPacket]

Slide 13

Slide 13 text

Decoder 13 trait Encoder[-A] { def encode(value: A): Attempt[BitVector] def sizeBound: SizeBound } trait Decoder[+A] { def decode(bits: BitVector): Attempt[DecodeResult[A]] } case class DecodeResult[+A](value: A, remainder: BitVector) trait Codec[A] extends Encoder[A] with Decoder[A]

Slide 14

Slide 14 text

Mapping decoders 14 trait Decoder[+A] { self => def decode(bits: BitVector): Attempt[DecodeResult[A]] def map[B](f: A => B): Decoder[B] = new Decoder[B] { def decode(bits: BitVector) = self.decode(bits).map { _.map(f) } } } val tuple: Decoder[(Int, Int)] = ??? val point: Decoder[Point] = tuple map { case (x, y) => Point(x, y) }

Slide 15

Slide 15 text

Functors 15 Where the behavior of map is constrained by the following laws: • Identity: mapping the identity function is an identity • Composition: given two functions, f and g, mapping f and then g gives the same results as mapping f andThen g import simulacrum.typeclass @typeclass trait Functor[F[_]] { def map[A, B](fa: F[A])(f: A => B): F[B] }

Slide 16

Slide 16 text

Functors 16 import simulacrum.typeclass @typeclass trait Functor[F[_]] { def map[A, B](fa: F[A])(f: A => B): F[B] } Can we represent the laws in code? Why would we want to? • We can write automated tests that verify the laws for a specified type constructor • e.g., automatically verify that a Functor[Decoder] is lawful • Definitions are more precise / less opportunity for misunderstanding • Laws are type checked • Laws are presented clearly and separate from test framework baggage

Slide 17

Slide 17 text

Encoding the functor laws with deferred equivalence 17 trait FunctorLaws[F[_]] { implicit val tc: Functor[F] import Functor.ops._ def functorIdentity[A](fa: F[A]): IsEqv[F[A]] = fa.map(identity) <-> fa def functorComposition[A, B, C]( fa: F[A], f: A => B, g: B => C): IsEqv[F[C]] = fa.map(f).map(g) <-> fa.map(f andThen g) } Type class instance that is being tested Simulacrum provided syntax (provides infix methods) Defers equivalence testing to the caller — equivalent to returning a Tuple2

Slide 18

Slide 18 text

Deferred Equivalence 18 case class IsEqv[A](lhs: A, rhs: A) { def isEqv(implicit e: Eqv[A]): Boolean = e.eqv(lhs, rhs) } implicit class IsEqvBuilder[A](val lhs: A) extends AnyVal { def <->(rhs: A): IsEqv[A] = IsEqv(lhs, rhs) } // Without an Eqv[Int] in scope val deferred = 1 <-> 2 // Later, with an Eqv[Int] in scope deferred.isEqv Laws test for equivalence, not object equality!

Slide 19

Slide 19 text

Deferred Equivalence 19 @typeclass trait Eqv[A] { @op("===", alias = true) def eqv(x: A, y: A): Boolean @op("=/=", alias = true) def neqv(x: A, y: A): Boolean = !eqv(x, y) } implicit val eqvInt: Eqv[Int] = new Eqv[Int] { def eqv(x: Int, y: Int) = x == y } import Eqv.ops._ 1 === 1; 1 =/= 2 Laws test for equivalence, not object equality!

Slide 20

Slide 20 text

Flat-mapping decoders 20 trait Decoder[+A] { self => def decode(bits: BitVector): Attempt[DecodeResult[A]] def flatMap[B](f: A => Decoder[B]): Decoder[B] = ??? } val size: Decoder[Int] = uint(4) val string: Decoder[String] = size flatMap { sz => fixedSizeBytes(sz, utf8) } 2 0 8 16 "OK" Remainder Models a dependency between decoders

Slide 21

Slide 21 text

decode( ) self.decode( ) bDecoder.decode( ) Flat-mapping decoders 21 trait Decoder[+A] { self => def decode(bits: BitVector): Attempt[DecodeResult[A]] def flatMap[B](f: A => Decoder[B]): Decoder[B] = new Decoder[B] { def decode(bits: BitVector) = for { aResult <- self.decode(bits) bDecoder = f(aResult.value) bResult <- bDecoder.decode(aResult.remainder) } yield bResult } } 0 8 16 0 8 16 8 16

Slide 22

Slide 22 text

Monads 22 @typeclass trait Monad[F[_]] extends Functor[F] { def pure[A](a: A): F[A] def flatMap[A, B](fa: F[A])(f: A => F[B]): F[B] override def map[A, B](fa: F[A])(f: A => B): F[B] = flatMap(fa)(f andThen pure) }

Slide 23

Slide 23 text

Monads 23 trait MonadLaws[F[_]] extends FunctorLaws[F] { implicit val tc: Monad[F] import Monad.ops._ def monadFlatMapAssociativity[A, B, C]( fa: F[A], f: A => F[B], g: B => F[C] ): IsEqv[F[C]] = fa.flatMap(f).flatMap(g) <-> fa.flatMap(a => f(a).flatMap(g)) def monadLeftIdentity[A, B](a: A, f: A => F[B]): IsEqv[F[B]] = typeClass.pure(a).flatMap(f) <-> f(a) def monadRightIdentity[A](fa: F[A]): IsEqv[F[A]] = fa.flatMap(typeClass.pure) <-> fa }

Slide 24

Slide 24 text

Monad instance for Decoder 24 implicit val decoderMonad: Monad[Decoder] = new Monad[Decoder] { def flatMap[A, B](fa: Decoder[A])(f: A => Decoder[B]) = fa flatMap f def pure[A](a: A): Decoder[A] = new Decoder[A] { def decode(b: BitVector) = Attempt.successful(DecodeResult(a, b)) } } Q: How do we write pure for the Decoder monad?

Slide 25

Slide 25 text

Derived methods on Monad 25 @typeclass trait Monad[F[_]] extends Functor[F] { def pure[A](a: A): F[A] def flatMap[A, B](fa: F[A])(f: A => F[B]): F[B] override def map[A, B](fa: F[A])(f: A => B): F[B] = flatMap(fa)(f andThen pure) def flatten[A](ffa: F[F[A]]): F[A] = flatMap(ffa)(identity) }

Slide 26

Slide 26 text

Derived methods on Monad 26 @typeclass trait Monad[F[_]] extends Functor[F] { def pure[A](a: A): F[A] def flatMap[A, B](fa: F[A])(f: A => F[B]): F[B] override def map[A, B](fa: F[A])(f: A => B): F[B] = flatMap(fa)(f andThen pure) def flatten[A](ffa: F[F[A]]): F[A] = flatMap(ffa)(identity) def ifM[A](fb: F[Boolean])(t: => F[A], f: => F[A]): F[A] = flatMap(fb)(b => if (b) t else f) }

Slide 27

Slide 27 text

Putting derived methods to use 27 val bool: Decoder[Boolean] = scodec.codecs.bool val uint8: Decoder[Int] = scodec.codecs.uint8 val uint8opt: Decoder[Option[Int]] = Monad[Decoder].ifM(bool)( uint8.map(Some.apply), Monad[Decoder].pure(None)) Is ifM useful when working with Decoders? 0 8 16 0 0 8 16 1

Slide 28

Slide 28 text

Derived methods on Monad 28 val uint8opt: Decoder[Option[Int]] = Monad[Decoder].ifM(bool)( uint8.map(Some.apply), Monad[Decoder].pure(None)) def optional[A]( guard: Decoder[Boolean], target: Decoder[A]): Decoder[Option[A]] = Monad[Decoder].ifM(guard)( target.map(Some.apply), Monad[Decoder].pure(None) ) val uint8opt: Decoder[Option[Int]] = optional(bool, uint8) Q: Can we generalize this pattern to work with any guard and value codecs?

Slide 29

Slide 29 text

Returning to Decoder’s flatMap… 29 trait Decoder[+A] { self => def decode(bits: BitVector): Attempt[DecodeResult[A]] def flatMap[B](f: A => Decoder[B]): Decoder[B] = new Decoder[B] { def decode(bits: BitVector) = for { aResult <- self.decode(bits) bDecoder = f(aResult.value) bResult <- bDecoder.decode(aResult.remainder) } yield bResult } }

Slide 30

Slide 30 text

Returning to Decoder’s flatMap… 30 trait Decoder[+A] { self => def decode(bits: BitVector): Attempt[DecodeResult[A]] def flatMap[B](f: A => Decoder[B]): Decoder[B] = new Decoder[B] { def decode(bits: BitVector) = for { aResult <- self.decode(bits) bDecoder = f(aResult.value) bResult <- bDecoder.decode(aResult.remainder) } yield bResult } } Threading of remainder

Slide 31

Slide 31 text

Progressively abstracting decode 31 def decode(bits: BitVector): Attempt[(BitVector, A)] def decode(bits: BitVector): Attempt[DecodeResult[A]] BitVector => Attempt[(BitVector, A)] S => Attempt[(S, A)] S => F[(S, A)] StateT[F, S, A] StateT[Attempt, BitVector, A]

Slide 32

Slide 32 text

Domain specific, fail fast, state monad 32 object Decoder { def get: Decoder[BitVector] = ??? def set(b: BitVector): Decoder[Unit] = ??? def modify(f: BitVector => BitVector): Decoder[Unit] = ??? } • Decoder is a domain specific, fail fast, state monad! • domain specific => fixes state type to BitVector • fail-fast => fixes type constructor to Attempt • state monad => allows composition of functions of the form S => F[(S, A)] • Hence, we can define common state monad constructors: get, set, modify

Slide 33

Slide 33 text

Example of using State monad constructors 33 def quoted[A](inner: Codec[A]): Codec[A] = new Codec[A] { private val quote = BitVector(0x22) def sizeBound = inner.sizeBound + SizeBound.exact(16) def encode(a: A) = inner.encode(a).map { b => quote ++ b ++ quote } def decode(b: BitVector) = (for { _ <- constant(0x22) b <- Decoder.get untilEndQuote = b.bytes.takeWhile(_ != 0x22.toByte).bits _ <- Decoder.set(untilEndQuote) value <- inner _ <- Decoder.set(b.drop(untilEndQuote.size)) _ <- constant(0x22) } yield value).decode(b) }

Slide 34

Slide 34 text

Applicative Functors 34 @typeclass trait Applicative[F[_]] extends Functor[F] { def pure[A](a: A): F[A] def ap[A, B](fa: F[A])(ff: F[A => B]): F[B] override def map[A, B](fa: F[A])(f: A => B): F[B] = ap(fa)(pure(f)) } @typeclass trait Monad[F[_]] extends Applicative[F] { def flatMap[A, B](fa: F[A])(f: A => F[B]): F[B] override def ap[A, B](fa: F[A])(ff: F[A => B]): F[B] = flatMap(ff)(f => fa.map(f)) } Weaker than a Monad, stronger than a Functor

Slide 35

Slide 35 text

Applicative Functors - Laws 35 trait ApplicativeLaws[F[_]] extends FunctorLaws[F] { implicit val tc: Applicative[F] import tc._ import Applicative.ops._ def applicativeIdentity[A](fa: F[A]): IsEqv[F[A]] = fa.ap(pure((a: A) => a)) <-> fa def applicativeComposition[A, B, C]( fa: F[A], fab: F[A => B], fbc: F[B => C]): IsEqv[F[C]] = fa.ap(fab).ap(fbc) <-> fa.ap(fab.ap(fbc.ap( pure((bc: B => C) => (ab: A => B) => ab andThen bc)))) def applicativeHomomorphism[A, B](a: A, f: A => B): IsEqv[F[B]] = pure(a).ap(pure(f)) <-> pure(f(a)) def applicativeInterchange[A, B](a: A, fab: F[A => B]): IsEqv[F[B]] = pure(a).ap(fab) <-> fab.ap(pure((f: A => B) => f(a))) }

Slide 36

Slide 36 text

Lax Monoidal Functors 36 @typeclass trait Monoidal[F[_]] extends Functor[F] { def pure[A](a: A): F[A] def zip[A, B](fa: F[A], fb: F[B]): F[(A, B)] }

Slide 37

Slide 37 text

Lax Monoidal Functors 37 trait MonoidalLaws[F[_]] extends FunctorLaws[F] { implicit val tc: Monoidal[F] import Monoidal.ops._ def monoidalLeftIdentity[A](fa: F[A]): IsEqv[F[A]] = tc.pure(()).zip(fa).map(_._2) <-> fa def monoidalRightIdentity[A](fa: F[A]): IsEqv[F[A]] = fa.zip(tc.pure(())).map(_._1) <-> fa def monoidalAssociativity[A, B, C]( fa: F[A], fb: F[B], fc: F[C]): IsEqv[F[(A, (B, C))]] = fa.zip(fb.zip(fc)) <-> (fa.zip(fb)).zip(fc).map { case ((a, b), c) => (a, (b, c)) } }

Slide 38

Slide 38 text

Lax Monoidal Functors => Applicative Functors 38 @typeclass trait Monoidal[F[_]] extends Functor[F] { self => def pure[A](a: A): F[A] def zip[A, B](fa: F[A], fb: F[B]): F[(A, B)] def toApplicative: Applicative[F] = new Applicative[F] { def pure[A](a: A): F[A] = self.pure(a) def ap[A, B](fa: F[A])(ff: F[A => B]): F[B] = self.map(self.zip(fa, ff)) { case (a, f) => f(a) } } } A lax monoidal functor gives rise to an applicative functor

Slide 39

Slide 39 text

Applicative Functors => Lax Monoidal Functors 39 @typeclass trait Applicative[F[_]] extends Functor[F] { self => def pure[A](a: A): F[A] def ap[A, B](fa: F[A])(ff: F[A => B]): F[B] override def map[A, B](fa: F[A])(f: A => B): F[B] = ap(fa)(pure(f)) def toMonoidal: Monoidal[F] = new Monoidal[F] { def pure[A](a: A): F[A] = self.pure(a) def zip[A, B](fa: F[A], fb: F[B]): F[(A, B)] = self.ap(fb)(self.map(fa)(a => (b: B) => (a, b))) def map[A, B](fa: F[A])(f: A => B): F[B] = self.map(fa)(f) } } An applicative functor gives rise to a lax monoidal functor

Slide 40

Slide 40 text

Decoder has a Lax Monoidal Functor 40 • Decoder has a lawful Monad instance, so we know it has a lawful applicative and lawful monoidal instance: • Do we get anything useful from using a decoder as a monoidal? • Alternatively, as an applicative: val a: Applicative[Decoder] = Monad[Decoder] val m: Monoidal[Decoder] = a.toMonoidal val point: Decoder[Point] = Monoidal[Decoder].zip(uint8, uint8).map { case (x, y) => Point(x, y) } val point: Decoder[Point] = Applicative[Decoder].map2(uint8, uint8) { 
 (x, y) => Point(x, y) }

Slide 41

Slide 41 text

Encoder 41 trait Encoder[-A] { def encode(value: A): Attempt[BitVector] def sizeBound: SizeBound } trait Decoder[+A] { def decode(bits: BitVector): Attempt[DecodeResult[A]] } case class DecodeResult[+A](value: A, remainder: BitVector) trait Codec[A] extends Encoder[A] with Decoder[A]

Slide 42

Slide 42 text

Encoder 42 trait Encoder[-A] { def encode(value: A): Attempt[BitVector] def sizeBound: SizeBound } Q: Can we map a function over an encoder?

Slide 43

Slide 43 text

Mapping Encoders 43 trait Encoder[-A] { self => def encode(value: A): Attempt[BitVector] def sizeBound: SizeBound def contramap[B](f: B => A): Encoder[B] = new Encoder[B] { def encode(value: B) = self.encode(f(b)) def sizeBound = self.sizeBound } } val tuple: Encoder[(Int, Int)] = ??? val point: Encoder[Point] = tuple.contramap(p => (p.x, p.y)) A: Yes, though not the function that first comes to mind! We can abstract over the ability to contramap…

Slide 44

Slide 44 text

Contravariant Functors 44 @typeclass trait Contravariant[F[_]] { def contramap[A, B](fa: F[A])(f: B => A): F[B] } trait ContravariantLaws[F[_]] { implicit val tc: Contravariant[F] import Contravariant.ops._ def contravariantIdentity[A](fa: F[A]): IsEqv[F[A]] = fa.contramap[A](identity) <-> fa def contravariantComposition[A, B, C]( fa: F[A], f: B => A, g: C => B): IsEqv[F[C]] = fa.contramap(f).contramap(g) <-> fa.contramap(g andThen f) }

Slide 45

Slide 45 text

Contravariant Functors 45 Other notable contravariant functors: • Eqv, Ordering, PartialOrder @typeclass Eqv[A] { def eqv(x: A, y: A): Boolean def contramap[B](f: B => A): Eqv[B] = new Eqv[B] { def eqv(x: B, y: B) = self.eqv(f(x), f(y)) } } • Function1[?, O] implicit def cf1[A, O](f: A => O) = new { def contramap[B](g: B => A): B => O = g andThen f }

Slide 46

Slide 46 text

Codec 46 trait Encoder[-A] { def encode(value: A): Attempt[BitVector] def sizeBound: SizeBound } trait Decoder[+A] { def decode(bits: BitVector): Attempt[DecodeResult[A]] } case class DecodeResult[+A](value: A, remainder: BitVector) trait Codec[A] extends Encoder[A] with Decoder[A]

Slide 47

Slide 47 text

Codec 47 trait Codec[A] extends Encoder[A] with Decoder[A] • Alternatively, can be thought of as a tuple (Encoder[A], Decoder[A]) • We can’t map or contramap a Codec • …and since we can’t map, we can’t have a Functor, Applicative, Monad, Monoidal, etc. • Yet, Codec is the primary type in scodec, and the pairing is intentional! • all built-in combinators operate on codecs • support for primitive datatypes come in the form of codecs • We’ve seen how to transform codecs using xmap…

Slide 48

Slide 48 text

xmapping Codecs 48 trait Codec[A] extends Encoder[A] with Decoder[A] { self => def xmap[B](f: A => B, g: B => A): Codec[B] = new Codec[B] { def sizeBound = self.sizeBound def encode(b: B) = self.encode(g(b)) def decode(b: BitVector) = self.decode(b).map(_.map(f)) } } val tuple: Codec[(Int, Int)] = ??? val point: Codec[Point] = tuple.xmap(Point.apply, p => (p.x, p.y))

Slide 49

Slide 49 text

Invariant Functors 49 @typeclass trait Invariant[F[_]] { def xmap[A, B](fa: F[A])(f: A => B)(g: B => A): F[B] } trait InvariantLaws[F[_]] { implicit val tc: Invariant[F] import Invariant.ops._ def invariantIdentity[A](fa: F[A]): IsEqv[F[A]] = fa.xmap(identity)(identity) <-> fa def invariantComposition[A, B, C](fa: F[A], f: A => B, fi: B => A, g: B => C, gi: C => B): IsEqv[F[C]] = fa.xmap(f)(fi).xmap(g)(gi) <-> fa.xmap(f andThen g)(gi andThen fi) }

Slide 50

Slide 50 text

Invariant Functors 50 • Every (covariant) functor is an invariant functor (which ignores the B => A) • Every contravariant functor is an invariant functor (which ignores the A => B) Invariant Contravariant Functor Applicative Monoidal Monad

Slide 51

Slide 51 text

Pairing Codecs 51 trait Codec[A] extends Encoder[A] with Decoder[A] { self => def pairWith[B](cb: Codec[B]): Codec[(A, B)] = new Codec[(A, B)] { def sizeBound = self.sizeBound + b.sizeBound def encode(ab: (A, B)) = for { encA <- self.encode(ab._1) encB <- cb.encode(ab._2) } yield encA ++ encB def decode(bv: BitVector) = (for { a <- self b <- cb } yield (a, b)).decode(bv) } } This provides all the power necessary to build Codec[TupleN[…]] for an arbitrary N! Q: Can we zip codecs? } } Encode each element independently and concatenate result Decode an A, decode a B, return result as tuple

Slide 52

Slide 52 text

Pairing Codecs 52 trait Codec[A] extends Encoder[A] with Decoder[A] { self => def pairWith[B](cb: Codec[B]): Codec[(A, B)] = ??? } This has the same signature as Monoidal[F].zip… …but we can’t have a Monoidal[Codec] because there’s no Functor[Codec] Still, what if we paired this zip operation with xmap instead of map?

Slide 53

Slide 53 text

Invariant Monoidal Functors 53 @typeclass trait InvariantMonoidal[F[_]] extends Invariant[F] { def pure[A](a: A): F[A] def zip[A, B](fa: F[A], fb: F[B]): F[(A, B)] }

Slide 54

Slide 54 text

Invariant Monoidal Functors 54 trait InvariantMonoidalLaws[F[_]] extends InvariantLaws[F] { implicit val tc: InvariantMonoidal[F] import InvariantMonoidal.ops._ def invmonoidalLeftIdentity[A](fa: F[A]): IsEqv[F[A]] = tc.pure(()).zip(fa).xmap(_._2)(a => ((), a)) <-> fa def invmonoidalRightIdentity[A](fa: F[A]): IsEqv[F[A]] = fa.zip(tc.pure(())).xmap(_._1)(a => (a, ())) <-> fa def invmonoidalAssociativity[A, B, C]( fa: F[A], fb: F[B], fc: F[C] ): IsEqv[F[(A, (B, C))]] = fa.zip(fb.zip(fc)) <-> (fa.zip(fb)).zip(fc). xmap { case ((a, b), c) => (a, (b, c)) } { case (a, (b, c)) => ((a, b), c) } } Laws are the same as the lax monoidal functor laws, but include an inverse

Slide 55

Slide 55 text

Zip 55 @typeclass trait Zip[F[_]] { def pure[A](a: A): F[A] def zip[A, B](fa: F[A], fb: F[B]): F[(A, B)] } @typeclass trait Monoidal[F[_]] extends Functor[F] with Zip[F] @typeclass trait InvariantMonoidal[F[_]] extends Invariant[F] with Zip[F] We could also extract a Zip type class, whose laws defer type normalization! • e.g., monoidal left identity would be expressed as a deferred equivalence of a F[(Unit, A)] and an F[A]

Slide 56

Slide 56 text

One final transformation (or two)… 56 2 0 8 16 "OK" Remainder When working with decoders, we saw how flatMap could be used to model a dependency: We know we can’t write flatMap for Codec due to the non-existence of a Functor… Can we find inspiration in this example instead? Note that when encoding, the size value can be derived from the encoded form of “OK”!

Slide 57

Slide 57 text

One final transformation (or two)… 57 trait Codec[A] extends Encoder[A] with Decoder[A] { self => def consume[B](f: A => Codec[B])(g: B => A): Codec[B] = new Codec[B] { def sizeBound = self.sizeBound.atLeast def decode(bv: BitVector) = (for { a <- self b <- f(a) } yield b).decode(bv) def encode(b: B) = { val a = g(b) for { encA <- self.encode(a) encB <- f(a).encode(b) } yield encA ++ encB } } }

Slide 58

Slide 58 text

Using Codec#consume 58 2 0 8 16 "OK" Remainder uint8.consume { sz => fixedSizeBytes(sz, utf8) } { str => numBytesFor(str) } uint8.consume { sz => fixedSizeBytes(sz, ascii) } { str => str.size }

Slide 59

Slide 59 text

Using Codec#consume 59 case class Flags(woozle: Boolean, wocket: Boolean) val woozle: Codec[Woozle] = ??? val wocket: Codec[Wocket] = ??? case class Grocket(woozle: Option[Woozle], wocket: Option[Wocket]) flags.consume { flgs => (conditional(flgs.woozle, woozle) :: conditional(flgs.wocket, wocket) ).as[Grocket] } { g => Flags(g.woozle.isDefined, g.wocket.isDefined) } 0 8 16 1 1 0 8 16 0 1 0 8 16 1 0 0 8 16 0 0

Slide 60

Slide 60 text

Deriving flatZip from consume 60 trait Codec[A] extends Encoder[A] with Decoder[A] { self => def consume[B](f: A => Codec[B])(g: B => A): Codec[B] = ??? def flatZip[B](f: A => Codec[B]): Codec[(A, B)] = consume { a => f(a).xmap(b => (a, b))(ab => ab._2) } { case (a, b) => a } }

Slide 61

Slide 61 text

Invariant Monads 61 @typeclass trait InvariantMonad[F[_]] extends InvariantMonoidal[F] { import InvariantMonad.ops._ private implicit def self = this def xflatMap[A, B](fa: F[A])(f: A => F[B])(g: B => A): F[B] def xflatMapProduct[A, B](fa: F[A])(f: A => F[B]): F[(A, B)] = fa.xflatMap { a => f(a).xmap { b => (a, b) } { case (a, b) => b } } { case (a, b) => a } override def xmap[A, B](fa: F[A])(f: A => B)(g: B => A): F[B] = xflatMap(fa)(f andThen pure)(g) override def zip[A, B](fa: F[A], fb: F[B]): F[(A, B)] = xflatMapProduct(fa)(_ => fb) }

Slide 62

Slide 62 text

Invariant Monads - Laws 62 trait InvariantMonadLaws[F[_]] extends InvariantMonoidalLaws[F] { implicit val tc: InvariantMonad[F] import InvariantMonad.ops._ def invMonadLeftIdentity[A, B]( a: A, f: A => F[B], fi: B => A): IsEqv[F[B]] = tc.pure(a).xflatMap(f)(fi) <-> f(a) def invMonadRightIdentity[A](fa: F[A]): IsEqv[F[A]] = fa.xflatMap { a => tc.pure(a) } { identity } <-> fa def invMonadXflatMapAssoc[A, B, C]( fa: F[A], f: A => F[B], fi: B => A, g: B => F[C], gi: C => B ): IsEqv[F[C]] = fa.xflatMap(f)(fi).xflatMap(g)(gi) <-> fa.xflatMap { a => f(a).xflatMap(g)(gi) } { gi andThen fi } }

Slide 63

Slide 63 text

Today’s Tour 63 Invariant Contravariant Functor Applicative Monoidal Monad Invariant Monoidal Zip Invariant Monad Disclaimer: Invariant (Lax) Monoidal Functors and Invariant Monads are not known in FP literature (AFAIK). It may be presumptuous to give these type classes these specific names, but the type classes and laws are completely legitimate.

Slide 64

Slide 64 text

Takeaways • Practical library design can be informed by FP theory …and FP theory can be investigated with inspiration from library design • Moving freely between working in the concrete and working in the abstract and vice-versa informs the design process • It is not necessary to learn formal category theory to reason with and apply FP structures • FP structures are useful — not exercises in needless generalization 64

Slide 65

Slide 65 text

Acknowledgements, Credits, and References • Special thanks to reviewers: Michael Smith, Matt Hughes, Erik Osheim, Jack Rudnick, Ben Rockstroh, and Paul Chiusano • Thanks to the functional Scala ecosystem, including libraries like non/algebra, Cats, Scalaz, and their respective communities • Libraries of interest: • https://github.com/non/cats • https://github.com/mpilquist/simulacrum • https://github.com/non/imp • https://github.com/typelevel/machinist • Information on invariant monoidals and monads: http://mpilquist.github.io/blog/2015/06/18/invariant-shadows/ http://mpilquist.github.io/blog/2015/06/22/invariant-shadows-part-2/ 65

Slide 66

Slide 66 text

No content