Easy JSON deserialization with Simple-JSON and Record Justin Woo Berlin Functional Programming Group 22 Feb 2018 

About me Korean-American guy in Helsinki, I enjoy sauna and unlimited mobile broadband You might have seen my Twitter memes PureScript user since Jan 2016 Haskell user since Mar 2016 

What is JSON? ● Annoying hashmap hell data format ● Can’t encode anything very usefully ● But it’s easy enough to work with, I guess ● Everyone uses it, like it or not 

How do we use JSON? ● Parse it out! ● Dynamic ○ parseJSON :: forall a. String -> a ○ A.k.a. Unsafe coercion hell ● Static - Manual ○ parseJSON :: String -> Either Error Foreign ○ readProperty :: Foreign -> Either Error Foreign ○ readString :: Foreign -> Either Error String ○ readInt :: Foreign -> Either Error Int ○ Safe, but annoying and manual 

Static types + manual decoding = tiring, error prone, meaningless ● type Config = { url :: String, id :: Int } ● All of the static information we need is right here: ○ “url” field of String ○ “id” field of Int ● { “url”: “google.com”, “id”: 1 } ○ Right Config ● { } ○ Left Error ● What if you could get JSON parsing for free? 

Simple-JSON ● No work needed for any type aliases of parsable types ● Simply newtype derive ReadForeign instances for newtypes of these newtype FilePath = FilePath String derive newtype instance rfFP :: ReadForeign FilePath newtype Id = Id Int derive newtype instance rfI :: ReadForeign Id type Config = { filePath :: FilePath, id :: Id } getConfig = readJSON <$> readTextFile UTF8 "./config.json" 

How? ● class ReadForeign a ● With instances for… ○ Foreign ○ Char, Int, Number, String, Boolean ○ Nullable, NullOrUndefined ○ Array ○ Record ○ Variant instance readArray :: ReadForeign a => ReadForeign (Array a) where readImpl = traverse readImpl <=< readArray 

Record Decoding ● Records in PureScript are parameterized with row types ● type Config = { filePath :: FilePath, id :: Id } ● type Config = Record ( filePath :: FilePath, id :: Id ) ● data Record :: # Type -> Type ○ # indicates row type where field is of kind provided ○ E.g. data Eff :: # Effect -> Type -> Type ● Record instance takes this row (unordered set) and converts it into a type-level RowList (ordered list) via RowToList RowToList (filePath :: FilePath, id :: Id) (Cons "filePath" FilePath (Cons "id" Id Nil)) 

“Gotcha” ● What if… ○ The type of the JSON values provided is wrong? ○ The name of a field is unpleasant/wrong? ● These are solved problems ● We can use the Record library for generic operations on records 

How? ● We can perform generic operations on records unlike most languages ● Remember { | row } ~ Record row ● PureScript-Record ○ E.g. what is “get” for records? ■ For all types row row’ l a ■ If l is a Symbol (type-level string) ■ And there exists a structural subtype row’ where a field has been added with label l and type a to form row ■ Then there is a function for a proxy carrying l to a record of type { | row } to a get :: forall r r' l a . IsSymbol l => RowCons l a r' r => SProxy l -> { | r } -> a 

Illustrated 

“Type is Wrong” ● Field might be Nullable but really you want an Array, where nulls are turned into empty arrays ● No problem, we can simply modify the field modify :: forall r1 r2 r l a b . IsSymbol l => RowCons l a r r1 => RowCons l b r r2 => SProxy l -> (a -> b) -> { | r1 } -> { | r2 } 

Modifying a field type MyThingy = { a :: String, b :: Array String } parseMyThingyJsonFromImperfectJsonButConvertTheDirtyProperty :: String -> Either (NonEmptyList ForeignError) MyThingy parseMyThingyJsonFromImperfectJsonButConvertTheDirtyProperty str = modify (SProxy :: SProxy "b") (fromMaybe [] <<< Nullable.toMaybe) <$> readJSON str ● So { "a": "hello", "b": null } works fine ● How does this work? 

Normal symbol substitution type MyThingy = { a :: String, b :: Array String } modify :: forall r1 (a :: String, b :: Array String) r "b" (Nullable (Array String)) (Array String) . RowCons "b" (Nullable (Array String)) r r1 => RowCons "b" (Array String) r (a :: String, b :: Array String) class RowCons (l :: Symbol) (a :: Type) (i :: # Type) (o :: # Type) | l a i -> o , l o -> a i We have l ("b") and o ((a :: String, b :: Array String)), so a and i can be solved for. We already know a, so it acts as an extra constraint, and then i is produced from the substitution. We know from the second constraint that r will then be (a :: String) without the field (b :: Array String) Then with r we can use the first constraint to insert (b :: Nullable (Array String)) So r1 is (a :: String, b :: Nullable (Array String)) And the input is parsed as { a :: String, b :: Nullable (Array String) } by ReadForeign instances 

Shortcut for single modification parseMyThingyJsonFromImperfectJsonButConvertTheDirtyProperty :: String -> Either (NonEmptyList ForeignError) MyThingy parseMyThingyJsonFromImperfectJsonButConvertTheDirtyProperty str = do json <- readJSON str let b = fromMaybe [] <<< Nullable.toMaybe $ json.b pure $ json { b = b } Using normal record update syntax 

Renaming a field More substitution fun rename :: forall prev next ty input inter output . IsSymbol prev -- previous name Symbol => IsSymbol next -- next name Symbol => RowCons prev ty inter input -- (prev, ty) + inter = input => RowLacks prev inter -- inter does not have a field with label prev => RowCons next ty inter output -- (next, ty) + inter = output => RowLacks next inter -- inter does not have a field with label next => SProxy prev -> SProxy next -> Record input -> Record output 

Renaming a field (cont.) type MyThing = { "fieldA" :: String, "fieldB" :: Int } decodeMyThingFromDirtyJSON :: String -> Either (NonEmptyList ForeignError) MyThing decodeMyThingFromDirtyJSON s = do parsed <- readJSON s pure $ rename (SProxy :: SProxy "MY_FIELD_A") (SProxy :: SProxy "fieldA") parsed Done! 

Conclusion ● Being able to work with records using generic row type information is cool ○ Records are not Hashmaps and vice versa ○ Always better to have stronger guarantees and not have invalid branches ■ “Make illegal states impossible” ● Nobody should have to decode JSON by hand ○ You should only need to do small tweaks using generic operations ● None of these techniques are specific to JSON ○ E.g. https://github.com/justinwoo/purescript-tortellini ○ Weird argument made by people who should know better ● People sometimes get really offended by this conclusion slide ○ “My favorite language can’t do this so this offends me” ○ You are not your tool 

Extra “What about Sum/Product types?” ● Let’s be honest, when has encoding them not been a giant mess ● Build your own solution with datatype generics using Generics-Rep ○ See my Simple-JSON-Generic-Sums repo ■ Sums encoded as { type, value } ■ Products encoded as heterogeneous array ■ Pre 0.12: converting reps record fields into fields and vice versa :barf: ■ Alas, this is not a talk about datatype generics ● Obvious solution: use Variants for sums, Records for products 

Thanks Links ● Simple-JSON: github.com / justinwoo / purescript-simple-json ● Record: github.com / purescript / purescript-record ● Me on Twitter: @jusrin00 ● Generics-Rep for ADTs github.com / justinwoo / simple-json-generic-sums