Easy JSON deserialization with Simple-JSON and Record

Presentation given for the Berlin Functional Programming Group Meetup, 22 Feb 2018

https://www.meetup.com/Berlin-Functional-Programming-Group/events/246441427/

Transcript

1. Easy JSON deserialization with Simple-JSON and Record Justin Woo Berlin Functional Programming Group 22 Feb 2018
2. About me Korean-American guy in Helsinki, I enjoy sauna and unlimited mobile broadband You might have seen my Twitter memes PureScript user since Jan 2016 Haskell user since Mar 2016
3. What is JSON? • Annoying hashmap hell data format • Can’t encode anything very usefully • But it’s easy enough to work with, I guess • Everyone uses it, like it or not
4. How do we use JSON? • Parse it out! • Dynamic ◦ parseJSON :: forall a. String -> a ◦ A.k.a. Unsafe coercion hell • Static - Manual ◦ parseJSON :: String -> Either Error Foreign ◦ readProperty :: Foreign -> Either Error Foreign ◦ readString :: Foreign -> Either Error String ◦ readInt :: Foreign -> Either Error Int ◦ Safe, but annoying and manual
5. Static types + manual decoding = tiring, error prone, meaningless • type Config = { url :: String, id :: Int } • All of the static information we need is right here: ◦ “url” field of String ◦ “id” field of Int • { “url”: “google.com”, “id”: 1 } ◦ Right Config • { } ◦ Left Error • What if you could get JSON parsing for free?
6. Simple-JSON • No work needed for any type aliases of parsable types • Simply newtype derive ReadForeign instances for newtypes of these newtype FilePath = FilePath String derive newtype instance rfFP :: ReadForeign FilePath newtype Id = Id Int derive newtype instance rfI :: ReadForeign Id type Config = { filePath :: FilePath, id :: Id } getConfig = readJSON <$> readTextFile UTF8 "./config.json"
7. How? • class ReadForeign a • With instances for… ◦ Foreign ◦ Char, Int, Number, String, Boolean ◦ Nullable, NullOrUndefined ◦ Array ◦ Record ◦ Variant instance readArray :: ReadForeign a => ReadForeign (Array a) where readImpl = traverse readImpl <=< readArray
8. Record Decoding • Records in PureScript are parameterized with row types • type Config = { filePath :: FilePath, id :: Id } • type Config = Record ( filePath :: FilePath, id :: Id ) • data Record :: # Type -> Type ◦ # indicates row type where field is of kind provided ◦ E.g. data Eff :: # Effect -> Type -> Type • Record instance takes this row (unordered set) and converts it into a type-level RowList (ordered list) via RowToList RowToList (filePath :: FilePath, id :: Id) (Cons "filePath" FilePath (Cons "id" Id Nil))
9. “Gotcha” • What if… ◦ The type of the JSON values provided is wrong? ◦ The name of a field is unpleasant/wrong? • These are solved problems • We can use the Record library for generic operations on records
10. How? • We can perform generic operations on records unlike most languages • Remember { | row } ~ Record row • PureScript-Record ◦ E.g. what is “get” for records? ▪ For all types row row’ l a ▪ If l is a Symbol (type-level string) ▪ And there exists a structural subtype row’ where a field has been added with label l and type a to form row ▪ Then there is a function for a proxy carrying l to a record of type { | row } to a get :: forall r r' l a . IsSymbol l => RowCons l a r' r => SProxy l -> { | r } -> a
11. Illustrated
12. “Type is Wrong” • Field might be Nullable but really you want an Array, where nulls are turned into empty arrays • No problem, we can simply modify the field modify :: forall r1 r2 r l a b . IsSymbol l => RowCons l a r r1 => RowCons l b r r2 => SProxy l -> (a -> b) -> { | r1 } -> { | r2 }
13. Modifying a field type MyThingy = { a :: String, b :: Array String } parseMyThingyJsonFromImperfectJsonButConvertTheDirtyProperty :: String -> Either (NonEmptyList ForeignError) MyThingy parseMyThingyJsonFromImperfectJsonButConvertTheDirtyProperty str = modify (SProxy :: SProxy "b") (fromMaybe [] <<< Nullable.toMaybe) <$> readJSON str • So { "a": "hello", "b": null } works fine • How does this work?
14. Normal symbol substitution type MyThingy = { a :: String, b :: Array String } modify :: forall r1 (a :: String, b :: Array String) r "b" (Nullable (Array String)) (Array String) . RowCons "b" (Nullable (Array String)) r r1 => RowCons "b" (Array String) r (a :: String, b :: Array String) class RowCons (l :: Symbol) (a :: Type) (i :: # Type) (o :: # Type) | l a i -> o , l o -> a i We have l ("b") and o ((a :: String, b :: Array String)), so a and i can be solved for. We already know a, so it acts as an extra constraint, and then i is produced from the substitution. We know from the second constraint that r will then be (a :: String) without the field (b :: Array String) Then with r we can use the first constraint to insert (b :: Nullable (Array String)) So r1 is (a :: String, b :: Nullable (Array String)) And the input is parsed as { a :: String, b :: Nullable (Array String) } by ReadForeign instances
15. Shortcut for single modification parseMyThingyJsonFromImperfectJsonButConvertTheDirtyProperty :: String -> Either (NonEmptyList ForeignError) MyThingy parseMyThingyJsonFromImperfectJsonButConvertTheDirtyProperty str = do json <- readJSON str let b = fromMaybe [] <<< Nullable.toMaybe $ json.b pure $ json { b = b } Using normal record update syntax
16. Renaming a field More substitution fun rename :: forall prev next ty input inter output . IsSymbol prev -- previous name Symbol => IsSymbol next -- next name Symbol => RowCons prev ty inter input -- (prev, ty) + inter = input => RowLacks prev inter -- inter does not have a field with label prev => RowCons next ty inter output -- (next, ty) + inter = output => RowLacks next inter -- inter does not have a field with label next => SProxy prev -> SProxy next -> Record input -> Record output
17. Renaming a field (cont.) type MyThing = { "fieldA" :: String, "fieldB" :: Int } decodeMyThingFromDirtyJSON :: String -> Either (NonEmptyList ForeignError) MyThing decodeMyThingFromDirtyJSON s = do parsed <- readJSON s pure $ rename (SProxy :: SProxy "MY_FIELD_A") (SProxy :: SProxy "fieldA") parsed Done!
18. Conclusion • Being able to work with records using generic row type information is cool ◦ Records are not Hashmaps and vice versa ◦ Always better to have stronger guarantees and not have invalid branches ▪ “Make illegal states impossible” • Nobody should have to decode JSON by hand ◦ You should only need to do small tweaks using generic operations • None of these techniques are specific to JSON ◦ E.g. https://github.com/justinwoo/purescript-tortellini ◦ Weird argument made by people who should know better • People sometimes get really offended by this conclusion slide ◦ “My favorite language can’t do this so this offends me” ◦ You are not your tool
19. Extra “What about Sum/Product types?” • Let’s be honest, when has encoding them not been a giant mess • Build your own solution with datatype generics using Generics-Rep ◦ See my Simple-JSON-Generic-Sums repo ▪ Sums encoded as { type, value } ▪ Products encoded as heterogeneous array ▪ Pre 0.12: converting reps record fields into fields and vice versa :barf: ▪ Alas, this is not a talk about datatype generics • Obvious solution: use Variants for sums, Records for products
20. Thanks Links • Simple-JSON: github.com / justinwoo / purescript-simple-json • Record: github.com / purescript / purescript-record • Me on Twitter: @jusrin00 • Generics-Rep for ADTs github.com / justinwoo / simple-json-generic-sums