Zero effort JSON de/serialization with Simple-JSON

A talk I gave for PureScript Helsinki on Simple-JSON https://www.meetup.com/PureScript-Helsinki/events/244635347/

Transcript

1. Zero effort JSON de/serialization with Simple-JSON Justin Woo PureScript Helsinki 28 Nov 2017
2. What is JSON? • Annoying data format, usually loses information • People like to put whatever in it, valid or not • Needs to be extracted from and validated • We’re stuck with it • But at least it’s kind of easy to work with
3. How is JSON read normally? • Dynamically ◦ JSON.parse :: String -> a ▪ How is the result always a? • It isn’t ▪ Very error prone • Cannot read property ‘apple’ of undefined ▪ Some approaches use schemas for validation • But you don’t get to use this information to automatically check the rest of your program!
4. Common JSON methods cont. • Statically typed approach via manual decoding ◦ Comes with specific methods ▪ parseJSON :: String -> Either Error JSObject ▪ readStringProperty :: String -> JSObject -> Either Error String ▪ readString :: JSForeign -> Either Error String ◦ But ends up being quite error-prone ▪ apple <- readStringProperty "appllee" jsObj ▪ banana <- readStringProperty "apple" jsObj
5. Common JSON methods cont. 2 • Reflection with annotations ◦ E.g. Java + Jackson ◦ Annotations can get out of sync quickly, but kind of works otherwise • Datatype generics ◦ E.g. via Generics-Rep, GHC Generics, Shapeless, etc. ◦ Uses the actual type information! ◦ Labels from the type information are used for reading records ▪ E.g. { a :: String } reads using property “a” • Metaprogramming and/or codegen ◦ More clumsy to use, but usually at least uses the type information or the AST
6. But what if we could directly use type information? • No more schema boilerplate e.g. schema, joi • No more manual boilerplate e.g. readString • No more generics boilerplate e.g. derive Generic, instance ReadJSON • Just use the type definition ◦ E.g. { a :: String, b :: Int } ◦ There’s already enough type information here!
7. Simple-JSON • Free JSON decoding with type aliases (and types in context, by annotation or usage) type MyType = { a :: String, b :: Int } result <- readJSON <$> readTextFile UTF8 "./config.json" case result of Right myType -> myFunction myType -- myFunction :: MyType -> ... • Works with int, string, boolean, arrays, nullables, records, and more • Records enabled by using row type information and RowToList (more on this later) Record ( a :: String, b :: Int ) RowToList, (Cons "a" String (Cons "b" Int Nil))
8. Flexibility of being able to use type information • We can return the correct type and modify the type used to parse • E.g. we can rename fields using purescript-record type MyThing = { "fieldA" :: String, "fieldB" :: Int } decodeMyThingFromDirtyJSON :: String -> Either _ MyThing decodeMyThingFromDirtyJSON s = do parsed <- readJSON s pure $ rename (SProxy :: SProxy "MY_FIELD_A") (SProxy :: SProxy "fieldA") parsed -- { MY_FIELD_A :: String, fieldB :: Int }
9. Example 2 - changing the type of a field type MyThingy = { a :: String , b :: Array String } parseMyThingyJsonFromDirtyJSON :: String -> Either _ MyThingy parseMyThingyJsonFromDirtyJSON str = do json <- readJSON str -- json read in as { a :: String, b :: Nullable (Array String) } let b = fromMaybe [] <<< Nullable.toMaybe $ json.b -- this value is Array String, and we can replace the existing field like so: pure $ json { b = b }
10. Implementation • Don’t really need to know this, but it can be useful to learn • Implemented using a type class defined for concrete types class ReadForeign a where readImpl :: Foreign -> F a instance readString :: ReadForeign String where readImpl = readString instance readArray :: ReadForeign a => ReadForeign (Array a) where readImpl = traverse readImpl <=< readArray
11. The Record Instance { a :: String, b :: Int } ~ Record ( a :: String, b :: Int ) RowToList ( a :: String, b :: Int ) (Cons "a" String (Cons "b" Int Nil)) instance readRecord :: ( RowToList fields fieldList , ReadForeignFields fieldList () fields ) => ReadForeign (Record fields) where readImpl o = do steps <- getFields (RLProxy :: RLProxy fieldList) o pure $ Builder.build steps {} Uses purescript-record Builder to build the whole record from an empty record
12. ReadForeignFields class ReadForeignFields (xs :: RowList) (from :: # Type) (to :: # Type) | xs -> from to where getFields :: RLProxy xs -> Foreign -> F (Builder (Record from) (Record to)) Uses the RowList to match instances of the type class and has the input and output row types as params The returned Builder in F (i.e. Except MultipleErrors) ultimately provides empty row to the full record
13. Nil case When there’s no more keys to add, the operation needed is identity instance readFieldsNil :: ReadForeignFields Nil () () where getFields _ _ = pure id
14. Cons case - the important parts instance readFieldsCons :: ( IsSymbol name -- need to do Symbol operations, like getting the value level string , ReadForeign ty -- reading the value from Foreign , ReadForeignFields tail from from' -- the rest of the fields, with the intermediate result , RowLacks name from' -- the current label does not exist in the intermediate result , RowCons name ty from' to -- adding the label with the type to the intermediate result -- gives us the final result we want ) => ReadForeignFields (Cons name ty tail) from to where getFields _ obj = do ...
15. Cons case - the mechanical parts getFields _ obj = do value :: ty <- withExcept' $ readImpl =<< readProp name obj -- read the field rest <- getFields tailP obj -- get the Builder for the rest of the fields let first :: Builder (Record from') (Record to) first = Builder.insert nameP value -- make the builder for the current field pure $ first <<< rest -- compose the builders to get the result builder where nameP = SProxy :: SProxy name tailP = RLProxy :: RLProxy tail name = reflectSymbol nameP withExcept' = withExcept <<< map $ ErrorAtProperty name
16. That’s it! Now you’ve seen some type-level programming in action ...and it solves real world problems more reliably than term-level programming
17. Takeaways • JSON decoding can be easy, if your language gives you tools to do it • Simple-JSON is fun • You shouldn’t have to manually do anything unless you have very specific needs ◦ We’ve already shown renaming is no big deal ◦ Neither is changing the type ◦ Removing nesting would be application of the two in various ways • Row types are fun • RowToList is fun • Type level programming is fun
18. Thanks for coming! Some Links • https://www.meetup.com/PureScript-Helsinki/ • https://github.com/justinwoo/purescript-simple-json • https://www.reddit.com/r/purescript/comments/7b5y7q/some_extra_examples_of_simplejson_us age/ • https://github.com/justinwoo/awesome-rowlist • https://speakerdeck.com/justinwoo/rowlist-fun-with-purescript-2nd-edition • https://twitter.com/jusrin00