Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Zero effort JSON de/serialization with Simple-JSON

Justin Woo
November 28, 2017

Zero effort JSON de/serialization with Simple-JSON

A talk I gave for PureScript Helsinki on Simple-JSON https://www.meetup.com/PureScript-Helsinki/events/244635347/

Justin Woo

November 28, 2017
Tweet

More Decks by Justin Woo

Other Decks in Programming

Transcript

  1. Zero effort JSON de/serialization
    with Simple-JSON
    Justin Woo
    PureScript Helsinki
    28 Nov 2017

    View full-size slide

  2. What is JSON?
    ● Annoying data format, usually loses information
    ● People like to put whatever in it, valid or not
    ● Needs to be extracted from and validated
    ● We’re stuck with it
    ● But at least it’s kind of easy to work with

    View full-size slide

  3. How is JSON read normally?
    ● Dynamically
    ○ JSON.parse :: String -> a
    ■ How is the result always a?
    ● It isn’t
    ■ Very error prone
    ● Cannot read property ‘apple’ of undefined
    ■ Some approaches use schemas for validation
    ● But you don’t get to use this information to automatically check
    the rest of your program!

    View full-size slide

  4. Common JSON methods cont.
    ● Statically typed approach via manual decoding
    ○ Comes with specific methods
    ■ parseJSON :: String -> Either Error JSObject
    ■ readStringProperty :: String -> JSObject -> Either Error String
    ■ readString :: JSForeign -> Either Error String
    ○ But ends up being quite error-prone
    ■ apple <- readStringProperty "appllee" jsObj
    ■ banana <- readStringProperty "apple" jsObj

    View full-size slide

  5. Common JSON methods cont. 2
    ● Reflection with annotations
    ○ E.g. Java + Jackson
    ○ Annotations can get out of sync quickly, but kind of works otherwise
    ● Datatype generics
    ○ E.g. via Generics-Rep, GHC Generics, Shapeless, etc.
    ○ Uses the actual type information!
    ○ Labels from the type information are used for reading records
    ■ E.g. { a :: String } reads using property “a”
    ● Metaprogramming and/or codegen
    ○ More clumsy to use, but usually at least uses the type information or the AST

    View full-size slide

  6. But what if we could directly use type information?
    ● No more schema boilerplate e.g. schema, joi
    ● No more manual boilerplate e.g. readString
    ● No more generics boilerplate e.g. derive Generic, instance ReadJSON
    ● Just use the type definition
    ○ E.g. { a :: String, b :: Int }
    ○ There’s already enough type information here!

    View full-size slide

  7. Simple-JSON
    ● Free JSON decoding with type aliases (and types in context, by annotation or usage)
    type MyType = { a :: String, b :: Int }
    result <- readJSON <$> readTextFile UTF8 "./config.json"
    case result of
    Right myType -> myFunction myType -- myFunction :: MyType -> ...
    ● Works with int, string, boolean, arrays, nullables, records, and more
    ● Records enabled by using row type information and RowToList (more on this later)
    Record ( a :: String, b :: Int )
    RowToList, (Cons "a" String (Cons "b" Int Nil))

    View full-size slide

  8. Flexibility of being able to use type information
    ● We can return the correct type and modify the type used to parse
    ● E.g. we can rename fields using purescript-record
    type MyThing = { "fieldA" :: String, "fieldB" :: Int }
    decodeMyThingFromDirtyJSON :: String -> Either _ MyThing
    decodeMyThingFromDirtyJSON s = do
    parsed <- readJSON s
    pure $ rename
    (SProxy :: SProxy "MY_FIELD_A")
    (SProxy :: SProxy "fieldA")
    parsed -- { MY_FIELD_A :: String, fieldB :: Int }

    View full-size slide

  9. Example 2 - changing the type of a field
    type MyThingy =
    { a :: String
    , b :: Array String
    }
    parseMyThingyJsonFromDirtyJSON :: String -> Either _ MyThingy
    parseMyThingyJsonFromDirtyJSON str = do
    json <- readJSON str
    -- json read in as { a :: String, b :: Nullable (Array String) }
    let b = fromMaybe [] <<< Nullable.toMaybe $ json.b
    -- this value is Array String, and we can replace the existing field like so:
    pure $ json { b = b }

    View full-size slide

  10. Implementation
    ● Don’t really need to know this, but it can be useful to learn
    ● Implemented using a type class defined for concrete types
    class ReadForeign a where
    readImpl :: Foreign -> F a
    instance readString :: ReadForeign String where
    readImpl = readString
    instance readArray :: ReadForeign a => ReadForeign (Array a) where
    readImpl = traverse readImpl <=< readArray

    View full-size slide

  11. The Record Instance
    { a :: String, b :: Int } ~ Record ( a :: String, b :: Int )
    RowToList ( a :: String, b :: Int ) (Cons "a" String (Cons "b" Int Nil))
    instance readRecord ::
    ( RowToList fields fieldList
    , ReadForeignFields fieldList () fields
    ) => ReadForeign (Record fields) where
    readImpl o = do
    steps <- getFields (RLProxy :: RLProxy fieldList) o
    pure $ Builder.build steps {}
    Uses purescript-record Builder to build the whole record from an empty record

    View full-size slide

  12. ReadForeignFields
    class ReadForeignFields (xs :: RowList) (from :: # Type) (to :: # Type)
    | xs -> from to where
    getFields :: RLProxy xs
    -> Foreign
    -> F (Builder (Record from) (Record to))
    Uses the RowList to match instances of the type class and has the input and output row types as params
    The returned Builder in F (i.e. Except MultipleErrors) ultimately provides empty row to the full record

    View full-size slide

  13. Nil case
    When there’s no more keys to add, the operation needed is identity
    instance readFieldsNil ::
    ReadForeignFields Nil () () where
    getFields _ _ =
    pure id

    View full-size slide

  14. Cons case - the important parts
    instance readFieldsCons ::
    ( IsSymbol name -- need to do Symbol operations, like getting the value level string
    , ReadForeign ty -- reading the value from Foreign
    , ReadForeignFields tail from from' -- the rest of the fields, with the intermediate result
    , RowLacks name from' -- the current label does not exist in the intermediate result
    , RowCons name ty from' to -- adding the label with the type to the intermediate result
    -- gives us the final result we want
    ) => ReadForeignFields (Cons name ty tail) from to where
    getFields _ obj = do
    ...

    View full-size slide

  15. Cons case - the mechanical parts
    getFields _ obj = do
    value :: ty <- withExcept' $ readImpl =<< readProp name obj -- read the field
    rest <- getFields tailP obj -- get the Builder for the rest of the fields
    let
    first :: Builder (Record from') (Record to)
    first = Builder.insert nameP value -- make the builder for the current field
    pure $ first <<< rest -- compose the builders to get the result builder
    where
    nameP = SProxy :: SProxy name
    tailP = RLProxy :: RLProxy tail
    name = reflectSymbol nameP
    withExcept' = withExcept <<< map $ ErrorAtProperty name

    View full-size slide

  16. That’s it!
    Now you’ve seen some type-level programming in action
    ...and it solves real world problems more reliably than term-level programming

    View full-size slide

  17. Takeaways
    ● JSON decoding can be easy, if your language gives you tools to do it
    ● Simple-JSON is fun
    ● You shouldn’t have to manually do anything unless you have very specific needs
    ○ We’ve already shown renaming is no big deal
    ○ Neither is changing the type
    ○ Removing nesting would be application of the two in various ways
    ● Row types are fun
    ● RowToList is fun
    ● Type level programming is fun

    View full-size slide

  18. Thanks for coming!
    Some Links
    ● https://www.meetup.com/PureScript-Helsinki/
    ● https://github.com/justinwoo/purescript-simple-json
    ● https://www.reddit.com/r/purescript/comments/7b5y7q/some_extra_examples_of_simplejson_us
    age/
    ● https://github.com/justinwoo/awesome-rowlist
    ● https://speakerdeck.com/justinwoo/rowlist-fun-with-purescript-2nd-edition
    ● https://twitter.com/jusrin00

    View full-size slide