Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Swift's Encoder and Decoder Protocols

Kaitlin Mahar
September 13, 2018
94

Swift's Encoder and Decoder Protocols

Ever wonder what’s actually going on inside JSONEncoder and JSONDecoder? This talk will be a deep dive into Swift’s Encoder and Decoder protocols. Drawing on a few different Encoders and Decoder examples, we’ll cover what the protocols require and how all the pieces fit together, as well as what design decisions are left up to the programmer and how to make those decisions.

Kaitlin Mahar

September 13, 2018
Tweet

Transcript

  1. Swift's Encoder
    and Decoder Protocols
    Kaitlin Mahar
    Software Engineer @ MongoDB
    @k__mahar @kmahar

    View Slide

  2. What is encoding?
    Encoder
    Swift type External
    representation
    etc...
    Scalar (e.g. Int)
    struct
    class
    enum

    View Slide

  3. What is encoding?
    JSON
    Encoder

    View Slide

  4. What is decoding?
    Decoder
    Swift type External
    representation
    etc...
    Scalar (e.g. Int)
    struct
    class
    enum

    View Slide

  5. What is decoding?
    JSON
    Decoder

    View Slide

  6. Swift 4 introduced a
    standardized approach to
    encoding and decoding.
    How does it actually work?

    View Slide

  7. Why does it matter?
    ● Customize how your types are encoded and decoded
    ○ Omit or rename properties, flatten nested structs...
    ● Write your own encoder and/or decoder
    ○ MongoDB driver
    ■ BSONEncoder: Swift type → MongoDB document
    ■ BSONDecoder: MongoDB document → Swift type

    View Slide

  8. What we'll talk about
    ● The public API
    ● Internals
    ● Limitations

    View Slide

  9. The Public API

    View Slide

  10. An Encodable type knows how to
    write itself to an Encoder.

    View Slide

  11. ● Automatic conformance if all properties are Encodable
    ● Types can provide custom implementations
    ● Format agnostic: write it once, works with any Encoder!

    View Slide

  12. A Decodable type knows how to
    initialize by reading from a Decoder.

    View Slide

  13. ● Automatic conformance if all properties are Decodable
    ● Types can provide custom implementations
    ● Write it once, works with any Decoder

    View Slide

  14. View Slide

  15. Types With Built-In Codable Support
    ● Numeric types
    ● Bool
    ● String
    ● If the values they contain are Encodable / Decodable:
    ○ Array
    ○ Set
    ○ Dictionary
    ○ Optional
    ● Common Foundation types: URL, Data, Date, etc.

    View Slide

  16. … and that's it!
    Making Types Codable

    View Slide

  17. Using Encoders and Decoders

    View Slide

  18. Using An Encoder
    JSONEncoder
    Encodable
    value of type T
    UTF-8
    encoded Data

    View Slide

  19. Using An Encoder

    View Slide

  20. Why doesn't the API match the Encodable protocol?

    View Slide

  21. _JSONEncoder:
    Encoder
    JSONEncoder
    Why doesn't the API match the Encodable protocol?
    private
    ● Not dictated by any protocol
    ● Allows setting top-level options
    (e.g. DateEncodingStrategy)
    ● Simple API, just one method
    ● Implements Encoder protocol
    ● Actually does the encoding work
    ● More complex API (stay tuned)

    View Slide

  22. Using A Decoder
    JSONDecoder
    Decodable
    value of type T
    Type to decode
    to, and UTF-8
    encoded Data

    View Slide

  23. Using A Decoder
    Data we got
    from encoding

    View Slide

  24. _JSONDecoder:
    Decoder
    JSONDecoder
    Why doesn't the API match the Decodable protocol?
    private

    View Slide

  25. Ok, so… what do these
    protocols actually require?

    View Slide

  26. Encoder
    Single
    Value
    Unkeyed Keyed
    one value sequence of values key/value pairs
    provides containers:
    views into storage

    View Slide

  27. In code...
    SingleValueEncodingContainer
    UnkeyedEncodingContainer
    KeyedEncodingContainer
    Encoder

    View Slide

  28. Encoding containers support storing three types of values.
    nil
    Bool, String, Double, Float
    all Int and UInt types
    Encodable type
    base case 1: nil
    base case 2:
    primitives
    recursive case

    View Slide

  29. nil
    primitive
    Encodable
    SingleValueEncodingContainer

    View Slide

  30. nil
    primitive
    Encodable
    SingleValueEncodingContainer
    UnkeyedEncodingContainer

    View Slide

  31. Encodable
    primitive
    nil
    KeyedEncodingContainerProtocol

    View Slide

  32. CodingKey
    KeyedEncodingContainer
    Encoder

    View Slide

  33. CodingKey
    Requirements:
    ● Initializable by String
    and/or Int
    ● Must have a String
    representation
    ● May have an Int
    representation
    Synthesized conformance for enums!

    View Slide

  34. Encoder
    Single
    Value
    Unkeyed Keyed
    Unkeyed Keyed Unkeyed Keyed
    recursive case

    View Slide

  35. Keyed
    Unkeyed
    UnkeyedEncodingContainer
    KeyedEncodingContainerProtocol
    Unkeyed
    Keyed

    View Slide

  36. So how are these
    containers used?

    View Slide

  37. View Slide

  38. Encoder
    KeyedEncodingContainer
    name color
    "Chester" "tan"

    View Slide

  39. Encoder
    KeyedEncodingContainer
    name color
    "Chester" "tan"

    View Slide

  40. Encoder
    KeyedEncodingContainer
    name color
    "Chester" "tan"
    Compiler generated!

    View Slide

  41. But what if we don't want the defaults?
    Encoder
    KeyedEncodingContainer
    firstName color
    "Chester" "tan"
    We can override just CodingKeys to
    rename keys but keep the default
    encode(to:) implementation.

    View Slide

  42. But what if we don't want the defaults?
    We can override just CodingKeys to
    omit keys altogether.
    Encoder
    KeyedEncodingContainer
    name
    "Chester"

    View Slide

  43. But what if we don't want the defaults?
    Encoder
    KeyedEncodingContainer
    name color
    "chester" "orange"
    Or we can override encode(to:) to
    further customize behavior.

    View Slide

  44. Let's make things more complicated...

    View Slide

  45. Encoder
    KeyedContainer
    name cats
    "Kaitlin"
    Unkeyed
    KeyedContainer
    name color
    "Chester" "tan"
    KeyedContainer
    name color
    "Roscoe" "orange"
    CatOwner
    cats
    cats[1]
    cats[0]

    View Slide

  46. KeyedContainer
    name color
    "Roscoe" "orange"
    Unkeyed
    Encoder
    name cats
    "Kaitlin"
    KeyedContainer
    KeyedContainer
    name color
    "Chester" "tan"

    View Slide

  47. KeyedContainer
    name color
    "Roscoe" "orange"
    Unkeyed
    KeyedContainer
    name color
    "Chester" "tan"
    Encoder
    name cats
    "Kaitlin"
    KeyedContainer

    View Slide

  48. Alternatively...

    View Slide

  49. KeyedContainer
    name color
    "Roscoe" "orange"
    Unkeyed
    name color
    "Chester" "tan"
    Encoder
    name cats
    "Kaitlin"
    KeyedContainer
    KeyedContainer
    Again, compiler generated!

    View Slide

  50. Weren't we also talking
    about decoding?

    View Slide

  51. Encoding: putting values into containers
    Decoding: taking values out of containers

    View Slide

  52. Decoder
    Single
    Value
    Unkeyed Keyed
    Unkeyed Keyed Unkeyed Keyed
    recursive case

    View Slide

  53. In code...
    Decoder
    SingleValueDecodingContainer
    UnkeyedDecodingContainer
    KeyedDecodingContainer

    View Slide

  54. Decoding containers support retrieving three types of values.
    nil
    Bool, String, Double, Float
    all Int and UInt types
    Decodable type
    base case 1: nil
    base case 2:
    primitives
    recursive case

    View Slide

  55. SingleValueDecodingContainer
    nil
    primitive
    Decodable

    View Slide

  56. nil
    primitive
    Decodable
    UnkeyedDecodingContainer

    View Slide

  57. KeyedDecodingContainerProtocol
    nil
    primitive
    Decodable

    View Slide

  58. Decoder
    KeyedDecodingContainer
    name color
    "Chester" "tan"

    View Slide

  59. Decoder
    KeyedDecodingContainer
    name color
    "Chester" "tan"

    View Slide

  60. Decoder
    KeyedDecodingContainer
    name color
    "Chester" "tan"
    Compiler generated!

    View Slide

  61. Public API Takeaways
    ● Types opt in by conforming to Encodable and/or Decodable
    ● Many types get conformance for free, but can customize when
    needed
    ● Public types (e.g. JSONEncoder) do *not* implement the
    corresponding protocols!
    ● Encoders and Decoders used container-based APIs for
    reading from/writing to storage
    ● The Encoder and Decoder APIs are very similar!

    View Slide

  62. Underneath the Hood

    View Slide

  63. Typical Structure
    _MyEncoder
    : Encoder
    storage
    private
    MyEncoder
    _MyDecoder
    : Decoder
    storage
    private
    MyDecoder

    View Slide

  64. Storage
    In what format do I store values while I am still in the middle
    of encoding/decoding?
    ● JSON: NS* types that work with JSONSerialization.
    ● PropertyList: NS* types that work with PropertyListSerialization.
    ● BSON: types conforming to BsonValue that work with MongoDB documents.
    How do I track where I am in the maze of nested containers?
    ● JSON, BSON, PropertyList: stack with whatever is backing the current
    container on top

    View Slide

  65. 4 more small but important things to note:
    1. SingleValueXContainer is often just implemented via an
    extension of the private Encoder type.
    2. The Encoder must enforce that only one value is written to a
    SingleValueEncodingContainer.
    3. There are various bookkeeping requirements on the protocols for
    tracking the path of keys taken so far.
    4. There are also superEncoder and superDecoder requirements for
    use with classes.

    View Slide

  66. Limitations
    ● Lots of boilerplate required for all of the containers, and
    the different encode and decode methods that they
    support
    ● Depending on design, you may end up duplicating a lot of
    code from JSONEncoder etc.
    ● Types cannot be extended to become Decodable (but
    should it be possible?)

    View Slide

  67. In conclusion...
    ● The API makes Codable conformance trivial in many cases, but also allows
    for very advanced customization when needed.
    ● The API makes it possible to write Encoders and Decoders that don't know
    what the types using them look like, and makes it possible for types to write
    encode methods without caring about the actual output format!
    ● Investigating the standard library source code and writing your own
    encoder/decoder reveals some hidden requirements.

    View Slide

  68. Thank you!
    Kaitlin Mahar
    Software Engineer @ MongoDB
    @k__mahar @kmahar

    View Slide