Swift's Encoder and Decoder Protocols

Ever wonder what’s actually going on inside JSONEncoder and JSONDecoder? This talk will be a deep dive into Swift’s Encoder and Decoder protocols. Drawing on a few different Encoders and Decoder examples, we’ll cover what the protocols require and how all the pieces fit together, as well as what design decisions are left up to the programmer and how to make those decisions.

Transcript

1. Swift's Encoder
   and Decoder Protocols
   Kaitlin Mahar
   Software Engineer @ MongoDB
   @k__mahar @kmahar
   

   View Slide

2. What is encoding?
   Encoder
   Swift type External
   representation
   etc...
   Scalar (e.g. Int)
   struct
   class
   enum
   

   View Slide

3. What is encoding?
   JSON
   Encoder
   

   View Slide

4. What is decoding?
   Decoder
   Swift type External
   representation
   etc...
   Scalar (e.g. Int)
   struct
   class
   enum
   

   View Slide

5. What is decoding?
   JSON
   Decoder
   

   View Slide

6. Swift 4 introduced a
   standardized approach to
   encoding and decoding.
   How does it actually work?
   

   View Slide

7. Why does it matter?
   ● Customize how your types are encoded and decoded
   ○ Omit or rename properties, flatten nested structs...
   ● Write your own encoder and/or decoder
   ○ MongoDB driver
   ■ BSONEncoder: Swift type → MongoDB document
   ■ BSONDecoder: MongoDB document → Swift type
   

   View Slide

8. What we'll talk about
   ● The public API
   ● Internals
   ● Limitations
   

   View Slide

9. The Public API
   

   View Slide

10. An Encodable type knows how to
    write itself to an Encoder.
    

    View Slide

11. ● Automatic conformance if all properties are Encodable
    ● Types can provide custom implementations
    ● Format agnostic: write it once, works with any Encoder!
    

    View Slide

12. A Decodable type knows how to
    initialize by reading from a Decoder.
    

    View Slide

13. ● Automatic conformance if all properties are Decodable
    ● Types can provide custom implementations
    ● Write it once, works with any Decoder
    

    View Slide

14. View Slide

15. Types With Built-In Codable Support
    ● Numeric types
    ● Bool
    ● String
    ● If the values they contain are Encodable / Decodable:
    ○ Array
    ○ Set
    ○ Dictionary
    ○ Optional
    ● Common Foundation types: URL, Data, Date, etc.
    

    View Slide

16. … and that's it!
    Making Types Codable
    

    View Slide

17. Using Encoders and Decoders
    

    View Slide

18. Using An Encoder
    JSONEncoder
    Encodable
    value of type T
    UTF-8
    encoded Data
    

    View Slide

19. Using An Encoder
    

    View Slide

20. Why doesn't the API match the Encodable protocol?
    

    View Slide

21. _JSONEncoder:
    Encoder
    JSONEncoder
    Why doesn't the API match the Encodable protocol?
    private
    ● Not dictated by any protocol
    ● Allows setting top-level options
    (e.g. DateEncodingStrategy)
    ● Simple API, just one method
    ● Implements Encoder protocol
    ● Actually does the encoding work
    ● More complex API (stay tuned)
    

    View Slide

22. Using A Decoder
    JSONDecoder
    Decodable
    value of type T
    Type to decode
    to, and UTF-8
    encoded Data
    

    View Slide

23. Using A Decoder
    Data we got
    from encoding
    

    View Slide

24. _JSONDecoder:
    Decoder
    JSONDecoder
    Why doesn't the API match the Decodable protocol?
    private
    

    View Slide

25. Ok, so… what do these
    protocols actually require?
    

    View Slide

26. Encoder
    Single
    Value
    Unkeyed Keyed
    one value sequence of values key/value pairs
    provides containers:
    views into storage
    

    View Slide

27. In code...
    SingleValueEncodingContainer
    UnkeyedEncodingContainer
    KeyedEncodingContainer
    Encoder
    

    View Slide

28. Encoding containers support storing three types of values.
    nil
    Bool, String, Double, Float
    all Int and UInt types
    Encodable type
    base case 1: nil
    base case 2:
    primitives
    recursive case
    

    View Slide

29. nil
    primitive
    Encodable
    SingleValueEncodingContainer
    

    View Slide

30. nil
    primitive
    Encodable
    SingleValueEncodingContainer
    UnkeyedEncodingContainer
    

    View Slide

31. Encodable
    primitive
    nil
    KeyedEncodingContainerProtocol
    

    View Slide

32. CodingKey
    KeyedEncodingContainer
    Encoder
    

    View Slide

33. CodingKey
    Requirements:
    ● Initializable by String
    and/or Int
    ● Must have a String
    representation
    ● May have an Int
    representation
    Synthesized conformance for enums!
    

    View Slide

34. Encoder
    Single
    Value
    Unkeyed Keyed
    Unkeyed Keyed Unkeyed Keyed
    recursive case
    

    View Slide

35. Keyed
    Unkeyed
    UnkeyedEncodingContainer
    KeyedEncodingContainerProtocol
    Unkeyed
    Keyed
    

    View Slide

36. So how are these
    containers used?
    

    View Slide

37. View Slide

38. Encoder
    KeyedEncodingContainer
    name color
    "Chester" "tan"
    

    View Slide

39. Encoder
    KeyedEncodingContainer
    name color
    "Chester" "tan"
    

    View Slide

40. Encoder
    KeyedEncodingContainer
    name color
    "Chester" "tan"
    Compiler generated!
    

    View Slide

41. But what if we don't want the defaults?
    Encoder
    KeyedEncodingContainer
    firstName color
    "Chester" "tan"
    We can override just CodingKeys to
    rename keys but keep the default
    encode(to:) implementation.
    

    View Slide

42. But what if we don't want the defaults?
    We can override just CodingKeys to
    omit keys altogether.
    Encoder
    KeyedEncodingContainer
    name
    "Chester"
    

    View Slide

43. But what if we don't want the defaults?
    Encoder
    KeyedEncodingContainer
    name color
    "chester" "orange"
    Or we can override encode(to:) to
    further customize behavior.
    

    View Slide

44. Let's make things more complicated...
    

    View Slide

45. Encoder
    KeyedContainer
    name cats
    "Kaitlin"
    Unkeyed
    KeyedContainer
    name color
    "Chester" "tan"
    KeyedContainer
    name color
    "Roscoe" "orange"
    CatOwner
    cats
    cats[1]
    cats[0]
    

    View Slide

46. KeyedContainer
    name color
    "Roscoe" "orange"
    Unkeyed
    Encoder
    name cats
    "Kaitlin"
    KeyedContainer
    KeyedContainer
    name color
    "Chester" "tan"
    

    View Slide

47. KeyedContainer
    name color
    "Roscoe" "orange"
    Unkeyed
    KeyedContainer
    name color
    "Chester" "tan"
    Encoder
    name cats
    "Kaitlin"
    KeyedContainer
    

    View Slide

48. Alternatively...
    

    View Slide

49. KeyedContainer
    name color
    "Roscoe" "orange"
    Unkeyed
    name color
    "Chester" "tan"
    Encoder
    name cats
    "Kaitlin"
    KeyedContainer
    KeyedContainer
    Again, compiler generated!
    

    View Slide

50. Weren't we also talking
    about decoding?
    

    View Slide

51. Encoding: putting values into containers
    Decoding: taking values out of containers
    

    View Slide

52. Decoder
    Single
    Value
    Unkeyed Keyed
    Unkeyed Keyed Unkeyed Keyed
    recursive case
    

    View Slide

53. In code...
    Decoder
    SingleValueDecodingContainer
    UnkeyedDecodingContainer
    KeyedDecodingContainer
    

    View Slide

54. Decoding containers support retrieving three types of values.
    nil
    Bool, String, Double, Float
    all Int and UInt types
    Decodable type
    base case 1: nil
    base case 2:
    primitives
    recursive case
    

    View Slide

55. SingleValueDecodingContainer
    nil
    primitive
    Decodable
    

    View Slide

56. nil
    primitive
    Decodable
    UnkeyedDecodingContainer
    

    View Slide

57. KeyedDecodingContainerProtocol
    nil
    primitive
    Decodable
    

    View Slide

58. Decoder
    KeyedDecodingContainer
    name color
    "Chester" "tan"
    

    View Slide

59. Decoder
    KeyedDecodingContainer
    name color
    "Chester" "tan"
    

    View Slide

60. Decoder
    KeyedDecodingContainer
    name color
    "Chester" "tan"
    Compiler generated!
    

    View Slide

61. Public API Takeaways
    ● Types opt in by conforming to Encodable and/or Decodable
    ● Many types get conformance for free, but can customize when
    needed
    ● Public types (e.g. JSONEncoder) do *not* implement the
    corresponding protocols!
    ● Encoders and Decoders used container-based APIs for
    reading from/writing to storage
    ● The Encoder and Decoder APIs are very similar!
    

    View Slide

62. Underneath the Hood
    

    View Slide

63. Typical Structure
    _MyEncoder
    : Encoder
    storage
    private
    MyEncoder
    _MyDecoder
    : Decoder
    storage
    private
    MyDecoder
    

    View Slide

64. Storage
    In what format do I store values while I am still in the middle
    of encoding/decoding?
    ● JSON: NS* types that work with JSONSerialization.
    ● PropertyList: NS* types that work with PropertyListSerialization.
    ● BSON: types conforming to BsonValue that work with MongoDB documents.
    How do I track where I am in the maze of nested containers?
    ● JSON, BSON, PropertyList: stack with whatever is backing the current
    container on top
    

    View Slide

65. 4 more small but important things to note:
    1. SingleValueXContainer is often just implemented via an
    extension of the private Encoder type.
    2. The Encoder must enforce that only one value is written to a
    SingleValueEncodingContainer.
    3. There are various bookkeeping requirements on the protocols for
    tracking the path of keys taken so far.
    4. There are also superEncoder and superDecoder requirements for
    use with classes.
    

    View Slide

66. Limitations
    ● Lots of boilerplate required for all of the containers, and
    the different encode and decode methods that they
    support
    ● Depending on design, you may end up duplicating a lot of
    code from JSONEncoder etc.
    ● Types cannot be extended to become Decodable (but
    should it be possible?)
    

    View Slide

67. In conclusion...
    ● The API makes Codable conformance trivial in many cases, but also allows
    for very advanced customization when needed.
    ● The API makes it possible to write Encoders and Decoders that don't know
    what the types using them look like, and makes it possible for types to write
    encode methods without caring about the actual output format!
    ● Investigating the standard library source code and writing your own
    encoder/decoder reveals some hidden requirements.
    

    View Slide

68. Thank you!
    Kaitlin Mahar
    Software Engineer @ MongoDB
    @k__mahar @kmahar
    

    View Slide