Swift's Encoder and Decoder Protocols

Ever wonder what’s actually going on inside JSONEncoder and JSONDecoder? This talk will be a deep dive into Swift’s Encoder and Decoder protocols. Drawing on a few different Encoders and Decoder examples, we’ll cover what the protocols require and how all the pieces fit together, as well as what design decisions are left up to the programmer and how to make those decisions.

Transcript

1. Swift's Encoder and Decoder Protocols Kaitlin Mahar Software Engineer @ MongoDB @k__mahar @kmahar
2. What is encoding? Encoder Swift type External representation etc... Scalar (e.g. Int) struct class enum
3. What is encoding? JSON Encoder
4. What is decoding? Decoder Swift type External representation etc... Scalar (e.g. Int) struct class enum
5. What is decoding? JSON Decoder
6. Swift 4 introduced a standardized approach to encoding and decoding. How does it actually work?
7. Why does it matter? • Customize how your types are encoded and decoded ◦ Omit or rename properties, flatten nested structs... • Write your own encoder and/or decoder ◦ MongoDB driver ▪ BSONEncoder: Swift type → MongoDB document ▪ BSONDecoder: MongoDB document → Swift type
8. What we'll talk about • The public API • Internals • Limitations
9. The Public API
10. An Encodable type knows how to write itself to an Encoder.
11. • Automatic conformance if all properties are Encodable • Types can provide custom implementations • Format agnostic: write it once, works with any Encoder!
12. A Decodable type knows how to initialize by reading from a Decoder.
13. • Automatic conformance if all properties are Decodable • Types can provide custom implementations • Write it once, works with any Decoder
15. Types With Built-In Codable Support • Numeric types • Bool • String • If the values they contain are Encodable / Decodable: ◦ Array ◦ Set ◦ Dictionary ◦ Optional • Common Foundation types: URL, Data, Date, etc.
16. … and that's it! Making Types Codable
17. Using Encoders and Decoders
18. Using An Encoder JSONEncoder Encodable value of type T UTF-8 encoded Data
19. Using An Encoder
20. Why doesn't the API match the Encodable protocol?
21. _JSONEncoder: Encoder JSONEncoder Why doesn't the API match the Encodable protocol? private • Not dictated by any protocol • Allows setting top-level options (e.g. DateEncodingStrategy) • Simple API, just one method • Implements Encoder protocol • Actually does the encoding work • More complex API (stay tuned)
22. Using A Decoder JSONDecoder Decodable value of type T Type to decode to, and UTF-8 encoded Data
23. Using A Decoder Data we got from encoding
24. _JSONDecoder: Decoder JSONDecoder Why doesn't the API match the Decodable protocol? private
25. Ok, so… what do these protocols actually require?
26. Encoder Single Value Unkeyed Keyed one value sequence of values key/value pairs provides containers: views into storage
27. In code... SingleValueEncodingContainer UnkeyedEncodingContainer KeyedEncodingContainer<Key> Encoder
28. Encoding containers support storing three types of values. nil Bool, String, Double, Float all Int and UInt types Encodable type base case 1: nil base case 2: primitives recursive case
29. nil primitive Encodable SingleValueEncodingContainer
30. nil primitive Encodable SingleValueEncodingContainer UnkeyedEncodingContainer
31. Encodable primitive nil KeyedEncodingContainerProtocol
32. CodingKey KeyedEncodingContainer<Key> Encoder
33. CodingKey Requirements: • Initializable by String and/or Int • Must have a String representation • May have an Int representation Synthesized conformance for enums!
34. Encoder Single Value Unkeyed Keyed Unkeyed Keyed Unkeyed Keyed recursive case
35. Keyed Unkeyed UnkeyedEncodingContainer KeyedEncodingContainerProtocol Unkeyed Keyed
36. So how are these containers used?
38. Encoder KeyedEncodingContainer name color "Chester" "tan"
39. Encoder KeyedEncodingContainer name color "Chester" "tan"
40. Encoder KeyedEncodingContainer name color "Chester" "tan" Compiler generated!
41. But what if we don't want the defaults? Encoder KeyedEncodingContainer firstName color "Chester" "tan" We can override just CodingKeys to rename keys but keep the default encode(to:) implementation.
42. But what if we don't want the defaults? We can override just CodingKeys to omit keys altogether. Encoder KeyedEncodingContainer name "Chester"
43. But what if we don't want the defaults? Encoder KeyedEncodingContainer name color "chester" "orange" Or we can override encode(to:) to further customize behavior.
44. Let's make things more complicated...
45. Encoder KeyedContainer name cats "Kaitlin" Unkeyed KeyedContainer name color "Chester" "tan" KeyedContainer name color "Roscoe" "orange" CatOwner cats cats[1] cats[0]
46. KeyedContainer name color "Roscoe" "orange" Unkeyed Encoder name cats "Kaitlin" KeyedContainer KeyedContainer name color "Chester" "tan"
47. KeyedContainer name color "Roscoe" "orange" Unkeyed KeyedContainer name color "Chester" "tan" Encoder name cats "Kaitlin" KeyedContainer
48. Alternatively...
49. KeyedContainer name color "Roscoe" "orange" Unkeyed name color "Chester" "tan" Encoder name cats "Kaitlin" KeyedContainer KeyedContainer Again, compiler generated!
50. Weren't we also talking about decoding?
51. Encoding: putting values into containers Decoding: taking values out of containers
52. Decoder Single Value Unkeyed Keyed Unkeyed Keyed Unkeyed Keyed recursive case
53. In code... Decoder SingleValueDecodingContainer UnkeyedDecodingContainer KeyedDecodingContainer<Key>
54. Decoding containers support retrieving three types of values. nil Bool, String, Double, Float all Int and UInt types Decodable type base case 1: nil base case 2: primitives recursive case
55. SingleValueDecodingContainer nil primitive Decodable
56. nil primitive Decodable UnkeyedDecodingContainer
57. KeyedDecodingContainerProtocol nil primitive Decodable
58. Decoder KeyedDecodingContainer name color "Chester" "tan"
59. Decoder KeyedDecodingContainer name color "Chester" "tan"
60. Decoder KeyedDecodingContainer name color "Chester" "tan" Compiler generated!
61. Public API Takeaways • Types opt in by conforming to Encodable and/or Decodable • Many types get conformance for free, but can customize when needed • Public types (e.g. JSONEncoder) do *not* implement the corresponding protocols! • Encoders and Decoders used container-based APIs for reading from/writing to storage • The Encoder and Decoder APIs are very similar!
62. Underneath the Hood
63. Typical Structure _MyEncoder : Encoder storage private MyEncoder _MyDecoder : Decoder storage private MyDecoder
64. Storage In what format do I store values while I am still in the middle of encoding/decoding? • JSON: NS* types that work with JSONSerialization. • PropertyList: NS* types that work with PropertyListSerialization. • BSON: types conforming to BsonValue that work with MongoDB documents. How do I track where I am in the maze of nested containers? • JSON, BSON, PropertyList: stack with whatever is backing the current container on top
65. 4 more small but important things to note: 1. SingleValueXContainer is often just implemented via an extension of the private Encoder type. 2. The Encoder must enforce that only one value is written to a SingleValueEncodingContainer. 3. There are various bookkeeping requirements on the protocols for tracking the path of keys taken so far. 4. There are also superEncoder and superDecoder requirements for use with classes.
66. Limitations • Lots of boilerplate required for all of the containers, and the different encode and decode methods that they support • Depending on design, you may end up duplicating a lot of code from JSONEncoder etc. • Types cannot be extended to become Decodable (but should it be possible?)
67. In conclusion... • The API makes Codable conformance trivial in many cases, but also allows for very advanced customization when needed. • The API makes it possible to write Encoders and Decoders that don't know what the types using them look like, and makes it possible for types to write encode methods without caring about the actual output format! • Investigating the standard library source code and writing your own encoder/decoder reveals some hidden requirements.
68. Thank you! Kaitlin Mahar Software Engineer @ MongoDB @k__mahar @kmahar