JSON Explained (Chicago Roboto 2019)

Video: https://vimeo.com/341115830
Code: https://github.com/swankjesse/jsonexplained

Java and Kotlin coders: note this talk. Its tran-
script describes some tools to encode any
object. I’ll review the libraries including an an-
notation that can make your app faster.

In this talk we’ll:

🦕 Be warned about the format’s gotchas
🦕 Watch a JSON denial of service attack
🦕 Compare streaming vs. trees vs. databinding
🦕 Appreciate optimizations in Jackson, Gson, and Moshi
🦕 Study the bugs in these same libraries

The talk’s key value will be showing how JSON libraries work.

Transcript

1. @jessewilson https://github.com/swankjesse/jsonexplained JSON Explained
2. JavaScript Object Notation
3. ============================================================================ Copyright (c) 2002 JSON.org Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. The Software shall be used for Good, not Evil.
4. The Software shall be used for Good, not Evil.
5. Open Source Definition
6. JSON • Specified by RFC 8259 • Four primitive types: booleans, numbers, strings, null • Two container types: arrays, objects
7. Booleans • Delightfully boring: just true, false • On iOS boolean is a NSNumber and they’d get 0 instead of false and 1 instead of true
8. Numbers • IEEE 754 double-precision binary floating point • Not boring…
9. Floating Point Refresher! • Float (32-bit) & Double (64-bit) • Specified by IEEE 754 • Supported by Kotlin, Java, and almost all programming languages • Implemented in CPUs and GPUs
10. Natural Numbers Integers Rational Numbers Real Numbers 1, 2, 3 -3, -2, -1, 0 -22/7, 51/100 √2, , 0 1 2 3 4 -4 -3 -2 -1 51/100 -22/7 √2 Math Class Numbers
11. • Byte: Subset of _Z covering -128..127 • Short: Subset of _Z covering -32,768..32,767 • Int: Subset of _Z covering -2,147,483,648..2,147,483,647 • Long: Subset of _Z covering
 -9,223,372,036,854,775,808..9,223,372,036,854,775,807 Computer Numbers
12. Computer Numbers • Float: Subset of , plus special values
 
 3.1415927 • Double: Larger subset of , plus the same special values
 
 3.141592653589793
13. Special Values println( 1.0 / 0.0 ) // Infinity println(-1.0 / 0.0 ) // -Infinity println( 0.0 / 1.0 ) // 0.0 println( 0.0 / -1.0 ) // -0.0 println( 0.0 / 0.0 ) // NaN
14. Infinity is Weird // Double.parseDouble() doesn't demand digits! val pos: Double = "Infinity".toDouble() println(pos) // Infinity val neg: Double = "-Infinity".toDouble() println(neg) // -Infinity
15. Negative Zero is Weird // Operators don't differentiate! println(-0.0 == 0.0) // true println(-0.0 < 0.0) // false println(-0.0 > 0.0) // false // But methods do. println((-0.0).equals(0.0)) // false println((-0.0).compareTo(0.0) == 0) // false
16. NaN is Weird // Operators are not reflexive! println(nan == nan) // false println(nan <= nan) // false println(nan >= nan) // false // But methods are. println(nan.compareTo(nan) == 0) // true println(nan.equals(nan)) // true // Parse "NaN" with or without a negative sign. println("NaN".toDouble()) // NaN println("-NaN".toDouble()) // NaN
17. Subset of Q • Every floating point value is an integer divided by a power of two 8.875 = 71/8 0.1 = 3,602,879,701,896,397/36,028,797,018,963,968 -127.5 = -255/2
18. Double Precision Floating Point • 1-bit sign • 11-bit exponent of denominator • 53-bit numerator (highest bit implied) 0 1 2 3 4 -4 -3 -2 -1
19. Doubles • Distance between values: • Between 1.0 and 2.0: 1/4,503,599,627,370,496 • Between 2.0 and 4.0: 1/2,251,799,813,685,248 • Between 4.0 and 8.0: 1/1,125,899,906,842,623
20. Doubles • Distance between values: • Between 2,251,799,813,685,248 and 4,503,599,627,370,496: 1/2 • Between 4,503,599,627,370,496 and 9,007,199,254,740,992: 1 • Between 9,007,199,254,740,992 and 18,014,398,509,481,984: 2
21. Mind the Gap val x = 9_007_199_254_740_992.0 println(x - 2.0 - x) // -2.0 println(x - 1.0 - x) // -1.0 println(x + 0.0 - x) // 0.0 println(x + 1.0 - x) // 0.0 wat. println(x + 2.0 - x) // 2.0
22. Mind the Gap val x = 9_007_199_254_740_992.0 println(x - 2.0 - x) // -2.0 println(x - 1.0 - x) // -1.0 println(x + 0.0 - x) // 0.0 println(x + 1.0 - x) // 0.0 wat. println(x + 2.0 - x) // 2.0
23. Mind the Gap val x = 9_007_199_254_740_992.0 println(x - 2.0 - x) // -2.0 println(x - 1.0 - x) // -1.0 println(x + 0.0 - x) // 0.0 println(x + 1.0 - x) // 0.0 wat. println(x + 2.0 - x) // 2.0
24. Beware of Decimal Approximation println(0.1 + 0.2); // 0.30000000000000004
25. Infinite Loop! • This value caused every JVM to enter an infinite loop:
 
 Double.parseDouble("2.2250738585072012e-308"); • Fixed very quickly once discovered in 2011
26. Double? Trouble. • Not like the numbers in math class! • Negative Zero, Infinity, NaN • Gaps! Like 9,007,199,254,740,993 = 253 + 1 • Decimal representation is approximate
27. JSON Numbers • JSON spec, RFC 8259 • “expect no more precision or range” than double precision • (-253 + 1) .. (253 - 1) is the safe range for integers • No Infinities or NaN, but negative zero is okay!
28. Is 253 Enough? • 253 bytes = 9 petabytes • 253 milliseconds = 285,427 years • 253 meters = 0.95 light-years
29. For Interop, Consider Double’s Limitations • APIs attempt to retain types and precision • But this won’t round-trip through systems that lack types • Relying in it is asking for trouble! • Don’t use Longs for IDs in JSON
30. Strings • UTF-8 encoded • Quote-delimited • Escape sequences for quotes \", slashes \\, newlines \n, etc.
31. Null • Should be boring?
32. Null vs. Absent? [ { "name": "Jupiter", "radius_km": 69911, "discovered_by": "Galileo Galilei" }, { "name": "Earth", "radius_km": 6371, "discovered_by": null }, { "name": "Mars", "radius_km": 3389 } ]
33. Arrays • Square brackets enclosed • Comma-separated values
34. Null vs. Absent vs. Empty [ { "name": "Mars", "inhabited_by": ["Sojourner", "Spirit", "Opportunity", "Curiosity"] }, { "name": "Jupiter", “inhabited_by": null }, { "name": "Earth" }, { "name": "Saturn", "inhabited_by": [] } ]
35. Objects • Curly braces enclosed • Comma-separated name/value pairs
36. { "j": "Jupiter", "v": "Venus", "n": "Neptune", "s": "Saturn" }
37. fun <K> hashBucket(key: K): Int { return key.hashCode() % buckets.size }
38. 0 1 2 3 4 5 6 7
39. 0 1 2 3 4 5 6 7 "j" "Jupiter" hashBucket("j") // 2
40. 0 1 2 3 4 5 6 7 "j" "Jupiter" "v" "Venus" hashBucket("v") // 6
41. 0 1 2 3 4 5 6 7 "j" "Jupiter" "v" "Venus" "n" "Neptune" hashBucket("n") // 6
42. 0 1 2 3 4 5 6 7 "j" "Jupiter" "s" "Saturn" "v" "Venus" "n" "Neptune" hashBucket("s") // 3
43. { "abcd": 0, "abdE": 0, "acDd": 0, "acEE": 0, "ad%d": 0, "bCcd": 0, "bCdE": 0, "bDDd": 0 }
44. 0 1 2 3 4 5 6 7
45. 0 1 2 3 4 5 6 7 hashBucket("abcd") // 7
46. 0 1 2 3 4 5 6 7 hashBucket("abdE") // 7
47. 0 1 2 3 4 5 6 7 hashBucket("acDd") // 7
48. 0 1 2 3 4 5 6 7 hashBucket("acEE") // 7
49. 0 1 2 3 4 5 6 7 hashBucket(“ad%d") // 7
50. 0 1 2 3 4 5 6 7 hashBucket(“bCcd") // 7
51. 0 1 2 3 4 5 6 7 hashBucket(“bCdE") // 7
52. 0 1 2 3 4 5 6 7 hashBucket(“bDDd") // 7
53. Hash DoS Impact • 1 MiB: saturate a CPU core for 1 minute • 2 MiB: saturate a CPU core for 4 minutes
54. Hash DoS Fixes • Fixed in Gson in 2012 • Fixed in OpenJDK 8 in 2013 • Fixed in Android Oreo in 2017 • Other platforms are still vulnerable
55. • A name could be repeated due to mistake or attack • Keep first? • Keep last? • Fail? Be Careful of Repeated Names { "alg": "HS256", "typ": "JWT", "alg": "none" }
56. Dates? • RFC 3339 is a good choice { "speaker": "Jesse Wilson", "title": "JSON Explained", "start_time": "2019-04-26T11:00:00-05:00" }
57. Binary? • Base64 is good inline • URLs are good too { "speaker": "Jesse Wilson", "title": "JSON Explained", "sha256": "J84t8GhVF5utZqMW8Nm5M/68Edc0+B7okSGIWnqgwbc=", "slides": "https://github.com/swankjesse/json/slides.pdf" }
58. Libraries
59. Jackson • Every feature you’ll ever want • Fast • Java library, Kotlin via a support module
60. Gson • Lots of features • Fast • Java library, Kotlin doesn’t really work
61. Moshi • Small • Fast • Java library, Kotlin via a support module
62. Kotlin Serialization • Small • No Streaming • Kotlin Multiplatform! But no Java
63. Streams
64. • Streaming starts decoding before the entire JSON is downloaded • Target either UTF-8 bytes or Java chars { " c o l o r " : " r e d " , " r a d i u s _ k m " : 3 3 3 9 } jsonBytes: ByteArray
65. fun readPlanet(reader: JsonReader): Planet { var color: String? = null var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { val name = reader.nextName() when (name) { "color" -> color = reader.nextString() "radius_km" -> radiusKm = reader.nextLong() else -> reader.skipValue() } } reader.endObject() return Planet(color, radiusKm) } Decode an Object
66. fun readPlanet(reader: JsonReader): Planet { var color: String? = null var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { val name = reader.nextName() when (name) { "color" -> color = reader.nextString() "radius_km" -> radiusKm = reader.nextLong() else -> reader.skipValue() } } reader.endObject() return Planet(color, radiusKm) } Decode an Object
67. fun readPlanet(reader: JsonReader): Planet { var color: String? = null var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { val name = reader.nextName() when (name) { "color" -> color = reader.nextString() "radius_km" -> radiusKm = reader.nextLong() else -> reader.skipValue() } } reader.endObject() return Planet(color, radiusKm) } Decode an Object
68. fun readPlanet(reader: JsonReader): Planet { var color: String? = null var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { val name = reader.nextName() when (name) { "color" -> color = reader.nextString() "radius_km" -> radiusKm = reader.nextLong() else -> reader.skipValue() } } reader.endObject() return Planet(color, radiusKm) } Decode an Object
69. fun readPlanet(reader: JsonReader): Planet { var color: String? = null var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { val name = reader.nextName() when (name) { "color" -> color = reader.nextString() "radius_km" -> radiusKm = reader.nextLong() else -> reader.skipValue() } } reader.endObject() return Planet(color, radiusKm) } Decode an Object
70. fun readPlanet(reader: JsonReader): Planet { var color: String? = null var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { val name = reader.nextName() when (name) { "color" -> color = reader.nextString() "radius_km" -> radiusKm = reader.nextLong() else -> reader.skipValue() } } reader.endObject() return Planet(color, radiusKm) } fun readPlanet(reader: JsonReader): Planet { var color: String? = null var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { val name = reader.nextName() when (name) { "color" -> color = reader.nextString() "radius_km" -> radiusKm = reader.nextLong() else -> reader.skipValue() } } reader.endObject() return Planet(color, radiusKm) } Decode an Object
71. { " c o l o r " : " r e d " , " r a d i u s _ k m " : 3 3 3 9 } begin 2 end 7 val name = String(jsonChars, begin, end) new String() jsonBytes: ByteArray
72. 0 ×××× ×××× ×××× 0 1 ×××× ×××× ×××× 0 2 ×××× ×××× ×××× 0 3 ×××× ×××× ×××× 0 4 ×××× ×××× ×××× 0 5 ×××× ×××× ×××× 0 6 ×××× ×××× ×××× 0 7 ×××× ×××× ×××× 0 0 1 2 3 4 5 6 7 Jackson’s ByteQuadsCanonicalizer
73. 0 ×××× ×××× ×××× 0 1 ×××× ×××× ×××× 0 2 ×××× ×××× ×××× 0 3 ×××× ×××× ×××× 0 4 ×××× ×××× ×××× 0 5 ×××× ×××× ×××× 0 6 colo r××× ×××× 5 7 ×××× ×××× ×××× 0 0 1 2 3 4 5 6 "color" 7 Jackson’s ByteQuadsCanonicalizer
74. 0 ×××× ×××× ×××× 0 1 ×××× ×××× ×××× 0 2 ×××× ×××× ×××× 0 3 ×××× ×××× ×××× 0 4 radi us_k m××× 9 5 ×××× ×××× ×××× 0 6 colo r××× ×××× 5 7 ×××× ×××× ×××× 0 0 1 2 3 4 "radius_km" 5 6 "color" 7 Jackson’s ByteQuadsCanonicalizer
75. Moshi’s Options Trie "path" "primary_artist" "header_image_url" "annotation_count" "header_image_thumbnail_url" "api_path" "url"
76. a… h… p… u… …notation_count 3 …n… …p… …a… …r… …i_path 4 …eader_image_… …t… …u… …humbnail_url 5 …rl 2 …imary_artist 1 …th 0 …rl 6
77. val FIELD_NAMES = JsonReader.Options.of("color", "radius_km") fun readPlanet(reader: JsonReader): Planet { var color: String? = null var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { when (reader.selectName(FIELD_NAMES)) { 0 -> color = reader.nextString() 1 -> radiusKm = reader.nextLong() else -> { reader.skipName() reader.skipValue() } } } reader.endObject() return Planet(color, radiusKm) }
78. val FIELD_NAMES = JsonReader.Options.of("color", "radius_km") fun readPlanet(reader: JsonReader): Planet { var color: String? = null var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { when (reader.selectName(FIELD_NAMES)) { 0 -> color = reader.nextString() 1 -> radiusKm = reader.nextLong() else -> { reader.skipName() reader.skipValue() } } } reader.endObject() return Planet(color, radiusKm) }
79. val FIELD_NAMES = JsonReader.Options.of("color", "radius_km") fun readPlanet(reader: JsonReader): Planet { var color: String? = null var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { when (reader.selectName(FIELD_NAMES)) { 0 -> color = reader.nextString() 1 -> radiusKm = reader.nextLong() else -> { reader.skipName() reader.skipValue() } } } reader.endObject() return Planet(color, radiusKm) }
80. val FIELD_NAMES = JsonReader.Options.of("color", "radius_km") fun readPlanet(reader: JsonReader): Planet { var color: String? = null var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { when (reader.selectName(FIELD_NAMES)) { 0 -> color = reader.nextString() 1 -> radiusKm = reader.nextLong() else -> { reader.skipName() reader.skipValue() } } } reader.endObject() return Planet(color, radiusKm) }
81. val FIELD_NAMES = JsonReader.Options.of("color", "radius_km") fun readPlanet(reader: JsonReader): Planet { var color: String? = null var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { when (reader.selectName(FIELD_NAMES)) { 0 -> color = reader.nextString() 1 -> radiusKm = reader.nextLong() else -> { reader.skipName() reader.skipValue() } } } reader.endObject() return Planet(color, radiusKm) }
82. Trees
83. Tree Models Jackson JsonNode, NullNode, MissingNode, POJONode Gson JsonElement, JsonPrimitive, JsonNull Moshi List, Map, String Kotlin Serialization JsonElement, JsonLiteral, JsonNull
84. Working with Trees • Lots of strings and casting • Two kinds of nulls • Makes me want types
85. Data Binding
86. Type Mapping JSON Kotlin / Java Boolean Boolean Number Byte, Short, Int, Long, Float, Double String String, Char, ByteString, Enums, Instant… Array Array, List, Set… Object Classes, Map
87. Adapters • Convert between Kotlin values and JSON values • Example: HttpUrl
88. Cycles class Planet( val name: String, val moons: List<Moon> ) class Moon( val name: String, val orbits: Planet ) { "name": "Saturn", "moons": [ { "name": "Titan", "orbits": { "name": "Saturn", "moons": [ { "name": "Titan", "orbits": { "name": "Saturn", "moons": [ { "name": "Titan", ...
89. Cycles { "name": "Saturn", "moons": [ { "name": "Titan", "orbits": { "name": "Saturn", "moons": [ { "name": "Titan", "orbits": { "name": "Saturn", "moons": [ { "name": "Titan", ... class Planet( val name: String, val moons: List<String> ) class Moon( val name: String, val orbits: String )
90. Cycles class Planet( val name: String, val moons: List<String> ) class Moon( val name: String, val orbits: String ) { "name": "Saturn", "moons": [ "Titan", "Calypso", "Tethys", "Rhea", "Dione", "Enceladus", "Mimas" ] }
91. Subclasses sealed class CosmicObject { data class Star( val name: String, val radius_km: Long ) : CosmicObject() data class Planet( val name: String, val orbits_star: String ) : CosmicObject() data class Moon( val name: String, val orbits_planet: String ) : CosmicObject() } { "destination": { "name": "Alpha Centauri", "radius_km": 851120 }, "origin": { "name": "Saturn", "oribits_star": "Sun" } }
92. Subclasses val moshi = Moshi.Builder() .add(PolymorphicJsonAdapterFactory.of(CosmicObject::class.java, "type") .withSubtype(CosmicObject.Star::class.java, "star") .withSubtype(CosmicObject.Planet::class.java, "planet") .withSubtype(CosmicObject.Moon::class.java, "moon")) .build()
93. Subclasses sealed class CosmicObject { data class Star( val name: String, val radius_km: Long ) : CosmicObject() data class Planet( val name: String, val orbits_star: String ) : CosmicObject() data class Moon( val name: String, val orbits_planet: String ) : CosmicObject() } { "destination": { "name": "Alpha Centauri", "radius_km": 851120 }, "origin": { "name": "Saturn", "oribits_star": "Sun" } }
94. Subclasses { "destination": { "type": "star", "name": "Alpha Centauri", "radius_km": 851120 }, "origin": { "type": "planet", "name": "Saturn", "oribits_star": "Sun" } } sealed class CosmicObject { data class Star( val name: String, val radius_km: Long ) : CosmicObject() data class Planet( val name: String, val orbits_star: String ) : CosmicObject() data class Moon( val name: String, val orbits_planet: String ) : CosmicObject() }
95. class Planet { final String name; final double radius_km; String color = "unknown"; List<String> moons = emptyList(); Planet( String name, double radiusKm) { this.name = name; this.radius_km = radiusKm; } ... } Reflection Magic val json = """ |{ | "name": "Mars", | "radius_km": 3389 |} """.trimMargin() val planet = adapter.fromJson(json)!! println(planet.name) // Mars println(planet.radius_km) // 3389.0 println(planet.color) // null println(planet.moons) // null
96. Reflection Magic val json = """ |{ | "name": "Mars", | "radius_km": 3389 |} """.trimMargin() val planet = adapter.fromJson(json)!! println(planet.name) // Mars println(planet.radius_km) // 3389.0 println(planet.color) // null println(planet.moons) // null @JsonClass(generateAdapter = true) class Planet( val name: String, val radius_km: Double ) { val color = "unknown" val moons = emptyList<String>() }
97. Reflection Magic val json = """ |{ | "name": "Mars", | "radius_km": 3389 |} """.trimMargin() val planet = adapter.fromJson(json)!! println(planet.name) // Mars println(planet.radius_km) // 3389.0 println(planet.color) // unknown println(planet.moons) // [] @JsonClass(generateAdapter = true) class Planet( val name: String, val radius_km: Double ) { val color = "unknown" val moons = emptyList<String>() }
98. Built-in Types data class Planet( val name: String, val surface_color: Color ) { "name" : "Saturn", "surface_color" : { "value" : -9677, "falpha" : 0 } }
99. Case Mapping val gson = GsonBuilder() .setFieldNamingPolicy(FieldNamingPolicy.LOWER_CASE_WITH_UNDERSCORES) .create()
100. I’m deleting old code. Is orbitsStar still used?
101. I’m deleting old code. Is orbitsStar still used? grep says no
102. Case Mapping val gson = GsonBuilder() .setFieldNamingPolicy(FieldNamingPolicy.LOWER_CASE_WITH_UNDERSCORES) .create()
103. Case Mapping data class Planet( val name: String, val orbitsStar: String, val radiusKm: Long, val moons: List<String> ) { "name": "Saturn", "oribits_star": "Sun", "radius_km": 58232, "moons": [ "Titan", "Calypso" ] }
104. Case Mapping data class Planet( val name: String, @SerializedName("orbits_star") val orbitsStar: String, @SerializedName("radius_km") val radiusKm: Long, val moons: List<String> ) { "name": "Saturn", "oribits_star": "Sun", "radius_km": 58232, "moons": [ "Titan", "Calypso" ] }
105. Case Mapping data class Planet( val name: String, val orbits_star: String, val radius_km: Long, val moons: List<String> ) { "name": "Saturn", "oribits_star": "Sun", "radius_km": 58232, "moons": [ "Titan", "Calypso" ] }
106. Generating Code Jackson Runtime bytecode generator
 (No Android support) Gson None Moshi Annotation Processor Kotlin Serialization Kotlin Compiler Plugin
107. Gson Jackson Moshi Kotlin S. ? ? ?
108. Jackson 2.9.8 (2018-12-15) 21 feature releases Gson 2.8.5 (2018-05-21) 17 feature releases Moshi 1.8.0 (2018-11-09) 9 feature releases Kotlin Serialization 0.11.0 (2019-04-12) pre-release Latest Release
109. Jackson 4,370 KiB includes kotlin-reflect (2,584 KiB) Gson 239 KiB Moshi 330 KiB + generated code includes Okio (178 KiB) Kotlin Serialization 762 KiB + generated code Size Before Shrinking
110. Jackson Fast stream encoder + decoder Allocation avoidance pool Gson Fast stream encoder + decoder Moshi Fast stream encoder + decoder Select trie Generated Kotlin adapters Kotlin Serialization Generated Kotlin adapters Performance Features
111. APIs
112. var mapper = jacksonObjectMapper().apply { disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES) } @Test fun decodeJson() { val inputStream: InputStream = ... val searchResponseBody = inputStream.use { mapper.readValue<SearchResponseBody>(it) } ... } Jackson
113. val gson = GsonBuilder().create() @Test fun decodeJson() { val reader: Reader = ... val searchResponseBody = reader.use { gson.fromJson(it, SearchResponseBody::class.java) } ... } Gson
114. val moshi = Moshi.Builder().build() val adapter = moshi.adapter(SearchResponseBody::class.java) @Test fun decodeJson() { val source: BufferedSource = ... val searchResponseBody = source.use { adapter.fromJson(it)!! } ... } Moshi
115. val json = Json(JsonConfiguration(strictMode = false)) @Test fun decodeJson() { val string: String = ... val searchResponseBody = json.parse( SearchResponseBody.serializer(), string) ... } Kotlin Serialization
116. Custom Adapters
117. object HttpUrlSerializer : StdSerializer<HttpUrl>(HttpUrl::class.java) { override fun serialize( value: HttpUrl, generator: JsonGenerator, provider: SerializerProvider ) { generator.writeString(value.toString()) } } object HttpUrlDeserializer : StdDeserializer<HttpUrl>(HttpUrl::class.java) { override fun deserialize( parser: JsonParser, context: DeserializationContext? ): HttpUrl { val text = parser.getValueAsString() return HttpUrl.get(text) } } var mapper = jacksonObjectMapper().apply { registerModule(SimpleModule().apply { addSerializer(HttpUrl::class.java, HttpUrlSerializer) addDeserializer(HttpUrl::class.java, HttpUrlDeserializer) }) } Jackson
118. object HttpUrlTypeAdapter : TypeAdapter<HttpUrl>() { override fun write(writer: JsonWriter, value: HttpUrl) { writer.value(value.toString()) } override fun read(reader: JsonReader): HttpUrl { val string = reader.nextString() return HttpUrl.get(string) } } val gson = GsonBuilder() .registerTypeAdapter(HttpUrl::class.java, HttpUrlTypeAdapter) .create() Gson
119. object CustomAdapters { @ToJson fun httpUrlToJson(httpUrl: HttpUrl): String { return httpUrl.toString() } @FromJson fun httpUrlFromJson(string: String): HttpUrl { return HttpUrl.get(string) } } val moshi = Moshi.Builder() .add(CustomAdapters) .build() Moshi
120. @Serializer(forClass = HttpUrl::class) object HttpUrlSerializer : KSerializer<HttpUrl> { override fun serialize(encoder: Encoder, httpUrl: HttpUrl) { encoder.encodeString(httpUrl.toString()) } override fun deserialize(decoder: Decoder): HttpUrl { val string = decoder.decodeString() return HttpUrl.get(string) } } @file:UseSerializers( serializerClasses = [HttpUrlSerializer::class]
 ) Kotlin Serialization
121. Usability
122. Jackson "Barnes & Noble" Gson "Barnes \u0026 Noble" Moshi "Barnes & Noble" Kotlin Serialization "Barnes & Noble" Special Characters
123. Jackson "Barnes & Noble" Gson "Barnes \u0026 Noble" Moshi "Barnes & Noble" Kotlin Serialization "Barnes & Noble" Special Characters
124. Jackson com.fasterxml.jackson.databind.exc.InvalidFormatException: Cannot deserialize value of type `long` from String "whoops": not a valid Long value at [Source: (BufferedInputStream); line: 302, column: 18] (through reference chain: com.jsonexplained.SearchResponseBody["response"]
 ->com.jsonexplained.SearchResponse["hits"]
 ->java.util.ArrayList[8]
 ->com.jsonexplained.Hit["result"]
 ->com.jsonexplained.SongResult["id"]) Gson com.google.gson.JsonSyntaxException: java.lang.NumberFormatException: For input string: "whoops" Moshi com.squareup.moshi.JsonDataException: Expected a long but was whoops at path $.response.hits[8].result.id Kotlin Serialization java.lang.NumberFormatException: For input string: "whoops" Type Error
125. Jackson com.fasterxml.jackson.databind.exc.InvalidFormatException: Cannot deserialize value of type `long` from String "whoops": not a valid Long value at [Source: (BufferedInputStream); line: 302, column: 18] (through reference chain: com.jsonexplained.SearchResponseBody["response"]
 ->com.jsonexplained.SearchResponse["hits"]
 ->java.util.ArrayList[8]
 ->com.jsonexplained.Hit["result"]
 ->com.jsonexplained.SongResult["id"]) Gson com.google.gson.JsonSyntaxException: java.lang.NumberFormatException: For input string: "whoops" Moshi com.squareup.moshi.JsonDataException: Expected a long but was whoops at path $.response.hits[8].result.id Kotlin Serialization java.lang.NumberFormatException: For input string: "whoops" Type Error
126. Jackson Success Gson Success Moshi Success Kotlin Serialization java.lang.NumberFormatException: For input string: "1.0" Type Acceptance
127. Jackson Success Gson Success Moshi Success Kotlin Serialization java.lang.NumberFormatException: For input string: "1.0" Type Acceptance
128. Jackson Long: completed normally, used 0
 String: crashed with a helpful message Gson Long: completed normally, used 0
 String: completed normally, used null Moshi com.squareup.moshi.JsonDataException: Required property 'id' missing at $.response.hits[8].result Kotlin Serialization kotlinx.serialization.MissingFieldException: Field 'id' is required, but it was missing Required Field Is Absent
129. Jackson Long: completed normally, used 0
 String: crashed with a helpful message Gson Long: completed normally, used 0
 String: completed normally, used null Moshi com.squareup.moshi.JsonDataException: Required property 'id' missing at $.response.hits[8].result Kotlin Serialization kotlinx.serialization.MissingFieldException: Field 'id' is required, but it was missing Required Field Is Absent
130. Jackson com.fasterxml.jackson.databind.exc.MismatchedInputException: No content to map due to end-of-input at [Source: (BufferedInputStream); line: 1, column: 0] Gson Completed normally, returned null Moshi java.io.EOFException: End of input Kotlin Serialization kotlinx.serialization.json.JsonParsingException: Invalid JSON at 0: Expected '{, kind: kotlinx.serialization.StructureKind$CLASS@221af3c0' Document is Empty
131. Jackson com.fasterxml.jackson.databind.exc.MismatchedInputException: No content to map due to end-of-input at [Source: (BufferedInputStream); line: 1, column: 0] Gson Completed normally, returned null Moshi java.io.EOFException: End of input Kotlin Serialization kotlinx.serialization.json.JsonParsingException: Invalid JSON at 0: Expected '{, kind: kotlinx.serialization.StructureKind$CLASS@221af3c0' Document is Empty
132. Jackson Completes normally, encoding implementation details of types like CompletableFuture Gson Completes normally, encoding implementation details of types like CompletableFuture Moshi Fails fast with an error Kotlin Serialization Fails fast with an error Write an Unexpected Type
133. Jackson Completes normally, encoding implementation details of types like CompletableFuture Gson Completes normally, encoding implementation details of types like CompletableFuture Moshi Fails fast with an error Kotlin Serialization Fails fast with an error Write an Unexpected Type
134. Jackson com.fasterxml.jackson.databind.JsonMappingException: Numeric value (2147483648) out of range of int at [Source: (BufferedInputStream); line: 15, column: 32] (through reference chain: com.jsonexplained.SearchResponseBody["response"]
 ->com.jsonexplained.SearchResponse["hits"]
 ->java.util.ArrayList[0]
 ->com.jsonexplained.Hit["result"]
 ->com.jsonexplained.SongResult["annotation_count"]) Gson com.google.gson.JsonSyntaxException: java.lang.NumberFormatException: Expected an int but was 2147483648 at line 15 column 42 path $.response.hits[0].result.annotation_count Moshi com.squareup.moshi.JsonDataException: Expected an int but was 2147483648 at path $.response.hits[0].result.annotation_count Kotlin Serialization java.lang.NumberFormatException: For input string: "2147483648" Too Large For An Int
135. Jackson com.fasterxml.jackson.databind.JsonMappingException: Numeric value (2147483648) out of range of int at [Source: (BufferedInputStream); line: 15, column: 32] (through reference chain: com.jsonexplained.SearchResponseBody["response"]
 ->com.jsonexplained.SearchResponse["hits"]
 ->java.util.ArrayList[0]
 ->com.jsonexplained.Hit["result"]
 ->com.jsonexplained.SongResult["annotation_count"]) Gson com.google.gson.JsonSyntaxException: java.lang.NumberFormatException: Expected an int but was 2147483648 at line 15 column 42 path $.response.hits[0].result.annotation_count Moshi com.squareup.moshi.JsonDataException: Expected an int but was 2147483648 at path $.response.hits[0].result.annotation_count Kotlin Serialization java.lang.NumberFormatException: For input string: "2147483648" Too Large For An Int
136. Gson MOST USED Jackson MOST CAPABLE Kotlin S. BEST FOR MULTI
 PLATFORM Moshi BEST API
137. Testing
138. Testing? • Confirm you’ve configured things properly • Give you confidence to refactor & make changes • Get your server team to code review!
139. @Test fun json() { val planet = Planet("Mercury", 2439, listOf("Messenger")) val json = """ |{ | "name": "Mercury", | "radius_km": 2439, | "visitors": [ | "Messenger" | ] |} """.trimMargin() val adapter = moshi.adapter(Planet::class.java).indent(" ") assertThat(adapter.toJson(planet)).isEqualTo(json) assertThat(adapter.fromJson(json)).isEqualTo(planet) }
140. Optimizing
141. Use Less JSON • No pretty-printed JSON • Delete unused fields: fewer bytes to send, store, encode, decode • Omit nulls
142. Avoid Reflection • Enable generated code • Don’t create a new JsonAdapter every time
143. Stream! • Download & decode in parallel • Streaming uploads? Make sure your models are immutable!
144. JSON + Protobuf • Try it as a schema language • Read + write JSON • Option to do binary encoding instead
145. @jessewilson Thanks https://github.com/swankjesse/jsonexplained