JSON Explained (Chicago Roboto 2019)

JSON Explained (Chicago Roboto 2019)

Video: https://vimeo.com/341115830
Code: https://github.com/swankjesse/jsonexplained

Java and Kotlin coders: note this talk. Its tran-
script describes some tools to encode any
object. I’ll review the libraries including an an-
notation that can make your app faster.

In this talk we’ll:

🦕 Be warned about the format’s gotchas
🦕 Watch a JSON denial of service attack
🦕 Compare streaming vs. trees vs. databinding
🦕 Appreciate optimizations in Jackson, Gson, and Moshi
🦕 Study the bugs in these same libraries

The talk’s key value will be showing how JSON libraries work.

69252b3de5cb7f464c09301d9a6b0401?s=128

Jesse Wilson

April 26, 2019
Tweet

Transcript

  1. @jessewilson https://github.com/swankjesse/jsonexplained JSON Explained

  2. JavaScript Object Notation

  3. ============================================================================ Copyright (c) 2002 JSON.org Permission is hereby granted, free

    of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. The Software shall be used for Good, not Evil.
  4. The Software shall be used for Good, not Evil.

  5. Open Source Definition

  6. JSON • Specified by RFC 8259 • Four primitive types:

    booleans, numbers, strings, null • Two container types: arrays, objects
  7. Booleans • Delightfully boring: just true, false • On iOS

    boolean is a NSNumber and they’d get 0 instead of false and 1 instead of true
  8. Numbers • IEEE 754 double-precision binary floating point • Not

    boring…
  9. Floating Point Refresher! • Float (32-bit) & Double (64-bit) •

    Specified by IEEE 754 • Supported by Kotlin, Java, and almost all programming languages • Implemented in CPUs and GPUs
  10. Natural Numbers Integers Rational Numbers Real Numbers 1, 2, 3

    -3, -2, -1, 0 -22/7, 51/100 √2, , 0 1 2 3 4 -4 -3 -2 -1 51/100 -22/7 √2 Math Class Numbers
  11. • Byte: Subset of _Z covering -128..127 • Short: Subset

    of _Z covering -32,768..32,767 • Int: Subset of _Z covering -2,147,483,648..2,147,483,647 • Long: Subset of _Z covering
 -9,223,372,036,854,775,808..9,223,372,036,854,775,807 Computer Numbers
  12. Computer Numbers • Float: Subset of , plus special values


    
 3.1415927 • Double: Larger subset of , plus the same special values
 
 3.141592653589793
  13. Special Values println( 1.0 / 0.0 ) // Infinity println(-1.0

    / 0.0 ) // -Infinity println( 0.0 / 1.0 ) // 0.0 println( 0.0 / -1.0 ) // -0.0 println( 0.0 / 0.0 ) // NaN
  14. Infinity is Weird // Double.parseDouble() doesn't demand digits! val pos:

    Double = "Infinity".toDouble() println(pos) // Infinity val neg: Double = "-Infinity".toDouble() println(neg) // -Infinity
  15. Negative Zero is Weird // Operators don't differentiate! println(-0.0 ==

    0.0) // true println(-0.0 < 0.0) // false println(-0.0 > 0.0) // false // But methods do. println((-0.0).equals(0.0)) // false println((-0.0).compareTo(0.0) == 0) // false
  16. NaN is Weird // Operators are not reflexive! println(nan ==

    nan) // false println(nan <= nan) // false println(nan >= nan) // false // But methods are. println(nan.compareTo(nan) == 0) // true println(nan.equals(nan)) // true // Parse "NaN" with or without a negative sign. println("NaN".toDouble()) // NaN println("-NaN".toDouble()) // NaN
  17. Subset of Q • Every floating point value is an

    integer divided by a power of two 8.875 = 71/8 0.1 = 3,602,879,701,896,397/36,028,797,018,963,968 -127.5 = -255/2
  18. Double Precision Floating Point • 1-bit sign • 11-bit exponent

    of denominator • 53-bit numerator (highest bit implied) 0 1 2 3 4 -4 -3 -2 -1
  19. Doubles • Distance between values: • Between 1.0 and 2.0:

    1/4,503,599,627,370,496 • Between 2.0 and 4.0: 1/2,251,799,813,685,248 • Between 4.0 and 8.0: 1/1,125,899,906,842,623
  20. Doubles • Distance between values: • Between 2,251,799,813,685,248 and 4,503,599,627,370,496:

    1/2 • Between 4,503,599,627,370,496 and 9,007,199,254,740,992: 1 • Between 9,007,199,254,740,992 and 18,014,398,509,481,984: 2
  21. Mind the Gap val x = 9_007_199_254_740_992.0 println(x - 2.0

    - x) // -2.0 println(x - 1.0 - x) // -1.0 println(x + 0.0 - x) // 0.0 println(x + 1.0 - x) // 0.0 wat. println(x + 2.0 - x) // 2.0
  22. Mind the Gap val x = 9_007_199_254_740_992.0 println(x - 2.0

    - x) // -2.0 println(x - 1.0 - x) // -1.0 println(x + 0.0 - x) // 0.0 println(x + 1.0 - x) // 0.0 wat. println(x + 2.0 - x) // 2.0
  23. Mind the Gap val x = 9_007_199_254_740_992.0 println(x - 2.0

    - x) // -2.0 println(x - 1.0 - x) // -1.0 println(x + 0.0 - x) // 0.0 println(x + 1.0 - x) // 0.0 wat. println(x + 2.0 - x) // 2.0
  24. Beware of Decimal Approximation println(0.1 + 0.2); // 0.30000000000000004

  25. Infinite Loop! • This value caused every JVM to enter

    an infinite loop:
 
 Double.parseDouble("2.2250738585072012e-308"); • Fixed very quickly once discovered in 2011
  26. Double? Trouble. • Not like the numbers in math class!

    • Negative Zero, Infinity, NaN • Gaps! Like 9,007,199,254,740,993 = 253 + 1 • Decimal representation is approximate
  27. JSON Numbers • JSON spec, RFC 8259 • “expect no

    more precision or range” than double precision • (-253 + 1) .. (253 - 1) is the safe range for integers • No Infinities or NaN, but negative zero is okay!
  28. Is 253 Enough? • 253 bytes = 9 petabytes •

    253 milliseconds = 285,427 years • 253 meters = 0.95 light-years
  29. For Interop, Consider Double’s Limitations • APIs attempt to retain

    types and precision • But this won’t round-trip through systems that lack types • Relying in it is asking for trouble! • Don’t use Longs for IDs in JSON
  30. Strings • UTF-8 encoded • Quote-delimited • Escape sequences for

    quotes \", slashes \\, newlines \n, etc.
  31. Null • Should be boring?

  32. Null vs. Absent? [ { "name": "Jupiter", "radius_km": 69911, "discovered_by":

    "Galileo Galilei" }, { "name": "Earth", "radius_km": 6371, "discovered_by": null }, { "name": "Mars", "radius_km": 3389 } ]
  33. Arrays • Square brackets enclosed • Comma-separated values

  34. Null vs. Absent vs. Empty [ { "name": "Mars", "inhabited_by":

    ["Sojourner", "Spirit", "Opportunity", "Curiosity"] }, { "name": "Jupiter", “inhabited_by": null }, { "name": "Earth" }, { "name": "Saturn", "inhabited_by": [] } ]
  35. Objects • Curly braces enclosed • Comma-separated name/value pairs

  36. { "j": "Jupiter", "v": "Venus", "n": "Neptune", "s": "Saturn" }

  37. fun <K> hashBucket(key: K): Int { return key.hashCode() % buckets.size

    }
  38. 0 1 2 3 4 5 6 7

  39. 0 1 2 3 4 5 6 7 "j" "Jupiter"

    hashBucket("j") // 2
  40. 0 1 2 3 4 5 6 7 "j" "Jupiter"

    "v" "Venus" hashBucket("v") // 6
  41. 0 1 2 3 4 5 6 7 "j" "Jupiter"

    "v" "Venus" "n" "Neptune" hashBucket("n") // 6
  42. 0 1 2 3 4 5 6 7 "j" "Jupiter"

    "s" "Saturn" "v" "Venus" "n" "Neptune" hashBucket("s") // 3
  43. { "abcd": 0, "abdE": 0, "acDd": 0, "acEE": 0, "ad%d":

    0, "bCcd": 0, "bCdE": 0, "bDDd": 0 }
  44. 0 1 2 3 4 5 6 7

  45. 0 1 2 3 4 5 6 7 hashBucket("abcd") //

    7
  46. 0 1 2 3 4 5 6 7 hashBucket("abdE") //

    7
  47. 0 1 2 3 4 5 6 7 hashBucket("acDd") //

    7
  48. 0 1 2 3 4 5 6 7 hashBucket("acEE") //

    7
  49. 0 1 2 3 4 5 6 7 hashBucket(“ad%d") //

    7
  50. 0 1 2 3 4 5 6 7 hashBucket(“bCcd") //

    7
  51. 0 1 2 3 4 5 6 7 hashBucket(“bCdE") //

    7
  52. 0 1 2 3 4 5 6 7 hashBucket(“bDDd") //

    7
  53. Hash DoS Impact • 1 MiB: saturate a CPU core

    for 1 minute • 2 MiB: saturate a CPU core for 4 minutes
  54. Hash DoS Fixes • Fixed in Gson in 2012 •

    Fixed in OpenJDK 8 in 2013 • Fixed in Android Oreo in 2017 • Other platforms are still vulnerable
  55. • A name could be repeated due to mistake or

    attack • Keep first? • Keep last? • Fail? Be Careful of Repeated Names { "alg": "HS256", "typ": "JWT", "alg": "none" }
  56. Dates? • RFC 3339 is a good choice { "speaker":

    "Jesse Wilson", "title": "JSON Explained", "start_time": "2019-04-26T11:00:00-05:00" }
  57. Binary? • Base64 is good inline • URLs are good

    too { "speaker": "Jesse Wilson", "title": "JSON Explained", "sha256": "J84t8GhVF5utZqMW8Nm5M/68Edc0+B7okSGIWnqgwbc=", "slides": "https://github.com/swankjesse/json/slides.pdf" }
  58. Libraries

  59. Jackson • Every feature you’ll ever want • Fast •

    Java library, Kotlin via a support module
  60. Gson • Lots of features • Fast • Java library,

    Kotlin doesn’t really work
  61. Moshi • Small • Fast • Java library, Kotlin via

    a support module
  62. Kotlin Serialization • Small • No Streaming • Kotlin Multiplatform!

    But no Java
  63. Streams

  64. • Streaming starts decoding before the entire JSON is downloaded

    • Target either UTF-8 bytes or Java chars { " c o l o r " : " r e d " , " r a d i u s _ k m " : 3 3 3 9 } jsonBytes: ByteArray
  65. fun readPlanet(reader: JsonReader): Planet { var color: String? = null

    var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { val name = reader.nextName() when (name) { "color" -> color = reader.nextString() "radius_km" -> radiusKm = reader.nextLong() else -> reader.skipValue() } } reader.endObject() return Planet(color, radiusKm) } Decode an Object
  66. fun readPlanet(reader: JsonReader): Planet { var color: String? = null

    var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { val name = reader.nextName() when (name) { "color" -> color = reader.nextString() "radius_km" -> radiusKm = reader.nextLong() else -> reader.skipValue() } } reader.endObject() return Planet(color, radiusKm) } Decode an Object
  67. fun readPlanet(reader: JsonReader): Planet { var color: String? = null

    var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { val name = reader.nextName() when (name) { "color" -> color = reader.nextString() "radius_km" -> radiusKm = reader.nextLong() else -> reader.skipValue() } } reader.endObject() return Planet(color, radiusKm) } Decode an Object
  68. fun readPlanet(reader: JsonReader): Planet { var color: String? = null

    var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { val name = reader.nextName() when (name) { "color" -> color = reader.nextString() "radius_km" -> radiusKm = reader.nextLong() else -> reader.skipValue() } } reader.endObject() return Planet(color, radiusKm) } Decode an Object
  69. fun readPlanet(reader: JsonReader): Planet { var color: String? = null

    var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { val name = reader.nextName() when (name) { "color" -> color = reader.nextString() "radius_km" -> radiusKm = reader.nextLong() else -> reader.skipValue() } } reader.endObject() return Planet(color, radiusKm) } Decode an Object
  70. fun readPlanet(reader: JsonReader): Planet { var color: String? = null

    var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { val name = reader.nextName() when (name) { "color" -> color = reader.nextString() "radius_km" -> radiusKm = reader.nextLong() else -> reader.skipValue() } } reader.endObject() return Planet(color, radiusKm) } fun readPlanet(reader: JsonReader): Planet { var color: String? = null var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { val name = reader.nextName() when (name) { "color" -> color = reader.nextString() "radius_km" -> radiusKm = reader.nextLong() else -> reader.skipValue() } } reader.endObject() return Planet(color, radiusKm) } Decode an Object
  71. { " c o l o r " : "

    r e d " , " r a d i u s _ k m " : 3 3 3 9 } begin 2 end 7 val name = String(jsonChars, begin, end) new String() jsonBytes: ByteArray
  72. 0 ×××× ×××× ×××× 0 1 ×××× ×××× ×××× 0

    2 ×××× ×××× ×××× 0 3 ×××× ×××× ×××× 0 4 ×××× ×××× ×××× 0 5 ×××× ×××× ×××× 0 6 ×××× ×××× ×××× 0 7 ×××× ×××× ×××× 0 0 1 2 3 4 5 6 7 Jackson’s ByteQuadsCanonicalizer
  73. 0 ×××× ×××× ×××× 0 1 ×××× ×××× ×××× 0

    2 ×××× ×××× ×××× 0 3 ×××× ×××× ×××× 0 4 ×××× ×××× ×××× 0 5 ×××× ×××× ×××× 0 6 colo r××× ×××× 5 7 ×××× ×××× ×××× 0 0 1 2 3 4 5 6 "color" 7 Jackson’s ByteQuadsCanonicalizer
  74. 0 ×××× ×××× ×××× 0 1 ×××× ×××× ×××× 0

    2 ×××× ×××× ×××× 0 3 ×××× ×××× ×××× 0 4 radi us_k m××× 9 5 ×××× ×××× ×××× 0 6 colo r××× ×××× 5 7 ×××× ×××× ×××× 0 0 1 2 3 4 "radius_km" 5 6 "color" 7 Jackson’s ByteQuadsCanonicalizer
  75. Moshi’s Options Trie "path" "primary_artist" "header_image_url" "annotation_count" "header_image_thumbnail_url" "api_path" "url"

  76. a… h… p… u… …notation_count 3 …n… …p… …a… …r…

    …i_path 4 …eader_image_… …t… …u… …humbnail_url 5 …rl 2 …imary_artist 1 …th 0 …rl 6
  77. val FIELD_NAMES = JsonReader.Options.of("color", "radius_km") fun readPlanet(reader: JsonReader): Planet {

    var color: String? = null var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { when (reader.selectName(FIELD_NAMES)) { 0 -> color = reader.nextString() 1 -> radiusKm = reader.nextLong() else -> { reader.skipName() reader.skipValue() } } } reader.endObject() return Planet(color, radiusKm) }
  78. val FIELD_NAMES = JsonReader.Options.of("color", "radius_km") fun readPlanet(reader: JsonReader): Planet {

    var color: String? = null var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { when (reader.selectName(FIELD_NAMES)) { 0 -> color = reader.nextString() 1 -> radiusKm = reader.nextLong() else -> { reader.skipName() reader.skipValue() } } } reader.endObject() return Planet(color, radiusKm) }
  79. val FIELD_NAMES = JsonReader.Options.of("color", "radius_km") fun readPlanet(reader: JsonReader): Planet {

    var color: String? = null var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { when (reader.selectName(FIELD_NAMES)) { 0 -> color = reader.nextString() 1 -> radiusKm = reader.nextLong() else -> { reader.skipName() reader.skipValue() } } } reader.endObject() return Planet(color, radiusKm) }
  80. val FIELD_NAMES = JsonReader.Options.of("color", "radius_km") fun readPlanet(reader: JsonReader): Planet {

    var color: String? = null var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { when (reader.selectName(FIELD_NAMES)) { 0 -> color = reader.nextString() 1 -> radiusKm = reader.nextLong() else -> { reader.skipName() reader.skipValue() } } } reader.endObject() return Planet(color, radiusKm) }
  81. val FIELD_NAMES = JsonReader.Options.of("color", "radius_km") fun readPlanet(reader: JsonReader): Planet {

    var color: String? = null var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { when (reader.selectName(FIELD_NAMES)) { 0 -> color = reader.nextString() 1 -> radiusKm = reader.nextLong() else -> { reader.skipName() reader.skipValue() } } } reader.endObject() return Planet(color, radiusKm) }
  82. Trees

  83. Tree Models Jackson JsonNode, NullNode, MissingNode, POJONode Gson JsonElement, JsonPrimitive,

    JsonNull Moshi List, Map, String Kotlin Serialization JsonElement, JsonLiteral, JsonNull
  84. Working with Trees • Lots of strings and casting •

    Two kinds of nulls • Makes me want types
  85. Data Binding

  86. Type Mapping JSON Kotlin / Java Boolean Boolean Number Byte,

    Short, Int, Long, Float, Double String String, Char, ByteString, Enums, Instant… Array Array, List, Set… Object Classes, Map
  87. Adapters • Convert between Kotlin values and JSON values •

    Example: HttpUrl
  88. Cycles class Planet( val name: String, val moons: List<Moon> )

    class Moon( val name: String, val orbits: Planet ) { "name": "Saturn", "moons": [ { "name": "Titan", "orbits": { "name": "Saturn", "moons": [ { "name": "Titan", "orbits": { "name": "Saturn", "moons": [ { "name": "Titan", ...
  89. Cycles { "name": "Saturn", "moons": [ { "name": "Titan", "orbits":

    { "name": "Saturn", "moons": [ { "name": "Titan", "orbits": { "name": "Saturn", "moons": [ { "name": "Titan", ... class Planet( val name: String, val moons: List<String> ) class Moon( val name: String, val orbits: String )
  90. Cycles class Planet( val name: String, val moons: List<String> )

    class Moon( val name: String, val orbits: String ) { "name": "Saturn", "moons": [ "Titan", "Calypso", "Tethys", "Rhea", "Dione", "Enceladus", "Mimas" ] }
  91. Subclasses sealed class CosmicObject { data class Star( val name:

    String, val radius_km: Long ) : CosmicObject() data class Planet( val name: String, val orbits_star: String ) : CosmicObject() data class Moon( val name: String, val orbits_planet: String ) : CosmicObject() } { "destination": { "name": "Alpha Centauri", "radius_km": 851120 }, "origin": { "name": "Saturn", "oribits_star": "Sun" } }
  92. Subclasses val moshi = Moshi.Builder() .add(PolymorphicJsonAdapterFactory.of(CosmicObject::class.java, "type") .withSubtype(CosmicObject.Star::class.java, "star") .withSubtype(CosmicObject.Planet::class.java,

    "planet") .withSubtype(CosmicObject.Moon::class.java, "moon")) .build()
  93. Subclasses sealed class CosmicObject { data class Star( val name:

    String, val radius_km: Long ) : CosmicObject() data class Planet( val name: String, val orbits_star: String ) : CosmicObject() data class Moon( val name: String, val orbits_planet: String ) : CosmicObject() } { "destination": { "name": "Alpha Centauri", "radius_km": 851120 }, "origin": { "name": "Saturn", "oribits_star": "Sun" } }
  94. Subclasses { "destination": { "type": "star", "name": "Alpha Centauri", "radius_km":

    851120 }, "origin": { "type": "planet", "name": "Saturn", "oribits_star": "Sun" } } sealed class CosmicObject { data class Star( val name: String, val radius_km: Long ) : CosmicObject() data class Planet( val name: String, val orbits_star: String ) : CosmicObject() data class Moon( val name: String, val orbits_planet: String ) : CosmicObject() }
  95. class Planet { final String name; final double radius_km; String

    color = "unknown"; List<String> moons = emptyList(); Planet( String name, double radiusKm) { this.name = name; this.radius_km = radiusKm; } ... } Reflection Magic val json = """ |{ | "name": "Mars", | "radius_km": 3389 |} """.trimMargin() val planet = adapter.fromJson(json)!! println(planet.name) // Mars println(planet.radius_km) // 3389.0 println(planet.color) // null println(planet.moons) // null
  96. Reflection Magic val json = """ |{ | "name": "Mars",

    | "radius_km": 3389 |} """.trimMargin() val planet = adapter.fromJson(json)!! println(planet.name) // Mars println(planet.radius_km) // 3389.0 println(planet.color) // null println(planet.moons) // null @JsonClass(generateAdapter = true) class Planet( val name: String, val radius_km: Double ) { val color = "unknown" val moons = emptyList<String>() }
  97. Reflection Magic val json = """ |{ | "name": "Mars",

    | "radius_km": 3389 |} """.trimMargin() val planet = adapter.fromJson(json)!! println(planet.name) // Mars println(planet.radius_km) // 3389.0 println(planet.color) // unknown println(planet.moons) // [] @JsonClass(generateAdapter = true) class Planet( val name: String, val radius_km: Double ) { val color = "unknown" val moons = emptyList<String>() }
  98. Built-in Types data class Planet( val name: String, val surface_color:

    Color ) { "name" : "Saturn", "surface_color" : { "value" : -9677, "falpha" : 0 } }
  99. Case Mapping val gson = GsonBuilder() .setFieldNamingPolicy(FieldNamingPolicy.LOWER_CASE_WITH_UNDERSCORES) .create()

  100. I’m deleting old code. Is orbitsStar still used?

  101. I’m deleting old code. Is orbitsStar still used? grep says

    no
  102. Case Mapping val gson = GsonBuilder() .setFieldNamingPolicy(FieldNamingPolicy.LOWER_CASE_WITH_UNDERSCORES) .create()

  103. Case Mapping data class Planet( val name: String, val orbitsStar:

    String, val radiusKm: Long, val moons: List<String> ) { "name": "Saturn", "oribits_star": "Sun", "radius_km": 58232, "moons": [ "Titan", "Calypso" ] }
  104. Case Mapping data class Planet( val name: String, @SerializedName("orbits_star") val

    orbitsStar: String, @SerializedName("radius_km") val radiusKm: Long, val moons: List<String> ) { "name": "Saturn", "oribits_star": "Sun", "radius_km": 58232, "moons": [ "Titan", "Calypso" ] }
  105. Case Mapping data class Planet( val name: String, val orbits_star:

    String, val radius_km: Long, val moons: List<String> ) { "name": "Saturn", "oribits_star": "Sun", "radius_km": 58232, "moons": [ "Titan", "Calypso" ] }
  106. Generating Code Jackson Runtime bytecode generator
 (No Android support) Gson

    None Moshi Annotation Processor Kotlin Serialization Kotlin Compiler Plugin
  107. Gson Jackson Moshi Kotlin S. ? ? ?

  108. Jackson 2.9.8 (2018-12-15) 21 feature releases Gson 2.8.5 (2018-05-21) 17

    feature releases Moshi 1.8.0 (2018-11-09) 9 feature releases Kotlin Serialization 0.11.0 (2019-04-12) pre-release Latest Release
  109. Jackson 4,370 KiB includes kotlin-reflect (2,584 KiB) Gson 239 KiB

    Moshi 330 KiB + generated code includes Okio (178 KiB) Kotlin Serialization 762 KiB + generated code Size Before Shrinking
  110. Jackson Fast stream encoder + decoder Allocation avoidance pool Gson

    Fast stream encoder + decoder Moshi Fast stream encoder + decoder Select trie Generated Kotlin adapters Kotlin Serialization Generated Kotlin adapters Performance Features
  111. APIs

  112. var mapper = jacksonObjectMapper().apply { disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES) } @Test fun decodeJson()

    { val inputStream: InputStream = ... val searchResponseBody = inputStream.use { mapper.readValue<SearchResponseBody>(it) } ... } Jackson
  113. val gson = GsonBuilder().create() @Test fun decodeJson() { val reader:

    Reader = ... val searchResponseBody = reader.use { gson.fromJson(it, SearchResponseBody::class.java) } ... } Gson
  114. val moshi = Moshi.Builder().build() val adapter = moshi.adapter(SearchResponseBody::class.java) @Test fun

    decodeJson() { val source: BufferedSource = ... val searchResponseBody = source.use { adapter.fromJson(it)!! } ... } Moshi
  115. val json = Json(JsonConfiguration(strictMode = false)) @Test fun decodeJson() {

    val string: String = ... val searchResponseBody = json.parse( SearchResponseBody.serializer(), string) ... } Kotlin Serialization
  116. Custom Adapters

  117. object HttpUrlSerializer : StdSerializer<HttpUrl>(HttpUrl::class.java) { override fun serialize( value: HttpUrl,

    generator: JsonGenerator, provider: SerializerProvider ) { generator.writeString(value.toString()) } } object HttpUrlDeserializer : StdDeserializer<HttpUrl>(HttpUrl::class.java) { override fun deserialize( parser: JsonParser, context: DeserializationContext? ): HttpUrl { val text = parser.getValueAsString() return HttpUrl.get(text) } } var mapper = jacksonObjectMapper().apply { registerModule(SimpleModule().apply { addSerializer(HttpUrl::class.java, HttpUrlSerializer) addDeserializer(HttpUrl::class.java, HttpUrlDeserializer) }) } Jackson
  118. object HttpUrlTypeAdapter : TypeAdapter<HttpUrl>() { override fun write(writer: JsonWriter, value:

    HttpUrl) { writer.value(value.toString()) } override fun read(reader: JsonReader): HttpUrl { val string = reader.nextString() return HttpUrl.get(string) } } val gson = GsonBuilder() .registerTypeAdapter(HttpUrl::class.java, HttpUrlTypeAdapter) .create() Gson
  119. object CustomAdapters { @ToJson fun httpUrlToJson(httpUrl: HttpUrl): String { return

    httpUrl.toString() } @FromJson fun httpUrlFromJson(string: String): HttpUrl { return HttpUrl.get(string) } } val moshi = Moshi.Builder() .add(CustomAdapters) .build() Moshi
  120. @Serializer(forClass = HttpUrl::class) object HttpUrlSerializer : KSerializer<HttpUrl> { override fun

    serialize(encoder: Encoder, httpUrl: HttpUrl) { encoder.encodeString(httpUrl.toString()) } override fun deserialize(decoder: Decoder): HttpUrl { val string = decoder.decodeString() return HttpUrl.get(string) } } @file:UseSerializers( serializerClasses = [HttpUrlSerializer::class]
 ) Kotlin Serialization
  121. Usability

  122. Jackson "Barnes & Noble" Gson "Barnes \u0026 Noble" Moshi "Barnes

    & Noble" Kotlin Serialization "Barnes & Noble" Special Characters
  123. Jackson "Barnes & Noble" Gson "Barnes \u0026 Noble" Moshi "Barnes

    & Noble" Kotlin Serialization "Barnes & Noble" Special Characters
  124. Jackson com.fasterxml.jackson.databind.exc.InvalidFormatException: Cannot deserialize value of type `long` from String

    "whoops": not a valid Long value at [Source: (BufferedInputStream); line: 302, column: 18] (through reference chain: com.jsonexplained.SearchResponseBody["response"]
 ->com.jsonexplained.SearchResponse["hits"]
 ->java.util.ArrayList[8]
 ->com.jsonexplained.Hit["result"]
 ->com.jsonexplained.SongResult["id"]) Gson com.google.gson.JsonSyntaxException: java.lang.NumberFormatException: For input string: "whoops" Moshi com.squareup.moshi.JsonDataException: Expected a long but was whoops at path $.response.hits[8].result.id Kotlin Serialization java.lang.NumberFormatException: For input string: "whoops" Type Error
  125. Jackson com.fasterxml.jackson.databind.exc.InvalidFormatException: Cannot deserialize value of type `long` from String

    "whoops": not a valid Long value at [Source: (BufferedInputStream); line: 302, column: 18] (through reference chain: com.jsonexplained.SearchResponseBody["response"]
 ->com.jsonexplained.SearchResponse["hits"]
 ->java.util.ArrayList[8]
 ->com.jsonexplained.Hit["result"]
 ->com.jsonexplained.SongResult["id"]) Gson com.google.gson.JsonSyntaxException: java.lang.NumberFormatException: For input string: "whoops" Moshi com.squareup.moshi.JsonDataException: Expected a long but was whoops at path $.response.hits[8].result.id Kotlin Serialization java.lang.NumberFormatException: For input string: "whoops" Type Error
  126. Jackson Success Gson Success Moshi Success Kotlin Serialization java.lang.NumberFormatException: For

    input string: "1.0" Type Acceptance
  127. Jackson Success Gson Success Moshi Success Kotlin Serialization java.lang.NumberFormatException: For

    input string: "1.0" Type Acceptance
  128. Jackson Long: completed normally, used 0
 String: crashed with a

    helpful message Gson Long: completed normally, used 0
 String: completed normally, used null Moshi com.squareup.moshi.JsonDataException: Required property 'id' missing at $.response.hits[8].result Kotlin Serialization kotlinx.serialization.MissingFieldException: Field 'id' is required, but it was missing Required Field Is Absent
  129. Jackson Long: completed normally, used 0
 String: crashed with a

    helpful message Gson Long: completed normally, used 0
 String: completed normally, used null Moshi com.squareup.moshi.JsonDataException: Required property 'id' missing at $.response.hits[8].result Kotlin Serialization kotlinx.serialization.MissingFieldException: Field 'id' is required, but it was missing Required Field Is Absent
  130. Jackson com.fasterxml.jackson.databind.exc.MismatchedInputException: No content to map due to end-of-input at

    [Source: (BufferedInputStream); line: 1, column: 0] Gson Completed normally, returned null Moshi java.io.EOFException: End of input Kotlin Serialization kotlinx.serialization.json.JsonParsingException: Invalid JSON at 0: Expected '{, kind: kotlinx.serialization.StructureKind$CLASS@221af3c0' Document is Empty
  131. Jackson com.fasterxml.jackson.databind.exc.MismatchedInputException: No content to map due to end-of-input at

    [Source: (BufferedInputStream); line: 1, column: 0] Gson Completed normally, returned null Moshi java.io.EOFException: End of input Kotlin Serialization kotlinx.serialization.json.JsonParsingException: Invalid JSON at 0: Expected '{, kind: kotlinx.serialization.StructureKind$CLASS@221af3c0' Document is Empty
  132. Jackson Completes normally, encoding implementation details of types like CompletableFuture

    Gson Completes normally, encoding implementation details of types like CompletableFuture Moshi Fails fast with an error Kotlin Serialization Fails fast with an error Write an Unexpected Type
  133. Jackson Completes normally, encoding implementation details of types like CompletableFuture

    Gson Completes normally, encoding implementation details of types like CompletableFuture Moshi Fails fast with an error Kotlin Serialization Fails fast with an error Write an Unexpected Type
  134. Jackson com.fasterxml.jackson.databind.JsonMappingException: Numeric value (2147483648) out of range of int

    at [Source: (BufferedInputStream); line: 15, column: 32] (through reference chain: com.jsonexplained.SearchResponseBody["response"]
 ->com.jsonexplained.SearchResponse["hits"]
 ->java.util.ArrayList[0]
 ->com.jsonexplained.Hit["result"]
 ->com.jsonexplained.SongResult["annotation_count"]) Gson com.google.gson.JsonSyntaxException: java.lang.NumberFormatException: Expected an int but was 2147483648 at line 15 column 42 path $.response.hits[0].result.annotation_count Moshi com.squareup.moshi.JsonDataException: Expected an int but was 2147483648 at path $.response.hits[0].result.annotation_count Kotlin Serialization java.lang.NumberFormatException: For input string: "2147483648" Too Large For An Int
  135. Jackson com.fasterxml.jackson.databind.JsonMappingException: Numeric value (2147483648) out of range of int

    at [Source: (BufferedInputStream); line: 15, column: 32] (through reference chain: com.jsonexplained.SearchResponseBody["response"]
 ->com.jsonexplained.SearchResponse["hits"]
 ->java.util.ArrayList[0]
 ->com.jsonexplained.Hit["result"]
 ->com.jsonexplained.SongResult["annotation_count"]) Gson com.google.gson.JsonSyntaxException: java.lang.NumberFormatException: Expected an int but was 2147483648 at line 15 column 42 path $.response.hits[0].result.annotation_count Moshi com.squareup.moshi.JsonDataException: Expected an int but was 2147483648 at path $.response.hits[0].result.annotation_count Kotlin Serialization java.lang.NumberFormatException: For input string: "2147483648" Too Large For An Int
  136. Gson MOST USED Jackson MOST CAPABLE Kotlin S. BEST FOR

    MULTI
 PLATFORM Moshi BEST API
  137. Testing

  138. Testing? • Confirm you’ve configured things properly • Give you

    confidence to refactor & make changes • Get your server team to code review!
  139. @Test fun json() { val planet = Planet("Mercury", 2439, listOf("Messenger"))

    val json = """ |{ | "name": "Mercury", | "radius_km": 2439, | "visitors": [ | "Messenger" | ] |} """.trimMargin() val adapter = moshi.adapter(Planet::class.java).indent(" ") assertThat(adapter.toJson(planet)).isEqualTo(json) assertThat(adapter.fromJson(json)).isEqualTo(planet) }
  140. Optimizing

  141. Use Less JSON • No pretty-printed JSON • Delete unused

    fields: fewer bytes to send, store, encode, decode • Omit nulls
  142. Avoid Reflection • Enable generated code • Don’t create a

    new JsonAdapter every time
  143. Stream! • Download & decode in parallel • Streaming uploads?

    Make sure your models are immutable!
  144. JSON + Protobuf • Try it as a schema language

    • Read + write JSON • Option to do binary encoding instead
  145. @jessewilson Thanks https://github.com/swankjesse/jsonexplained