Slide 1

Slide 1 text

@jessewilson https://github.com/swankjesse/jsonexplained JSON Explained

Slide 2

Slide 2 text

JavaScript Object Notation

Slide 3

Slide 3 text

============================================================================ Copyright (c) 2002 JSON.org Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. The Software shall be used for Good, not Evil.

Slide 4

Slide 4 text

The Software shall be used for Good, not Evil.

Slide 5

Slide 5 text

Open Source Definition

Slide 6

Slide 6 text

JSON • Specified by RFC 8259 • Four primitive types: booleans, numbers, strings, null • Two container types: arrays, objects

Slide 7

Slide 7 text

Booleans • Delightfully boring: just true, false • On iOS boolean is a NSNumber and they’d get 0 instead of false and 1 instead of true

Slide 8

Slide 8 text

Numbers • IEEE 754 double-precision binary floating point • Not boring…

Slide 9

Slide 9 text

Floating Point Refresher! • Float (32-bit) & Double (64-bit) • Specified by IEEE 754 • Supported by Kotlin, Java, and almost all programming languages • Implemented in CPUs and GPUs

Slide 10

Slide 10 text

Natural Numbers Integers Rational Numbers Real Numbers 1, 2, 3 -3, -2, -1, 0 -22/7, 51/100 √2, , 0 1 2 3 4 -4 -3 -2 -1 51/100 -22/7 √2 Math Class Numbers

Slide 11

Slide 11 text

• Byte: Subset of _Z covering -128..127 • Short: Subset of _Z covering -32,768..32,767 • Int: Subset of _Z covering -2,147,483,648..2,147,483,647 • Long: Subset of _Z covering
 -9,223,372,036,854,775,808..9,223,372,036,854,775,807 Computer Numbers

Slide 12

Slide 12 text

Computer Numbers • Float: Subset of , plus special values
 
 3.1415927 • Double: Larger subset of , plus the same special values
 
 3.141592653589793

Slide 13

Slide 13 text

Special Values println( 1.0 / 0.0 ) // Infinity println(-1.0 / 0.0 ) // -Infinity println( 0.0 / 1.0 ) // 0.0 println( 0.0 / -1.0 ) // -0.0 println( 0.0 / 0.0 ) // NaN

Slide 14

Slide 14 text

Infinity is Weird // Double.parseDouble() doesn't demand digits! val pos: Double = "Infinity".toDouble() println(pos) // Infinity val neg: Double = "-Infinity".toDouble() println(neg) // -Infinity

Slide 15

Slide 15 text

Negative Zero is Weird // Operators don't differentiate! println(-0.0 == 0.0) // true println(-0.0 < 0.0) // false println(-0.0 > 0.0) // false // But methods do. println((-0.0).equals(0.0)) // false println((-0.0).compareTo(0.0) == 0) // false

Slide 16

Slide 16 text

NaN is Weird // Operators are not reflexive! println(nan == nan) // false println(nan <= nan) // false println(nan >= nan) // false // But methods are. println(nan.compareTo(nan) == 0) // true println(nan.equals(nan)) // true // Parse "NaN" with or without a negative sign. println("NaN".toDouble()) // NaN println("-NaN".toDouble()) // NaN

Slide 17

Slide 17 text

Subset of Q • Every floating point value is an integer divided by a power of two 8.875 = 71/8 0.1 = 3,602,879,701,896,397/36,028,797,018,963,968 -127.5 = -255/2

Slide 18

Slide 18 text

Double Precision Floating Point • 1-bit sign • 11-bit exponent of denominator • 53-bit numerator (highest bit implied) 0 1 2 3 4 -4 -3 -2 -1

Slide 19

Slide 19 text

Doubles • Distance between values: • Between 1.0 and 2.0: 1/4,503,599,627,370,496 • Between 2.0 and 4.0: 1/2,251,799,813,685,248 • Between 4.0 and 8.0: 1/1,125,899,906,842,623

Slide 20

Slide 20 text

Doubles • Distance between values: • Between 2,251,799,813,685,248 and 4,503,599,627,370,496: 1/2 • Between 4,503,599,627,370,496 and 9,007,199,254,740,992: 1 • Between 9,007,199,254,740,992 and 18,014,398,509,481,984: 2

Slide 21

Slide 21 text

Mind the Gap val x = 9_007_199_254_740_992.0 println(x - 2.0 - x) // -2.0 println(x - 1.0 - x) // -1.0 println(x + 0.0 - x) // 0.0 println(x + 1.0 - x) // 0.0 wat. println(x + 2.0 - x) // 2.0

Slide 22

Slide 22 text

Mind the Gap val x = 9_007_199_254_740_992.0 println(x - 2.0 - x) // -2.0 println(x - 1.0 - x) // -1.0 println(x + 0.0 - x) // 0.0 println(x + 1.0 - x) // 0.0 wat. println(x + 2.0 - x) // 2.0

Slide 23

Slide 23 text

Mind the Gap val x = 9_007_199_254_740_992.0 println(x - 2.0 - x) // -2.0 println(x - 1.0 - x) // -1.0 println(x + 0.0 - x) // 0.0 println(x + 1.0 - x) // 0.0 wat. println(x + 2.0 - x) // 2.0

Slide 24

Slide 24 text

Beware of Decimal Approximation println(0.1 + 0.2); // 0.30000000000000004

Slide 25

Slide 25 text

Infinite Loop! • This value caused every JVM to enter an infinite loop:
 
 Double.parseDouble("2.2250738585072012e-308"); • Fixed very quickly once discovered in 2011

Slide 26

Slide 26 text

Double? Trouble. • Not like the numbers in math class! • Negative Zero, Infinity, NaN • Gaps! Like 9,007,199,254,740,993 = 253 + 1 • Decimal representation is approximate

Slide 27

Slide 27 text

JSON Numbers • JSON spec, RFC 8259 • “expect no more precision or range” than double precision • (-253 + 1) .. (253 - 1) is the safe range for integers • No Infinities or NaN, but negative zero is okay!

Slide 28

Slide 28 text

Is 253 Enough? • 253 bytes = 9 petabytes • 253 milliseconds = 285,427 years • 253 meters = 0.95 light-years

Slide 29

Slide 29 text

For Interop, Consider Double’s Limitations • APIs attempt to retain types and precision • But this won’t round-trip through systems that lack types • Relying in it is asking for trouble! • Don’t use Longs for IDs in JSON

Slide 30

Slide 30 text

Strings • UTF-8 encoded • Quote-delimited • Escape sequences for quotes \", slashes \\, newlines \n, etc.

Slide 31

Slide 31 text

Null • Should be boring?

Slide 32

Slide 32 text

Null vs. Absent? [ { "name": "Jupiter", "radius_km": 69911, "discovered_by": "Galileo Galilei" }, { "name": "Earth", "radius_km": 6371, "discovered_by": null }, { "name": "Mars", "radius_km": 3389 } ]

Slide 33

Slide 33 text

Arrays • Square brackets enclosed • Comma-separated values

Slide 34

Slide 34 text

Null vs. Absent vs. Empty [ { "name": "Mars", "inhabited_by": ["Sojourner", "Spirit", "Opportunity", "Curiosity"] }, { "name": "Jupiter", “inhabited_by": null }, { "name": "Earth" }, { "name": "Saturn", "inhabited_by": [] } ]

Slide 35

Slide 35 text

Objects • Curly braces enclosed • Comma-separated name/value pairs

Slide 36

Slide 36 text

{ "j": "Jupiter", "v": "Venus", "n": "Neptune", "s": "Saturn" }

Slide 37

Slide 37 text

fun hashBucket(key: K): Int { return key.hashCode() % buckets.size }

Slide 38

Slide 38 text

0 1 2 3 4 5 6 7

Slide 39

Slide 39 text

0 1 2 3 4 5 6 7 "j" "Jupiter" hashBucket("j") // 2

Slide 40

Slide 40 text

0 1 2 3 4 5 6 7 "j" "Jupiter" "v" "Venus" hashBucket("v") // 6

Slide 41

Slide 41 text

0 1 2 3 4 5 6 7 "j" "Jupiter" "v" "Venus" "n" "Neptune" hashBucket("n") // 6

Slide 42

Slide 42 text

0 1 2 3 4 5 6 7 "j" "Jupiter" "s" "Saturn" "v" "Venus" "n" "Neptune" hashBucket("s") // 3

Slide 43

Slide 43 text

{ "abcd": 0, "abdE": 0, "acDd": 0, "acEE": 0, "ad%d": 0, "bCcd": 0, "bCdE": 0, "bDDd": 0 }

Slide 44

Slide 44 text

0 1 2 3 4 5 6 7

Slide 45

Slide 45 text

0 1 2 3 4 5 6 7 hashBucket("abcd") // 7

Slide 46

Slide 46 text

0 1 2 3 4 5 6 7 hashBucket("abdE") // 7

Slide 47

Slide 47 text

0 1 2 3 4 5 6 7 hashBucket("acDd") // 7

Slide 48

Slide 48 text

0 1 2 3 4 5 6 7 hashBucket("acEE") // 7

Slide 49

Slide 49 text

0 1 2 3 4 5 6 7 hashBucket(“ad%d") // 7

Slide 50

Slide 50 text

0 1 2 3 4 5 6 7 hashBucket(“bCcd") // 7

Slide 51

Slide 51 text

0 1 2 3 4 5 6 7 hashBucket(“bCdE") // 7

Slide 52

Slide 52 text

0 1 2 3 4 5 6 7 hashBucket(“bDDd") // 7

Slide 53

Slide 53 text

Hash DoS Impact • 1 MiB: saturate a CPU core for 1 minute • 2 MiB: saturate a CPU core for 4 minutes

Slide 54

Slide 54 text

Hash DoS Fixes • Fixed in Gson in 2012 • Fixed in OpenJDK 8 in 2013 • Fixed in Android Oreo in 2017 • Other platforms are still vulnerable

Slide 55

Slide 55 text

• A name could be repeated due to mistake or attack • Keep first? • Keep last? • Fail? Be Careful of Repeated Names { "alg": "HS256", "typ": "JWT", "alg": "none" }

Slide 56

Slide 56 text

Dates? • RFC 3339 is a good choice { "speaker": "Jesse Wilson", "title": "JSON Explained", "start_time": "2019-04-26T11:00:00-05:00" }

Slide 57

Slide 57 text

Binary? • Base64 is good inline • URLs are good too { "speaker": "Jesse Wilson", "title": "JSON Explained", "sha256": "J84t8GhVF5utZqMW8Nm5M/68Edc0+B7okSGIWnqgwbc=", "slides": "https://github.com/swankjesse/json/slides.pdf" }

Slide 58

Slide 58 text

Libraries

Slide 59

Slide 59 text

Jackson • Every feature you’ll ever want • Fast • Java library, Kotlin via a support module

Slide 60

Slide 60 text

Gson • Lots of features • Fast • Java library, Kotlin doesn’t really work

Slide 61

Slide 61 text

Moshi • Small • Fast • Java library, Kotlin via a support module

Slide 62

Slide 62 text

Kotlin Serialization • Small • No Streaming • Kotlin Multiplatform! But no Java

Slide 63

Slide 63 text

Streams

Slide 64

Slide 64 text

• Streaming starts decoding before the entire JSON is downloaded • Target either UTF-8 bytes or Java chars { " c o l o r " : " r e d " , " r a d i u s _ k m " : 3 3 3 9 } jsonBytes: ByteArray

Slide 65

Slide 65 text

fun readPlanet(reader: JsonReader): Planet { var color: String? = null var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { val name = reader.nextName() when (name) { "color" -> color = reader.nextString() "radius_km" -> radiusKm = reader.nextLong() else -> reader.skipValue() } } reader.endObject() return Planet(color, radiusKm) } Decode an Object

Slide 66

Slide 66 text

fun readPlanet(reader: JsonReader): Planet { var color: String? = null var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { val name = reader.nextName() when (name) { "color" -> color = reader.nextString() "radius_km" -> radiusKm = reader.nextLong() else -> reader.skipValue() } } reader.endObject() return Planet(color, radiusKm) } Decode an Object

Slide 67

Slide 67 text

fun readPlanet(reader: JsonReader): Planet { var color: String? = null var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { val name = reader.nextName() when (name) { "color" -> color = reader.nextString() "radius_km" -> radiusKm = reader.nextLong() else -> reader.skipValue() } } reader.endObject() return Planet(color, radiusKm) } Decode an Object

Slide 68

Slide 68 text

fun readPlanet(reader: JsonReader): Planet { var color: String? = null var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { val name = reader.nextName() when (name) { "color" -> color = reader.nextString() "radius_km" -> radiusKm = reader.nextLong() else -> reader.skipValue() } } reader.endObject() return Planet(color, radiusKm) } Decode an Object

Slide 69

Slide 69 text

fun readPlanet(reader: JsonReader): Planet { var color: String? = null var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { val name = reader.nextName() when (name) { "color" -> color = reader.nextString() "radius_km" -> radiusKm = reader.nextLong() else -> reader.skipValue() } } reader.endObject() return Planet(color, radiusKm) } Decode an Object

Slide 70

Slide 70 text

fun readPlanet(reader: JsonReader): Planet { var color: String? = null var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { val name = reader.nextName() when (name) { "color" -> color = reader.nextString() "radius_km" -> radiusKm = reader.nextLong() else -> reader.skipValue() } } reader.endObject() return Planet(color, radiusKm) } fun readPlanet(reader: JsonReader): Planet { var color: String? = null var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { val name = reader.nextName() when (name) { "color" -> color = reader.nextString() "radius_km" -> radiusKm = reader.nextLong() else -> reader.skipValue() } } reader.endObject() return Planet(color, radiusKm) } Decode an Object

Slide 71

Slide 71 text

{ " c o l o r " : " r e d " , " r a d i u s _ k m " : 3 3 3 9 } begin 2 end 7 val name = String(jsonChars, begin, end) new String() jsonBytes: ByteArray

Slide 72

Slide 72 text

0 ×××× ×××× ×××× 0 1 ×××× ×××× ×××× 0 2 ×××× ×××× ×××× 0 3 ×××× ×××× ×××× 0 4 ×××× ×××× ×××× 0 5 ×××× ×××× ×××× 0 6 ×××× ×××× ×××× 0 7 ×××× ×××× ×××× 0 0 1 2 3 4 5 6 7 Jackson’s ByteQuadsCanonicalizer

Slide 73

Slide 73 text

0 ×××× ×××× ×××× 0 1 ×××× ×××× ×××× 0 2 ×××× ×××× ×××× 0 3 ×××× ×××× ×××× 0 4 ×××× ×××× ×××× 0 5 ×××× ×××× ×××× 0 6 colo r××× ×××× 5 7 ×××× ×××× ×××× 0 0 1 2 3 4 5 6 "color" 7 Jackson’s ByteQuadsCanonicalizer

Slide 74

Slide 74 text

0 ×××× ×××× ×××× 0 1 ×××× ×××× ×××× 0 2 ×××× ×××× ×××× 0 3 ×××× ×××× ×××× 0 4 radi us_k m××× 9 5 ×××× ×××× ×××× 0 6 colo r××× ×××× 5 7 ×××× ×××× ×××× 0 0 1 2 3 4 "radius_km" 5 6 "color" 7 Jackson’s ByteQuadsCanonicalizer

Slide 75

Slide 75 text

Moshi’s Options Trie "path" "primary_artist" "header_image_url" "annotation_count" "header_image_thumbnail_url" "api_path" "url"

Slide 76

Slide 76 text

a… h… p… u… …notation_count 3 …n… …p… …a… …r… …i_path 4 …eader_image_… …t… …u… …humbnail_url 5 …rl 2 …imary_artist 1 …th 0 …rl 6

Slide 77

Slide 77 text

val FIELD_NAMES = JsonReader.Options.of("color", "radius_km") fun readPlanet(reader: JsonReader): Planet { var color: String? = null var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { when (reader.selectName(FIELD_NAMES)) { 0 -> color = reader.nextString() 1 -> radiusKm = reader.nextLong() else -> { reader.skipName() reader.skipValue() } } } reader.endObject() return Planet(color, radiusKm) }

Slide 78

Slide 78 text

val FIELD_NAMES = JsonReader.Options.of("color", "radius_km") fun readPlanet(reader: JsonReader): Planet { var color: String? = null var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { when (reader.selectName(FIELD_NAMES)) { 0 -> color = reader.nextString() 1 -> radiusKm = reader.nextLong() else -> { reader.skipName() reader.skipValue() } } } reader.endObject() return Planet(color, radiusKm) }

Slide 79

Slide 79 text

val FIELD_NAMES = JsonReader.Options.of("color", "radius_km") fun readPlanet(reader: JsonReader): Planet { var color: String? = null var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { when (reader.selectName(FIELD_NAMES)) { 0 -> color = reader.nextString() 1 -> radiusKm = reader.nextLong() else -> { reader.skipName() reader.skipValue() } } } reader.endObject() return Planet(color, radiusKm) }

Slide 80

Slide 80 text

val FIELD_NAMES = JsonReader.Options.of("color", "radius_km") fun readPlanet(reader: JsonReader): Planet { var color: String? = null var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { when (reader.selectName(FIELD_NAMES)) { 0 -> color = reader.nextString() 1 -> radiusKm = reader.nextLong() else -> { reader.skipName() reader.skipValue() } } } reader.endObject() return Planet(color, radiusKm) }

Slide 81

Slide 81 text

val FIELD_NAMES = JsonReader.Options.of("color", "radius_km") fun readPlanet(reader: JsonReader): Planet { var color: String? = null var radiusKm: Long? = null reader.beginObject() while (reader.hasNext()) { when (reader.selectName(FIELD_NAMES)) { 0 -> color = reader.nextString() 1 -> radiusKm = reader.nextLong() else -> { reader.skipName() reader.skipValue() } } } reader.endObject() return Planet(color, radiusKm) }

Slide 82

Slide 82 text

Trees

Slide 83

Slide 83 text

Tree Models Jackson JsonNode, NullNode, MissingNode, POJONode Gson JsonElement, JsonPrimitive, JsonNull Moshi List, Map, String Kotlin Serialization JsonElement, JsonLiteral, JsonNull

Slide 84

Slide 84 text

Working with Trees • Lots of strings and casting • Two kinds of nulls • Makes me want types

Slide 85

Slide 85 text

Data Binding

Slide 86

Slide 86 text

Type Mapping JSON Kotlin / Java Boolean Boolean Number Byte, Short, Int, Long, Float, Double String String, Char, ByteString, Enums, Instant… Array Array, List, Set… Object Classes, Map

Slide 87

Slide 87 text

Adapters • Convert between Kotlin values and JSON values • Example: HttpUrl

Slide 88

Slide 88 text

Cycles class Planet( val name: String, val moons: List ) class Moon( val name: String, val orbits: Planet ) { "name": "Saturn", "moons": [ { "name": "Titan", "orbits": { "name": "Saturn", "moons": [ { "name": "Titan", "orbits": { "name": "Saturn", "moons": [ { "name": "Titan", ...

Slide 89

Slide 89 text

Cycles { "name": "Saturn", "moons": [ { "name": "Titan", "orbits": { "name": "Saturn", "moons": [ { "name": "Titan", "orbits": { "name": "Saturn", "moons": [ { "name": "Titan", ... class Planet( val name: String, val moons: List ) class Moon( val name: String, val orbits: String )

Slide 90

Slide 90 text

Cycles class Planet( val name: String, val moons: List ) class Moon( val name: String, val orbits: String ) { "name": "Saturn", "moons": [ "Titan", "Calypso", "Tethys", "Rhea", "Dione", "Enceladus", "Mimas" ] }

Slide 91

Slide 91 text

Subclasses sealed class CosmicObject { data class Star( val name: String, val radius_km: Long ) : CosmicObject() data class Planet( val name: String, val orbits_star: String ) : CosmicObject() data class Moon( val name: String, val orbits_planet: String ) : CosmicObject() } { "destination": { "name": "Alpha Centauri", "radius_km": 851120 }, "origin": { "name": "Saturn", "oribits_star": "Sun" } }

Slide 92

Slide 92 text

Subclasses val moshi = Moshi.Builder() .add(PolymorphicJsonAdapterFactory.of(CosmicObject::class.java, "type") .withSubtype(CosmicObject.Star::class.java, "star") .withSubtype(CosmicObject.Planet::class.java, "planet") .withSubtype(CosmicObject.Moon::class.java, "moon")) .build()

Slide 93

Slide 93 text

Subclasses sealed class CosmicObject { data class Star( val name: String, val radius_km: Long ) : CosmicObject() data class Planet( val name: String, val orbits_star: String ) : CosmicObject() data class Moon( val name: String, val orbits_planet: String ) : CosmicObject() } { "destination": { "name": "Alpha Centauri", "radius_km": 851120 }, "origin": { "name": "Saturn", "oribits_star": "Sun" } }

Slide 94

Slide 94 text

Subclasses { "destination": { "type": "star", "name": "Alpha Centauri", "radius_km": 851120 }, "origin": { "type": "planet", "name": "Saturn", "oribits_star": "Sun" } } sealed class CosmicObject { data class Star( val name: String, val radius_km: Long ) : CosmicObject() data class Planet( val name: String, val orbits_star: String ) : CosmicObject() data class Moon( val name: String, val orbits_planet: String ) : CosmicObject() }

Slide 95

Slide 95 text

class Planet { final String name; final double radius_km; String color = "unknown"; List moons = emptyList(); Planet( String name, double radiusKm) { this.name = name; this.radius_km = radiusKm; } ... } Reflection Magic val json = """ |{ | "name": "Mars", | "radius_km": 3389 |} """.trimMargin() val planet = adapter.fromJson(json)!! println(planet.name) // Mars println(planet.radius_km) // 3389.0 println(planet.color) // null println(planet.moons) // null

Slide 96

Slide 96 text

Reflection Magic val json = """ |{ | "name": "Mars", | "radius_km": 3389 |} """.trimMargin() val planet = adapter.fromJson(json)!! println(planet.name) // Mars println(planet.radius_km) // 3389.0 println(planet.color) // null println(planet.moons) // null @JsonClass(generateAdapter = true) class Planet( val name: String, val radius_km: Double ) { val color = "unknown" val moons = emptyList() }

Slide 97

Slide 97 text

Reflection Magic val json = """ |{ | "name": "Mars", | "radius_km": 3389 |} """.trimMargin() val planet = adapter.fromJson(json)!! println(planet.name) // Mars println(planet.radius_km) // 3389.0 println(planet.color) // unknown println(planet.moons) // [] @JsonClass(generateAdapter = true) class Planet( val name: String, val radius_km: Double ) { val color = "unknown" val moons = emptyList() }

Slide 98

Slide 98 text

Built-in Types data class Planet( val name: String, val surface_color: Color ) { "name" : "Saturn", "surface_color" : { "value" : -9677, "falpha" : 0 } }

Slide 99

Slide 99 text

Case Mapping val gson = GsonBuilder() .setFieldNamingPolicy(FieldNamingPolicy.LOWER_CASE_WITH_UNDERSCORES) .create()

Slide 100

Slide 100 text

I’m deleting old code. Is orbitsStar still used?

Slide 101

Slide 101 text

I’m deleting old code. Is orbitsStar still used? grep says no

Slide 102

Slide 102 text

Case Mapping val gson = GsonBuilder() .setFieldNamingPolicy(FieldNamingPolicy.LOWER_CASE_WITH_UNDERSCORES) .create()

Slide 103

Slide 103 text

Case Mapping data class Planet( val name: String, val orbitsStar: String, val radiusKm: Long, val moons: List ) { "name": "Saturn", "oribits_star": "Sun", "radius_km": 58232, "moons": [ "Titan", "Calypso" ] }

Slide 104

Slide 104 text

Case Mapping data class Planet( val name: String, @SerializedName("orbits_star") val orbitsStar: String, @SerializedName("radius_km") val radiusKm: Long, val moons: List ) { "name": "Saturn", "oribits_star": "Sun", "radius_km": 58232, "moons": [ "Titan", "Calypso" ] }

Slide 105

Slide 105 text

Case Mapping data class Planet( val name: String, val orbits_star: String, val radius_km: Long, val moons: List ) { "name": "Saturn", "oribits_star": "Sun", "radius_km": 58232, "moons": [ "Titan", "Calypso" ] }

Slide 106

Slide 106 text

Generating Code Jackson Runtime bytecode generator
 (No Android support) Gson None Moshi Annotation Processor Kotlin Serialization Kotlin Compiler Plugin

Slide 107

Slide 107 text

Gson Jackson Moshi Kotlin S. ? ? ?

Slide 108

Slide 108 text

Jackson 2.9.8 (2018-12-15) 21 feature releases Gson 2.8.5 (2018-05-21) 17 feature releases Moshi 1.8.0 (2018-11-09) 9 feature releases Kotlin Serialization 0.11.0 (2019-04-12) pre-release Latest Release

Slide 109

Slide 109 text

Jackson 4,370 KiB includes kotlin-reflect (2,584 KiB) Gson 239 KiB Moshi 330 KiB + generated code includes Okio (178 KiB) Kotlin Serialization 762 KiB + generated code Size Before Shrinking

Slide 110

Slide 110 text

Jackson Fast stream encoder + decoder Allocation avoidance pool Gson Fast stream encoder + decoder Moshi Fast stream encoder + decoder Select trie Generated Kotlin adapters Kotlin Serialization Generated Kotlin adapters Performance Features

Slide 111

Slide 111 text

APIs

Slide 112

Slide 112 text

var mapper = jacksonObjectMapper().apply { disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES) } @Test fun decodeJson() { val inputStream: InputStream = ... val searchResponseBody = inputStream.use { mapper.readValue(it) } ... } Jackson

Slide 113

Slide 113 text

val gson = GsonBuilder().create() @Test fun decodeJson() { val reader: Reader = ... val searchResponseBody = reader.use { gson.fromJson(it, SearchResponseBody::class.java) } ... } Gson

Slide 114

Slide 114 text

val moshi = Moshi.Builder().build() val adapter = moshi.adapter(SearchResponseBody::class.java) @Test fun decodeJson() { val source: BufferedSource = ... val searchResponseBody = source.use { adapter.fromJson(it)!! } ... } Moshi

Slide 115

Slide 115 text

val json = Json(JsonConfiguration(strictMode = false)) @Test fun decodeJson() { val string: String = ... val searchResponseBody = json.parse( SearchResponseBody.serializer(), string) ... } Kotlin Serialization

Slide 116

Slide 116 text

Custom Adapters

Slide 117

Slide 117 text

object HttpUrlSerializer : StdSerializer(HttpUrl::class.java) { override fun serialize( value: HttpUrl, generator: JsonGenerator, provider: SerializerProvider ) { generator.writeString(value.toString()) } } object HttpUrlDeserializer : StdDeserializer(HttpUrl::class.java) { override fun deserialize( parser: JsonParser, context: DeserializationContext? ): HttpUrl { val text = parser.getValueAsString() return HttpUrl.get(text) } } var mapper = jacksonObjectMapper().apply { registerModule(SimpleModule().apply { addSerializer(HttpUrl::class.java, HttpUrlSerializer) addDeserializer(HttpUrl::class.java, HttpUrlDeserializer) }) } Jackson

Slide 118

Slide 118 text

object HttpUrlTypeAdapter : TypeAdapter() { override fun write(writer: JsonWriter, value: HttpUrl) { writer.value(value.toString()) } override fun read(reader: JsonReader): HttpUrl { val string = reader.nextString() return HttpUrl.get(string) } } val gson = GsonBuilder() .registerTypeAdapter(HttpUrl::class.java, HttpUrlTypeAdapter) .create() Gson

Slide 119

Slide 119 text

object CustomAdapters { @ToJson fun httpUrlToJson(httpUrl: HttpUrl): String { return httpUrl.toString() } @FromJson fun httpUrlFromJson(string: String): HttpUrl { return HttpUrl.get(string) } } val moshi = Moshi.Builder() .add(CustomAdapters) .build() Moshi

Slide 120

Slide 120 text

@Serializer(forClass = HttpUrl::class) object HttpUrlSerializer : KSerializer { override fun serialize(encoder: Encoder, httpUrl: HttpUrl) { encoder.encodeString(httpUrl.toString()) } override fun deserialize(decoder: Decoder): HttpUrl { val string = decoder.decodeString() return HttpUrl.get(string) } } @file:UseSerializers( serializerClasses = [HttpUrlSerializer::class]
 ) Kotlin Serialization

Slide 121

Slide 121 text

Usability

Slide 122

Slide 122 text

Jackson "Barnes & Noble" Gson "Barnes \u0026 Noble" Moshi "Barnes & Noble" Kotlin Serialization "Barnes & Noble" Special Characters

Slide 123

Slide 123 text

Jackson "Barnes & Noble" Gson "Barnes \u0026 Noble" Moshi "Barnes & Noble" Kotlin Serialization "Barnes & Noble" Special Characters

Slide 124

Slide 124 text

Jackson com.fasterxml.jackson.databind.exc.InvalidFormatException: Cannot deserialize value of type `long` from String "whoops": not a valid Long value at [Source: (BufferedInputStream); line: 302, column: 18] (through reference chain: com.jsonexplained.SearchResponseBody["response"]
 ->com.jsonexplained.SearchResponse["hits"]
 ->java.util.ArrayList[8]
 ->com.jsonexplained.Hit["result"]
 ->com.jsonexplained.SongResult["id"]) Gson com.google.gson.JsonSyntaxException: java.lang.NumberFormatException: For input string: "whoops" Moshi com.squareup.moshi.JsonDataException: Expected a long but was whoops at path $.response.hits[8].result.id Kotlin Serialization java.lang.NumberFormatException: For input string: "whoops" Type Error

Slide 125

Slide 125 text

Jackson com.fasterxml.jackson.databind.exc.InvalidFormatException: Cannot deserialize value of type `long` from String "whoops": not a valid Long value at [Source: (BufferedInputStream); line: 302, column: 18] (through reference chain: com.jsonexplained.SearchResponseBody["response"]
 ->com.jsonexplained.SearchResponse["hits"]
 ->java.util.ArrayList[8]
 ->com.jsonexplained.Hit["result"]
 ->com.jsonexplained.SongResult["id"]) Gson com.google.gson.JsonSyntaxException: java.lang.NumberFormatException: For input string: "whoops" Moshi com.squareup.moshi.JsonDataException: Expected a long but was whoops at path $.response.hits[8].result.id Kotlin Serialization java.lang.NumberFormatException: For input string: "whoops" Type Error

Slide 126

Slide 126 text

Jackson Success Gson Success Moshi Success Kotlin Serialization java.lang.NumberFormatException: For input string: "1.0" Type Acceptance

Slide 127

Slide 127 text

Jackson Success Gson Success Moshi Success Kotlin Serialization java.lang.NumberFormatException: For input string: "1.0" Type Acceptance

Slide 128

Slide 128 text

Jackson Long: completed normally, used 0
 String: crashed with a helpful message Gson Long: completed normally, used 0
 String: completed normally, used null Moshi com.squareup.moshi.JsonDataException: Required property 'id' missing at $.response.hits[8].result Kotlin Serialization kotlinx.serialization.MissingFieldException: Field 'id' is required, but it was missing Required Field Is Absent

Slide 129

Slide 129 text

Jackson Long: completed normally, used 0
 String: crashed with a helpful message Gson Long: completed normally, used 0
 String: completed normally, used null Moshi com.squareup.moshi.JsonDataException: Required property 'id' missing at $.response.hits[8].result Kotlin Serialization kotlinx.serialization.MissingFieldException: Field 'id' is required, but it was missing Required Field Is Absent

Slide 130

Slide 130 text

Jackson com.fasterxml.jackson.databind.exc.MismatchedInputException: No content to map due to end-of-input at [Source: (BufferedInputStream); line: 1, column: 0] Gson Completed normally, returned null Moshi java.io.EOFException: End of input Kotlin Serialization kotlinx.serialization.json.JsonParsingException: Invalid JSON at 0: Expected '{, kind: kotlinx.serialization.StructureKind$CLASS@221af3c0' Document is Empty

Slide 131

Slide 131 text

Jackson com.fasterxml.jackson.databind.exc.MismatchedInputException: No content to map due to end-of-input at [Source: (BufferedInputStream); line: 1, column: 0] Gson Completed normally, returned null Moshi java.io.EOFException: End of input Kotlin Serialization kotlinx.serialization.json.JsonParsingException: Invalid JSON at 0: Expected '{, kind: kotlinx.serialization.StructureKind$CLASS@221af3c0' Document is Empty

Slide 132

Slide 132 text

Jackson Completes normally, encoding implementation details of types like CompletableFuture Gson Completes normally, encoding implementation details of types like CompletableFuture Moshi Fails fast with an error Kotlin Serialization Fails fast with an error Write an Unexpected Type

Slide 133

Slide 133 text

Jackson Completes normally, encoding implementation details of types like CompletableFuture Gson Completes normally, encoding implementation details of types like CompletableFuture Moshi Fails fast with an error Kotlin Serialization Fails fast with an error Write an Unexpected Type

Slide 134

Slide 134 text

Jackson com.fasterxml.jackson.databind.JsonMappingException: Numeric value (2147483648) out of range of int at [Source: (BufferedInputStream); line: 15, column: 32] (through reference chain: com.jsonexplained.SearchResponseBody["response"]
 ->com.jsonexplained.SearchResponse["hits"]
 ->java.util.ArrayList[0]
 ->com.jsonexplained.Hit["result"]
 ->com.jsonexplained.SongResult["annotation_count"]) Gson com.google.gson.JsonSyntaxException: java.lang.NumberFormatException: Expected an int but was 2147483648 at line 15 column 42 path $.response.hits[0].result.annotation_count Moshi com.squareup.moshi.JsonDataException: Expected an int but was 2147483648 at path $.response.hits[0].result.annotation_count Kotlin Serialization java.lang.NumberFormatException: For input string: "2147483648" Too Large For An Int

Slide 135

Slide 135 text

Jackson com.fasterxml.jackson.databind.JsonMappingException: Numeric value (2147483648) out of range of int at [Source: (BufferedInputStream); line: 15, column: 32] (through reference chain: com.jsonexplained.SearchResponseBody["response"]
 ->com.jsonexplained.SearchResponse["hits"]
 ->java.util.ArrayList[0]
 ->com.jsonexplained.Hit["result"]
 ->com.jsonexplained.SongResult["annotation_count"]) Gson com.google.gson.JsonSyntaxException: java.lang.NumberFormatException: Expected an int but was 2147483648 at line 15 column 42 path $.response.hits[0].result.annotation_count Moshi com.squareup.moshi.JsonDataException: Expected an int but was 2147483648 at path $.response.hits[0].result.annotation_count Kotlin Serialization java.lang.NumberFormatException: For input string: "2147483648" Too Large For An Int

Slide 136

Slide 136 text

Gson MOST USED Jackson MOST CAPABLE Kotlin S. BEST FOR MULTI
 PLATFORM Moshi BEST API

Slide 137

Slide 137 text

Testing

Slide 138

Slide 138 text

Testing? • Confirm you’ve configured things properly • Give you confidence to refactor & make changes • Get your server team to code review!

Slide 139

Slide 139 text

@Test fun json() { val planet = Planet("Mercury", 2439, listOf("Messenger")) val json = """ |{ | "name": "Mercury", | "radius_km": 2439, | "visitors": [ | "Messenger" | ] |} """.trimMargin() val adapter = moshi.adapter(Planet::class.java).indent(" ") assertThat(adapter.toJson(planet)).isEqualTo(json) assertThat(adapter.fromJson(json)).isEqualTo(planet) }

Slide 140

Slide 140 text

Optimizing

Slide 141

Slide 141 text

Use Less JSON • No pretty-printed JSON • Delete unused fields: fewer bytes to send, store, encode, decode • Omit nulls

Slide 142

Slide 142 text

Avoid Reflection • Enable generated code • Don’t create a new JsonAdapter every time

Slide 143

Slide 143 text

Stream! • Download & decode in parallel • Streaming uploads? Make sure your models are immutable!

Slide 144

Slide 144 text

JSON + Protobuf • Try it as a schema language • Read + write JSON • Option to do binary encoding instead

Slide 145

Slide 145 text

@jessewilson Thanks https://github.com/swankjesse/jsonexplained