Data classes and parsing JSON - A story about converting Models to Kotlin

Since the announcement of official Kotlin support at Google I/O we can be sure that Kotlin is here to stay and you can already feel the boost this language is getting. At Kitchen Stories we love this uprise and we are migrating our code base gradually to leverage the new features this language provides us.
In this session, I want to take you along the journey on how and why we migrated all our POJOs to POKOs. You will learn how Kotlins data classes, constructor default values and non-nullability can enable a more robust JSON parsing and how you can improve parsing with Moshi.

Some more context to the slides can be found here:
https://engineering.kitchenstories.io/caf8a599df9e

The Live-Code-Example is available here:
https://github.com/stefanmedack/MoshiKotlinExample

Transcript

1. Data classes and parsing JSON - A story about converting Models to Kotlin Stefan Medack Zonic03
3. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) How to make your PO happy What we have: ▸ API that can not be (easily) changed ▸ JSON can be different (to your expectations) What we want: ▸ Less content related crashes ▸ get rid of tedious, nested null checks ▸ filter „broken“ content
4. Quiz Time How many different ways are there to define a String field in JSON?
5. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) { "field": "has some text" }
6. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) { „field": null }
7. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) { // field is omitted from JSON }
8. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) { "id": "article_1", "title": "Data classes and parsing JSON", "subtitle": "A story about converting Models to Kotlin", "cell_image": { "url": "https://images.kitchenstories.de/example.jpg" }, "author": { "username": "Stefan Medack" }, "comments": [ ... ], "content": [ ... ] ... }
9. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) public class Article { public String id; public String title; public Image cell_image; public User author; public Video video; public String subtitle; public List<Content> content; public int like_count; public List<Comment> comments; public Date published_at; // Constructors, Getters, Setters, ... }
10. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) data class Article( val id: String, val title: String, val cell_image: Image, val author: User, val video: Video? = null, val subtitle: String = EMPTY_STRING, val content: List<Content> = listOf(), val like_count: Int = 0, val comments: List<Comment> = listOf(), val published_at: Date = Date(0) )
11. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) data class Article( val id: String, val title: String, val cell_image: Image, val author: User, val video: Video? = null, val subtitle: String = EMPTY_STRING, val content: List<Content> = listOf(), val like_count: Int = 0, val comments: List<Comment> = listOf(), val published_at: Date = Date(0) )
12. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) data class Article( val id: String, val title: String, val image: Image, val author: User, val video: Video? = null, val subtitle: String = EMPTY_STRING, val content: List<Content> = listOf(), val likeCount: Int = 0, val comments: List<Comment> = listOf(), val publishedAt: Date = Date(0) )
13. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) data class Article( val id: String, val title: String, @Json(name = "cell_image") val image: Image, val author: User, val video: Video? = null, val subtitle: String = EMPTY_STRING, val content: List<Content> = listOf(), @Json(name = "like_count") val likeCount: Int = 0, val comments: List<Comment> = listOf(), @Json(name = "published_at") val publishedAt: Date = Date(0) )
14. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) data class Article( val id: String, val title: String, val image: Image, val author: User, val video: Video? = null, val subtitle: String = EMPTY_STRING, val content: List<Content> = listOf(), val likeCount: Int = 0, val comments: List<Comment> = listOf(), val publishedAt: Date = Date(0) )
15. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) data class Article( val id: String, val title: String, val image: Image, val author: User, val video: Video? = null, val subtitle: String = EMPTY_STRING, val content: List<Content> = listOf(), val likeCount: Int = 0, val comments: List<Comment> = listOf(), val publishedAt: Date = Date(0) )
16. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) data class Article( val id: String, val title: String, val image: Image, val author: User, val video: Video? = null, val subtitle: String = EMPTY_STRING, val content: List<Content> = listOf(), val likeCount: Int = 0, val comments: List<Comment> = listOf(), val publishedAt: Date = Date(0) )
17. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) data class Article( val id: String, val title: String, val image: Image, val author: User, val video: Video? = null, val subtitle: String = EMPTY_STRING, val content: List<Content> = listOf(), val likeCount: Int = 0, val comments: List<Comment> = listOf(), val publishedAt: Date = Date(0) )
18. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) data class Article( val id: String, val title: String, val image: Image, val author: User, val video: Video? = null, val subtitle: String = EMPTY_STRING, val content: List<Content> = listOf(), val likeCount: Int = 0, val comments: List<Comment> = listOf(), val publishedAt: Date = Date(0) )
19. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) data class Article( val id: String, val title: String, val image: Image, val author: User, val video: Video? = null, val subtitle: String = EMPTY_STRING, val content: List<Content> = listOf(), val likeCount: Int = 0, val comments: List<Comment> = listOf(), val publishedAt: Date = Date(0) )
20. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) data class Article( val id: String, val title: String, val image: Image, val author: User, val video: Video? = null, val subtitle: String = EMPTY_STRING, val content: List<Content> = listOf(), val likeCount: Int = 0, val comments: List<Comment> = listOf(), val publishedAt: Date = Date(0) )
21. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) JSON <-> Data Class ▸ Kotson ▸ Jackson ▸ Moshi
22. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) How to setup Moshi 1
23. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) How to setup Moshi implementation „com.squareup.moshi:moshi:1.6.0“ // Version A) implementation „com.squareup.moshi:moshi-kotlin:1.6.0“ // adds kotlin.reflect library !
24. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) How to setup Moshi implementation „com.squareup.moshi:moshi:1.6.0“ // Version A) implementation „com.squareup.moshi:moshi-kotlin:1.6.0“ // Version B) Kapt „com.squareup.moshi:moshi-kotlin-codegen:1.6.0“
25. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) How to setup Moshi 2
26. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) How to setup Moshi val moshi = Moshi.Builder() // "... here be adapters# .build()
27. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) How to setup Moshi - with reflection val moshi = Moshi.Builder() // "... here be adapters# .add(KotlinJsonAdapterFactory()) //constructor default values$ .build() https://github.com/square/moshi/issues/315
28. OR… use version B (B stands for BETTER! %)
29. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) How to setup Moshi - with code generation data class Article( val title: String, val image: Image, // … ) val moshi = Moshi.Builder() // no Kotlin Adapter needed .build()
30. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) How to setup Moshi - with code generation @JsonClass(generateAdapter = true) data class Article( val title: String, val image: Image, // … ) val moshi = Moshi.Builder() // no Kotlin Adapter needed .build()
31. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) How to setup Moshi - with code generation @JsonClass(generateAdapter = true) data class Article( val title: String, val image: Image, // … ) val moshi = Moshi.Builder() // no Kotlin Adapter needed .build() Step 3) use Moshi with Retrofit https://proandroiddev.com/a69c2621708b
32. But is it the end??? sadly, no
33. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) Let’s test https://github.com/stefanmedack/MoshiKotlinExample
34. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) Let’s test ▸ What is Date ! -> Crash & ▸ Not all mandatory fields are present? ! -> Crash & ▸ There can be nulls in non-nullable lists? ! -> Crash &
35. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) val moshi = Moshi.Builder() // "... here be adapters# .build()
36. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) val moshi = Moshi.Builder() .add( Date::class.java, Rfc3339DateJsonAdapter().nullSafe() ) .build()
37. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) Let’s test ▸ What is Date ! -> Successful ✅ ▸ Not all mandatory fields are present? ! -> Crash & ▸ There can be nulls in non-nullable lists? ! -> Crash &
38. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) Let’s test ▸ What is Date ! -> Successful ✅ ▸ Not all mandatory fields are present? ! -> Crash & ▸ There can be nulls in non-nullable lists? ! -> Crash &
39. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) @JsonClass(generateAdapter = true) data class Article( val id: String, val title: String, val image: Image, val author: User, val video: Video? = null, val subtitle: String = EMPTY_STRING, val content: List<Content> = listOf(), val likeCount: Int = 0, val comments: List<Comment> = listOf(), val publishedAt: Date = Date(0) )
40. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) class DefaultOnDataMismatchAdapter<T> private constructor( private val delegate: JsonAdapter<T>, private val defaultValue: T? ) : JsonAdapter<T>() { @Throws(IOException::class) override fun fromJson(reader: JsonReader): T? = try { delegate.fromJsonValue(reader.readJsonValue()) } catch (e: Exception) { Timber.w("Wrongful content - could not parse delegate $delegate") defaultValue } @Throws(IOException::class) override fun toJson(writer: JsonWriter, value: T?) { delegate.toJson(writer, value) } companion object { @JvmStatic fun <T> newFactory(type: Class<T>, defaultValue: T?): JsonAdapter.Factory { return object : JsonAdapter.Factory { override fun create(requestedType: Type, annotations: Set<Annotation>, moshi: Moshi): JsonAdapter<*>? { if (type != requestedType) { return null } val delegate = moshi.nextAdapter<T>(this, type, annotations) return DefaultOnDataMismatchAdapter(delegate, defaultValue) } } } } } https://gist.github.com/stefanmedack/719fd8d905c6be982e1b4f807cbda3fa
41. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) class DefaultOnDataMismatchAdapter<T> private constructor( private val delegate: JsonAdapter<T>, private val defaultValue: T? ) : JsonAdapter<T>() { @Throws(IOException::class) override fun fromJson(reader: JsonReader): T? = try { delegate.fromJsonValue(reader.readJsonValue()) } catch (e: Exception) { Timber.w("Wrongful content - could not parse delegate $delegate") defaultValue } @Throws(IOException::class) override fun toJson(writer: JsonWriter, value: T?) { delegate.toJson(writer, value) } } https://gist.github.com/stefanmedack/719fd8d905c6be982e1b4f807cbda3fa
42. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) class DefaultOnDataMismatchAdapter<T> private constructor( private val delegate: JsonAdapter<T>, private val defaultValue: T? ) : JsonAdapter<T>() { @Throws(IOException::class) override fun fromJson(reader: JsonReader): T? = try { delegate.fromJsonValue(reader.readJsonValue()) } catch (e: Exception) { Timber.w("Wrongful content - could not parse delegate $delegate") defaultValue } @Throws(IOException::class) override fun toJson(writer: JsonWriter, value: T?) { delegate.toJson(writer, value) } } https://gist.github.com/stefanmedack/719fd8d905c6be982e1b4f807cbda3fa
43. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) val moshi = Moshi.Builder() // "... here be adapters# .add(KotlinJsonAdapterFactory()) .build()
44. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) val moshi = Moshi.Builder() // "... here be adapters# .add( DefaultOnDataMismatchAdapter.newFactory( Article::class.java, null ) ) .add(KotlinJsonAdapterFactory()) .build()
45. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) val moshi = Moshi.Builder() // "... here be adapters# .add( DefaultOnDataMismatchAdapter.newFactory( AspectRatio::class.java, AspectRatio.unknown ) ) .add(KotlinJsonAdapterFactory()) .build()
46. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) val moshi = Moshi.Builder() // "... here be adapters# .add( DefaultOnDataMismatchAdapter.newFactory( ArticlePage::class.java, ArticlePage(…) ) ) .add(KotlinJsonAdapterFactory()) .build()
47. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) Let’s test ▸ What is Date ! -> Successful ✅ ▸ Not all mandatory fields are present? ! -> Successful ✅ ▸ filter general parsing errors ▸ more robust parsing ▸ There can be nulls in non-nullable lists? ! -> Crash &
48. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) Let’s test ▸ What is Date ! -> Successful ✅ ▸ Not all mandatory fields are present? ! -> Successful ✅ ▸ filter general parsing errors ▸ more robust parsing ▸ There can be nulls in non-nullable lists?* ! -> Crash & *https://github.com/square/moshi/issues/331
49. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) @JsonClass(generateAdapter = true) data class Article( val id: String, val title: String, val cell_image: Image, val author: User, val video: Video? = null, val subtitle: String = EMPTY_STRING, val content: List<Content> = listOf(), val like_count: Int = 0, val comments: List<Comment> = listOf(), val published_at: Date = Date(0) )
50. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) @JsonClass(generateAdapter = true) data class Article( val id: String, val title: String, val cell_image: Image, val author: User, val video: Video? = null, val subtitle: String = EMPTY_STRING, val content: List<Content?> = listOf(), val like_count: Int = 0, val comments: List<Comment?> = listOf(), val published_at: Date = Date(0) )
51. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) @JsonClass(generateAdapter = true) data class Article( val id: String, val title: String, val cell_image: Image, val author: User, val video: Video? = null, val subtitle: String = EMPTY_STRING, val content: List<Content!> = listOf(), val like_count: Int = 0, val comments: List<Comment!> = listOf(), val published_at: Date = Date(0) )
52. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) class FilterNullValuesFromListAdapter<T : Any> private constructor( private val delegate: JsonAdapter<List<T?>> ) : JsonAdapter<List<T>>() { @Throws(IOException::class) override fun fromJson(reader: JsonReader): List<T> = delegate.fromJsonValue(reader.readJsonValue())?.filterNotNull() ?: listOf() @Throws(IOException::class) override fun toJson(writer: JsonWriter, value: List<T>?) { delegate.toJson(writer, value) } companion object { @JvmStatic fun <T : Any> newFactory(type: Class<T>): JsonAdapter.Factory { return object : JsonAdapter.Factory { override fun create(requestedType: Type,annotations: Set<Annotation>,moshi: Moshi): JsonAdapter<*>? { val listType = Types.newParameterizedType(List::class.java, type) if (listType != requestedType) { return null } val delegate = moshi.nextAdapter<List<T?>>(this, listType, annotations) return FilterNullValuesFromListAdapter(delegate) } } } } } https://gist.github.com/stefanmedack/005971818edbf56e059f121cd4908c76
53. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) class FilterNullValuesFromListAdapter<T : Any> private constructor( private val delegate: JsonAdapter<List<T?>> ) : JsonAdapter<List<T>>() { @Throws(IOException::class) override fun fromJson(reader: JsonReader): List<T> = delegate.fromJsonValue(reader.readJsonValue()) ?.filterNotNull() ?: listOf() @Throws(IOException::class) override fun toJson(writer: JsonWriter, value: List<T>?) { delegate.toJson(writer, value) } } https://gist.github.com/stefanmedack/005971818edbf56e059f121cd4908c76
54. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) class FilterNullValuesFromListAdapter<T : Any> private constructor( private val delegate: JsonAdapter<List<T?>> ) : JsonAdapter<List<T>>() { @Throws(IOException::class) override fun fromJson(reader: JsonReader): List<T> = delegate.fromJsonValue(reader.readJsonValue()) ?.filterNotNull() ?: listOf() @Throws(IOException::class) override fun toJson(writer: JsonWriter, value: List<T>?) { delegate.toJson(writer, value) } } https://gist.github.com/stefanmedack/005971818edbf56e059f121cd4908c76
55. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) class FilterNullValuesFromListAdapter<T : Any> private constructor( private val delegate: JsonAdapter<List<T?>> ) : JsonAdapter<List<T>>() { @Throws(IOException::class) override fun fromJson(reader: JsonReader): List<T> = delegate.fromJsonValue(reader.readJsonValue()) ?.filterNotNull() ?: listOf() @Throws(IOException::class) override fun toJson(writer: JsonWriter, value: List<T>?) { delegate.toJson(writer, value) } } https://gist.github.com/stefanmedack/005971818edbf56e059f121cd4908c76
56. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) class FilterNullValuesFromListAdapter<T : Any> private constructor( private val delegate: JsonAdapter<List<T?>> ) : JsonAdapter<List<T>>() { @Throws(IOException::class) override fun fromJson(reader: JsonReader): List<T> = delegate.fromJsonValue(reader.readJsonValue()) ?.filterNotNull() ?: listOf() @Throws(IOException::class) override fun toJson(writer: JsonWriter, value: List<T>?) { delegate.toJson(writer, value) } } https://gist.github.com/stefanmedack/005971818edbf56e059f121cd4908c76
57. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) val moshi = Moshi.Builder() // "... here be adapters# .add(KotlinJsonAdapterFactory()) .build()
58. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) val moshi = Moshi.Builder() .add( FilterNullValuesFromListAdapter.newFactory( Content::class.java ) ) .add(KotlinJsonAdapterFactory()) .build()
59. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) val moshi = Moshi.Builder() .add( FilterNullValuesFromListAdapter.newFactory( Comment::class.java ) ) .add(KotlinJsonAdapterFactory()) .build()
60. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) Let’s test ▸ What is Date ! -> Successful ✅ ▸ Not all mandatory fields are present? ! -> Successful ✅ ▸ There is null in the list? ! -> Successful ✅
61. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) TL;FA ▸ Kotlin is fun ▸ Immutable, non-nullable and constructor default values ▸ Parse JSON into Data Classes with Moshi ▸ Filter „broken“ content from bottom up ▸ Workaround for List-Bug in Moshi ▸ maybe not suited for very big set of models
62. Data Classes and Parsing JSON - Stefan Medack (@Zonic03) Links ▸ Blogpost: https://engineering.kitchenstories.io/caf8a599df9e ▸ MoshiKotlinExample: https://github.com/stefanmedack/MoshiKotlinExample ▸ Moshi: https://github.com/square/moshi ▸ Moshi with Kotlin: https://medium.com/square-corner-blog/fcd6ef99256b ▸ Exploring Moshi’s Kotlin Code Gen: https://medium.com/@sweers/dec09d72de5e ▸ why to include moshi-kotlin: https://github.com/square/moshi/issues/315 ▸ discussion on using kotlin-reflect: https://github.com/square/moshi/issues/307 ▸ Moshi Bug nulls in non-nullable lists: https://github.com/square/moshi/issues/331 ▸ Adapter1: https://gist.github.com/stefanmedack/719fd8d905c6be982e1b4f807cbda3fa ▸ Adapter2: https://gist.github.com/stefanmedack/005971818edbf56e059f121cd4908c76
63. We are hiring! Deutsch - kitchenstories.io/de/jobs-de English - kitchenstories.io/en/jobs
64. Thank you for listening!
 
 Questions? Stefan Medack Zonic03 https://engineering.kitchenstories.io/ caf8a599df9e