Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Kotlin @ Coupang Backend 2017

Kotlin @ Coupang Backend 2017

Experience for adopting Kotlin to Coupang Catalog backend system.

Sunghyouk Bae

April 23, 2018
Tweet

More Decks by Sunghyouk Bae

Other Decks in Programming

Transcript

  1. Kotlin Backend @ Coupang
    Spring Camp 2018

    [email protected]

    View full-size slide

  2. Who is @debop
    • Since 1993

    • Robotics, BPM Solution, Healthcare

    • C/C++, Object Pascal (Delphi), C#, Java, Scala, Kotlin

    • Use Kotlin since 2016.08

    • Sr. Principle Software Engineer in Coupang (2017~)

    View full-size slide

  3. Agenda
    • Kotlin - The Good, The Bad, The Ugly

    • Kotlin Coroutines

    • Kotlin Backend 도입 전략 / 사례

    • Lesson & Learn

    • Resources

    View full-size slide

  4. Kotlin
    The Good, The Bad and The Ugly

    View full-size slide

  5. Kotlin - The Good (1/3)
    • Java Interoperability (@JvmXXXX)

    • Familiar Syntax (like Scala, C#, Groovy, Swift …)

    • Type Inference (like Scala, C#)

    • Smart Casts (`is`, `as`)

    • Data Class (Scala case class, POJO with @Data)

    View full-size slide

  6. Kotlin - The Good (1/3)
    when (x) {
    is Int -> print(x + 1)
    is String -> print(x.length + 1)
    is IntArray -> print(x.sum())
    !is BigDecimal -> print(x)
    1L..10L -> print(x)
    }
    val x: String? = y as String (Unsafe)
    val x: String? = y as String? (Unsafe)
    val x: String? = y as? String (Safe)
    is / !is
    as / as?

    View full-size slide

  7. Kotlin - The Good (1/3)
    public class User {
    private String name;
    private Integer age;
    public User(String name) {
    this(name, 0);
    }
    public User(String name, Integer age) {
    this.name = name;
    this.age = age;
    }
    public String getName() {return this.name;}
    public Integer getAge() {return this.age;}
    public void setName(String name) {this.name = name; }
    public void setAge(Integer age) {this.age = age; }
    public boolean equals(Object o) { /* ࢤۚ */ }
    public int hashCode() {
    return Objects.hash(name, age);
    }
    protected boolean canEqual(Object other) {return other instanceof User;}
    public String toString() {return "User(name=" + this.getName() + ", age=" + this.getAge() + ")";}
    }

    View full-size slide

  8. Kotlin - The Good (1/3)
    public class User {
    private String name;
    private Integer age;
    public User(String name) {
    this(name, 0);
    }
    public User(String name, Integer age) {
    this.name = name;
    this.age = age;
    }
    public String getName() {return this.name;}
    public Integer getAge() {return this.age;}
    public void setName(String name) {this.name = name; }
    public void setAge(Integer age) {this.age = age; }
    public boolean equals(Object o) { /* ࢤۚ */ }
    public int hashCode() {
    return Objects.hash(name, age);
    }
    protected boolean canEqual(Object other) {return other instanceof User;}
    public String toString() {return "User(name=" + this.getName() + ", age=" + this.getAge() + ")";}
    }
    data class User(var name:String) {
    var age:Int = 0
    }

    View full-size slide

  9. Kotlin - The Good (2/3)
    • Default Arguments

    • String Templates (Scala - string interpolation)

    • Properties (C#, No Getter, Setter)

    View full-size slide

  10. val tomAge = map["tom"] ?: 0
    map["jane"] = 29
    val query = """
    | CREATE KEYSPACE IF NOT EXISTS $keyspaceName
    | WITH replication = { 'class': '$replicationStrategy',
    | 'replication_factor':$replicationFactor }
    """.trimMargin()
    val s = "abc"
    val str = "$s.length is ${s.length}" // prints "abc.length is 3"
    Kotlin - The Good (2/3)
    fun read(b:ByteArray, offset:Int = 0, length:Int = b.size) {
    TODO(”Not Implemented”)
    }

    View full-size slide

  11. val tomAge = map["tom"] ?: 0
    map["jane"] = 29
    val query = """
    | CREATE KEYSPACE IF NOT EXISTS $keyspaceName
    | WITH replication = { 'class': '$replicationStrategy',
    | 'replication_factor':$replicationFactor }
    """.trimMargin()
    val s = "abc"
    val str = "$s.length is ${s.length}" // prints "abc.length is 3"
    Kotlin - The Good (2/3)
    fun read(b:ByteArray, offset:Int = 0, length:Int = b.size) {
    TODO(”Not Implemented”)
    }

    View full-size slide

  12. val tomAge = map["tom"] ?: 0
    map["jane"] = 29
    val query = """
    | CREATE KEYSPACE IF NOT EXISTS $keyspaceName
    | WITH replication = { 'class': '$replicationStrategy',
    | 'replication_factor':$replicationFactor }
    """.trimMargin()
    val s = "abc"
    val str = "$s.length is ${s.length}" // prints "abc.length is 3"
    Kotlin - The Good (2/3)
    fun read(b:ByteArray, offset:Int = 0, length:Int = b.size) {
    TODO(”Not Implemented”)
    }

    View full-size slide

  13. val tomAge = map["tom"] ?: 0
    map["jane"] = 29
    val query = """
    | CREATE KEYSPACE IF NOT EXISTS $keyspaceName
    | WITH replication = { 'class': '$replicationStrategy',
    | 'replication_factor':$replicationFactor }
    """.trimMargin()
    val s = "abc"
    val str = "$s.length is ${s.length}" // prints "abc.length is 3"
    Kotlin - The Good (2/3)
    fun read(b:ByteArray, offset:Int = 0, length:Int = b.size) {
    TODO(”Not Implemented”)
    }

    View full-size slide

  14. Kotlin - The Good (3/3)
    • Destructuring Declarations

    • Class, Property Delegates

    • Extension Functions (C#)

    • Null Safety (C#)

    • Better Lambdas than Java (same Scala)

    • Scala functional features (use arrow)

    • Coroutines (kotlinx-coroutines)

    View full-size slide

  15. Kotlin - The Good (3/3)
    fun Int.millis(): Duration = Duration.ofMillis(this.toLong())
    fun Int.seconds(): Duration = Duration.ofSeconds(this.toLong())
    // 5.seconds() + 400.millis() —> 5400 msec
    if (text != null) text.length else -1
    text?.length ?: -1
    suspend fun Period.daySequence(): Sequence = buildSequence {
    var day = 0
    val days = [email protected]
    while (day < days) {
    yield(day++)
    }
    }
    }

    View full-size slide

  16. Kotlin - The Good (3/3)
    if(command!=null) {
    if(command.getThreadPoolProperties()!=null) {
    if(command.getThreadPoolProperties().maxQueueSize()!=null) {
    val x = command.getThreadPoolProperties().maxQueueSize().get()
    }
    }
    }

    View full-size slide

  17. val x = command?.getThreadPoolProperties()?.maxQueueSize()?.get() ?: 0
    Kotlin - The Good (3/3)
    if(command!=null) {
    if(command.getThreadPoolProperties()!=null) {
    if(command.getThreadPoolProperties().maxQueueSize()!=null) {
    val x = command.getThreadPoolProperties().maxQueueSize().get()
    }
    }
    }

    View full-size slide

  18. Compile time

    View full-size slide

  19. Compile time
    Scala >>>>> Kotlin > Java

    View full-size slide

  20. Kotlin - The Bad
    • No namespace (package variable, methods)

    • No static modifier (use companion object)

    View full-size slide

  21. Kotlin - The Bad
    class OffHeapOutputStream(private var ohm: OffHeapMemory) : OutputStream() {
    companion object {
    private val bax = ByteArrayTool.Factory.create()
    @JvmField val EMPTY = OffHeapOutputStream(0)
    }
    }
    @file:JvmName("Localex")
    /** ૑੿ೠ Locale ੉ null ੉ݶ ӝࠄ Locale ਸ ߈ജ೤פ׮ */
    fun Locale?.orDefault(): Locale = this ?: Locale.getDefault()

    View full-size slide

  22. Kotlin - The Ugly
    • SAM conversion

    • Unit returning lambdas (Use in java)

    • Final by default (안정성? 성능? A bit about picking defaults)

    • Open: 50%, Final : 50%
    registerCallback(() -> {
    /** do stuff */
    return Unit.INSTANCE;
    })

    View full-size slide

  23. Killer feature - Coroutines
    • Readable, clean code

    • Escape Callback hell

    • Imperative programming

    • Not exclusive, Cooperative

    • Lightweight thread (fiber, goroutines)

    • Note: Experimental !!!

    View full-size slide

  24. @GetMapping("/composeasync2")
    DeferredResult asyncCompose2() {
    DeferredResult dr = new DeferredResult();
    ListenableFuture f1 = myService.async();
    f1.addCallback(res1 -> {
    ListenableFuture f2 = myService.async2(res1);
    f2.addCallback(res2 -> {
    ListenableFuture f3 = myService.async3(res2);
    f3.addCallback(res3 -> {
    dr.setResult(res3);
    }, e -> {
    dr.setErrorResult(e);
    });
    }, e -> {
    dr.setErrorResult(e);
    });
    }, e -> {
    dr.setErrorResult(e);
    });
    return dr;
    }

    View full-size slide

  25. @GetMapping("/composeasync2")
    DeferredResult asyncCompose2() {
    DeferredResult dr = new DeferredResult();
    ListenableFuture f1 = myService.async();
    f1.addCallback(res1 -> {
    ListenableFuture f2 = myService.async2(res1);
    f2.addCallback(res2 -> {
    ListenableFuture f3 = myService.async3(res2);
    f3.addCallback(res3 -> {
    dr.setResult(res3);
    }, e -> {
    dr.setErrorResult(e);
    });
    }, e -> {
    dr.setErrorResult(e);
    });
    }, e -> {
    dr.setErrorResult(e);
    });
    return dr;
    }
    Callback Hell

    View full-size slide

  26. lock.lockAsync(3000, TimeUnit.MILLISECONDS)
    .thenComposeAsync(rl -> updateProductItemMapping(productId, itemId))
    .thenComposeAsync(updated -> adultInverseIndex.fastPutAsync(itemId, productId))
    .thenComposeAsync(added -> addItemIdToList(productId, itemId))
    .whenCompleteAsync((result, error) -> lock.forceUnlock());
    CompletableFuture
    RxJava
    Observable
    .defer { Observable.just(timer.time()) }
    .flatMap { timerCtx ->
    external.doOnCompleted { timerCtx.stop() }
    }
    .subscribe { println(it) }

    View full-size slide

  27. C# Way
    async Task work() {
    Thread.sleep(200);
    return “done”;
    }
    async Task moreWork() {
    Console.WriteLine(“Work started”);
    var str = await work();
    Console.WriteLine($“Work completed: {str}”);
    }

    View full-size slide

  28. C# Way
    async Task work() {
    Thread.sleep(200);
    return “done”;
    }
    async Task moreWork() {
    Console.WriteLine(“Work started”);
    var str = await work();
    Console.WriteLine($“Work completed: {str}”);
    }

    View full-size slide

  29. C# Way
    async Task work() {
    Thread.sleep(200);
    return “done”;
    }
    async Task moreWork() {
    Console.WriteLine(“Work started”);
    var str = await work();
    Console.WriteLine($“Work completed: {str}”);
    }

    View full-size slide

  30. C# Way
    async Task work() {
    Thread.sleep(200);
    return “done”;
    }
    async Task moreWork() {
    Console.WriteLine(“Work started”);
    var str = await work();
    Console.WriteLine($“Work completed: {str}”);
    }

    View full-size slide

  31. Kotlin Way
    suspend fun work(): String {
    delay(200L)
    return "done"
    }
    fun moreWork(): Deferred = async {
    println("Work started")
    val str = work()
    println("Work completed. $str")
    }
    runBlocking {
    moreWork().await()
    }

    View full-size slide

  32. Kotlin Way
    suspend fun work(): String {
    delay(200L)
    return "done"
    }
    fun moreWork(): Deferred = async {
    println("Work started")
    val str = work()
    println("Work completed. $str")
    }
    runBlocking {
    moreWork().await()
    }

    View full-size slide

  33. Kotlin Way
    suspend fun work(): String {
    delay(200L)
    return "done"
    }
    fun moreWork(): Deferred = async {
    println("Work started")
    val str = work()
    println("Work completed. $str")
    }
    runBlocking {
    moreWork().await()
    }

    View full-size slide

  34. Kotlin Way
    suspend fun work(): String {
    delay(200L)
    return "done"
    }
    fun moreWork(): Deferred = async {
    println("Work started")
    val str = work()
    println("Work completed. $str")
    }
    runBlocking {
    moreWork().await()
    }

    View full-size slide

  35. Go Way
    runtime.GOMAXPROCS(runtime.NumCPU())
    wg:= new(sync.WaitGroup)
    for i:=0; i < 100000; i++ {
    wg.Add(1)
    go func() {
    defer wg.Done()
    time.Sleep(1000 * time.Millisecond)
    print(".")
    }()
    }
    wg.Wait()

    View full-size slide

  36. Go Way
    runtime.GOMAXPROCS(runtime.NumCPU())
    wg:= new(sync.WaitGroup)
    for i:=0; i < 100000; i++ {
    wg.Add(1)
    go func() {
    defer wg.Done()
    time.Sleep(1000 * time.Millisecond)
    print(".")
    }()
    }
    wg.Wait()

    View full-size slide

  37. Go Way
    runtime.GOMAXPROCS(runtime.NumCPU())
    wg:= new(sync.WaitGroup)
    for i:=0; i < 100000; i++ {
    wg.Add(1)
    go func() {
    defer wg.Done()
    time.Sleep(1000 * time.Millisecond)
    print(".")
    }()
    }
    wg.Wait()

    View full-size slide

  38. Go Way
    runtime.GOMAXPROCS(runtime.NumCPU())
    wg:= new(sync.WaitGroup)
    for i:=0; i < 100000; i++ {
    wg.Add(1)
    go func() {
    defer wg.Done()
    time.Sleep(1000 * time.Millisecond)
    print(".")
    }()
    }
    wg.Wait()

    View full-size slide

  39. Quiz - How much time takes?
    runBlocking {
    val jobs = List(100_000) {
    launch(CommonPool) {
    delay(1000L)
    print(".")
    }
    }
    jobs.forEach { it.join() }
    }

    View full-size slide

  40. Quiz - How much time takes?
    runBlocking {
    val jobs = List(100_000) {
    launch(CommonPool) {
    delay(1000L)
    print(".")
    }
    }
    jobs.forEach { it.join() }
    }
    100,000 Seconds?

    View full-size slide

  41. Quiz - How much time takes?
    runBlocking {
    val jobs = List(100_000) {
    launch(CommonPool) {
    delay(1000L)
    print(".")
    }
    }
    jobs.forEach { it.join() }
    }
    100,000 Seconds? Or 1~3 seconds?

    View full-size slide

  42. Quiz - How much time takes?
    runBlocking {
    val jobs = List(100_000) {
    launch(CommonPool) {
    delay(1000L)
    print(".")
    }
    }
    jobs.forEach { it.join() }
    }
    100,000 Seconds? Or 1~3 seconds?

    View full-size slide

  43. Kotlin بੑ ੹ۚ

    View full-size slide

  44. What is our problem?
    • 개발자 수준 편차가 크다

    • 조직 수준은 최하위 개발자 수준의 의해 결정된다

    View full-size slide

  45. 변화를 추구하려면?
    • 객관적 시각 유지

    • History 및 현실 상황 대한 이해 (기술부채)

    • 현 조직의 개발 역량 평가

    • 동기부여 - 필요성 설득보다는 자각할 수 있도록 자극

    • 충분한 학습 시간

    • 변화 경험 공유

    • ASP -> C#, Java -> Scala

    View full-size slide

  46. Environments 준비
    • 개발 Tool 지원 - IntelliJ IDEA

    • Static code analysis - ktlint, detekt

    • Test code coverage - jacoco (sample)

    • Sonarcube - sonar-kotlin plugin

    View full-size slide

  47. Kotlin Backend بੑ ࢎ۹

    View full-size slide

  48. Use cases
    • Common Library (2017. 6 ~)

    • Kotlin extension library

    • Components (2017.08 ~ 2017.12)

    • Aho-corasick, Korean Tokenizer, Kafka Client

    • Standalone Web Application (2017.10~~2017.12)

    • Audit Tool

    • Large scale system (2017.09~2018.01)

    • Catalog creation pipeline system

    View full-size slide

  49. 1. Kotlin Extension library
    • Coupang Standard Framework 보조

    • Kotlin Best practices 제공

    • Object / IO / Utils / Cache / JDBC / Spring …

    • Kotlin 학습 예제 제공 (Test cases / Examples)

    • Based debop4k (personal project)

    View full-size slide

  50. 2. Korean Tokenizer
    • 중복상품 Merge 위한 Tokenizer 필요 (명사 위주)

    • Twitter 에서 개발한 open-korean-text 를 Customizing

    • Scala vs Kotlin 성능 비교

    • Kotlin version is 1.5 ~ 3X faster with Coroutines

    • 효과

    • Full Indexing per Day 부하 감소 : 30%

    • Elastic Search 질의 부하 : 80%

    View full-size slide

  51. “ઁపझ MSW-3928PR թࢿ ࠺஖౟ۦ௼, 95(L), ࠶ۑ࠶ۑ҅ৌ”
    ਷੹ೠ׭ Twitter
    ࠶ۑ
    ࠺஖
    ࠺஖౟ۦ௼
    ҅ৌ
    ઁపझ
    3928PR
    ࠶ۑ࠶ۑ҅ৌ
    PR
    MSW
    ,
    (
    )
    -
    ౟ۦ௼
    3928
    95
    ઁపझ
    թࢿ

    View full-size slide

  52. “ઁపझ MSW-3928PR թࢿ ࠺஖౟ۦ௼, 95(L), ࠶ۑ࠶ۑ҅ৌ”
    ਷੹ೠ׭ Twitter
    ࠶ۑ
    ࠺஖
    ࠺஖౟ۦ௼
    ҅ৌ
    ઁపझ
    3928PR
    ࠶ۑ࠶ۑ҅ৌ
    PR
    MSW
    ,
    (
    )
    -
    ౟ۦ௼
    3928
    95
    ઁపझ
    թࢿ

    View full-size slide

  53. “ઁపझ MSW-3928PR թࢿ ࠺஖౟ۦ௼, 95(L), ࠶ۑ࠶ۑ҅ৌ”
    ਷੹ೠ׭ Twitter
    ࠶ۑ
    ࠺஖
    ࠺஖౟ۦ௼
    ҅ৌ
    ઁపझ
    3928PR
    ࠶ۑ࠶ۑ҅ৌ
    PR
    MSW
    ,
    (
    )
    L -
    ౟ۦ௼
    3928
    95
    ઁపझ
    թࢿ

    View full-size slide

  54. Tokenizer Benchmark
    은전한닢 Twitter RATIO
    RPS 73.2 429.5 5.87 X
    Avg Latency 626 ms 106 ms 5.91 X
    Avg CPU Load 90% 55% 35 % Down
    Twitter Tokenizer 는 한음절을 분석하지 못하는 단점이 있다

    View full-size slide

  55. Scala Kotlin RATIO
    Tokenize 668.620 ms 197.632 ms 3.38 X
    Phrase extractor 644.902 ms 212.500 ms 3.13 X
    ҳޙ : “ز೧ޛҗ ߔف࢑੉ ݃ܰҊ ׷ب۾“
    Benchmark by jmh

    Linux Mint 18.2 Intel I7, 32GB, SSD

    View full-size slide

  56. Why Kotlin is faster?
    • Scala 의 loop는 느리다 - 아주 느릴 수 있다

    • eclipse-collections 사용

    • 메모리 절약

    • Primitive type collection 지원

    • Kotlin Coroutines

    • 비동기 / Non-Blocking

    View full-size slide

  57. 3. Kafka Client - Wilson
    • 동기

    • 안정된 Kafka 사용을 위해 Offset 관리가 필수

    • 각 팀별 중복된 Client 구현 및 삽질

    • 효과

    • Message 중복 / 유실 없음 (Latest strategy)

    • Retry / Circuit breaker 지원

    • Metrics 를 활용한 Ack 지원

    • 전사 확대 적용 중

    View full-size slide

  58. Wilson message flows
    Producer Consumer
    Kafka
    Retry
    Circuit Breaker
    Metrics
    Retry
    Circuit Breaker
    Metrics
    Dead letters
    Sending box
    Redis
    MySQL
    Couchbase
    Dead letters
    Received box
    Last sent timestamp Kafka Offset Manager
    Message Managements
    • Metrics
    • Recovery / Retry
    • Deadletter handling

    View full-size slide

  59. Wilson Dashboard

    View full-size slide

  60. 4. Audit Tool
    • 상품 정보 Audit System

    • developers : 1 senior, 2 junior developer

    • Software stack

    • React

    • Spring Boot 1.5 on Vitamin Framework

    • jOOQ (향후 requery로 변환 예정)

    • Pilot 로 시작, 개발자들의 노력으로 정식 시스템으로 승격

    View full-size slide

  61. 4. Audit Tool
    Kotlin Coroutines + Spring MVC

    View full-size slide

  62. 5. Creation Pipeline System
    • 상품 정보 생성 프로세스 관리 시스템

    • Features

    • Workflow (Heavy use Kafka)

    • Asynchronous / Non-blocking System

    • Need High throughput

    View full-size slide

  63. Seller
    Retail
    Kafka
    Deduplication
    Refining
    Processing
    Creation
    Logging
    Message Dispatcher
    Creation Pipeline flow

    View full-size slide

  64. 5. Creation Pipeline System
    Spring Boot 1.x Kafka 0.10.x
    Kotlin 1.2.x on JVM 8
    Couchbase 4.x / Aurora
    Zookeeper
    3.x

    View full-size slide

  65. Lesson & Learns

    View full-size slide

  66. 반성해야 하는 점
    • 기본기 학습 - 닥치고 코딩 (X)

    • Java와 차이점 및 Kotlin best practices 검토 필요

    • 첫 술에 배부를 수 없다

    • 실망하지 말라, Refactoring 은 필수다

    • Coroutines 에 대한 학습 및 테스트

    • 어차피 비동기는 어렵다.

    • Coroutines는 하나의 방법일 뿐이다

    View full-size slide

  67. 안정적 도입을 위한 Tips
    • 충분한 학습 기회 & 실습

    • Code Quality 중요성 인식

    • Upsource 전사 활용 중

    • 강력한 동기 부여

    • Tech Leader의 지속적인 Leading & Coach

    • 성공 사례 만들기 (작은 것부터)

    View full-size slide

  68. 효과
    • Safety Code

    • Readable Code

    • 성능 향상

    • Latency 감소, High Throughput 달성

    • Asynchronous/Non-Blocking 적용 용이

    • 유지보수성 향상 (장애 감소)

    View full-size slide

  69. 언어별 Catalog Tribe 개발자 비율
    (2017)
    5%
    Kotlin
    10%
    Scala
    15%
    Java
    70%
    Java Scala Kotlin Python
    언어별 Catalog Tribe 개발자 비율
    (2018 예상)
    7%
    Kotlin
    25%
    Scala
    13%
    Java
    55%
    Java Scala Kotlin Python

    View full-size slide

  70. Aurora Cassandra ElasticSearch ArangoDB
    JVM 8
    Spring Data
    Requery JPA/Hibernate
    Virtualization

    kubernetes
    Docker
    Spring Framework 5.x
    Kodein
    Kotlin Coroutines with Reactor / RxJava
    Services

    Kafka
    Redis
    Spring Boot 2.x Webflux with Netty
    Common Backend Stack (2018)

    View full-size slide

  71. Boo1 vs Boo2 Performance

    View full-size slide

  72. Spring MVC + Cassandra
    Spring WebFlux + Cassandra Reactive
    출처: Reactive Java Performance Comparison

    View full-size slide

  73. Resources
    • Kotlin Resources

    • kotlinx.coroutines

    • spring-kotlin-coroutine

    • 20 Excellent Resources for learning Kotlin

    • Books

    • Kotlin in action

    • Try Kotlin

    View full-size slide