Who is @debop
• Since 1993
• Robotics, BPM Solution, Healthcare
• C/C++, Object Pascal (Delphi), C#, Java, Scala, Kotlin
• Use Kotlin since 2016.08
• Sr. Principle Software Engineer in Coupang (2017~)
Slide 3
Slide 3 text
Agenda
• Kotlin - The Good, The Bad, The Ugly
• Kotlin Coroutines
• Kotlin Backend 도입 전략 / 사례
• Lesson & Learn
• Resources
Slide 4
Slide 4 text
Kotlin
The Good, The Bad and The Ugly
Slide 5
Slide 5 text
No content
Slide 6
Slide 6 text
No content
Slide 7
Slide 7 text
No content
Slide 8
Slide 8 text
No content
Slide 9
Slide 9 text
Kotlin - The Good (1/3)
• Java Interoperability (@JvmXXXX)
• Familiar Syntax (like Scala, C#, Groovy, Swift …)
• Type Inference (like Scala, C#)
• Smart Casts (`is`, `as`)
• Data Class (Scala case class, POJO with @Data)
Slide 10
Slide 10 text
Kotlin - The Good (1/3)
when (x) {
is Int -> print(x + 1)
is String -> print(x.length + 1)
is IntArray -> print(x.sum())
!is BigDecimal -> print(x)
1L..10L -> print(x)
}
val x: String? = y as String (Unsafe)
val x: String? = y as String? (Unsafe)
val x: String? = y as? String (Safe)
is / !is
as / as?
Slide 11
Slide 11 text
Kotlin - The Good (1/3)
public class User {
private String name;
private Integer age;
public User(String name) {
this(name, 0);
}
public User(String name, Integer age) {
this.name = name;
this.age = age;
}
public String getName() {return this.name;}
public Integer getAge() {return this.age;}
public void setName(String name) {this.name = name; }
public void setAge(Integer age) {this.age = age; }
public boolean equals(Object o) { /* ࢤۚ */ }
public int hashCode() {
return Objects.hash(name, age);
}
protected boolean canEqual(Object other) {return other instanceof User;}
public String toString() {return "User(name=" + this.getName() + ", age=" + this.getAge() + ")";}
}
Slide 12
Slide 12 text
Kotlin - The Good (1/3)
public class User {
private String name;
private Integer age;
public User(String name) {
this(name, 0);
}
public User(String name, Integer age) {
this.name = name;
this.age = age;
}
public String getName() {return this.name;}
public Integer getAge() {return this.age;}
public void setName(String name) {this.name = name; }
public void setAge(Integer age) {this.age = age; }
public boolean equals(Object o) { /* ࢤۚ */ }
public int hashCode() {
return Objects.hash(name, age);
}
protected boolean canEqual(Object other) {return other instanceof User;}
public String toString() {return "User(name=" + this.getName() + ", age=" + this.getAge() + ")";}
}
data class User(var name:String) {
var age:Int = 0
}
Slide 13
Slide 13 text
Kotlin - The Good (2/3)
• Default Arguments
• String Templates (Scala - string interpolation)
• Properties (C#, No Getter, Setter)
Slide 14
Slide 14 text
val tomAge = map["tom"] ?: 0
map["jane"] = 29
val query = """
| CREATE KEYSPACE IF NOT EXISTS $keyspaceName
| WITH replication = { 'class': '$replicationStrategy',
| 'replication_factor':$replicationFactor }
""".trimMargin()
val s = "abc"
val str = "$s.length is ${s.length}" // prints "abc.length is 3"
Kotlin - The Good (2/3)
fun read(b:ByteArray, offset:Int = 0, length:Int = b.size) {
TODO(”Not Implemented”)
}
Slide 15
Slide 15 text
val tomAge = map["tom"] ?: 0
map["jane"] = 29
val query = """
| CREATE KEYSPACE IF NOT EXISTS $keyspaceName
| WITH replication = { 'class': '$replicationStrategy',
| 'replication_factor':$replicationFactor }
""".trimMargin()
val s = "abc"
val str = "$s.length is ${s.length}" // prints "abc.length is 3"
Kotlin - The Good (2/3)
fun read(b:ByteArray, offset:Int = 0, length:Int = b.size) {
TODO(”Not Implemented”)
}
Slide 16
Slide 16 text
val tomAge = map["tom"] ?: 0
map["jane"] = 29
val query = """
| CREATE KEYSPACE IF NOT EXISTS $keyspaceName
| WITH replication = { 'class': '$replicationStrategy',
| 'replication_factor':$replicationFactor }
""".trimMargin()
val s = "abc"
val str = "$s.length is ${s.length}" // prints "abc.length is 3"
Kotlin - The Good (2/3)
fun read(b:ByteArray, offset:Int = 0, length:Int = b.size) {
TODO(”Not Implemented”)
}
Slide 17
Slide 17 text
val tomAge = map["tom"] ?: 0
map["jane"] = 29
val query = """
| CREATE KEYSPACE IF NOT EXISTS $keyspaceName
| WITH replication = { 'class': '$replicationStrategy',
| 'replication_factor':$replicationFactor }
""".trimMargin()
val s = "abc"
val str = "$s.length is ${s.length}" // prints "abc.length is 3"
Kotlin - The Good (2/3)
fun read(b:ByteArray, offset:Int = 0, length:Int = b.size) {
TODO(”Not Implemented”)
}
Slide 18
Slide 18 text
Kotlin - The Good (3/3)
• Destructuring Declarations
• Class, Property Delegates
• Extension Functions (C#)
• Null Safety (C#)
• Better Lambdas than Java (same Scala)
• Scala functional features (use arrow)
• Coroutines (kotlinx-coroutines)
Slide 19
Slide 19 text
No content
Slide 20
Slide 20 text
Kotlin - The Good (3/3)
fun Int.millis(): Duration = Duration.ofMillis(this.toLong())
fun Int.seconds(): Duration = Duration.ofSeconds(this.toLong())
// 5.seconds() + 400.millis() —> 5400 msec
if (text != null) text.length else -1
text?.length ?: -1
suspend fun Period.daySequence(): Sequence = buildSequence {
var day = 0
val days = [email protected]
while (day < days) {
yield(day++)
}
}
}
Slide 21
Slide 21 text
Kotlin - The Good (3/3)
if(command!=null) {
if(command.getThreadPoolProperties()!=null) {
if(command.getThreadPoolProperties().maxQueueSize()!=null) {
val x = command.getThreadPoolProperties().maxQueueSize().get()
}
}
}
Slide 22
Slide 22 text
val x = command?.getThreadPoolProperties()?.maxQueueSize()?.get() ?: 0
Kotlin - The Good (3/3)
if(command!=null) {
if(command.getThreadPoolProperties()!=null) {
if(command.getThreadPoolProperties().maxQueueSize()!=null) {
val x = command.getThreadPoolProperties().maxQueueSize().get()
}
}
}
Slide 23
Slide 23 text
Compile time
Slide 24
Slide 24 text
Compile time
Scala >>>>> Kotlin > Java
Slide 25
Slide 25 text
the bad
Slide 26
Slide 26 text
Kotlin - The Bad
• No namespace (package variable, methods)
• No static modifier (use companion object)
Slide 27
Slide 27 text
Kotlin - The Bad
class OffHeapOutputStream(private var ohm: OffHeapMemory) : OutputStream() {
companion object {
private val bax = ByteArrayTool.Factory.create()
@JvmField val EMPTY = OffHeapOutputStream(0)
}
}
@file:JvmName("Localex")
/** ೠ Locale null ݶ ӝࠄ Locale ਸ ߈ജפ */
fun Locale?.orDefault(): Locale = this ?: Locale.getDefault()
Slide 28
Slide 28 text
the ugly
Slide 29
Slide 29 text
Kotlin - The Ugly
• SAM conversion
• Unit returning lambdas (Use in java)
• Final by default (안정성? 성능? A bit about picking defaults)
• Open: 50%, Final : 50%
registerCallback(() -> {
/** do stuff */
return Unit.INSTANCE;
})
Kotlin Way
suspend fun work(): String {
delay(200L)
return "done"
}
fun moreWork(): Deferred = async {
println("Work started")
val str = work()
println("Work completed. $str")
}
runBlocking {
moreWork().await()
}
Slide 39
Slide 39 text
Kotlin Way
suspend fun work(): String {
delay(200L)
return "done"
}
fun moreWork(): Deferred = async {
println("Work started")
val str = work()
println("Work completed. $str")
}
runBlocking {
moreWork().await()
}
Slide 40
Slide 40 text
Kotlin Way
suspend fun work(): String {
delay(200L)
return "done"
}
fun moreWork(): Deferred = async {
println("Work started")
val str = work()
println("Work completed. $str")
}
runBlocking {
moreWork().await()
}
Slide 41
Slide 41 text
Kotlin Way
suspend fun work(): String {
delay(200L)
return "done"
}
fun moreWork(): Deferred = async {
println("Work started")
val str = work()
println("Work completed. $str")
}
runBlocking {
moreWork().await()
}
Slide 42
Slide 42 text
Go Way
runtime.GOMAXPROCS(runtime.NumCPU())
wg:= new(sync.WaitGroup)
for i:=0; i < 100000; i++ {
wg.Add(1)
go func() {
defer wg.Done()
time.Sleep(1000 * time.Millisecond)
print(".")
}()
}
wg.Wait()
Slide 43
Slide 43 text
Go Way
runtime.GOMAXPROCS(runtime.NumCPU())
wg:= new(sync.WaitGroup)
for i:=0; i < 100000; i++ {
wg.Add(1)
go func() {
defer wg.Done()
time.Sleep(1000 * time.Millisecond)
print(".")
}()
}
wg.Wait()
Slide 44
Slide 44 text
Go Way
runtime.GOMAXPROCS(runtime.NumCPU())
wg:= new(sync.WaitGroup)
for i:=0; i < 100000; i++ {
wg.Add(1)
go func() {
defer wg.Done()
time.Sleep(1000 * time.Millisecond)
print(".")
}()
}
wg.Wait()
Slide 45
Slide 45 text
Go Way
runtime.GOMAXPROCS(runtime.NumCPU())
wg:= new(sync.WaitGroup)
for i:=0; i < 100000; i++ {
wg.Add(1)
go func() {
defer wg.Done()
time.Sleep(1000 * time.Millisecond)
print(".")
}()
}
wg.Wait()
Slide 46
Slide 46 text
Quiz - How much time takes?
runBlocking {
val jobs = List(100_000) {
launch(CommonPool) {
delay(1000L)
print(".")
}
}
jobs.forEach { it.join() }
}
Slide 47
Slide 47 text
Quiz - How much time takes?
runBlocking {
val jobs = List(100_000) {
launch(CommonPool) {
delay(1000L)
print(".")
}
}
jobs.forEach { it.join() }
}
100,000 Seconds?
Slide 48
Slide 48 text
Quiz - How much time takes?
runBlocking {
val jobs = List(100_000) {
launch(CommonPool) {
delay(1000L)
print(".")
}
}
jobs.forEach { it.join() }
}
100,000 Seconds? Or 1~3 seconds?
Slide 49
Slide 49 text
Quiz - How much time takes?
runBlocking {
val jobs = List(100_000) {
launch(CommonPool) {
delay(1000L)
print(".")
}
}
jobs.forEach { it.join() }
}
100,000 Seconds? Or 1~3 seconds?
Slide 50
Slide 50 text
Kotlin بੑ ۚ
Slide 51
Slide 51 text
What is our problem?
• 개발자 수준 편차가 크다
• 조직 수준은 최하위 개발자 수준의 의해 결정된다
Slide 52
Slide 52 text
변화를 추구하려면?
• 객관적 시각 유지
• History 및 현실 상황 대한 이해 (기술부채)
• 현 조직의 개발 역량 평가
• 동기부여 - 필요성 설득보다는 자각할 수 있도록 자극
• 충분한 학습 시간
• 변화 경험 공유
• ASP -> C#, Java -> Scala
Slide 53
Slide 53 text
Environments 준비
• 개발 Tool 지원 - IntelliJ IDEA
• Static code analysis - ktlint, detekt
• Test code coverage - jacoco (sample)
• Sonarcube - sonar-kotlin plugin
Slide 54
Slide 54 text
Kotlin Backend بੑ ࢎ۹
Slide 55
Slide 55 text
Use cases
• Common Library (2017. 6 ~)
• Kotlin extension library
• Components (2017.08 ~ 2017.12)
• Aho-corasick, Korean Tokenizer, Kafka Client
• Standalone Web Application (2017.10~~2017.12)
• Audit Tool
• Large scale system (2017.09~2018.01)
• Catalog creation pipeline system
Slide 56
Slide 56 text
1. Kotlin Extension library
• Coupang Standard Framework 보조
• Kotlin Best practices 제공
• Object / IO / Utils / Cache / JDBC / Spring …
• Kotlin 학습 예제 제공 (Test cases / Examples)
• Based debop4k (personal project)
Slide 57
Slide 57 text
2. Korean Tokenizer
• 중복상품 Merge 위한 Tokenizer 필요 (명사 위주)
• Twitter 에서 개발한 open-korean-text 를 Customizing
• Scala vs Kotlin 성능 비교
• Kotlin version is 1.5 ~ 3X faster with Coroutines
• 효과
• Full Indexing per Day 부하 감소 : 30%
• Elastic Search 질의 부하 : 80%
Tokenizer Benchmark
은전한닢 Twitter RATIO
RPS 73.2 429.5 5.87 X
Avg Latency 626 ms 106 ms 5.91 X
Avg CPU Load 90% 55% 35 % Down
Twitter Tokenizer 는 한음절을 분석하지 못하는 단점이 있다
Slide 62
Slide 62 text
Scala Kotlin RATIO
Tokenize 668.620 ms 197.632 ms 3.38 X
Phrase extractor 644.902 ms 212.500 ms 3.13 X
ҳޙ : “ز೧ޛҗ ߔف ݃ܰҊ ب۾“
Benchmark by jmh
Linux Mint 18.2 Intel I7, 32GB, SSD
Slide 63
Slide 63 text
Why Kotlin is faster?
• Scala 의 loop는 느리다 - 아주 느릴 수 있다
• eclipse-collections 사용
• 메모리 절약
• Primitive type collection 지원
• Kotlin Coroutines
• 비동기 / Non-Blocking
Slide 64
Slide 64 text
3. Kafka Client - Wilson
• 동기
• 안정된 Kafka 사용을 위해 Offset 관리가 필수
• 각 팀별 중복된 Client 구현 및 삽질
• 효과
• Message 중복 / 유실 없음 (Latest strategy)
• Retry / Circuit breaker 지원
• Metrics 를 활용한 Ack 지원
• 전사 확대 적용 중
Slide 65
Slide 65 text
Wilson message flows
Producer Consumer
Kafka
Retry
Circuit Breaker
Metrics
Retry
Circuit Breaker
Metrics
Dead letters
Sending box
Redis
MySQL
Couchbase
Dead letters
Received box
Last sent timestamp Kafka Offset Manager
Message Managements
• Metrics
• Recovery / Retry
• Deadletter handling
Slide 66
Slide 66 text
Wilson Dashboard
Slide 67
Slide 67 text
4. Audit Tool
• 상품 정보 Audit System
• developers : 1 senior, 2 junior developer
• Software stack
• React
• Spring Boot 1.5 on Vitamin Framework
• jOOQ (향후 requery로 변환 예정)
• Pilot 로 시작, 개발자들의 노력으로 정식 시스템으로 승격
Slide 68
Slide 68 text
4. Audit Tool
Kotlin Coroutines + Spring MVC
Slide 69
Slide 69 text
5. Creation Pipeline System
• 상품 정보 생성 프로세스 관리 시스템
• Features
• Workflow (Heavy use Kafka)
• Asynchronous / Non-blocking System
• Need High throughput
5. Creation Pipeline System
Spring Boot 1.x Kafka 0.10.x
Kotlin 1.2.x on JVM 8
Couchbase 4.x / Aurora
Zookeeper
3.x
Slide 72
Slide 72 text
%
Slide 73
Slide 73 text
%
Slide 74
Slide 74 text
Lesson & Learns
Slide 75
Slide 75 text
반성해야 하는 점
• 기본기 학습 - 닥치고 코딩 (X)
• Java와 차이점 및 Kotlin best practices 검토 필요
• 첫 술에 배부를 수 없다
• 실망하지 말라, Refactoring 은 필수다
• Coroutines 에 대한 학습 및 테스트
• 어차피 비동기는 어렵다.
• Coroutines는 하나의 방법일 뿐이다
Slide 76
Slide 76 text
안정적 도입을 위한 Tips
• 충분한 학습 기회 & 실습
• Code Quality 중요성 인식
• Upsource 전사 활용 중
• 강력한 동기 부여
• Tech Leader의 지속적인 Leading & Coach
• 성공 사례 만들기 (작은 것부터)
Slide 77
Slide 77 text
효과
• Safety Code
• Readable Code
• 성능 향상
• Latency 감소, High Throughput 달성
• Asynchronous/Non-Blocking 적용 용이
• 유지보수성 향상 (장애 감소)