$30 off During Our Annual Pro Sale. View Details »

Scala performance tips -ScalaMatsuri2017-

petitviolet
February 25, 2017

Scala performance tips -ScalaMatsuri2017-

Introduction of performance tips of Scala.
This presentation performed at ScalaMatsuri2017 in Japan.
[petitviolet/scalamatsuri2017: sample code project for ScalaMatsuri2017 presentation](https://github.com/petitviolet/scalamatsuri2017/tree/master)

petitviolet

February 25, 2017
Tweet

More Decks by petitviolet

Other Decks in Technology

Transcript

  1. Performance tips
    on advertising service
    Fringe81 Co., Ltd.
    Hiroki Komurasaki @petitviolet
    広告配信システムで積み重ねてきたパフォーマンスTips

    View Slide

  2. Fringe81 Co., Ltd.
    Advertising service
    • AdNetwork implemented by Scala, Play.
    • Average ~10ms response.
    • High performance is required.

    高いパフォーマンスが要求される広告ネットワークを
    Scalaで実装している

    View Slide

  3. Fringe81 Co., Ltd.
    Introduction
    • “Every little bit counts.”
    • To build `responsive` system, it’s necessary
    and important to improve performance
    even little.

    ちりも積もれば山となる
    即応性のあるシステムの実現には小さな改善が欠かせない

    View Slide

  4. Fringe81 Co., Ltd.
    Introduction
    • “Every little bit counts.”
    • To build `responsive` system, it’s necessary
    and important to improve performance
    even little.

    そうやって積み重ねて来たScalaで出来る改善の
    一部を紹介します
    I will introduce a few performance tips on
    implementing such ad-network service.

    View Slide

  5. Fringe81 Co., Ltd.
    Disclaimer
    • This talk is meant for beginners.
    • The topics covered are not surprising, but
    are basics we should all learn.
    • These will help you when performance is
    not as you expected.

    知っている人には当たり前のことだが、知っておきたい話

    View Slide

  6. Useful utility classes
    知っておきたい便利クラスの話

    View Slide

  7. Fringe81 Co., Ltd.
    1. Sorting

    ソート

    View Slide

  8. Fringe81 Co., Ltd.
    Sorting sequence

    普通こうするよね
    I usually do like this.
    val NUM = 10000

    val target: Seq[Int] = 

    (1 to NUM).map { _ => Random.nextInt(NUM) }

    target.sorted

    View Slide

  9. Fringe81 Co., Ltd.
    scala.util.Sorting?
    val NUM = 10000

    val target: Seq[Int] = 

    (1 to NUM).map { _ => Random.nextInt(NUM) }

    import scala.util.Sorting

    Sorting.stableSort(target)

    Sortingクラスもある
    scala.util.Sorting can be used to sort sequence.

    View Slide

  10. Fringe81 Co., Ltd.
    Sorting is much faster

    scala.util.Sortingめっちゃ速い
    0
    250000
    500000
    750000
    1000000
    100
    Seq#sorted Sorting
    0
    40
    80
    120
    160
    100000
    Seq#sorted Sorting
    sequence size sequence size
    throughput(ops/s) throughput(ops/s)

    View Slide

  11. Fringe81 Co., Ltd.
    Sorting is optimized to
    primitive types

    Sortingはプリミティブ型に対してのみ有効で、
    独自型に対して用いても特に高速ではない
    0
    70000
    140000
    210000
    280000
    100
    Seq#sorted Sorting
    0
    8.75
    17.5
    26.25
    35
    100000
    Seq#sorted Sorting
    sequence size sequence size
    Sorting throughput comparison for non-primitive type
    source: https://goo.gl/KLMaqp

    View Slide

  12. Fringe81 Co., Ltd.
    2. Random

    乱数

    View Slide

  13. Fringe81 Co., Ltd.
    Random

    これ以上何かある?
    What else can be done here?
    import scala.util.Random

    Random.nextInt(NUM)

    View Slide

  14. Fringe81 Co., Ltd.
    ThreadLocalRandomというクラスがある
    import scala.concurrent.forkjoin.ThreadLocalRandom
    ThreadLocalRandom.current().nextInt(NUM)

    View Slide

  15. Fringe81 Co., Ltd.
    ThreadLocalRandomめっちゃ速い
    0
    45000000
    90000000
    135000000
    180000000
    throughput
    Random ThreadLocalRandom
    import scala.concurrent.forkjoin.ThreadLocalRandom
    ThreadLocalRandom.current().nextInt(NUM)
    throughput(ops/s)

    View Slide

  16. Collection
    コレクションにまつわる話

    View Slide

  17. Fringe81 Co., Ltd.
    3. Seq#map or Set#map
    which is faster?

    SeqとSetのmapはどちらが速い?

    View Slide

  18. Fringe81 Co., Ltd.
    map -Seq or Set-
    // just example
    val N: Int = ???

    val func: Int => Int = ???

    val seq: Seq[Int] = (1 to N).toSeq

    val set: Set[Int] = (1 to N).toSet


    seq map func

    set map func

    SeqとSetのmapでどれくらい速度が違うでしょう?
    • How much performance will be different?

    View Slide

  19. Fringe81 Co., Ltd.
    val N: Int = ??? // 100 or 10000
    val seq: Seq[Int] = (1 to N).toSeq
    val set: Set[Int] = (1 to N).toSet
    seq map { _ % 2 }
    seq map { _ % 1000 }
    // set size N to 2
    set map { _ % 2 }
    // set size N to 1000
    set map { _ % 1000 }

    SeqとSetのmapでどれくらい速度が違うでしょう?

    View Slide

  20. Fringe81 Co., Ltd.
    スループットでSeqの圧勝
    0
    275000
    550000
    825000
    1100000
    N = 100
    % 2 % 1000
    Set Seq
    throughput(ops/s)
    0
    2000
    4000
    6000
    8000
    N = 10000
    % 2 % 1000
    Set Seq
    throughput(ops/s)

    View Slide

  21. Fringe81 Co., Ltd.
    map
    • Avoid to use Set#map as possible
    • call `hashCode` and `equals` to eliminate duplication
    • https://gist.github.com/petitviolet/dc5b44fae57277ae915bd770ba4f2435
    • also flatMap, collect, etc.
    • Collections - Performance Characteristics
    • http://docs.scala-lang.org/overviews/collections/performance-characteristics
    • `Set#contains` is faster than `Seq`

    Setであるための計算が裏側に隠れて遅くなってしまうので注意

    View Slide

  22. Fringe81 Co., Ltd.
    4. construct Seq

    Seqを作る時

    View Slide

  23. Fringe81 Co., Ltd.
    たったこれだけで何が変わるか
    How different?
    val a: Seq[Int] = Seq(1)

    val b: Seq[Int] = List(1)

    val c: Seq[Int] = 1 :: Nil

    View Slide

  24. Fringe81 Co., Ltd.
    :: Nil is great

    :: Nil めっちゃ速い
    0
    27500000
    55000000
    82500000
    110000000
    troughput
    List#apply Seq#apply :: Nill
    val a: Seq[Int] = Seq(1)

    val b: Seq[Int] = List(1)

    val c: Seq[Int] = 1 :: Nil

    View Slide

  25. Fringe81 Co., Ltd.
    • `::` is just a class
    • `:: Nil` just create new instance
    • `Seq()` and `List()` append to `ListBuffer`

    :: Nilはただnewしてるだけだから速い
    1 :: Nil == new ::(1, Nil)
    val b = newBuilder[A] // mutable.ListBuffer

    b ++= elems

    b.result()

    View Slide

  26. Scala language features
    Scalaの知っておきたい言語機能の話

    View Slide

  27. Fringe81 Co., Ltd.
    5. call-by-name

    名前渡し

    View Slide

  28. Fringe81 Co., Ltd.
    call-by-name?

    名前渡しされた引数は使用される直前に評価される
    Call-by-name is an argument type like `=> T`.
    This argument will be evaluated just before being used.
    def doSomething[T](arg: => T) = ???

    View Slide

  29. Fringe81 Co., Ltd.
    call-by-name

    上下で第一引数の型が異なるだけで、後は全く同じ
    Only the type of the first argument is different.
    def byName(value: => String, flag: Boolean): String =

    if (flag) value else ""


    def byValue(value: String, flag: Boolean): String =

    if (flag) value else ""

    View Slide

  30. Fringe81 Co., Ltd.
    引数を使わなければめっちゃ速い
    0
    100000000
    200000000
    300000000
    400000000
    true false
    call-by-value
    call-by-name
    def byName(value: => String, flag: Boolean): String = if (flag) value else ""


    def byValue(value: String, flag: Boolean): String = if (flag) value else ""
    throughput(ops/s)

    View Slide

  31. Fringe81 Co., Ltd.
    call-by-nameは例えばloggerの実装などに便利
    // just example

    class MyLogger(logger: Logger, isDebugEnabled: Boolean) {

    def debug(msg: => String) =
    if (isDebugEnabled) logger.debug(msg)


    def info(msg: => String) = logger.info(msg)

    }
    Call-by-name can be used to implement, such as a logger.

    View Slide

  32. Fringe81 Co., Ltd.
    6. Value class

    値クラス

    View Slide

  33. Fringe81 Co., Ltd.
    Value-class?

    値クラスはAnyValを継承し、ただ1つのvalを持つもの
    A class declaration with extends `AnyVal` and
    has just one public `val` field.
    class Awesome(val value: Int) extends AnyVal {

    def double = value * 2

    }

    View Slide

  34. Fringe81 Co., Ltd.
    Value-class?

    実行時にはそのvalフィールドの値として表現される
    At runtime, a value-class instance will be
    represented as its `val` filed type.
    Therefore, `Awesome` instance is just an `Int`.
    class Awesome(val value: Int) extends AnyVal {

    def double = value * 2

    }

    View Slide

  35. Fringe81 Co., Ltd.
    パフォーマンス観点ではどれくらい変わる?
    How different is it from a performance point of view?
    object Normal {

    case class User(id: Id, name: Name)

    case class Id(value: Long)

    case class Name(value: String)

    }

    Normal.User(Normal.Id(1L), Normal.Name("hoge"))
    object Value {

    // not Value-class

    case class UserValue(id: IdValue, name: NameValue)

    case class IdValue(value: Long) extends AnyVal

    case class NameValue(value: String) extends AnyVal

    }

    Value.UserValue(Value.IdValue(1L), Value.NameValue("hoge"))

    View Slide

  36. Fringe81 Co., Ltd.
    インスタンス化は圧倒的に値クラスの方が速い
    0
    40000000
    80000000
    120000000
    160000000
    instantiation ops/sec
    Normal class Value class
    Normal.User(Normal.Id(1L), Normal.Name("hoge"))

    Value.UserValue(Value.IdValue(1L), Value.NameValue("hoge"))

    View Slide

  37. Fringe81 Co., Ltd.
    値クラスはDDDやimplicit classなど使いどころは多い
    Value-class will suits Domain-Driven-Design.
    object Value {

    // not Value class

    case class UserValue(id: IdValue, name: NameValue)

    case class IdValue(value: Long) extends AnyVal

    case class NameValue(value: String) extends AnyVal

    }
    Value.UserValue(Value.IdValue(1L), Value.NameValue("hoge"))
    implicit class AwesomeInt(val n: Int) extends AnyVal {

    def double = n * 2

    }
    In addition, value-class helps implicit conversion
    performance.

    View Slide

  38. Fringe81 Co., Ltd.
    7. Future trap

    Futureの罠

    View Slide

  39. Fringe81 Co., Ltd.
    Futureとfor式
    import Thread.sleep

    val result: Future[Int] = for {

    a <- Future { sleep(100); println("finish 100"); 100 }

    b <- Future { sleep(50); println("finish 50"); 50 }

    } yield a + b


    result.onComplete(println)

    View Slide

  40. Fringe81 Co., Ltd.
    どっち?
    finish 100
    finish 50
    150
    import Thread.sleep

    val result: Future[Int] = for {

    a <- Future { sleep(100); println("finish 100"); 100 }

    b <- Future { sleep(50); println("finish 50"); 50 }

    } yield a + b


    result.onComplete(println)
    finish 50
    finish 100
    150

    View Slide

  41. Fringe81 Co., Ltd.
    上から順に実行されていく
    import Thread.sleep

    val result: Future[Int] = for {

    a <- Future { sleep(100); println("finish 100"); 100 }

    b <- Future { sleep(50); println("finish 50"); 50 }

    } yield a + b


    result.onComplete(println)
    finish 100
    finish 50
    150
    Executed in order from top.

    View Slide

  42. Fringe81 Co., Ltd.
    Future.applyの実行タイミングに注意しましょう
    // call Future.apply before `for`

    val aF = Future { sleep(100); 100 }

    val bF = Future { sleep(50); 50 }

    val result: Future[Int] = for {

    a <- aF

    b <- bF

    } yield a + b
    Pay attention to the timing to call `Future.apply`.
    // use `Future#zip`

    val result: Future[Int] = for {

    (a, b) <- Future { sleep(100); 100 } zip Future { sleep(50); 50 }

    } yield a + b

    View Slide

  43. Fringe81 Co., Ltd.
    others
    • Collection.breakOut
    • structural sub typing
    • PartialFunction#orElse is expensive
    • recursion function
    • etc.

    その他、紹介したかったものたち

    View Slide

  44. Summary
    まとめ

    View Slide

  45. Fringe81 Co., Ltd.
    Summary
    • Sorting is faster than Seq#sorted
    • ThreadLocalRandom is faster than Random
    • Avoid to use Set#map as possible
    • Use `:: Nil` to create `Seq[T]`
    • Call-by-name make an arguments lazy
    • Value-class is useful in various situation
    • Be careful about timing of `Future.apply`

    まとめ

    View Slide

  46. Fringe81 Co., Ltd.
    detail link
    • Sorting is faster than Seq#sorted
    • https://gist.github.com/petitviolet/07f8459f8a40afd54bea00343548b080
    • ThreadLocalRandom is faster than Random
    • https://gist.github.com/petitviolet/89886339e028779172a293a01e18d8f0
    • Avoid to use Set#map as possible
    • https://gist.github.com/petitviolet/dc5b44fae57277ae915bd770ba4f2435
    • Use `:: Nil` to create `Seq[T]`
    • https://gist.github.com/petitviolet/b67d63aad1a23350f5fa5266d077efca
    • Call-by-name make an arguments lazy
    • https://gist.github.com/petitviolet/bb0f14aad27021c46efecd972690d9b3
    • Value-class is useful in various situation
    • https://gist.github.com/petitviolet/026979105dbe447c93d47a778a604619
    • Be careful about timing of `Future.apply`
    • https://gist.github.com/petitviolet/d5ed010ce4ce12bc250b3f259c97ff02

    もうちょっと詳しく

    View Slide

  47. Fringe81 Co., Ltd.
    Summary
    • Before relying on such tips, there should be
    other things to do.
    • e.g.) caching to avoid I/O, RDBMS index,…
    • Faster is better than slower
    • May such tips help you

    tipsに頼る前に何かしらやるべきことはある
    遅いよりは速い方が良いので、tipsでも何かしら役に立てば

    View Slide

  48. Fringe81 Co., Ltd.

    View Slide