Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Scala performance tips -ScalaMatsuri2017-

petitviolet
February 25, 2017

Scala performance tips -ScalaMatsuri2017-

Introduction of performance tips of Scala.
This presentation performed at ScalaMatsuri2017 in Japan.
[petitviolet/scalamatsuri2017: sample code project for ScalaMatsuri2017 presentation](https://github.com/petitviolet/scalamatsuri2017/tree/master)

petitviolet

February 25, 2017
Tweet

More Decks by petitviolet

Other Decks in Technology

Transcript

  1. Performance tips on advertising service Fringe81 Co., Ltd. Hiroki Komurasaki

    @petitviolet 広告配信システムで積み重ねてきたパフォーマンスTips
  2. Fringe81 Co., Ltd. Advertising service • AdNetwork implemented by Scala,

    Play. • Average ~10ms response. • High performance is required.  高いパフォーマンスが要求される広告ネットワークを Scalaで実装している
  3. Fringe81 Co., Ltd. Introduction • “Every little bit counts.” •

    To build `responsive` system, it’s necessary and important to improve performance even little.  ちりも積もれば山となる 即応性のあるシステムの実現には小さな改善が欠かせない
  4. Fringe81 Co., Ltd. Introduction • “Every little bit counts.” •

    To build `responsive` system, it’s necessary and important to improve performance even little.  そうやって積み重ねて来たScalaで出来る改善の 一部を紹介します I will introduce a few performance tips on implementing such ad-network service.
  5. Fringe81 Co., Ltd. Disclaimer • This talk is meant for

    beginners. • The topics covered are not surprising, but are basics we should all learn. • These will help you when performance is not as you expected.  知っている人には当たり前のことだが、知っておきたい話
  6. Fringe81 Co., Ltd. Sorting sequence  普通こうするよね I usually do

    like this. val NUM = 10000
 val target: Seq[Int] = 
 (1 to NUM).map { _ => Random.nextInt(NUM) }
 target.sorted
  7. Fringe81 Co., Ltd. scala.util.Sorting? val NUM = 10000
 val target:

    Seq[Int] = 
 (1 to NUM).map { _ => Random.nextInt(NUM) }
 import scala.util.Sorting
 Sorting.stableSort(target)  Sortingクラスもある scala.util.Sorting can be used to sort sequence.
  8. Fringe81 Co., Ltd. Sorting is much faster  scala.util.Sortingめっちゃ速い 0

    250000 500000 750000 1000000 100 Seq#sorted Sorting 0 40 80 120 160 100000 Seq#sorted Sorting sequence size sequence size throughput(ops/s) throughput(ops/s)
  9. Fringe81 Co., Ltd. Sorting is optimized to primitive types 

    Sortingはプリミティブ型に対してのみ有効で、 独自型に対して用いても特に高速ではない 0 70000 140000 210000 280000 100 Seq#sorted Sorting 0 8.75 17.5 26.25 35 100000 Seq#sorted Sorting sequence size sequence size Sorting throughput comparison for non-primitive type source: https://goo.gl/KLMaqp
  10. Fringe81 Co., Ltd. Random  これ以上何かある? What else can be

    done here? import scala.util.Random
 Random.nextInt(NUM)
  11. Fringe81 Co., Ltd.  ThreadLocalRandomめっちゃ速い 0 45000000 90000000 135000000 180000000

    throughput Random ThreadLocalRandom import scala.concurrent.forkjoin.ThreadLocalRandom ThreadLocalRandom.current().nextInt(NUM) throughput(ops/s)
  12. Fringe81 Co., Ltd. 3. Seq#map or Set#map which is faster?

     SeqとSetのmapはどちらが速い?
  13. Fringe81 Co., Ltd. map -Seq or Set- // just example

    val N: Int = ???
 val func: Int => Int = ???
 val seq: Seq[Int] = (1 to N).toSeq
 val set: Set[Int] = (1 to N).toSet
 
 seq map func
 set map func  SeqとSetのmapでどれくらい速度が違うでしょう? • How much performance will be different?
  14. Fringe81 Co., Ltd. val N: Int = ??? // 100

    or 10000 val seq: Seq[Int] = (1 to N).toSeq val set: Set[Int] = (1 to N).toSet seq map { _ % 2 } seq map { _ % 1000 } // set size N to 2 set map { _ % 2 } // set size N to 1000 set map { _ % 1000 }  SeqとSetのmapでどれくらい速度が違うでしょう?
  15. Fringe81 Co., Ltd.  スループットでSeqの圧勝 0 275000 550000 825000 1100000

    N = 100 % 2 % 1000 Set Seq throughput(ops/s) 0 2000 4000 6000 8000 N = 10000 % 2 % 1000 Set Seq throughput(ops/s)
  16. Fringe81 Co., Ltd. map • Avoid to use Set#map as

    possible • call `hashCode` and `equals` to eliminate duplication • https://gist.github.com/petitviolet/dc5b44fae57277ae915bd770ba4f2435 • also flatMap, collect, etc. • Collections - Performance Characteristics • http://docs.scala-lang.org/overviews/collections/performance-characteristics • `Set#contains` is faster than `Seq`  Setであるための計算が裏側に隠れて遅くなってしまうので注意
  17. Fringe81 Co., Ltd.  たったこれだけで何が変わるか How different? val a: Seq[Int]

    = Seq(1)
 val b: Seq[Int] = List(1)
 val c: Seq[Int] = 1 :: Nil
  18. Fringe81 Co., Ltd. :: Nil is great  :: Nil

    めっちゃ速い 0 27500000 55000000 82500000 110000000 troughput List#apply Seq#apply :: Nill val a: Seq[Int] = Seq(1)
 val b: Seq[Int] = List(1)
 val c: Seq[Int] = 1 :: Nil
  19. Fringe81 Co., Ltd. • `::` is just a class •

    `:: Nil` just create new instance • `Seq()` and `List()` append to `ListBuffer`  :: Nilはただnewしてるだけだから速い 1 :: Nil == new ::(1, Nil) val b = newBuilder[A] // mutable.ListBuffer
 b ++= elems
 b.result()
  20. Fringe81 Co., Ltd. call-by-name?  名前渡しされた引数は使用される直前に評価される Call-by-name is an argument

    type like `=> T`. This argument will be evaluated just before being used. def doSomething[T](arg: => T) = ???
  21. Fringe81 Co., Ltd. call-by-name  上下で第一引数の型が異なるだけで、後は全く同じ Only the type of

    the first argument is different. def byName(value: => String, flag: Boolean): String =
 if (flag) value else ""
 
 def byValue(value: String, flag: Boolean): String =
 if (flag) value else ""
  22. Fringe81 Co., Ltd.  引数を使わなければめっちゃ速い 0 100000000 200000000 300000000 400000000

    true false call-by-value call-by-name def byName(value: => String, flag: Boolean): String = if (flag) value else ""
 
 def byValue(value: String, flag: Boolean): String = if (flag) value else "" throughput(ops/s)
  23. Fringe81 Co., Ltd.  call-by-nameは例えばloggerの実装などに便利 // just example
 class MyLogger(logger:

    Logger, isDebugEnabled: Boolean) {
 def debug(msg: => String) = if (isDebugEnabled) logger.debug(msg)
 
 def info(msg: => String) = logger.info(msg)
 } Call-by-name can be used to implement, such as a logger.
  24. Fringe81 Co., Ltd. Value-class?  値クラスはAnyValを継承し、ただ1つのvalを持つもの A class declaration with

    extends `AnyVal` and has just one public `val` field. class Awesome(val value: Int) extends AnyVal {
 def double = value * 2
 }
  25. Fringe81 Co., Ltd. Value-class?  実行時にはそのvalフィールドの値として表現される At runtime, a value-class

    instance will be represented as its `val` filed type. Therefore, `Awesome` instance is just an `Int`. class Awesome(val value: Int) extends AnyVal {
 def double = value * 2
 }
  26. Fringe81 Co., Ltd.  パフォーマンス観点ではどれくらい変わる? How different is it from

    a performance point of view? object Normal {
 case class User(id: Id, name: Name)
 case class Id(value: Long)
 case class Name(value: String)
 }
 Normal.User(Normal.Id(1L), Normal.Name("hoge")) object Value {
 // not Value-class
 case class UserValue(id: IdValue, name: NameValue)
 case class IdValue(value: Long) extends AnyVal
 case class NameValue(value: String) extends AnyVal
 } 
 Value.UserValue(Value.IdValue(1L), Value.NameValue("hoge"))
  27. Fringe81 Co., Ltd.  インスタンス化は圧倒的に値クラスの方が速い 0 40000000 80000000 120000000 160000000

    instantiation ops/sec Normal class Value class Normal.User(Normal.Id(1L), Normal.Name("hoge"))
 Value.UserValue(Value.IdValue(1L), Value.NameValue("hoge"))
  28. Fringe81 Co., Ltd.  値クラスはDDDやimplicit classなど使いどころは多い Value-class will suits Domain-Driven-Design.

    object Value {
 // not Value class
 case class UserValue(id: IdValue, name: NameValue)
 case class IdValue(value: Long) extends AnyVal
 case class NameValue(value: String) extends AnyVal
 } Value.UserValue(Value.IdValue(1L), Value.NameValue("hoge")) implicit class AwesomeInt(val n: Int) extends AnyVal {
 def double = n * 2
 } In addition, value-class helps implicit conversion performance.
  29. Fringe81 Co., Ltd.  Futureとfor式 import Thread.sleep
 val result: Future[Int]

    = for {
 a <- Future { sleep(100); println("finish 100"); 100 }
 b <- Future { sleep(50); println("finish 50"); 50 }
 } yield a + b
 
 result.onComplete(println)
  30. Fringe81 Co., Ltd.  どっち? finish 100 finish 50 150

    import Thread.sleep
 val result: Future[Int] = for {
 a <- Future { sleep(100); println("finish 100"); 100 }
 b <- Future { sleep(50); println("finish 50"); 50 }
 } yield a + b
 
 result.onComplete(println) finish 50 finish 100 150
  31. Fringe81 Co., Ltd.  上から順に実行されていく import Thread.sleep
 val result: Future[Int]

    = for {
 a <- Future { sleep(100); println("finish 100"); 100 }
 b <- Future { sleep(50); println("finish 50"); 50 }
 } yield a + b
 
 result.onComplete(println) finish 100 finish 50 150 Executed in order from top.
  32. Fringe81 Co., Ltd.  Future.applyの実行タイミングに注意しましょう // call Future.apply before `for`


    val aF = Future { sleep(100); 100 }
 val bF = Future { sleep(50); 50 }
 val result: Future[Int] = for {
 a <- aF
 b <- bF
 } yield a + b Pay attention to the timing to call `Future.apply`. // use `Future#zip`
 val result: Future[Int] = for {
 (a, b) <- Future { sleep(100); 100 } zip Future { sleep(50); 50 }
 } yield a + b
  33. Fringe81 Co., Ltd. others • Collection.breakOut • structural sub typing

    • PartialFunction#orElse is expensive • recursion function • etc.  その他、紹介したかったものたち
  34. Fringe81 Co., Ltd. Summary • Sorting is faster than Seq#sorted

    • ThreadLocalRandom is faster than Random • Avoid to use Set#map as possible • Use `:: Nil` to create `Seq[T]` • Call-by-name make an arguments lazy • Value-class is useful in various situation • Be careful about timing of `Future.apply`  まとめ
  35. Fringe81 Co., Ltd. detail link • Sorting is faster than

    Seq#sorted • https://gist.github.com/petitviolet/07f8459f8a40afd54bea00343548b080 • ThreadLocalRandom is faster than Random • https://gist.github.com/petitviolet/89886339e028779172a293a01e18d8f0 • Avoid to use Set#map as possible • https://gist.github.com/petitviolet/dc5b44fae57277ae915bd770ba4f2435 • Use `:: Nil` to create `Seq[T]` • https://gist.github.com/petitviolet/b67d63aad1a23350f5fa5266d077efca • Call-by-name make an arguments lazy • https://gist.github.com/petitviolet/bb0f14aad27021c46efecd972690d9b3 • Value-class is useful in various situation • https://gist.github.com/petitviolet/026979105dbe447c93d47a778a604619 • Be careful about timing of `Future.apply` • https://gist.github.com/petitviolet/d5ed010ce4ce12bc250b3f259c97ff02  もうちょっと詳しく
  36. Fringe81 Co., Ltd. Summary • Before relying on such tips,

    there should be other things to do. • e.g.) caching to avoid I/O, RDBMS index,… • Faster is better than slower • May such tips help you  tipsに頼る前に何かしらやるべきことはある 遅いよりは速い方が良いので、tipsでも何かしら役に立てば