Share
Embed
Scala performance tips -ScalaMatsuri2017-
Introduction of performance tips of Scala.
This presentation performed at ScalaMatsuri2017 in Japan.
[petitviolet/scalamatsuri2017: sample code project for ScalaMatsuri2017 presentation](https://github.com/petitviolet/scalamatsuri2017/tree/master)
petitviolet
February 25, 2017
Transcript
48 slides
-
Performance tips on advertising service Fringe81 Co., Ltd. Hiroki Komurasaki
@petitviolet 広告配信システムで積み重ねてきたパフォーマンスTips -
Fringe81 Co., Ltd. Advertising service • AdNetwork implemented by Scala,
Play. • Average ~10ms response. • High performance is required. 高いパフォーマンスが要求される広告ネットワークを Scalaで実装している -
Fringe81 Co., Ltd. Introduction • “Every little bit counts.” •
To build `responsive` system, it’s necessary and important to improve performance even little. ちりも積もれば山となる 即応性のあるシステムの実現には小さな改善が欠かせない -
Fringe81 Co., Ltd. Introduction • “Every little bit counts.” •
To build `responsive` system, it’s necessary and important to improve performance even little. そうやって積み重ねて来たScalaで出来る改善の 一部を紹介します I will introduce a few performance tips on implementing such ad-network service. -
Fringe81 Co., Ltd. Disclaimer • This talk is meant for
beginners. • The topics covered are not surprising, but are basics we should all learn. • These will help you when performance is not as you expected. 知っている人には当たり前のことだが、知っておきたい話 -
Fringe81 Co., Ltd. Sorting sequence 普通こうするよね I usually do
like this. val NUM = 10000 val target: Seq[Int] = (1 to NUM).map { _ => Random.nextInt(NUM) } target.sorted -
Fringe81 Co., Ltd. scala.util.Sorting? val NUM = 10000 val target:
Seq[Int] = (1 to NUM).map { _ => Random.nextInt(NUM) } import scala.util.Sorting Sorting.stableSort(target) Sortingクラスもある scala.util.Sorting can be used to sort sequence. -
Fringe81 Co., Ltd. Sorting is much faster scala.util.Sortingめっちゃ速い 0
250000 500000 750000 1000000 100 Seq#sorted Sorting 0 40 80 120 160 100000 Seq#sorted Sorting sequence size sequence size throughput(ops/s) throughput(ops/s) -
Fringe81 Co., Ltd. Sorting is optimized to primitive types
Sortingはプリミティブ型に対してのみ有効で、 独自型に対して用いても特に高速ではない 0 70000 140000 210000 280000 100 Seq#sorted Sorting 0 8.75 17.5 26.25 35 100000 Seq#sorted Sorting sequence size sequence size Sorting throughput comparison for non-primitive type source: https://goo.gl/KLMaqp -
Fringe81 Co., Ltd. Random これ以上何かある? What else can be
done here? import scala.util.Random Random.nextInt(NUM) -
Fringe81 Co., Ltd. ThreadLocalRandomめっちゃ速い 0 45000000 90000000 135000000 180000000
throughput Random ThreadLocalRandom import scala.concurrent.forkjoin.ThreadLocalRandom ThreadLocalRandom.current().nextInt(NUM) throughput(ops/s) -
Fringe81 Co., Ltd. 3. Seq#map or Set#map which is faster?
SeqとSetのmapはどちらが速い? -
Fringe81 Co., Ltd. map -Seq or Set- // just example
val N: Int = ??? val func: Int => Int = ??? val seq: Seq[Int] = (1 to N).toSeq val set: Set[Int] = (1 to N).toSet seq map func set map func SeqとSetのmapでどれくらい速度が違うでしょう? • How much performance will be different? -
Fringe81 Co., Ltd. val N: Int = ??? // 100
or 10000 val seq: Seq[Int] = (1 to N).toSeq val set: Set[Int] = (1 to N).toSet seq map { _ % 2 } seq map { _ % 1000 } // set size N to 2 set map { _ % 2 } // set size N to 1000 set map { _ % 1000 } SeqとSetのmapでどれくらい速度が違うでしょう? -
Fringe81 Co., Ltd. スループットでSeqの圧勝 0 275000 550000 825000 1100000
N = 100 % 2 % 1000 Set Seq throughput(ops/s) 0 2000 4000 6000 8000 N = 10000 % 2 % 1000 Set Seq throughput(ops/s) -
Fringe81 Co., Ltd. map • Avoid to use Set#map as
possible • call `hashCode` and `equals` to eliminate duplication • https://gist.github.com/petitviolet/dc5b44fae57277ae915bd770ba4f2435 • also flatMap, collect, etc. • Collections - Performance Characteristics • http://docs.scala-lang.org/overviews/collections/performance-characteristics • `Set#contains` is faster than `Seq` Setであるための計算が裏側に隠れて遅くなってしまうので注意 -
Fringe81 Co., Ltd. たったこれだけで何が変わるか How different? val a: Seq[Int]
= Seq(1) val b: Seq[Int] = List(1) val c: Seq[Int] = 1 :: Nil -
Fringe81 Co., Ltd. :: Nil is great :: Nil
めっちゃ速い 0 27500000 55000000 82500000 110000000 troughput List#apply Seq#apply :: Nill val a: Seq[Int] = Seq(1) val b: Seq[Int] = List(1) val c: Seq[Int] = 1 :: Nil -
Fringe81 Co., Ltd. • `::` is just a class •
`:: Nil` just create new instance • `Seq()` and `List()` append to `ListBuffer` :: Nilはただnewしてるだけだから速い 1 :: Nil == new ::(1, Nil) val b = newBuilder[A] // mutable.ListBuffer b ++= elems b.result() -
Fringe81 Co., Ltd. call-by-name? 名前渡しされた引数は使用される直前に評価される Call-by-name is an argument
type like `=> T`. This argument will be evaluated just before being used. def doSomething[T](arg: => T) = ??? -
Fringe81 Co., Ltd. call-by-name 上下で第一引数の型が異なるだけで、後は全く同じ Only the type of
the first argument is different. def byName(value: => String, flag: Boolean): String = if (flag) value else "" def byValue(value: String, flag: Boolean): String = if (flag) value else "" -
Fringe81 Co., Ltd. 引数を使わなければめっちゃ速い 0 100000000 200000000 300000000 400000000
true false call-by-value call-by-name def byName(value: => String, flag: Boolean): String = if (flag) value else "" def byValue(value: String, flag: Boolean): String = if (flag) value else "" throughput(ops/s) -
Fringe81 Co., Ltd. call-by-nameは例えばloggerの実装などに便利 // just example class MyLogger(logger:
Logger, isDebugEnabled: Boolean) { def debug(msg: => String) = if (isDebugEnabled) logger.debug(msg) def info(msg: => String) = logger.info(msg) } Call-by-name can be used to implement, such as a logger. -
Fringe81 Co., Ltd. Value-class? 値クラスはAnyValを継承し、ただ1つのvalを持つもの A class declaration with
extends `AnyVal` and has just one public `val` field. class Awesome(val value: Int) extends AnyVal { def double = value * 2 } -
Fringe81 Co., Ltd. Value-class? 実行時にはそのvalフィールドの値として表現される At runtime, a value-class
instance will be represented as its `val` filed type. Therefore, `Awesome` instance is just an `Int`. class Awesome(val value: Int) extends AnyVal { def double = value * 2 } -
Fringe81 Co., Ltd. パフォーマンス観点ではどれくらい変わる? How different is it from
a performance point of view? object Normal { case class User(id: Id, name: Name) case class Id(value: Long) case class Name(value: String) } Normal.User(Normal.Id(1L), Normal.Name("hoge")) object Value { // not Value-class case class UserValue(id: IdValue, name: NameValue) case class IdValue(value: Long) extends AnyVal case class NameValue(value: String) extends AnyVal } Value.UserValue(Value.IdValue(1L), Value.NameValue("hoge")) -
Fringe81 Co., Ltd. インスタンス化は圧倒的に値クラスの方が速い 0 40000000 80000000 120000000 160000000
instantiation ops/sec Normal class Value class Normal.User(Normal.Id(1L), Normal.Name("hoge")) Value.UserValue(Value.IdValue(1L), Value.NameValue("hoge")) -
Fringe81 Co., Ltd. 値クラスはDDDやimplicit classなど使いどころは多い Value-class will suits Domain-Driven-Design.
object Value { // not Value class case class UserValue(id: IdValue, name: NameValue) case class IdValue(value: Long) extends AnyVal case class NameValue(value: String) extends AnyVal } Value.UserValue(Value.IdValue(1L), Value.NameValue("hoge")) implicit class AwesomeInt(val n: Int) extends AnyVal { def double = n * 2 } In addition, value-class helps implicit conversion performance. -
Fringe81 Co., Ltd. Futureとfor式 import Thread.sleep val result: Future[Int]
= for { a <- Future { sleep(100); println("finish 100"); 100 } b <- Future { sleep(50); println("finish 50"); 50 } } yield a + b result.onComplete(println) -
Fringe81 Co., Ltd. どっち? finish 100 finish 50 150
import Thread.sleep val result: Future[Int] = for { a <- Future { sleep(100); println("finish 100"); 100 } b <- Future { sleep(50); println("finish 50"); 50 } } yield a + b result.onComplete(println) finish 50 finish 100 150 -
Fringe81 Co., Ltd. 上から順に実行されていく import Thread.sleep val result: Future[Int]
= for { a <- Future { sleep(100); println("finish 100"); 100 } b <- Future { sleep(50); println("finish 50"); 50 } } yield a + b result.onComplete(println) finish 100 finish 50 150 Executed in order from top. -
Fringe81 Co., Ltd. Future.applyの実行タイミングに注意しましょう // call Future.apply before `for`
val aF = Future { sleep(100); 100 } val bF = Future { sleep(50); 50 } val result: Future[Int] = for { a <- aF b <- bF } yield a + b Pay attention to the timing to call `Future.apply`. // use `Future#zip` val result: Future[Int] = for { (a, b) <- Future { sleep(100); 100 } zip Future { sleep(50); 50 } } yield a + b -
Fringe81 Co., Ltd. others • Collection.breakOut • structural sub typing
• PartialFunction#orElse is expensive • recursion function • etc. その他、紹介したかったものたち -
Fringe81 Co., Ltd. Summary • Sorting is faster than Seq#sorted
• ThreadLocalRandom is faster than Random • Avoid to use Set#map as possible • Use `:: Nil` to create `Seq[T]` • Call-by-name make an arguments lazy • Value-class is useful in various situation • Be careful about timing of `Future.apply` まとめ -
Fringe81 Co., Ltd. detail link • Sorting is faster than
Seq#sorted • https://gist.github.com/petitviolet/07f8459f8a40afd54bea00343548b080 • ThreadLocalRandom is faster than Random • https://gist.github.com/petitviolet/89886339e028779172a293a01e18d8f0 • Avoid to use Set#map as possible • https://gist.github.com/petitviolet/dc5b44fae57277ae915bd770ba4f2435 • Use `:: Nil` to create `Seq[T]` • https://gist.github.com/petitviolet/b67d63aad1a23350f5fa5266d077efca • Call-by-name make an arguments lazy • https://gist.github.com/petitviolet/bb0f14aad27021c46efecd972690d9b3 • Value-class is useful in various situation • https://gist.github.com/petitviolet/026979105dbe447c93d47a778a604619 • Be careful about timing of `Future.apply` • https://gist.github.com/petitviolet/d5ed010ce4ce12bc250b3f259c97ff02 もうちょっと詳しく -
Fringe81 Co., Ltd. Summary • Before relying on such tips,
there should be other things to do. • e.g.) caching to avoid I/O, RDBMS index,… • Faster is better than slower • May such tips help you tipsに頼る前に何かしらやるべきことはある 遅いよりは速い方が良いので、tipsでも何かしら役に立てば