ScalaBlitz

Optimizing the parallel collection operations.

Transcript

1. ScalaBlitz Efficient Collections Framework
2. What’s a Blitz?
3. Blitz-chess is a style of rapid chess play.
4. Blitz-chess is a style of rapid chess play.
5. Knights have horses.
6. Horses run fast.
7. def mean(xs: Array[Float]): Float = xs.par.reduce(_ + _) / xs.length
10. With Lists, operations can only be executed from left to right
11. 1 2 4 8
12. 1 2 4 8 Not your typical list.
13. Bon app.
18. Apparently not enough
20. No amount of documentation is apparently enough
22. The reduceLeft guarantees operations are executed from left to right
23. Parallel and sequential collections sharing operations
24. There are several problems here
25. How we see users
26. How users see the docs
27. Bending the truth.
28. And sometimes we were just slow
29. So, we have a new API now def findDoe(names: Array[String]): Option[String] = { names.toPar.find(_.endsWith(“Doe”)) }
30. Wait, you renamed a method? def findDoe(names: Array[String]): Option[String] = { names.toPar.find(_.endsWith(“Doe”)) }
31. Yeah, par already exists. But, toPar is different. def findDoe(names: Array[String]): Option[String] = { names.toPar.find(_.endsWith(“Doe”)) }
32. def findDoe(names: Array[String]): Option[String] = { names.toPar.find(_.endsWith(“Doe”)) } implicit class ParOps[Repr](val r: Repr) extends AnyVal { def toPar = new Par(r) }
33. def findDoe(names: Array[String]): Option[String] = { ParOps(names).toPar.find(_.endsWith(“Doe”)) } implicit class ParOps[Repr](val r: Repr) extends AnyVal { def toPar = new Par(r) }
34. def findDoe(names: Array[String]): Option[String] = { ParOps(names).toPar.find(_.endsWith(“Doe”)) } implicit class ParOps[Repr](val r: Repr) extends AnyVal { def toPar = new Par(r) } class Par[Repr](r: Repr)
35. def findDoe(names: Array[String]): Option[String] = { (new Par(names)).find(_.endsWith(“Doe”)) } implicit class ParOps[Repr](val r: Repr) extends AnyVal { def toPar = new Par(r) } class Par[Repr](r: Repr)
36. def findDoe(names: Array[String]): Option[String] = { (new Par(names)).find(_.endsWith(“Doe”)) } class Par[Repr](r: Repr) But, Par[Repr] does not have the find method!
37. True, but Par[Array[String]] does have a find method. def findDoe(names: Array[String]): Option[String] = { (new Par(names)).find(_.endsWith(“Doe”)) } class Par[Repr](r: Repr)
38. def findDoe(names: Array[String]): Option[String] = { (new Par(names)).find(_.endsWith(“Doe”)) } class Par[Repr](r: Repr) implicit class ParArrayOps[T](pa: Par[Array[T]]) { ... def find(p: T => Boolean): Option[T] ... }
39. More flexible!
40. More flexible! • does not have to implement methods that make no sense in parallel
41. More flexible! • does not have to implement methods that make no sense in parallel • slow conversions explicit
42. No standard library collections were hurt doing this. No standard library collections were hurt doing this.
43. More flexible! • does not have to implement methods that make no sense in parallel • slow conversions explicit • non-intrusive addition to standard library
44. More flexible! • does not have to implement methods that make no sense in parallel • slow conversions explicit • non-intrusive addition to standard library • easy to add new methods and collections
45. More flexible! • does not have to implement methods that make no sense in parallel • slow conversions explicit • non-intrusive addition to standard library • easy to add new methods and collections • import switches between implementations
46. def findDoe(names: Seq[String]): Option[String] = { names.toPar.find(_.endsWith(“Doe”)) }
47. def findDoe(names: Seq[String]): Option[String] = { names.toPar.find(_.endsWith(“Doe”)) }
48. But how do I write generic code? def findDoe(names: Seq[String]): Option[String] = { names.toPar.find(_.endsWith(“Doe”)) }
49. def findDoe[Repr[_]](names: Par[Repr[String]]) = { names.toPar.find(_.endsWith(“Doe”)) }
50. Par[Repr[String]]does not have a find def findDoe[Repr[_]](names: Par[Repr[String]]) = { names.toPar.find(_.endsWith(“Doe”)) }
51. def findDoe[Repr[_]: Ops](names: Par[Repr[String]]) = { names.toPar.find(_.endsWith(“Doe”)) }
52. def findDoe[Repr[_]: Ops](names: Par[Repr[String]]) = { names.toPar.find(_.endsWith(“Doe”)) } We don’t do this.
53. Make everything as simple as possible, but not simpler.
54. def findDoe(names: Reducable[String])= { names.find(_.endsWith(“Doe”)) }
55. def findDoe(names: Reducable[String])= { names.find(_.endsWith(“Doe”)) } findDoe(Array(1, 2, 3).toPar)
56. def findDoe(names: Reducable[String])= { names.find(_.endsWith(“Doe”)) } findDoe(toReducable(Array(1, 2, 3).toPar))
57. def findDoe(names: Reducable[String])= { names.find(_.endsWith(“Doe”)) } findDoe(toReducable(Array(1, 2, 3).toPar)) def arrayIsReducable[T]: IsReducable[T] = { … }
58. So let’s write a program!
60. import scala.collection.par._ val pixels = new Array[Int](wdt * hgt) for (idx <- (0 until (wdt * hgt)).toPar) { }
61. import scala.collection.par._ val pixels = new Array[Int](wdt * hgt) for (idx <- (0 until (wdt * hgt)).toPar) { val x = idx % wdt val y = idx / wdt }
62. import scala.collection.par._ val pixels = new Array[Int](wdt * hgt) for (idx <- (0 until (wdt * hgt)).toPar) { val x = idx % wdt val y = idx / wdt pixels(idx) = computeColor(x, y) }
63. import scala.collection.par._ val pixels = new Array[Int](wdt * hgt) for (idx <- (0 until (wdt * hgt)).toPar) { val x = idx % wdt val y = idx / wdt pixels(idx) = computeColor(x, y) } Scheduler not found!
64. import scala.collection.par._ import Scheduler.Implicits.global val pixels = new Array[Int](wdt * hgt) for (idx <- (0 until (wdt * hgt)).toPar) { val x = idx % wdt val y = idx / wdt pixels(idx) = computeColor(x, y) }
65. import scala.collection.par._ import Scheduler.Implicits.global val pixels = new Array[Int](wdt * hgt) for (idx <- (0 until (wdt * hgt)).toPar) { val x = idx % wdt val y = idx / wdt pixels(idx) = computeColor(x, y) }
66. New parallel collections 33% faster! Now 103 ms Previously 148 ms
67. Workstealing tree scheduler rocks!
68. Workstealing tree scheduler rocks! But, are there other interesting
69. Fine-grained uniform workloads are on the opposite side of the spectrum.
70. def mean(xs: Array[Float]): Float = { val sum = xs.toPar.fold(0)(_ + _) sum / xs.length }
71. def mean(xs: Array[Float]): Float = { val sum = xs.toPar.fold(0)(_ + _) sum / xs.length } Now 15 ms Previously 565 ms
72. But how?
73. def fold[T](a: Iterable[T])(z:T)(op: (T, T) => T) = { var it = a.iterator var acc = z while (it.hasNext) { acc = op(acc, it.next) } acc }
74. def fold[T](a: Iterable[T])(z:T)(op: (T, T) => T) = { var it = a.iterator var acc = z while (it.hasNext) { acc = box(op(acc, it.next)) } acc }
75. def fold[T](a: Iterable[T])(z:T)(op: (T, T) => T) = { var it = a.iterator var acc = z while (it.hasNext) { acc = box(op(acc, it.next)) } acc } Generic methods cause boxing of primitives
76. def mean(xs: Array[Float]): Float = { val sum = xs.toPar.fold(0)(_ + _) sum / xs.length }
77. def mean(xs: Array[Float]): Float = { val sum = xs.toPar.fold(0)(_ + _) sum / xs.length } Generic methods hurt performance What can we do instead?
78. def mean(xs: Array[Float]): Float = { val sum = xs.toPar.fold(0)(_ + _) sum / xs.length } Generic methods hurt performance What can we do instead? Inline method body!
79. def mean(xs: Array[Float]): Float = { val sum = { var it = xs.iterator var acc = 0 while (it.hasNext) { acc = acc + it.next } acc } sum / xs.length }
80. def mean(xs: Array[Float]): Float = { val sum = { var it = xs.iterator var acc = 0 while (it.hasNext) { acc = acc + it.next } acc } sum / xs.length } Specific type No boxing! No memory allocation!
81. def mean(xs: Array[Float]): Float = { val sum = { var it = xs.iterator var acc = 0 while (it.hasNext) { acc = acc + it.next } acc } sum / xs.length } Specific type No boxing! No memory allocation! 2X speedup 565 ms → 281 ms
82. def mean(xs: Array[Float]): Float = { val sum = { var it = xs.iterator var acc = 0 while (it.hasNext) { acc = acc + it.next } acc } sum / xs.length }
83. def mean(xs: Array[Float]): Float = { val sum = { var it = xs.iterator var acc = 0 while (it.hasNext) { acc = acc + it.next } acc } sum / xs.length } Iterators? For Array? We don’t need them!
84. def mean(xs: Array[Float]): Float = { val sum = { var i = 0 val until = xs.size var acc = 0 while (i < until) { acc = acc + a(i) i = i + 1 } acc } sum / xs.length } Use index-based access!
85. def mean(xs: Array[Float]): Float = { val sum = { var i = 0 val until = xs.size var acc = 0 while (i < until) { acc = acc + a(i) i = i + 1 } acc } sum / xs.length } 19x speedup Use index-based access! 281 ms → 15 ms
86. Are those optimizations parallel-collections specific?
87. Are those optimizations parallel-collections specific? No
88. Are those optimizations parallel-collections specific? No You can use them on sequential collections
89. def mean(xs: Array[Float]): Float = { val sum = xs.fold(0)(_ + _) sum / xs.length }
90. import scala.collections.optimizer._ def mean(xs: Array[Float]): Float = optimize{ val sum = xs.fold(0)(_ + _) sum / xs.length }
91. import scala.collections.optimizer._ def mean(xs: Array[Float]): Float = optimize{ val sum = xs.fold(0)(_ + _) sum / xs.length } You get 38 times speedup!
92. Future work
93. @specialized collections • Maps • Sets • Lists • Vectors Both faster & consuming less memory
94. @specialized collections • Maps • Sets • Lists • Vectors Both faster & consuming less memory Expect to get this for free inside optimize{} block
95. jdk8-style streams(parallel views) • Fast • Lightweight • Expressive API • Optimized Lazy data-parallel operations made easy
96. Future’s based asynchronous API val sum = future{ xs.sum } val normalized = sum.andThen(sum => sum/xs.size) Boilerplate code, ugly
97. Future’s based asynchronous API val sum = xs.toFuture.sum val scaled = xs.map(_ / sum) • Simple to use • Lightweight • Expressive API • Optimized Asynchronous dat parallel operations made easy
98. Current research: operation fusion val minMaleAge = people.filter(_.isMale) .map(_.age).min val minFemaleAge = people.filter(_.isFemale) .map(_.age).min
99. Current research: operation fusion val minMaleAge = people.filter(_.isMale) .map(_.age).min val minFemaleAge = people.filter(_.isFemale) .map(_.age).min • Requires up to 3 times more memory than original collection • Requires 6 traversals of collections
100. Current research: operation fusion val minMaleAge = people.filter(_.isMale) .map(_.age).min val minFemaleAge = people.filter(_.isFemale) .map(_.age).min • Requires up to 3 times more memory than original collection • Requires 6 traversals of collections We aim to reduce this to single traversal with no additional memory. Without you changing your code