Aleksandar Prokopec
June 14, 2011
69

# Parallel Collections Overview

Parallel Collections presentation from ScalaDays 2011.

June 14, 2011

## Transcript

2. ### Scala collections for { s <- surnames n <- names

if s endsWith n } yield (n, s) McDonald
3. ### Scala collections for { s <- surnames n <- names

if s endsWith n } yield (n, s) 1040 ms
4. ### Scala parallel collections for { s <- surnames n <-

names if s endsWith n } yield (n, s)
5. ### Scala parallel collections for { s <- surnames.par n <-

names.par if s endsWith n } yield (n, s)
6. ### Scala parallel collections for { s <- surnames.par n <-

names.par if s endsWith n } yield (n, s) 2 cores 575 ms
7. ### Scala parallel collections for { s <- surnames.par n <-

names.par if s endsWith n } yield (n, s) 4 cores 305 ms
8. ### for comprehensions surnames.par.flatMap { s => names.par .filter(n => s

endsWith n) .map(n => (n, s)) }
9. ### for comprehensions nested parallelized bulk operations surnames.par.flatMap { s =>

names.par .filter(n => s endsWith n) .map(n => (n, s)) }

11. ### Nested parallelism parallel within parallel composition surnames.par.flatMap { s =>

surnameToCollection(s) // may invoke parallel ops }

13. ### Nested parallelism going recursive def vowel(c: Char): Boolean = ...

def gen(n: Int, acc: Seq[String]): Seq[String] = if (n == 0) acc
14. ### Nested parallelism going recursive def vowel(c: Char): Boolean = ...

def gen(n: Int, acc: Seq[String]): Seq[String] = if (n == 0) acc else for (s <- gen(n - 1, acc); c <- 'a' to 'z') yield recursive algorithms
15. ### Nested parallelism going recursive def vowel(c: Char): Boolean = ...

def gen(n: Int, acc: Seq[String]): Seq[String] = if (n == 0) acc else for (s <- gen(n - 1, acc); c <- 'a' to 'z') yield if (s.length == 0) s + c
16. ### Nested parallelism going recursive def vowel(c: Char): Boolean = ...

def gen(n: Int, acc: Seq[String]): Seq[String] = if (n == 0) acc else for (s <- gen(n - 1, acc); c <- 'a' to 'z') yield if (s.length == 0) s + c else if (vowel(s.last) && !vowel(c)) s + c else if (!vowel(s.last) && vowel(c)) s + c
17. ### Nested parallelism going recursive def vowel(c: Char): Boolean = ...

def gen(n: Int, acc: Seq[String]): Seq[String] = if (n == 0) acc else for (s <- gen(n - 1, acc); c <- 'a' to 'z') yield if (s.length == 0) s + c else if (vowel(s.last) && !vowel(c)) s + c else if (!vowel(s.last) && vowel(c)) s + c else s gen(5, Array(""))
18. ### Nested parallelism going recursive def vowel(c: Char): Boolean = ...

def gen(n: Int, acc: Seq[String]): Seq[String] = if (n == 0) acc else for (s <- gen(n - 1, acc); c <- 'a' to 'z') yield if (s.length == 0) s + c else if (vowel(s.last) && !vowel(c)) s + c else if (!vowel(s.last) && vowel(c)) s + c else s gen(5, Array("")) 1545 ms
19. ### Nested parallelism going recursive def vowel(c: Char): Boolean = ...

def gen(n: Int, acc: ParSeq[String]): ParSeq[String] = if (n == 0) acc else for (s <- gen(n - 1, acc); c <- 'a' to 'z') yield if (s.length == 0) s + c else if (vowel(s.last) && !vowel(c)) s + c else if (!vowel(s.last) && vowel(c)) s + c else s gen(5, ParArray(""))
20. ### Nested parallelism going recursive def vowel(c: Char): Boolean = ...

def gen(n: Int, acc: ParSeq[String]): ParSeq[String] = if (n == 0) acc else for (s <- gen(n - 1, acc); c <- 'a' to 'z') yield if (s.length == 0) s + c else if (vowel(s.last) && !vowel(c)) s + c else if (!vowel(s.last) && vowel(c)) s + c else s gen(5, ParArray("")) 1 core 1575 ms
21. ### Nested parallelism going recursive def vowel(c: Char): Boolean = ...

def gen(n: Int, acc: ParSeq[String]): ParSeq[String] = if (n == 0) acc else for (s <- gen(n - 1, acc); c <- 'a' to 'z') yield if (s.length == 0) s + c else if (vowel(s.last) && !vowel(c)) s + c else if (!vowel(s.last) && vowel(c)) s + c else s gen(5, ParArray("")) 2 cores 809 ms
22. ### Nested parallelism going recursive def vowel(c: Char): Boolean = ...

def gen(n: Int, acc: ParSeq[String]): ParSeq[String] = if (n == 0) acc else for (s <- gen(n - 1, acc); c <- 'a' to 'z') yield if (s.length == 0) s + c else if (vowel(s.last) && !vowel(c)) s + c else if (!vowel(s.last) && vowel(c)) s + c else s gen(5, ParArray("")) 4 cores 530 ms

25. ### Character count use case for foldLeft val txt: String =

... txt.foldLeft(0) { case (a, ‘ ‘) => a case (a, c) => a + 1 }
26. ### 6 5 4 3 2 1 0 Character count use

case for foldLeft txt.foldLeft(0) { case (a, ‘ ‘) => a case (a, c) => a + 1 } going left to right - not parallelizable! A B C D E F _ + 1
27. ### Character count use case for foldLeft txt.foldLeft(0) { case (a,

‘ ‘) => a case (a, c) => a + 1 } going left to right – not really necessary 3 2 1 0 A B C _ + 1 3 2 1 0 D E F _ + 1 _ + _ 6
28. ### Character count in parallel txt.fold(0) { case (a, ‘ ‘)

=> a case (a, c) => a + 1 }
29. ### Character count in parallel txt.fold(0) { case (a, ‘ ‘)

=> a case (a, c) => a + 1 } 3 2 1 1 A B C _ + 1 3 2 1 1 A B C : (Int, Char) => Int
30. ### Character count fold not applicable txt.fold(0) { case (a, ‘

‘) => a case (a, c) => a + 1 } 3 2 1 3 A B C _ + _ 3 3 3 2 1 3 A B C ! (Int, Int) => Int
31. ### Character count use case for aggregate txt.aggregate(0)({ case (a, ‘

‘) => a case (a, c) => a + 1 }, _ + _)
32. ### 3 2 1 1 A B C Character count use

case for aggregate txt.aggregate(0)({ case (a, ‘ ‘) => a case (a, c) => a + 1 }, _ + _) _ + _ 3 3 3 2 1 3 A B C _ + 1
33. ### Character count use case for aggregate aggregation  element 3

2 1 1 A B C _ + _ 3 3 3 2 1 3 A B C txt.aggregate(0)({ case (a, ‘ ‘) => a case (a, c) => a + 1 }, _ + _) B _ + 1
34. ### Character count use case for aggregate aggregation  aggregation aggregation

 element 3 2 1 1 A B C _ + _ 3 3 3 2 1 3 A B C txt.aggregate(0)({ case (a, ‘ ‘) => a case (a, c) => a + 1 }, _ + _) B _ + 1
35. ### Word count another use case for foldLeft txt.foldLeft((0, true)) {

case ((wc, _), ' ') => (wc, true) case ((wc, true), x) => (wc + 1, false) case ((wc, false), x) => (wc, false) }
36. ### Word count initial accumulation txt.foldLeft((0, true)) { case ((wc, _),

' ') => (wc, true) case ((wc, true), x) => (wc + 1, false) case ((wc, false), x) => (wc, false) } 0 words so far last character was a space “Folding me softly.”
37. ### Word count a space txt.foldLeft((0, true)) { case ((wc, _),

' ') => (wc, true) case ((wc, true), x) => (wc + 1, false) case ((wc, false), x) => (wc, false) } “Folding me softly.” last seen character is a space
38. ### Word count a non space txt.foldLeft((0, true)) { case ((wc,

_), ' ') => (wc, true) case ((wc, true), x) => (wc + 1, false) case ((wc, false), x) => (wc, false) } “Folding me softly.” last seen character was a space – a new word
39. ### Word count a non space txt.foldLeft((0, true)) { case ((wc,

_), ' ') => (wc, true) case ((wc, true), x) => (wc + 1, false) case ((wc, false), x) => (wc, false) } “Folding me softly.” last seen character wasn’t a space – no new word

41. ### Word count in parallel “softly.“ “Folding me “ wc =

2; rs = 1 wc = 1; ls = 0  P1 P2
42. ### Word count in parallel “softly.“ “Folding me “ wc =

2; rs = 1 wc = 1; ls = 0  wc = 3 P1 P2
43. ### Word count must assume arbitrary partitions “g me softly.“ “Foldin“

wc = 1; rs = 0 wc = 3; ls = 0  P1 P2
44. ### Word count must assume arbitrary partitions “g me softly.“ “Foldin“

wc = 1; rs = 0 wc = 3; ls = 0  P1 P2 wc = 3

46. ### Word count initial aggregation txt.par.aggregate((0, 0, 0)) # spaces on

the left # spaces on the right #words
47. ### Word count initial aggregation txt.par.aggregate((0, 0, 0)) # spaces on

the left # spaces on the right #words ””
48. ### Word count aggregation  aggregation ... }, { case ((0,

0, 0), res) => res case (res, (0, 0, 0)) => res ““ “Folding me“  “softly.“ ““ 
49. ### Word count aggregation  aggregation ... }, { case ((0,

0, 0), res) => res case (res, (0, 0, 0)) => res case ((lls, lwc, 0), (0, rwc, rrs)) => (lls, lwc + rwc - 1, rrs) “e softly.“ “Folding m“ 
50. ### Word count aggregation  aggregation ... }, { case ((0,

0, 0), res) => res case (res, (0, 0, 0)) => res case ((lls, lwc, 0), (0, rwc, rrs)) => (lls, lwc + rwc - 1, rrs) case ((lls, lwc, _), (_, rwc, rrs)) => (lls, lwc + rwc, rrs) “ softly.“ “Folding me” 
51. ### Word count aggregation  element txt.par.aggregate((0, 0, 0))({ case ((ls,

0, _), ' ') => (ls + 1, 0, ls + 1) ”_” 0 words and a space – add one more space each side
52. ### Word count aggregation  element txt.par.aggregate((0, 0, 0))({ case ((ls,

0, _), ' ') => (ls + 1, 0, ls + 1) case ((ls, 0, _), c) => (ls, 1, 0) ” m” 0 words and a non-space – one word, no spaces on the right side
53. ### Word count aggregation  element txt.par.aggregate((0, 0, 0))({ case ((ls,

0, _), ' ') => (ls + 1, 0, ls + 1) case ((ls, 0, _), c) => (ls, 1, 0) case ((ls, wc, rs), ' ') => (ls, wc, rs + 1) ” me_” nonzero words and a space – one more space on the right side
54. ### Word count aggregation  element txt.par.aggregate((0, 0, 0))({ case ((ls,

0, _), ' ') => (ls + 1, 0, ls + 1) case ((ls, 0, _), c) => (ls, 1, 0) case ((ls, wc, rs), ' ') => (ls, wc, rs + 1) case ((ls, wc, 0), c) => (ls, wc, 0) ” me sof” nonzero words, last non-space and current non-space – no change
55. ### Word count aggregation  element txt.par.aggregate((0, 0, 0))({ case ((ls,

0, _), ' ') => (ls + 1, 0, ls + 1) case ((ls, 0, _), c) => (ls, 1, 0) case ((ls, wc, rs), ' ') => (ls, wc, rs + 1) case ((ls, wc, 0), c) => (ls, wc, 0) case ((ls, wc, rs), c) => (ls, wc + 1, 0) ” me s” nonzero words, last space and current non-space – one more word
56. ### Word count in parallel txt.par.aggregate((0, 0, 0))({ case ((ls, 0,

_), ' ') => (ls + 1, 0, ls + 1) case ((ls, 0, _), c) => (ls, 1, 0) case ((ls, wc, rs), ' ') => (ls, wc, rs + 1) case ((ls, wc, 0), c) => (ls, wc, 0) case ((ls, wc, rs), c) => (ls, wc + 1, 0) }, { case ((0, 0, 0), res) => res case (res, (0, 0, 0)) => res case ((lls, lwc, 0), (0, rwc, rrs)) => (lls, lwc + rwc - 1, rrs) case ((lls, lwc, _), (_, rwc, rrs)) => (lls, lwc + rwc, rrs) })
57. ### Word count using parallel strings? txt.par.aggregate((0, 0, 0))({ case ((ls,

0, _), ' ') => (ls + 1, 0, ls + 1) case ((ls, 0, _), c) => (ls, 1, 0) case ((ls, wc, rs), ' ') => (ls, wc, rs + 1) case ((ls, wc, 0), c) => (ls, wc, 0) case ((ls, wc, rs), c) => (ls, wc + 1, 0) }, { case ((0, 0, 0), res) => res case (res, (0, 0, 0)) => res case ((lls, lwc, 0), (0, rwc, rrs)) => (lls, lwc + rwc - 1, rrs) case ((lls, lwc, _), (_, rwc, rrs)) => (lls, lwc + rwc, rrs) })

59. ### Word count string not really parallelizable scala> (txt: String).par collection.parallel.ParSeq[Char]

= ParArray(…)
60. ### Word count string not really parallelizable scala> (txt: String).par collection.parallel.ParSeq[Char]

= ParArray(…) different internal representation!
61. ### Word count string not really parallelizable scala> (txt: String).par collection.parallel.ParSeq[Char]

= ParArray(…) different internal representation! ParArray
62. ### Word count string not really parallelizable scala> (txt: String).par collection.parallel.ParSeq[Char]

= ParArray(…) different internal representation! ParArray  copy string contents into an array
63. ### Conversions going parallel // `par` is efficient for... mutable.{Array, ArrayBuffer,

ArraySeq} mutable.{HashMap, HashSet} immutable.{Vector, Range} immutable.{HashMap, HashSet}
64. ### Conversions going parallel // `par` is efficient for... mutable.{Array, ArrayBuffer,

ArraySeq} mutable.{HashMap, HashSet} immutable.{Vector, Range} immutable.{HashMap, HashSet} most other collections construct a new parallel collection!
65. ### Conversions going parallel sequential parallel Array, ArrayBuffer, ArraySeq mutable.ParArray mutable.HashMap

mutable.ParHashMap mutable.HashSet mutable.ParHashSet immutable.Vector immutable.ParVector immutable.Range immutable.ParRange immutable.HashMap immutable.ParHashMap immutable.HashSet immutable.ParHashSet
66. ### Conversions going parallel // `seq` is always efficient ParArray(1, 2,

3).seq List(1, 2, 3, 4).seq ParHashMap(1 -> 2, 3 -> 4).seq ”abcd”.seq // `par` may not be... ”abcd”.par

70. ### Custom collection class ParString(val str: String) extends parallel.immutable.ParSeq[Char] { def

apply(i: Int) = str.charAt(i) def length = str.length
71. ### Custom collection class ParString(val str: String) extends parallel.immutable.ParSeq[Char] { def

apply(i: Int) = str.charAt(i) def length = str.length def seq = new WrappedString(str)
72. ### Custom collection class ParString(val str: String) extends parallel.immutable.ParSeq[Char] { def

apply(i: Int) = str.charAt(i) def length = str.length def seq = new WrappedString(str) def splitter: Splitter[Char]
73. ### Custom collection class ParString(val str: String) extends parallel.immutable.ParSeq[Char] { def

apply(i: Int) = str.charAt(i) def length = str.length def seq = new WrappedString(str) def splitter = new ParStringSplitter(0, str.length)
74. ### Custom collection splitter definition class ParStringSplitter(var i: Int, len: Int)

extends Splitter[Char] {
75. ### Custom collection splitters are iterators class ParStringSplitter(i: Int, len: Int)

extends Splitter[Char] { def hasNext = i < len def next = { val r = str.charAt(i) i += 1 r }
76. ### Custom collection splitters must be duplicated ... def dup =

new ParStringSplitter(i, len)
77. ### Custom collection splitters know how many elements remain ... def

dup = new ParStringSplitter(i, len) def remaining = len - i
78. ### Custom collection splitters can be split ... def psplit(sizes: Int*):

Seq[ParStringSplitter] = { val splitted = new ArrayBuffer[ParStringSplitter] for (sz <- sizes) { val next = (i + sz) min ntl splitted += new ParStringSplitter(i, next) i = next } splitted }
79. ### Word count now with parallel strings new ParString(txt).aggregate((0, 0, 0))({

case ((ls, 0, _), ' ') => (ls + 1, 0, ls + 1) case ((ls, 0, _), c) => (ls, 1, 0) case ((ls, wc, rs), ' ') => (ls, wc, rs + 1) case ((ls, wc, 0), c) => (ls, wc, 0) case ((ls, wc, rs), c) => (ls, wc + 1, 0) }, { case ((0, 0, 0), res) => res case (res, (0, 0, 0)) => res case ((lls, lwc, 0), (0, rwc, rrs)) => (lls, lwc + rwc - 1, rrs) case ((lls, lwc, _), (_, rwc, rrs)) => (lls, lwc + rwc, rrs) })
80. ### Word count performance txt.foldLeft((0, true)) { case ((wc, _), '

') => (wc, true) case ((wc, true), x) => (wc + 1, false) case ((wc, false), x) => (wc, false) } new ParString(txt).aggregate((0, 0, 0))({ case ((ls, 0, _), ' ') => (ls + 1, 0, ls + 1) case ((ls, 0, _), c) => (ls, 1, 0) case ((ls, wc, rs), ' ') => (ls, wc, rs + 1) case ((ls, wc, 0), c) => (ls, wc, 0) case ((ls, wc, rs), c) => (ls, wc + 1, 0) }, { case ((0, 0, 0), res) => res case (res, (0, 0, 0)) => res case ((lls, lwc, 0), (0, rwc, rrs)) => (lls, lwc + rwc - 1, rrs) case ((lls, lwc, _), (_, rwc, rrs)) => (lls, lwc + rwc, rrs) }) 100 ms cores: 1 2 4 time: 137 ms 70 ms 35 ms

82. ### Hierarchy def nonEmpty(sq: Seq[String]) = { val res = new

mutable.ArrayBuffer[String]() for (s <- sq) { if (s.nonEmpty) res += s } res }
83. ### Hierarchy def nonEmpty(sq: ParSeq[String]) = { val res = new

mutable.ArrayBuffer[String]() for (s <- sq) { if (s.nonEmpty) res += s } res }
84. ### Hierarchy def nonEmpty(sq: ParSeq[String]) = { val res = new

mutable.ArrayBuffer[String]() for (s <- sq) { if (s.nonEmpty) res += s } res } side-effects! ArrayBuffer is not synchronized!
85. ### Hierarchy def nonEmpty(sq: ParSeq[String]) = { val res = new

mutable.ArrayBuffer[String]() for (s <- sq) { if (s.nonEmpty) res += s } res } side-effects! ArrayBuffer is not synchronized! ParSeq Seq
86. ### Hierarchy def nonEmpty(sq: GenSeq[String]) = { val res = new

mutable.ArrayBuffer[String]() for (s <- sq) { if (s.nonEmpty) res.synchronized { res += s } } res }
87. ### Accessors vs. transformers some methods need more than just splitters

foreach, reduce, find, sameElements, indexOf, corresponds, forall, exists, max, min, sum, count, … map, flatMap, filter, partition, ++, take, drop, span, zip, patch, padTo, …
88. ### Accessors vs. transformers some methods need more than just splitters

foreach, reduce, find, sameElements, indexOf, corresponds, forall, exists, max, min, sum, count, … map, flatMap, filter, partition, ++, take, drop, span, zip, patch, padTo, … These return collections!
89. ### Accessors vs. transformers some methods need more than just splitters

foreach, reduce, find, sameElements, indexOf, corresponds, forall, exists, max, min, sum, count, … map, flatMap, filter, partition, ++, take, drop, span, zip, patch, padTo, … Sequential collections – builders
90. ### Accessors vs. transformers some methods need more than just splitters

foreach, reduce, find, sameElements, indexOf, corresponds, forall, exists, max, min, sum, count, … map, flatMap, filter, partition, ++, take, drop, span, zip, patch, padTo, … Sequential collections – builders Parallel collections – combiners
91. ### Builders building a sequential collection 1 2 3 4 5

6 7 Nil 2 4 6 Nil ListBuilder += += += result

93. ### Combiners building parallel collections trait Combiner[-Elem, +To] extends Builder[Elem, To]

{ def combine[N <: Elem, NewTo >: To] (other: Combiner[N, NewTo]): Combiner[N, NewTo] }
94. ### Combiners building parallel collections trait Combiner[-Elem, +To] extends Builder[Elem, To]

{ def combine[N <: Elem, NewTo >: To] (other: Combiner[N, NewTo]): Combiner[N, NewTo] } Combiner Combiner Combiner
95. ### Combiners building parallel collections trait Combiner[-Elem, +To] extends Builder[Elem, To]

{ def combine[N <: Elem, NewTo >: To] (other: Combiner[N, NewTo]): Combiner[N, NewTo] } Should be efficient – O(log n) worst case
96. ### Combiners building parallel collections trait Combiner[-Elem, +To] extends Builder[Elem, To]

{ def combine[N <: Elem, NewTo >: To] (other: Combiner[N, NewTo]): Combiner[N, NewTo] } How to implement this combine?
97. ### Parallel arrays 1, 2, 3, 4 5, 6, 7, 8

2, 4 6, 8 3, 1, 8, 0 2, 2, 1, 9 8, 0 2, 2 merge merge merge copy allocate 2 4 6 8 8 0 2 2

99. ### Parallel hash tables ParHashMap 0 1 2 4 5 7

8 9 e.g. calling filter
100. ### Parallel hash tables ParHashMap 0 1 2 4 5 7

8 9 ParHashCombiner ParHashCombiner e.g. calling filter 0 5 1 7 9 4
101. ### Parallel hash tables ParHashMap 0 1 2 4 5 7

8 9 ParHashCombiner 0 1 4 ParHashCombiner 5 7 9
102. ### Parallel hash tables ParHashMap 0 1 2 4 5 7

8 9 ParHashCombiner 0 1 4 ParHashCombiner 5 9 5 7 0 1 4 7 9

7 0 1 4 9
104. ### 5 7 8 9 1 4 0 Parallel hash tables

buckets! ParHashCombiner ParHashCombiner 0 1 4 9 7 5 ParHashMap 2 0 = 00002 1 = 00012 4 = 01002

5 combine
106. ### Parallel hash tables ParHashCombiner ParHashCombiner 9 7 5 0 1

4 ParHashCombiner no copying!

109. ### Custom combiners for methods returning custom collections new ParString(txt).filter(_ !=

‘ ‘) What is the return type here?
110. ### Custom combiners for methods returning custom collections new ParString(txt).filter(_ !=

‘ ‘) creates a ParVector!
111. ### Custom combiners for methods returning custom collections new ParString(txt).filter(_ !=

‘ ‘) creates a ParVector! class ParString(val str: String) extends parallel.immutable.ParSeq[Char] { def apply(i: Int) = str.charAt(i) ...
112. ### Custom combiners for methods returning custom collections class ParString(val str:

String) extends immutable.ParSeq[Char] with ParSeqLike[Char, ParString, WrappedString] { def apply(i: Int) = str.charAt(i) ...
113. ### Custom combiners for methods returning custom collections class ParString(val str:

String) extends immutable.ParSeq[Char] with ParSeqLike[Char, ParString, WrappedString] { def apply(i: Int) = str.charAt(i) ... protected[this] override def newCombiner : Combiner[Char, ParString]
114. ### Custom combiners for methods returning custom collections class ParString(val str:

String) extends immutable.ParSeq[Char] with ParSeqLike[Char, ParString, WrappedString] { def apply(i: Int) = str.charAt(i) ... protected[this] override def newCombiner = new ParStringCombiner
115. ### Custom combiners for methods returning custom collections class ParStringCombiner extends

Combiner[Char, ParString] {
116. ### Custom combiners for methods returning custom collections class ParStringCombiner extends

Combiner[Char, ParString] { var size = 0
117. ### Custom combiners for methods returning custom collections class ParStringCombiner extends

Combiner[Char, ParString] { var size = 0 size
118. ### Custom combiners for methods returning custom collections class ParStringCombiner extends

Combiner[Char, ParString] { var size = 0 val chunks = ArrayBuffer(new StringBuilder) size
119. ### Custom combiners for methods returning custom collections class ParStringCombiner extends

Combiner[Char, ParString] { var size = 0 val chunks = ArrayBuffer(new StringBuilder) size chunks
120. ### Custom combiners for methods returning custom collections class ParStringCombiner extends

Combiner[Char, ParString] { var size = 0 val chunks = ArrayBuffer(new StringBuilder) var lastc = chunks.last size chunks
121. ### Custom combiners for methods returning custom collections class ParStringCombiner extends

Combiner[Char, ParString] { var size = 0 val chunks = ArrayBuffer(new StringBuilder) var lastc = chunks.last size lastc chunks
122. ### Custom combiners for methods returning custom collections class ParStringCombiner extends

Combiner[Char, ParString] { var size = 0 val chunks = ArrayBuffer(new StringBuilder) var lastc = chunks.last def +=(elem: Char) = { lastc += elem size += 1 this }
123. ### Custom combiners for methods returning custom collections class ParStringCombiner extends

Combiner[Char, ParString] { var size = 0 val chunks = ArrayBuffer(new StringBuilder) var lastc = chunks.last def +=(elem: Char) = { lastc += elem size += 1 this } size lastc chunks +1
124. ### Custom combiners for methods returning custom collections ... def combine[U

<: Char, NewTo >: ParString] (other: Combiner[U, NewTo]) = other match { case psc: ParStringCombiner => sz += that.sz chunks ++= that.chunks lastc = chunks.last this }
125. ### Custom combiners for methods returning custom collections ... def combine[U

<: Char, NewTo >: ParString] (other: Combiner[U, NewTo]) lastc chunks lastc chunks
126. ### Custom combiners for methods returning custom collections ... def result

= { val rsb = new StringBuilder for (sb <- chunks) rsb.append(sb) new ParString(rsb.toString) } ...
127. ### Custom combiners for methods returning custom collections ... def result

= ... lastc chunks StringBuilder
128. ### Custom combiners for methods expecting implicit builder factories // only

for big boys ... with GenericParTemplate[T, ParColl] ... object ParColl extends ParFactory[ParColl] { implicit def canCombineFrom[T] = new GenericCanCombineFrom[T] ...

!= ‘ ‘)
130. ### txt.filter(_ != ‘ ‘) new ParString(txt).filter(_ != ‘ ‘) 106

ms Custom combiners performance measurement
131. ### txt.filter(_ != ‘ ‘) new ParString(txt).filter(_ != ‘ ‘) 106

ms 1 core 125 ms Custom combiners performance measurement
132. ### txt.filter(_ != ‘ ‘) new ParString(txt).filter(_ != ‘ ‘) 106

ms 1 core 125 ms 2 cores 81 ms Custom combiners performance measurement
133. ### txt.filter(_ != ‘ ‘) new ParString(txt).filter(_ != ‘ ‘) 106

ms 1 core 125 ms 2 cores 81 ms 4 cores 56 ms Custom combiners performance measurement
134. ### 1 core 125 ms 2 cores 81 ms 4 cores

56 ms t/ms proc 125 ms 1 2 4 81 ms 56 ms Custom combiners performance measurement
135. ### 1 core 125 ms 2 cores 81 ms 4 cores

56 ms t/ms proc 125 ms 1 2 4 81 ms 56 ms def result (not parallelized) Custom combiners performance measurement
136. ### Custom combiners tricky! • two-step evaluation – parallelize the result

method in combiners • efficient merge operation – binomial heaps, ropes, etc. • concurrent data structures – non-blocking scalable insertion operation – we’re working on this
137. ### Future work coming up • concurrent data structures • more

efficient vectors • custom task pools • user defined scheduling • parallel bulk in-place modifications