Functional Parallel Architecture

How we use similar functional abstractions for different parallelisation aspects. Part 1 of my talk at parallel 2013.

1. Die Parallelisierung von Anwendungen als Architekturaufgabe Tobias Neef / / innoQ parallel 2013 / / Karlsruhe
2. Theory Languages Applications Parallel Computing Architecture
3. Context
4. (Mostly) Server-Side development on the JVM
5. Architecture
6. Components are part of your architecture
7. Is a Framework part of you Architecture?
8. Is a Language part of your Architecture?
9. “Architecture is about the important stuff” Martin Fowler
10. Aspects of a parallel Architecture
11. Processes Logic Foundation
12. Parallel (Distributed) Batch Processing
13. MapReduce
14. map (k1,v1) → list(k2,v2) reduce (k2,list(v2)) → list(v3)
15. Distributed WordCount http:/ /commons.wikimedia.org/wiki/File:Rosetta_Stone.JPG
16. map(String key, String value): for each word w in value: EmitIntermediate(w, "1");
17. reduce(String key, Iterator values): int result = 0; for each v in values: result += ParseInt(v); Emit(AsString(result));
18. http:/ /de.wikipedia.org/wiki/Datei:Mapreduce_(Ville_Tuulos).png
19. Is MR an important part of your Architecture?
20. Lets get real
21. 1 package org.myorg; 2 3 import java.io.IOException; 4 import java.util.*; 5 6 import org.apache.hadoop.fs.Path; 7 import org.apache.hadoop.conf.*; 8 import org.apache.hadoop.io.*; 9 import org.apache.hadoop.mapreduce.*; 10 import org.apache.hadoop.mapreduce.lib.input.FileInputFormat; 11 import org.apache.hadoop.mapreduce.lib.input.TextInputFormat; 12 import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat; 13 import org.apache.hadoop.mapreduce.lib.output.TextOutputFormat; 14 15 public class WordCount { 16 17 public static class Map extends Mapper<LongWritable, Text, Text, IntWritable> { 18 private final static IntWritable one = new IntWritable(1); 19 private Text word = new Text(); 20 21 public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException { 22 String line = value.toString(); 23 StringTokenizer tokenizer = new StringTokenizer(line); 24 while (tokenizer.hasMoreTokens()) { 25 word.set(tokenizer.nextToken()); 26 context.write(word, one); 27 } 28 } 29 } 30 31 public static class Reduce extends Reducer<Text, IntWritable, Text, IntWritable> { 32 33 public void reduce(Text key, Iterable<IntWritable> values, Context context) 34 throws IOException, InterruptedException { 35 int sum = 0; 36 for (IntWritable val : values) { 37 sum += val.get(); 38 } 39 context.write(key, new IntWritable(sum)); 40 } 41 } 42 43 public static void main(String[] args) throws Exception { 44 Configuration conf = new Configuration(); 45 46 Job job = new Job(conf, "wordcount"); 47 48 job.setOutputKeyClass(Text.class); 49 job.setOutputValueClass(IntWritable.class); 50 51 job.setMapperClass(Map.class); 52 job.setReducerClass(Reduce.class); 53 54 job.setInputFormatClass(TextInputFormat.class); 55 job.setOutputFormatClass(TextOutputFormat.class); 56 57 FileInputFormat.addInputPath(job, new Path(args[0])); 58 FileOutputFormat.setOutputPath(job, new Path(args[1])); 59 60 job.waitForCompletion(true); 61 } 62 63 }
22. Map / Reduce are functional concepts but implemented procedurally by Hadoop
23. It’s just math
24. Functors and Monads
25. trait Functor[F[_]] { def map[A,B](fa: F[A])(f: A => B) : F[B] }
26. List(1,2,3).map(_*2) == List(2,4,6)
27. trait Monad[F[_]] extends Functor { def flatMap[A, B](fa: F[A])(f : A => F[B]) : F[B] }
28. List("Tobias","Neef") .flatMap(_.toList) == List(T, o, b, i, a, s, N, e, e, f)
29. What does that buy us?
30. val lines = fromTextFile("hdfs:/ /in/...") val counts = lines.flatMap(line => line.split(" ")) .map(word => (word, 1)) .groupByKey .combine(_+_) persist(counts.toTextFile("hdfs:/ /out/...", overwrite=true)) A Monad in a Hadoop context
31. MapReduce is a pattern for distributed and parallel processing of data It is based on functional concepts Functional concepts are easier to express in a functional language
32. Is FP part of your architecture?
33. Bulk Computation
34. Parallel Sum
35. ForkJoin
36. class Sum extends RecursiveTask<Long> { static final int SEQ_THRESHOLD = 5000; int low; int high; int[] array; Sum(int[] arr, int lo, int hi) { array = arr; low = lo; high = hi; } http:/ /homes.cs.washington.edu/~djg/teachingMaterials/grossmanSPAC_forkJoinFramework.html
37. protected Long compute() { if(high - low <= SEQ_THRESHOLD) { long sum = 0; for(int i=low; i < high; ++i) sum += array[i]; return sum; } else { ... } } http:/ /homes.cs.washington.edu/~djg/teachingMaterials/grossmanSPAC_forkJoinFramework.html
38. int mid = low + (high - low) / 2; Sum left = new Sum(array, low, mid); Sum right = new Sum(array, mid, high); left.fork(); long rightAns = right.compute(); long leftAns = left.join(); return leftAns + rightAns; http:/ /homes.cs.washington.edu/~djg/teachingMaterials/grossmanSPAC_forkJoinFramework.html
39. ForkJoin is a functional abstraction
40. def sum(numbers:Seq[Long]) = numbers.reduce(_+_)
41. A Scala ForkJoin Interface
42. def sum(numbers:Seq[Long]) = numbers.par.reduce(_+_) A Monad in a parallel collection context
43. ForkJoin is a pattern for expressing parallel bulk operations It is based on functional concepts Functional concepts are easier to express in a functional language
44. Reduce Blocking -> Increase parallelism
45. Async IO Processing
47. 0 20 40 60 80 Deal2 Deal1 Deal5 Deal4 Deal3 Deals Berlin
49. App Index Page Deal 1 Deal 2 Deal X HTML Parser Output Gen
50. App Index Get D1 Get D2 Get DX Result Parse D1 Parse D2 Parse DX Merge Time
51. Futures
52. java.util.concurrent
55. scala.concurrent
56. map flatMap filter forEach zip andThen ...
57. val f = for { a ← Future(10 / 2) / / 10 / 2 = 5 b ← Future(a + 1) / / 5 + 1 = 6 c ← Future(a - 1) / / 5 - 1 = 4 if c > 3 } yield b * c
58. App Index Get D1 Get D2 Get DX Result Parse D1 Parse D2 Parse DX Merge Time
59. def dealOverviewData(city: String) = { dealLinksForCity(city).flatMap { links => val listOfDealFutures = links.map { link => dealData(link) } Future.sequence(listOfDealFutures) } }
60. def dealLinksForCity(city: String) = { WS.url(url+city).get() .map(_.body.toString) .map(extractLinks) }
61. def dealOverviewData(city: String) = { dealLinksForCity(city).flatMap { links => val listOfDealFutures = links.map { link => dealData(link) } Future.sequence(listOfDealFutures) } }
62. App Index Get D1 Get D2 Get DX Result Parse D1 Parse D2 Parse DX Merge Time
63. def dealData(link: String) = { WS.url(link).get() .map(_.body.toString) .map(extractDealData) }
64. App Index Get D1 Get D2 Get DX Result Parse D1 Parse D2 Parse DX Merge Time
65. def dealOverviewData(city: String) = { dealLinksForCity(city).flatMap { links => val listOfDealFutures = links.map { link => dealData(link) } Future.sequence(listOfDealFutures) } }
66. Abstraction?
67. Monads but with different contexts
68. dealLinksForCity(city).flatMap { links => val listOfDealFutures = links.map { link => getBody(link) .map(extractDealData) } future context collection context
69. Async IO can increase the concurrency of you app and thus the potential for parallelism Functional abstractions can be used to describe asynchronous workflows
70. A lot of parallel abstractions are based on functional concepts MapReduce ForkJoin Async Workflows
71. Some of these concepts can be expressed in a monadic way This style of programming is well suited for functional languages like Scala, Haskell, Clojure or F#
72. Processes Logic Foundation
73. parallelization tool vs. architecture approach
74. Questions?