Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Procesanso datos con Hadoop: MapReduce y YARN

Procesanso datos con Hadoop: MapReduce y YARN

Charla dada bajo el marco de las Jornadas Técnicas Uex-CIEMAT: "Procesando grandes volúmenes de datos con Hadoop"

**NOTA** Algunas transparencias aparecen cortadas en el visor. La versión descargable funciona correctamente.

267ee9b8bfeb0c72b5dbe643bbc4433b?s=128

César Suárez Ortega

February 11, 2015
Tweet

Transcript

  1. Jornadas Técnicas Uex-CIEMAT // 10-12 Febrero 2015 Procesando grandes volúmenes

    de datos con HADOOP César Suárez Ortega cesar.suarez@externos.ciemat.es Procesando datos con Hadoop: MapReduce y YARN
  2. Software Developer / Researcher César Suárez Ortega tharandur csuarez

  3. None
  4. Software Engineer / Researcher César Suárez Ortega

  5. Software Engineer / Researcher César Suárez Ortega

  6. Índice 1.  2.  3.  z  z  4.  Exprimiendo al máximo

    MapReduce 5. 
  7. Introducción a MapReduce

  8. Modelo de programación para el procesamiento de datos

  9. None
  10. ¿Qué es MapReduce? z  z  MUCHOS z  z 

  11. by MapReduce: Simplified Data Processing on Large Clusters. Dean J.

    and Ghemawat S. (2004)!
  12. Aprendiendo con ejemplos

  13. National Climatic Data Center z  z  z  z  z 

  14. 1958 -0021 1959 +0065 1960 +0054

  15. Soluciones 1.  Script z  2.  Paralelización por año z  3. 

    Paralelización por partes iguales z  4.  MapReduce z 
  16. MapReduce 101 z  z  <Key, Value> z  Hay que definir

    una función para cada etapa. MAP REDUCE
  17. Map MAP <0, 005733213…> //line1 <160, 006844324…> //line2

  18. Map MAP <K1, V1>

  19. Map MAP <K1, V1> list(K2, V2)

  20. 1958 -0021 1959 +0065 1960 +0054

  21. Map MAP <0, 005733213…> //line1 <160, 006844324…> //line2 <1958, -21>

    <1959, +65>
  22. Shuffle <1958, -21> <1959, +65> <1958, -34> <1959, +28> <1958,

    [-21, -34]> <1959, [+65, +28]>
  23. Shuffle list(K2, V2) list(K2, list(V2))

  24. Reduce REDUCE list(K2, list(V2)) list(K3, V3)

  25. Reduce REDUCE <1958, [-21, -34]> <1959, [+65, +28]> <1958, -21>

    <1959, +65>
  26. Map & Reduce REDUCE list(K2, list(V2)) list(K3, V3)) MAP <K1,

    V1> list(K2, V2)
  27. Map&Reduce z  z  à z  à Job z  z  z 

  28. split 0 split 1 split 2 map map map reduce

    output HDFS replication
  29. split 0 split 1 split 2 map map map reduce

    part0 HDFS replication reduce part1 HDFS replication
  30. Práctica 1 Mandando nuestro primer trabajo

  31. MapReduce API z  z  1.  Blabla! z  2.  BlablaMapper! z 

    3.  BlablaReducer! z 
  32. FlightsByCarrier z  z  ! ! https://github.com/csuarez/seminario-mapreduce!   flights-by-carrier/  

  33. Year,Month,DayofMonth,DayOfWeek,DepTime,CRSDepTime,ArrTime,CRSArrTime,UniqueCarrier,FlightNum,Tai lNum,ActualElapsedTime,CRSElapsedTime,AirTime,ArrDelay,DepDelay,Origin,Dest,Distance,TaxiIn,TaxiO ut,Cancelled,CancellationCode,Diverted,CarrierDelay,WeatherDelay,NASDelay,SecurityDelay,LateAircr aftDelay! 1987,10,14,3,741,730,912,849,PS,1451,NA,91,79,NA,23,11,SAN,SFO,447,NA,NA,0,NA,0,NA,NA,NA,NA,NA! 1987,10,15,4,729,730,903,849,PS,1451,NA,94,79,NA,14,-1,SAN,SFO,447,NA,NA,0,NA,0,NA,NA,NA,NA,NA! 1987,10,17,6,741,730,918,849,PS,1451,NA,97,79,NA,29,11,SAN,SFO,447,NA,NA,0,NA,0,NA,NA,NA,NA,NA! 1987,10,18,7,729,730,847,849,PS,1451,NA,78,79,NA,-2,-1,SAN,SFO,447,NA,NA,0,NA,0,NA,NA,NA,NA,NA! 1987,10,19,1,749,730,922,849,PS,1451,NA,93,79,NA,33,19,SAN,SFO,447,NA,NA,0,NA,0,NA,NA,NA,NA,NA! 1987,10,21,3,728,730,848,849,PS,1451,NA,80,79,NA,-1,-2,SAN,SFO,447,NA,NA,0,NA,0,NA,NA,NA,NA,NA!

    1987,10,22,4,728,730,852,849,PS,1451,NA,84,79,NA,3,-2,SAN,SFO,447,NA,NA,0,NA,0,NA,NA,NA,NA,NA! 1987,10,23,5,731,730,902,849,PS,1451,NA,91,79,NA,13,1,SAN,SFO,447,NA,NA,0,NA,0,NA,NA,NA,NA,NA! 1987,10,24,6,744,730,908,849,PS,1451,NA,84,79,NA,19,14,SAN,SFO,447,NA,NA,0,NA,0,NA,NA,NA,NA,NA! 1987,10,25,7,729,730,851,849,PS,1451,NA,82,79,NA,2,-1,SAN,SFO,447,NA,NA,0,NA,0,NA,NA,NA,NA,NA! 1987,10,26,1,735,730,904,849,PS,1451,NA,89,79,NA,15,5,SAN,SFO,447,NA,NA,0,NA,0,NA,NA,NA,NA,NA! 1987,10,28,3,741,725,919,855,PS,1451,NA,98,90,NA,24,16,SAN,SFO,447,NA,NA,0,NA,0,NA,NA,NA,NA,NA! 1987,10,29,4,742,725,906,855,PS,1451,NA,84,90,NA,11,17,SAN,SFO,447,NA,NA,0,NA,0,NA,NA,NA,NA,NA! 1987,10,31,6,726,725,848,855,PS,1451,NA,82,90,NA,-7,1,SAN,SFO,447,NA,NA,0,NA,0,NA,NA,NA,NA,NA! 1987,10,1,4,936,915,1035,1001,PS,1451,NA,59,46,NA,34,21,SFO,RNO,192,NA,NA,0,NA,0,NA,NA,NA,NA,NA! 1987,10,2,5,918,915,1017,1001,PS,1451,NA,59,46,NA,16,3,SFO,RNO,192,NA,NA,0,NA,0,NA,NA,NA,NA,NA! 1987,10,3,6,928,915,1037,1001,PS,1451,NA,69,46,NA,36,13,SFO,RNO,192,NA,NA,0,NA,0,NA,NA,NA,NA,NA! 1987,10,4,7,914,915,1003,1001,PS,1451,NA,49,46,NA,2,-1,SFO,RNO,192,NA,NA,0,NA,0,NA,NA,NA,NA,NA! 1987,10,5,1,1042,915,1129,1001,PS,1451,NA,47,46,NA,88,87,SFO,RNO,192,NA,NA,0,NA,0,NA,NA,NA,NA,NA! 1987,10,6,2,934,915,1024,1001,PS,1451,NA,50,46,NA,23,19,SFO,RNO,192,NA,NA,0,NA,0,NA,NA,NA,NA,NA! 1987,10,7,3,946,915,1037,1001,PS,1451,NA,51,46,NA,36,31,SFO,RNO,192,NA,NA,0,NA,0,NA,NA,NA,NA,NA! 1987,10,8,4,932,915,1033,1001,PS,1451,NA,61,46,NA,32,17,SFO,RNO,192,NA,NA,0,NA,0,NA,NA,NA,NA,NA! 1987,10,9,5,947,915,1036,1001,PS,1451,NA,49,46,NA,35,32,SFO,RNO,192,NA,NA,0,NA,0,NA,NA,NA,NA,NA! 1987,10,10,6,915,915,1022,1001,PS,1451,NA,67,46,NA,21,0,SFO,RNO,192,NA,NA,0,NA,0,NA,NA,NA,NA,NA! 1987,10,11,7,916,915,1006,1001,PS,1451,NA,50,46,NA,5,1,SFO,RNO,192,NA,NA,0,NA,0,NA,NA,NA,NA,NA! 1987,10,12,1,944,915,1027,1001,PS,1451,NA,43,46,NA,26,29,SFO,RNO,192,NA,NA,0,NA,0,NA,NA,NA,NA,NA! 1987,10,13,2,941,915,1036,1001,PS,1451,NA,55,46,NA,35,26,SFO,RNO,192,NA,NA,0,NA,0,NA,NA,NA,NA,NA! 1987,10,14,3,930,915,1029,1001,PS,1451,NA,59,46,NA,28,15,SFO,RNO,192,NA,NA,0,NA,0,NA,NA,NA,NA,NA! 1987,10,15,4,920,915,1023,1001,PS,1451,NA,63,46,NA,22,5,SFO,RNO,192,NA,NA,0,NA,0,NA,NA,NA,NA,NA! 1987,10,17,6,1009,915,1104,1001,PS,1451,NA,55,46,NA,63,54,SFO,RNO,192,NA,NA,0,NA,0,NA,NA,NA,NA,NA! 1987,10,18,7,915,915,1008,1001,PS,1451,NA,53,46,NA,7,0,SFO,RNO,192,NA,NA,0,NA,0,NA,NA,NA,NA,NA! 1987,10,19,1,940,915,1032,1001,PS,1451,NA,52,46,NA,31,25,SFO,RNO,192,NA,NA,0,NA,0,NA,NA,NA,NA,NA! 1987,10,21,3,913,915,1003,1001,PS,1451,NA,50,46,NA,2,-2,SFO,RNO,192,NA,NA,0,NA,0,NA,NA,NA,NA,NA!
  34. public class FlightsByCarrier {! public static void main (String[] args)

    throws Exception {! ! Job job = new Job();! job.setJarByClass(FlightsByCarrier.class); ! ! job.setJobName("FlightsByCarrier”);! ! ! TextInputFormat.addInputPath(job, new Path(args[0])); TextOutputFormat.setOutputPath(job, new Path(args[1]));! job.setInputFormatClass(TextInputFormat.class);! job.setOutputKeyClass(Text.class);! job.setOutputValueClass(IntWritable.class);! ! ! job.setMapperClass(FlightsByCarrierMapper.class);! job.setReducerClass(FlightsByCarrierReducer.class); ! job.setOutputFormatClass(TextOutputFormat.class); ! ! job.addFileToClassPath(new Path("/user/root/opencsv-2.3.jar")); ! job.waitForCompletion(true);! }! }!
  35. //Mapper<KeyIn, ValueIn, KeyOut, ValueOut>! public class FlightsByCarrierMapper extends Mapper<LongWritable, Text,

    Text, IntWritable> {! ! @Override! protected void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {! ! if (key.get() > 0) { //Ignora la primera linea! String[] lines = new CSVParser().parseLine(value.toString());! ! context.write(new Text(lines[8]), new IntWritable(1));! }! }! }!
  36. public class FlightsByCarrierReducer extends Reducer<Text, IntWritable, Text, IntWritable> {! !

    @Override! protected void reduce(Text token, Iterable<IntWritable> counts, Context context) throws IOException, InterruptedException {! ! int sum = 0;! ! for (IntWritable count : counts) {! sum += count.get();! }! ! context.write(token, new IntWritable(sum));! }! }! !
  37. ! ! $ git clone https://github.com/csuarez/seminario-mapreduce.git! ! [...]! ! $

    tar xvzf 1987.tar.gz! ! $ hdfs dfs –copyFromLocal lib/opencsv-2.3.jar /user/root! ! $ hdfs dfs –copyFromLocal 1987.csv /user/root! ! $ sh build.sh! ! $ hadoop jar FlightsByCarrier.jar FlightsByCarrier /user/root/ 1987.csv /user/root/output/flightsCount! ! $ hdfs dfs -cat /user/root/output/flightsCount/part-r-00000! ! ! Ejecución
  38. FlightsByCarrier output INFO mapreduce.JobSubmitter: number of splits:2! ! ...! !

    INFO mapreduce.Job: map 0% reduce 0%! INFO mapreduce.Job: map 22% reduce 0%! INFO mapreduce.Job: map 41% reduce 0%! INFO mapreduce.Job: map 83% reduce 0%! INFO mapreduce.Job: map 100% reduce 0%! INFO mapreduce.Job: map 100% reduce 100%! ! ...! ! Job Counters! Launched map tasks=2! Launched reduce tasks=1! Rack-local map tasks=2! Total time spent by all maps in occupied slots (ms)=42442! Total time spent by all reduces in occupied slots (ms)=13465! !
  39. MapReduce Cómo funciona

  40. MapReduce v1

  41. MapReduce Program Job Client Job Tracker Task Tracker HDFS Child

    M/R task TASKTRACKER NODE JVM 1. run 2. get id 3. copy 4. submit job 5. init. job 6. get splits 7. heartbeats 9. launch 10. run 8. get resources JVM CLIENT NODE JOB TRACKER NODE
  42. Actores 1.  2.  3.  4.  MapReduce Program Job Client Job

    Tracker Task Tracker HDFS Child M/R task TASKTRACKER NODE JVM 1. run 2. get id 3. copy 4. submit job 5. init. job 6. get splits 7. heartbeats 9. launch 10. run 8. get resources JVM CLIENT NODE JOB TRACKER NODE
  43. Ejecución MapReduce z  z  z  z  z  z  z 

  44. Paso 1 Job Submission z  z  z  z  MapReduce Program

    Job Client Job Tracker Task Tracker HDFS Child M/R task TASKTRACKER NODE JVM 1. run 2. get id 3. copy 4. submit job 5. init. job 6. get splits 7. heartbeats 9. launch 10. run 8. get resources JVM CLIENT NODE JOB TRACKER NODE
  45. Paso 2 Job Initialization z  z  z  z  z  MapReduce

    Program Job Client Job Tracker Task Tracker HDFS Child M/R task TASKTRACKER NODE JVM 1. run 2. get id 3. copy 4. submit job 5. init. job 6. get splits 7. heartbeats 9. launch 10. run 8. get resources JVM CLIENT NODE JOB TRACKER NODE
  46. Paso 3 Task Assignment z  z  z  z  MapReduce Program

    Job Client Job Tracker Task Tracker HDFS Child M/R task TASKTRACKER NODE JVM 1. run 2. get id 3. copy 4. submit job 5. init. job 6. get splits 7. heartbeats 9. launch 10. run 8. get resources JVM CLIENT NODE JOB TRACKER NODE
  47. Paso 4 Task Execution z  z  z  MapReduce Program Job

    Client Job Tracker Task Tracker HDFS Child M/R task TASKTRACKER NODE JVM 1. run 2. get id 3. copy 4. submit job 5. init. job 6. get splits 7. heartbeats 9. launch 10. run 8. get resources JVM CLIENT NODE JOB TRACKER NODE
  48. Paso 5 Progress & Status z  z  z  z  MapReduce

    Program Job Client Job Tracker Task Tracker HDFS Child M/R task TASKTRACKER NODE JVM 1. run 2. get id 3. copy 4. submit job 5. init. job 6. get splits 7. heartbeats 9. launch 10. run 8. get resources JVM CLIENT NODE JOB TRACKER NODE
  49. Paso 6 Job Completion z  z  MapReduce Program Job Client

    Job Tracker Task Tracker HDFS Child M/R task TASKTRACKER NODE JVM 1. run 2. get id 3. copy 4. submit job 5. init. job 6. get splits 7. heartbeats 9. launch 10. run 8. get resources JVM CLIENT NODE JOB TRACKER NODE
  50. MapReduce v2 + YARN

  51. ¿Por qué YARN? z  z  z  z  z 

  52. MapReduce Progran Job Client Resource Manager Node Manager HDFS YARN

    Child M/R task JVM 1. run JVM CLIENT NODE RESOURCE MANAGER NODE Node Manager Application Master 2. get ID 3. copy job resources 4. submit application 5a. start container 5b. launch 6. init. job 7. get splits 8. allocate resources 9a. start container 9b. launch 10. get data NODE MANAGER NODE NODE MANAGER NODE
  53. YARN: Actores 1.  z  2.  z  3.  z  4.  z 

    5. 
  54. vs. MapReduce v1: Actores 1.  2.  3.  4.  MapReduce Progran

    Job Client Job Tracker Task Tracker HDFS Child M/R task TASKRACKER NODE JVM 1. run 2. get id 3. copy 4. submit job 5. init. job 6. get splits 7. heartbeats 9. launch 10. run 8. get resources JVM CLIENT NODE JOB TRACKER NODE 1.  YARN Resource Manager 2.  YARN Node Manager 3.  Application Master
  55. None
  56. MapReduce v1 API Processing FW Resource Manager Storage MR MapReduce

    v1   HDFS   PIG   HIVE   HBASE  
  57. YARN API Processing FW Resource Manager Storage MR YARN  

    HDFS   PIG   STORM   MR v2 TEZ   MPI  
  58. MapReduce Progran Job Client Resource Manager Node Manager HDFS YARN

    Child M/R task JVM 1. run JVM CLIENT NODE RESOURCE MANAGER NODE Node Manager Application Master 2. get ID 3. copy job resources 4. submit application 5a. start container 5b. launch 6. init. job 7. get splits 8. allocate resources 9a. start container 9b. launch 10. get data NODE MANAGER NODE NODE MANAGER NODE
  59. Paso 1 Job Submission z  z  z  z  z  z 

  60. MapReduce Progran Job Client Resource Manager Node Manager HDFS YARN

    Child M/R task JVM 1. run JVM CLIENT NODE RESOURCE MANAGER NODE Node Manager Application Master 2. get ID 3. copy job resources 4. submit application 5a. start container 5b. launch 6. init. job 7. get splits 8. allocate resources 9a. start container 9b. launch 10. get data NODE MANAGER NODE NODE MANAGER NODE
  61. Paso 2 Job Initialization z  z  z  z  z  z 

    z 
  62. Paso 2** Uber Mode z  z  z  z  z 

  63. MapReduce Progran Job Client Resource Manager Node Manager HDFS YARN

    Child M/R task JVM 1. run JVM CLIENT NODE RESOURCE MANAGER NODE Node Manager Application Master 2. get ID 3. copy job resources 4. submit application 5a. start container 5b. launch 6. init. job 7. get splits 8. allocate resources 9a. start container 9b. launch 10. get data NODE MANAGER NODE NODE MANAGER NODE
  64. Paso 3 Task Assignment z  z  z  z  z  z 

  65. MapReduce Progran Job Client Resource Manager Node Manager HDFS YARN

    Child M/R task JVM 1. run JVM CLIENT NODE RESOURCE MANAGER NODE Node Manager Application Master 2. get ID 3. copy job resources 4. submit application 5a. start container 5b. launch 6. init. job 7. get splits 8. allocate resources 9a. start container 9b. launch 10. get data NODE MANAGER NODE NODE MANAGER NODE
  66. Paso 4 Task Execution z  z  z  z  z 

  67. MapReduce Progran Job Client Resource Manager Node Manager HDFS YARN

    Child M/R task JVM 1. run JVM CLIENT NODE RESOURCE MANAGER NODE Node Manager Application Master 2. get ID 3. copy job resources 4. submit application 5a. start container 5b. launch 6. init. job 7. get splits 8. allocate resources 9a. start container 9b. launch 10. get data NODE MANAGER NODE NODE MANAGER NODE
  68. Paso 5 Progress & Status z  z  z  z  z 

  69. MapReduce Progran Job Client Resource Manager Node Manager HDFS YARN

    Child M/R task JVM 1. run JVM CLIENT NODE RESOURCE MANAGER NODE Node Manager Application Master 2. get ID 3. copy job resources 4. submit application 5a. start container 5b. launch 6. init. job 7. get splits 8. allocate resources 9a. start container 9b. launch 10. get data NODE MANAGER NODE NODE MANAGER NODE
  70. Exprimiendo MapReduce

  71. Apache Pig

  72. Apache Pig z  z  z  z  z 

  73. records = LOAD '1987.csv' USING PigStorage(',') AS! (Year,Month,DayofMonth,DayOfWeek,DepTime,CRSDepTime,ArrTime,CRSArrTime,UniqueCarrie r,FlightNum,TailNum,ActualElapsedTime,CRSElapsedTime,AirTime,ArrDelay,DepDelay,Orig in,Dest,Distance:int,TaxiIn,TaxiOut,Cancelled,CancellationCode,Diverted,CarrierDela

    y,WeatherDelay,NASDelay,DecurityDelay,LateAircraftDelay);! ! milage_recs = GROUP records ALL;! ! tot_miles = FOREACH milage_recs GENERATE SUM(records.Distance);! ! STORE tot_miles INTO '/user/root/totalmiles';! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! https://github.com/csuarez/seminario-mapreduce! $ cd pig-total-miles/! $ pig totalmiles.pig!
  74. Hadoop Streaming

  75. ! ! ! ! ! ! ! ! ! !

    ! ! ! ! $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar \! -input myInputDirs \! -output myOutputDir \! -mapper myPythonScript.py \! -reducer /bin/wc \! -file myPythonScript.py!
  76. None
  77. None
  78. Práctica 2 MapReduce Job

  79. GreekGodCounter (Estándar) z  z  ! ! https://github.com/csuarez/seminario-mapreduce!   greek-god-counter-standard/  

  80. ! ! ! ! ! private final static String[] gods

    = {! "Zeus",! "Hera",! "Poseidón",! "Dioniso",! "Apolo",! "Artemisa",! "Hermes",! "Atenea",! "Ares",! "Afrodita",! "Hefesto",! "Deméter”! };! ! ! ! ! ! !
  81. //Initializing the initial structure! for (String god : gods) {!

    godMap.put(god, 0);! }! ! try {! //Reading input! br = new BufferedReader(new FileReader(args[0]));! String line = br.readLine();! while (line != null) {! StringTokenizer tokenizer = new StringTokenizer(line);! while (tokenizer.hasMoreTokens()) {! String token = tokenizer.nextToken();! if (godMap.containsKey(token)) {! godMap.put(token, godMap.get(token) + 1);! }! }! line = br.readLine();! }! ! //Writing output! Writer writer = new BufferedWriter(new FileWriter("gods.txt"));! for (Entry<String, Integer> entry : godMap.entrySet()) {! writer.write(entry.getKey() + " = " + entry.getValue());! writer.write(System.lineSeparator());! }! writer.close();! }!
  82. ¡¡¡TAREA!!!

  83. GreekGodCounter (MapReduce) z  z  ! ! https://github.com/csuarez/seminario-mapreduce!   greek-god-counter-mapreduce/  

  84. import org.apache.hadoop.fs.Path;! import org.apache.hadoop.io.*;! import org.apache.hadoop.mapreduce.Job;! import org.apache.hadoop.mapreduce.lib.input.TextInputFormat;! import org.apache.hadoop.mapreduce.lib.output.TextOutputFormat;!

    import org.apache.hadoop.filecache.DistributedCache;! ! ! public class GreekGodCounterMapReduce {! public static void main (String[] args) throws Exception {! ! Job job = new Job();! job.setJarByClass(GreekGodCounterMapReduce.class);! job.setJobName("GreekGodCounterMapReduce");! TextInputFormat.addInputPath(job, new Path(args[0])); ! job.setInputFormatClass(TextInputFormat.class);! job.setMapperClass(GreekGodCounterMapReduceMapper.class);! job.setReducerClass(GreekGodCounterMapReduceReducer.class); ! TextOutputFormat.setOutputPath(job, new Path(args[1]));! job.setOutputFormatClass(TextOutputFormat.class);! job.setOutputKeyClass(Text.class);! job.setOutputValueClass(IntWritable.class);! job.waitForCompletion(true);! ! }! }!
  85. import java.io.IOException;! import org.apache.hadoop.io.*;! import org.apache.hadoop.mapreduce.Mapper;! import java.util.HashMap;! import java.util.Map.Entry;!

    import java.util.StringTokenizer;! ! public class GreekGodCounterMapReduceMapper ! extends Mapper<LongWritable, Text, Text, IntWritable> {! private final static String[] gods = {! "Zeus",! "Hera",! "Poseidón",! "Dioniso",! "Apolo",! "Artemisa",! "Hermes",! "Atenea",! "Ares",! "Afrodita",! "Hefesto",! "Deméter"! };! ! @Override! protected void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {! ! }! }!
  86. import java.io.IOException;! import org.apache.hadoop.io.*;! import org.apache.hadoop.mapreduce.Reducer;! ! public class GreekGodCounterMapReduceReducer

    ! extends Reducer<Text, IntWritable, Text, IntWritable> {! @Override! protected void reduce! (Text token, Iterable<IntWritable> counts, Context context) ! throws IOException, InterruptedException {! ! }! }!
  87. Recursos

  88. Recursos

  89. ¡Gracias! ¿Alguna pregunta? cesar.suarez@externos.ciemat.es