10.14

Fill in the blanks below to complete the following Apache Spark program which computes the number of occurrences of each word in a file. For simplicity we assume that words only occur in lowercase, and there are no punctuations marks.

JavaRDD<String> textFile = sc.textFile("hdfs://...");
JavaPairRDD<String, Integer> counts = textFile.____(s->Arrays.asList(s.split(" "))._____())
    .mapToPair(word -> new _______).reduceByKey((a,b) -> a + b); 

---

JavaRDD<String> textFile = sc.textFile("hdfs://...");
JavaPairRDD<String, Integer> counts = textFile.flatMap(s->Arrays.asList(s.split(" ")).iterator())
    .mapToPair(word -> new Tuple2<>(word, 1)).reduceByKey((a,b) -> a + b);