Java 8: Mehrere Prozessorkerne effizient nutzen mit Lambdas und Streams

Lambda-Ausdrücke und Streams sind wohl die beiden wichtigsten Neuerungen in Java 8. Viele Lösungen lassen sich mit Lambdas kürzer und prägnanter ausdrücken, als das in vorherigen Java-Versionen möglich war. Streams ermöglichen die Anwendung der neuen Lambda-Ausdrücke auf klassische Datenstrukturen, wie z.B. ArrayList und HashMap.

Die Kombination von Lambdas und Streams bietet jedoch mehr als nur eine elegantere Schreibweise für dieselbe Funktionalität. Ein grundlegender Mehrwert besteht besteht darin, dass die Verarbeitung von Streams parallelisierbar ist.

Diese Parallelisierbarkeit ermöglicht es, moderne Mehrkernprozessoren ohne viel Aufwand effektiv zu nutzen.

Im Vortrag wird die parallele Streamverarbeitung von Java 8 genauer unter die Lupe genommen:

Wie kann man die Parrallelisierbarkeit der neuen Standarddatentypen nutzen?
Wie kann man eigene Datenstrukturen für parallele Verarbeitung implementieren?
Wann ist der Einsatz von paralleler Streamverarbeitung sinnvoll und wann sollte darauf verzichtet werden?
Welche Mechanismen werden unter der Haube verwendet, um parallele Streams zu realisieren?

---

Example Code: https://github.com/ConSol/parallel-streams-example

---

JavaLand 2014, http://www.javaland.eu

Transcript

1. Wir unternehmen IT. Efficient use of multi-core processors with lambdas and streams Dr. Fabian Stäber Kundenlogo
2. Slide 14.03.2014 www.consol.de 2 λ λ λ λ λ
3. Slide Example 26 March 2014 www.consol.de 3 Business Data Aggregation
4. Slide Example • Boilerplate – pom.xml – RestConfig (modern web.xml) • Model – Contract – State enum – TestDataGenerator • View – JavaScript, HTML • Controller – Dashboard. 26 March 2014 www.consol.de 4
5. Slide • Boilerplate – pom.xml – RestConfig (modern web.xml) • Model – Contract – State enum – TestDataGenerator • View – JavaScript, HTML • Controller – Dashboard. <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>de.consol.research</groupId> <artifactId>lambda</artifactId> <version>1.0-SNAPSHOT</version> <packaging>war</packaging> <properties> <maven.compiler.source>1.8</maven.compiler.source> <maven.compiler.target>1.8</maven.compiler.target> <failOnMissingWebXml>false</failOnMissingWebXml> </properties> <dependencies> <dependency> <groupId>javax</groupId> <artifactId>javaee-api</artifactId> <version>7.0</version> </dependency> </dependencies> <build> <finalName>lambda</finalName> </build> </project> Example 26 March 2014 www.consol.de 5 <dependency> <groupId>javax</groupId> <artifactId>javaee-api</artifactId> <version>7.0</version> </dependency>
6. Slide package de.consol.research; import ... @ApplicationPath("api") public class RestConfig extends Application {} Example • Boilerplate – pom.xml – RestConfig (modern web.xml) • Model – Contract – State enum – TestDataGenerator • View – JavaScript, HTML • Controller – Dashboard. 26 March 2014 www.consol.de 6 @ApplicationPath("api") public class RestConfig extends Application {}
7. Slide Example • Boilerplate – pom.xml – RestConfig (modern web.xml) • Model – Contract – State enum – TestDataGenerator • View – JavaScript, HTML • Controller – Dashboard. 26 March 2014 www.consol.de 7 package de.consol.research; import java.time.LocalDate; public class Contract { private final State state; private final int priceInCent; private final LocalDate date; public Contract(State state, int priceInCent, LocalDate date) { this.state = state; this.priceInCent = priceInCent; this.date = date; } public State getState() { return state; } public int getPriceInCent() { return priceInCent; } public LocalDate getDate() { return date; } } public class Contract { private final State state; private final int priceInCent; private final LocalDate date;
8. Slide Example • Boilerplate – pom.xml – RestConfig (modern web.xml) • Model – Contract – State enum – TestDataGenerator • View – JavaScript, HTML • Controller – Dashboard. 26 March 2014 www.consol.de 8 package de.consol.research; public enum State { BW, // Baden-Württemberg BY, // Bayern BE, // Berlin BB, // Brandenburg HB, // Bremen HH, // Hamburg HE, // Hessen MV, // Mecklenburg-Vorpommern NI, // Niedersachsen NW, // Nordrhein-Westfalen RP, // Rheinland-Pfalz SL, // Saarland SN, // Sachsen ST, // Sachsen-Anhalt SH, // Schleswig-Holstein TH // Thüringen } public enum State { BW, // Baden-Württemberg BY, // Bayern BE, // Berlin BB, // Brandenburg
9. Slide Example • Boilerplate – pom.xml – RestConfig (modern web.xml) • Model – Contract – State enum – TestDataGenerator • View – JavaScript, HTML • Controller – Dashboard. 26 March 2014 www.consol.de 9 package de.consol.research; import ... public class TestDataGenerator { public static List<Contract> makeTestData() { return Arrays.asList( new Contract(BY, 2000, now()), new Contract(BY, 3500, now().minusDays(1)), new Contract(NW, 7000, now().minusMonths(1)), new Contract(HE, 400, now().minusMonths(1) .minusDays(2)) ); } } public class TestDataGenerator { public static List<Contract> makeTestData() { return Arrays.asList( new Contract(BY, 2000, now()), new Contract(BY, 3500, now().minusDays(1)), new Contract(NW, 7000, now().minusMonths(1)),
10. Slide Example • Boilerplate – pom.xml – RestConfig (modern web.xml) • Model – Contract – State enum – TestDataGenerator • View – JavaScript, HTML • Controller – Dashboard. 26 March 2014 www.consol.de 10
11. Slide Example • Boilerplate – pom.xml – RestConfig (modern web.xml) • Model – Contract – State enum – TestDataGenerator • View – JavaScript, HTML • Controller – Dashboard. 26 March 2014 www.consol.de 11 package de.consol.research; import ... @Path("dashboard") public class Dashboard { private List<Contract> contracts = TestDataGenerator.makeTestData(); } @Path("dashboard") public class Dashboard { private List<Contract> contracts = TestDataGenerator.makeTestData(); }
12. Slide Stream API 26 March 2014 www.consol.de 12 Business Data Aggregation with Stream API
13. Slide GET /lambda/api/dashboard/total-sales Stream API 26 March 2014 www.consol.de 13 Task 1: Total Sales 12900
14. Slide Stream API 26 March 2014 www.consol.de 14 @GET @Path("total-sales") @Produces(APPLICATION_JSON) public int totalSales() { return contracts .stream() .filter(contract -> contract.getDate().isAfter(now().minusYears(1))) .mapToInt(Contract::getPriceInCent) .sum(); }
15. Slide Stream API 26 March 2014 www.consol.de 15 Task 2: Sales by State GET /lambda/api/dashboard/sales-by-state { "BY": 5500, "NW": 7000, "HE": 400 }
16. Slide Stream API 26 March 2014 www.consol.de 16 @GET @Path("sales-by-state") @Produces(APPLICATION_JSON) public Map<State, Integer> salesByState() { return contracts .stream() .filter(contract -> contract.getDate().isAfter(now().minusYears(1))) .collect(Collectors.groupingBy( Contract::getState, Collectors.summingInt(Contract::getPriceInCent))); }
17. Slide Stream API 26 March 2014 www.consol.de 17 @GET @Path("sales-by-state") @Produces(APPLICATION_JSON) public Map<State, Integer> salesByState() { return contracts .stream() .filter(contract -> contract.getDate().isAfter(now().minusYears(1))) .collect(Collectors.groupingBy( Contract::getState, Collectors.summingInt(Contract::getPriceInCent))); } Can we re-use Lambda expressions?
18. Slide Stream API 26 March 2014 www.consol.de 18 private Predicate<Contract> last12months = contract -> contract.getDate().isAfter(now().minusYears(1)); @GET @Path("sales-by-state") @Produces(APPLICATION_JSON) public Map<State, Integer> salesByState() { return contracts .stream() .filter(last12months) .collect(Collectors.groupingBy( Contract::getState, Collectors.summingInt(Contract::getPriceInCent))); }
19. Slide GET /lambda/api/dashboard/sales-per-month Stream API 26 March 2014 www.consol.de 19 Task 3: Sales per Month { “2014-03": 5500, “2014-02": 7400 }
20. Slide Stream API 26 March 2014 www.consol.de 20 @GET @Path("sales-per-month") @Produces(APPLICATION_JSON) public Map<String, Integer> salesPerMonth() { return contracts .stream() .filter(last12months) .collect(Collectors.groupingBy( c -> c.getDate().getYear() + "-" + c.getDate().getMonthValue(), Collectors.summingInt(Contract::getPriceInCent))); }
21. Slide Stream API Stream API 26 March 2014 www.consol.de 21 Get things done in few lines of code… …really  ?
22. Slide Stream API 26 March 2014 www.consol.de 22 public int totalSales() { return contracts .stream() .mapToInt(Contract::getPriceInCent) .sum(); } public int totalSales() { int resultInCent = 0; for (Contract contract : contracts) { resultInCent += contract.getPriceInCent(); } return resultInCent; } Java 8 Java 7
23. Slide Stream API 26 March 2014 www.consol.de 23 Why all that hype?
24. Slide Stream API 26 March 2014 www.consol.de 24 Faster !!!
25. Slide How To Make Things Faster • Keep Data in RAM • Avoid DB Roundtrips • Snappy Lightweight AJAX Front-End • Use Multiple Cores 26 March 2014 www.consol.de 25    ? ?
26. Slide 14.03.2014 www.consol.de 26 λ λ λ λ λ
27. Slide Use Multiple Cores parallelize 26 March 2014 www.consol.de 27
28. Slide Use Multiple Cores 26 March 2014 www.consol.de 28 public int totalSales() { return contracts .stream() .mapToInt(Contract::getPriceInCent) .sum(); } public int totalSales() { int resultInCent = 0; for (Contract contract : contracts) { resultInCent += contract.getPriceInCent(); } return resultInCent; } Java 8 Java 7
29. Slide Use Multiple Cores 26 March 2014 www.consol.de 29 public int totalSales() { return contracts .stream() .mapToInt(Contract::getPriceInCent) .sum(); } public int totalSales() { int resultInCent = 0; for (Contract contract : contracts) { resultInCent += contract.getPriceInCent(); } return resultInCent; } Do It Yourself synchronized, volatile, Lock, Executor, Callable, … Let the Library Do It For You
30. Slide Use Multiple Cores 26 March 2014 www.consol.de 30 public int totalSales() { return contracts .parallelStream() .mapToInt(Contract::getPriceInCent) .sum(); } public int totalSales() { int resultInCent = 0; for (Contract contract : contracts) { resultInCent += contract.getPriceInCent(); } return resultInCent; } Do It Yourself synchronized, volatile, Lock, Executor, Callable, … Let the Library Do It For You
31. Slide Use Multiple Cores 26 March 2014 www.consol.de 31 // Set of all states in contract data Set<State> states = new HashSet<State>(); contracts .parallelStream() .map(Contract::getState) .forEach(s -> states.add(s)); Thread Safe Operations Not Thread Safe
32. Slide Use Multiple Cores 26 March 2014 www.consol.de 32 // Set of all states in contract data Set<State> states = contracts .parallelStream() .map(Contract::getState) .collect(Collectors.toSet()); Thread Safe Operations Thread Safe 
33. Slide Use Multiple Cores Under the Hood (internal stream API implementation) 26 March 2014 www.consol.de 33
34. Slide Under the Hood 26 March 2014 www.consol.de 34 (2 + 8 + 7 + 3) + (3 + 5 + 1 + 7) (2 + 8) + (7 + 3) 7 + 3 2 + 8 3 + 5 1 + 7 (3 + 5) + (1 + 7)
35. Slide Under the Hood 26 March 2014 www.consol.de 35 if (my portion of the work is small enough) do the work directly else split my work into two pieces invoke the two pieces and wait for the results Java 7: Fork/Join Framework
36. Slide Under the Hood 26 March 2014 www.consol.de 36 public class LegacyContractsData implements Iterable<Contract> { private State[] states; private int[] pricesInCent; private LocalDate[] dates; // ... } Can I use this with my legacy data?
37. Slide Under the Hood Tell the stream API: • how to split your data into chunks • how to iterate over a chunk. 26 March 2014 www.consol.de 37 Implement a Spliterator. Can I use this with my legacy data?
38. Slide Under the Hood 26 March 2014 www.consol.de 38 public class MyOwnSpliterator implements Spliterator<Contract> { @Override public Spliterator<Contract> trySplit() { // split data into chunks } @Override public boolean tryAdvance(Consumer<? super Contract> action) { // iterate over a chunk } @Override public long estimateSize() { // optional metadata } @Override public int characteristics() { // optional metadata } }
39. Slide 14.03.2014 www.consol.de 39 λ λ λ λ λ λ
40. Slide Summary 26 March 2014 www.consol.de 40 Efficient use of multi-core processors with lambdas and streams: • Library does it for you • Use thread safe operations • Implement Spliterators