Peter Burka on Crossing the Gap from Imperative to Functional Programming through Refactoring

The introduction of lambdas to Java 8 might be the most significant change to the Java language since Java 2 was released in 1998. Lambdas and the accompanying functional operations like map and filter promise to allow Java programmers to write clearer, simpler code, and to take better advantage of parallelism.

While developers of new code will be able to start using the features immediately, what should we do with the billions of lines of code that have already been written? This paper proposes that we can automatically translate the existing body of Java code to make use of the new features. This improves the readability of the old code, maintains consistency between new and old code, and potentially improves performance.

This paper addresses technical debt, an area of software engineering which is only starting to be addressed in academic literature. The authors solve a practical and important problem with a powerful technical tool.

Transcript

1. Crossing the Gap from Imperative to Functional Programming through Refactoring Gyori, Franklin, Dig and Lahoda Presented to Papers We Love New York Peter Burka (@pburka), Two Sigma Labs September 23, 2014
2. About me • Member of the Two Sigma Lab • Interested in software engineering • Concerned by technical debt
3. Why do I love this paper? • Practical problem • Practical solution
4. Why do I love this paper? • Practical problem • Practical solution Crossing the Gap from Imperative to Functional Programming through Refactoring
5. Java 8: Project Lambda • Java 8’s lambdas might be the biggest change to Java ever • Alleviate DRY problem • Lambdas fit into Java ecosystem • Cleverly disguised inner classes • Implement one method interfaces
6. Java 8: Project Lambda • New stream library • Based on functional language list ops • Use filter, map, reduce on Java collections and arrays • Encourage Java programmers to think functionally • Easy-to-use parallelism • Replace stream() with parallelStream()
7. Example • Imperative for-loop: • Functional-style with stream and lambda:
8. Example • Java 7 parallelism:
9. Example • Java 8 parallelism with streams and lambdas:
10. Benefits of streams and lambdas • Simpler to read, • Simpler to reason about, and • Simpler to parallelize! Cost of refactoring • Time and effort • Risk of error ...but if a tool does it for you...
11. What did they do? “We designed and implemented a tool which automates two refactorings: 1. Convert anonymous inner classes to lambda expressions 2. Convert for loops that iterate over Collections to functional operations that use lambda expressions ”
12. Converting anonymous classes • Convert this: • To this:
13. Converting anonymous classes • Fairly straight-forward • Handles a bunch of tedious special cases • Different scoping rules • Must be functional interface • No this or super • No recursion • Type inferencing
14. Converting for loops • Convert this: • To this:
15. Converting for loops • Converts for loops into chains of composable, streaming operations • Currently support: • map(mapper) • filter(predicate) • reduce(identity, reducer) • anyMatch(predicate) • noneMatch(predicate) • forEach(consumer)
16. How do they do it? • Convert continue to if • Convert if to filter • Convert everything else to map • Detect terminal operation • return true/false → anyMatch/noneMatch • +=, *=, etc. → reduce • Otherwise, forEach
17. More details • Use fine grained operations • Gives more control over parallelism • Last op in chain must be eager! • Unlike for loops, some stream ops are lazy • Does not parallelize • But it’s just a one-word change!
18. Does it work? • Tested 9 OSS projects • Over 1 million lines of code, 4300 tests • 2718 refactorings • No test failures
19. But how well does it work? • Compared tool to human programmers • Used information retrieval metrics • Precision: • Does the refactoring match the best, most fine-grained refactorings of an expert? • Recall: • How many of the possible refactorings identified by an expert did the tool apply? • Also measured ΔLOC and ΔAST size
20. Results • Both refactorings had high recall and precision • Refactorings were not clustered • More work to do by hand • Converting to lambdas simplified code
21. More opportunities • Support arrays • Convert return to findFirst() • Convert nested for to flatMap() • Simplify reduce() to count() • Use collect() for building collections
22. Lessons • Refactoring old code is important • Computers can do some refactorings better than humans • When we program computers to do human tasks, we should measure them against humans
23. Question for you • What is the sed of refactoring? Google Refaster Eclipse AST MoDisco
24. Backup
25. Why Java? • Why not use Scala, Clojure, Haskell? • Too late – billions of lines of existing Java • Huge ecosystem – great value in scale • Java is very, very well tested • Java is learning from these trail-blazers • Java is easy to refactor • Strong type system, well-defined compiler facilitate static analysis