Tracking GHC Performance

Presented at "HIW 2017 Haskell Implementors' Workshop": https://icfp17.sigplan.org/track/hiw-2017

A recurring topic of concern among users of GHC is GHC’s correctness and performance, both regarding compilation time and runtime of the generated code, including how GHC’s performance evolves across GHC releases. That concern is aggravated by the long release cycle and the small footprint of libraries in CI builds. As a datapoint that lends credibility to this concern, consider that, although an explicit goal for GHC 8.2 is to improve compilation times, there is evidence that compilation times (for a release candidate) degraded for some packages. Moreover, in the past, releases have often been delayed due to regressions found late in the release cycle.

In this talk, we will analyse the current situation and propose a scheme that enables the (a) continuous monitoring of GHC correctness and performance on (b) a wider range of packages. As our scheme is based on Stackage package sets, we will also argue that widening the range of packages under CI will accelerate the adoption of new GHC releases by the wider community. In particular, we detail our own plans to work towards an implementation of the proposed scheme.

Transcript

1. Manuel M T Chakravarty & Mathieu Boespflug Tweag I/O Tracking GHC Performance And Other Regressions mchakravarty TacticalGrace justtesting.org tweag.io
2. Metrics of Performance Functional Correctness Execution Speed Compilation Speed
3. “Given GHC’s large surface area, getting all of this right is hard?”
4. Functional Correctness GHC 7.8.2
5. Functional Correctness GHC 8.2.1
6. Execution Speed GHC 7.0.1
7. Compilation Speed GHC 8.2.1
8. “What do these issues have in common?”
9. accelerate inline-java clr-inline attoparsec vector
10. “Can we use Stackage for regression testing?”
12. GHC Regression Testing with Stackage ubuntu stackage:nightly docker build docker build stackage:head New
13. ubuntu stackage:nightly docker build docker build stackage:head build-constraints.yaml stackage-curator build-plan.yaml stackage-curator stackage-nightly
14. ubuntu stackage:nightly docker build docker build stackage:head build-constraints.yaml stackage-curator build-plan.yaml stackage-curator Version Conflicts HEAD
15. ubuntu stackage:nightly docker build docker build stackage:head build-constraints.yaml ộ Version Conflicts HEAD Build Results PruneConstraints pruned-constraints.yaml stackage-curator pruned-plan.yaml stackage-curator New
16. Functional Correctness Execution Speed Compilation Speed
17. Functional Correctness build-constraints.yaml ộ pruned-plan.yaml pruned-plan.yaml stackage-curator Build Results stackage-curator Build Results Compare
18. Functional Correctness Execution Speed Compilation Speed
19. Compilation Speed Compare build times of master vs branch Track performance over time “Tracking Performance over the entire software lifecycle” http://www.tweag.io/posts/2017-09-06-hyperion.html
20. Functional Correctness Execution Speed Compilation Speed
21. Execution Speed Packages with benchmark sections PRs and track over time
22. Functional Correctness Execution Speed Compilation Speed
23. Image Attribution https://pixabay.com/photo-2387317/ https://github.com/fpco/stackage/blob/master/DATA-FLOW.md