Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Tracking GHC Performance

Tracking GHC Performance

Presented at "HIW 2017 Haskell Implementors' Workshop": https://icfp17.sigplan.org/track/hiw-2017

A recurring topic of concern among users of GHC is GHC’s correctness and performance, both regarding compilation time and runtime of the generated code, including how GHC’s performance evolves across GHC releases. That concern is aggravated by the long release cycle and the small footprint of libraries in CI builds. As a datapoint that lends credibility to this concern, consider that, although an explicit goal for GHC 8.2 is to improve compilation times, there is evidence that compilation times (for a release candidate) degraded for some packages. Moreover, in the past, releases have often been delayed due to regressions found late in the release cycle.

In this talk, we will analyse the current situation and propose a scheme that enables the (a) continuous monitoring of GHC correctness and performance on (b) a wider range of packages. As our scheme is based on Stackage package sets, we will also argue that widening the range of packages under CI will accelerate the adoption of new GHC releases by the wider community. In particular, we detail our own plans to work towards an implementation of the proposed scheme.

2296a6bdc7779fe4017a23d268c8a79d?s=128

Manuel Chakravarty
PRO

September 11, 2017
Tweet

Transcript

  1. Manuel M T Chakravarty & Mathieu Boespflug Tweag I/O Tracking

    GHC Performance And Other Regressions mchakravarty TacticalGrace justtesting.org tweag.io
  2. Metrics of Performance Functional Correctness Execution Speed Compilation Speed

  3. “Given GHC’s large surface area, getting all of this right

    is hard?”
  4. Functional Correctness GHC 7.8.2

  5. Functional Correctness GHC 8.2.1

  6. Execution Speed GHC 7.0.1

  7. Compilation Speed GHC 8.2.1

  8. “What do these issues have in common?”

  9. accelerate inline-java clr-inline attoparsec vector

  10. “Can we use Stackage for regression testing?”

  11. None
  12. GHC Regression Testing with Stackage ubuntu stackage:nightly docker build docker

    build stackage:head New
  13. ubuntu stackage:nightly docker build docker build stackage:head build-constraints.yaml stackage-curator build-plan.yaml

    stackage-curator stackage-nightly
  14. ubuntu stackage:nightly docker build docker build stackage:head build-constraints.yaml stackage-curator build-plan.yaml

    stackage-curator Version Conflicts HEAD
  15. ubuntu stackage:nightly docker build docker build stackage:head build-constraints.yaml ộ Version

    Conflicts HEAD Build Results PruneConstraints pruned-constraints.yaml stackage-curator pruned-plan.yaml stackage-curator New
  16. Functional Correctness Execution Speed Compilation Speed

  17. Functional Correctness build-constraints.yaml ộ pruned-plan.yaml pruned-plan.yaml stackage-curator Build Results stackage-curator

    Build Results Compare
  18. Functional Correctness Execution Speed Compilation Speed

  19. Compilation Speed Compare build times of master vs branch Track

    performance over time “Tracking Performance over the entire software lifecycle” http://www.tweag.io/posts/2017-09-06-hyperion.html
  20. Functional Correctness Execution Speed Compilation Speed

  21. Execution Speed Packages with benchmark sections PRs and track over

    time
  22. Functional Correctness Execution Speed Compilation Speed

  23. Image Attribution https://pixabay.com/photo-2387317/ https://github.com/fpco/stackage/blob/master/DATA-FLOW.md