Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Tracking GHC Performance

Tracking GHC Performance

Presented at "HIW 2017 Haskell Implementors' Workshop": https://icfp17.sigplan.org/track/hiw-2017

A recurring topic of concern among users of GHC is GHC’s correctness and performance, both regarding compilation time and runtime of the generated code, including how GHC’s performance evolves across GHC releases. That concern is aggravated by the long release cycle and the small footprint of libraries in CI builds. As a datapoint that lends credibility to this concern, consider that, although an explicit goal for GHC 8.2 is to improve compilation times, there is evidence that compilation times (for a release candidate) degraded for some packages. Moreover, in the past, releases have often been delayed due to regressions found late in the release cycle.

In this talk, we will analyse the current situation and propose a scheme that enables the (a) continuous monitoring of GHC correctness and performance on (b) a wider range of packages. As our scheme is based on Stackage package sets, we will also argue that widening the range of packages under CI will accelerate the adoption of new GHC releases by the wider community. In particular, we detail our own plans to work towards an implementation of the proposed scheme.

Manuel Chakravarty
PRO

September 11, 2017
Tweet

More Decks by Manuel Chakravarty

Other Decks in Programming

Transcript

  1. Manuel M T Chakravarty & Mathieu Boespflug

    Tweag I/O
    Tracking GHC Performance
    And Other Regressions
    mchakravarty
    TacticalGrace
    justtesting.org
    tweag.io

    View Slide

  2. Metrics of Performance
    Functional
    Correctness
    Execution
    Speed
    Compilation
    Speed

    View Slide

  3. “Given GHC’s large surface area,
    getting all of this right is hard?”

    View Slide

  4. Functional Correctness
    GHC 7.8.2

    View Slide

  5. Functional Correctness
    GHC 8.2.1

    View Slide

  6. Execution Speed
    GHC 7.0.1

    View Slide

  7. Compilation Speed
    GHC 8.2.1

    View Slide

  8. “What do these issues
    have in common?”

    View Slide

  9. accelerate inline-java
    clr-inline
    attoparsec
    vector

    View Slide

  10. “Can we use Stackage for
    regression testing?”

    View Slide

  11. View Slide

  12. GHC Regression Testing with Stackage
    ubuntu
    stackage:nightly
    docker build
    docker build
    stackage:head
    New

    View Slide

  13. ubuntu
    stackage:nightly
    docker build
    docker build
    stackage:head
    build-constraints.yaml
    stackage-curator
    build-plan.yaml
    stackage-curator
    stackage-nightly

    View Slide

  14. ubuntu
    stackage:nightly
    docker build
    docker build
    stackage:head
    build-constraints.yaml
    stackage-curator
    build-plan.yaml
    stackage-curator
    Version Conflicts
    HEAD

    View Slide

  15. ubuntu
    stackage:nightly
    docker build
    docker build
    stackage:head
    build-constraints.yaml

    Version Conflicts
    HEAD
    Build Results
    PruneConstraints
    pruned-constraints.yaml
    stackage-curator
    pruned-plan.yaml
    stackage-curator
    New

    View Slide

  16. Functional
    Correctness
    Execution
    Speed
    Compilation
    Speed

    View Slide

  17. Functional
    Correctness
    build-constraints.yaml

    pruned-plan.yaml pruned-plan.yaml
    stackage-curator
    Build Results
    stackage-curator
    Build Results
    Compare

    View Slide

  18. Functional
    Correctness
    Execution
    Speed
    Compilation
    Speed

    View Slide

  19. Compilation
    Speed
    Compare build times of master vs branch
    Track performance over time
    “Tracking Performance over the entire software lifecycle”
    http://www.tweag.io/posts/2017-09-06-hyperion.html

    View Slide

  20. Functional
    Correctness
    Execution
    Speed
    Compilation
    Speed

    View Slide

  21. Execution
    Speed
    Packages with benchmark sections
    PRs and track over time

    View Slide

  22. Functional
    Correctness
    Execution
    Speed
    Compilation
    Speed

    View Slide

  23. Image Attribution
    https://pixabay.com/photo-2387317/
    https://github.com/fpco/stackage/blob/master/DATA-FLOW.md

    View Slide