Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Rust.Tokyo 2021: How Rust triages regressions and prevents them from reaching to stable

1f7f6943810323d2ad8ac736617047d5?s=47 Yuki Okushi
September 18, 2021

Rust.Tokyo 2021: How Rust triages regressions and prevents them from reaching to stable

1f7f6943810323d2ad8ac736617047d5?s=128

Yuki Okushi

September 18, 2021
Tweet

Transcript

  1. How Rust triages regressions and prevents them from reaching to

    stable Yuki Okushi
  2. Yuki Okushi - Rust contributor - Work at Huawei -

    Working on diagnostics and some docs (Rustnomicon, rustc-dev-guide, ...) - Maintaining some crates (libc, actix-*, ...) - A member of prioritization WG
  3. Rust’s release channels

  4. Rust’s release channels - Rust has three release channels -

    Stable - Beta - Nightly - Usually, people use Stable, or Nightly (for unstable and/or latest changes) - So, when is Beta used?
  5. What are the different types of regressions?

  6. What are the different types of regressions? - Correctness changes

    (ICE, miscompilation, ...) - Performance changes (binary size, memory usage, ...) - Rustdoc-related - ...
  7. Regression types - regression-from-stable-to-nightly: exists only on nightly - regression-from-stable-to-beta:

    exists on beta - regression-from-stable-to-stable: exists on stable😱 - We should definitely prevent this!
  8. How to find regressions before releasing stable?

  9. 1. Crater

  10. What’s Crater? Find regressions by building or testing code on

    Beta or particular PR
  11. How to find? 1. Run `cargo check/build/test` and some tools

    (Clippy, Rustdoc) to a number of crates available on crates.io and GitHub 2. Compare the results with the latest Stable (or the current master branch if it’s PR)
  12. Example: Crater on PR - #82565: Re-stabilize `integer::BITS`

  13. Crater result

  14. Example case: #82565 - Found one crate’s build failed -

    Confirmed it was fixed on newer version - The team confirmed that the change was safe → The PR has been merged with less pain
  15. Check “Compatibility Notes” out on relnotes!

  16. Example: Crater on Beta #86597: Crater runs for 1.54

  17. Example regression #86831: infallible residual could not convert error

  18. 2. Performance monitoring

  19. Rustc's compile-time issue - Rustc is slow to compile (as

    you know) - Well, don’t the team care about it? → Yes, there are some effort!
  20. Performance monitoring - perf.rust-lang.org - Run benchmarks on all the

    merged PRs - Triage them every week - Run benchmarks on perf-sensitive PRs before merging
  21. Performance monitoring

  22. Performance triage

  23. Performance triage It’s also mentioned on TWiR

  24. Example: benchmarks on PR #86385: Use `AttrVec` for `Arm`, `FieldDef`,

    and `Variant`
  25. Regression triage

  26. Regression triage - We’ve been doing our best but we

    cannot reduce the number of regressions to 0. - What should we do if we find them? → This is where the Prioritization WG comes in!
  27. What’s the prioritization working group? - Prioritize the regressions reported

    on GitHub issue tracker - Monitor critical ones not to slip out - Nominate some of them to t-compiler meetings if necessary - ...so that we do not ship regressions to stable mistakenly More details about WG can be found here
  28. Prioritization? - rust-lang/rust has four priority levels - Assign them

    to regressions
  29. Priority levels - Critical - A release blocker, must be

    resolved/discussed until next release (e.g. unsound issue) - High - Not a blocker but should be resolved/triaged ASAP (e.g. working -> broken regression) - Medium - Low
  30. Prioritization workflow: prioritizing regressions

  31. Put them on the t-compiler meeting agenda

  32. Beta backport

  33. How to contribute it?

  34. How to contribute it? Prioritization WG welcomes everyone! - How

    to participate? Can I actually contribute? - Anyone can join the Zulip stream and say “Hi”! - See this blog post for more details
  35. But it’s not the only way!

  36. Other ways: “Cleanup Crew” - Prepare Minimal Complete and Verifiable

    Example (MCVE) - Bisect a commit or nightly that introduced a regression - Find related/duplicate issues if exists More details can be found here
  37. Other ways: Use Rust - Everyone could find a regression

    by using Rust. - Report it to the issue tracker if you find it and have some time Early reporting allows contributors more (earlier) time to work on a fix 🚢
  38. How to create a great bug report? - Fill out

    the issue template as possible - Try the latest Nightly if you’re on Stable or an old Nightly - MCVE would be helpful! - There’s a useful blog post by Felix S Klock II - Backtrace would also be helpful
  39. Conclusion

  40. Conclusion - Rust has several ways to prevent regressions -

    Crater, performance monitoring, prioritization WG - The discussion is open - Everyone can read a log and leave a comment - Prioritizing is one of good ways to get your feet wet at contributing
  41. Thanks!