Slide 1
Slide 1 text
How Rust triages regressions
and prevents them from
reaching to stable
Yuki Okushi
Slide 2
Slide 2 text
Yuki Okushi
- Rust contributor
- Work at Huawei
- Working on diagnostics and some docs (Rustnomicon,
rustc-dev-guide, ...)
- Maintaining some crates (libc, actix-*, ...)
- A member of prioritization WG
Slide 3
Slide 3 text
Rust’s release channels
Slide 4
Slide 4 text
Rust’s release channels
- Rust has three release channels
- Stable
- Beta
- Nightly
- Usually, people use Stable, or Nightly (for unstable and/or
latest changes)
- So, when is Beta used?
Slide 5
Slide 5 text
What are the different types of regressions?
Slide 6
Slide 6 text
What are the different types of regressions?
- Correctness changes (ICE,
miscompilation, ...)
- Performance changes (binary size,
memory usage, ...)
- Rustdoc-related
- ...
Slide 7
Slide 7 text
Regression types
- regression-from-stable-to-nightly: exists only
on nightly
- regression-from-stable-to-beta: exists on beta
-
regression-from-stable-to-stable: exists on
stable😱
- We should definitely prevent this!
Slide 8
Slide 8 text
How to find regressions
before releasing stable?
Slide 9
Slide 9 text
1. Crater
Slide 10
Slide 10 text
What’s Crater?
Find regressions by building or testing code on Beta or
particular PR
Slide 11
Slide 11 text
How to find?
1. Run `cargo check/build/test` and some tools (Clippy,
Rustdoc) to a number of crates available on crates.io and
GitHub
2. Compare the results with the latest Stable (or the current
master branch if it’s PR)
Slide 12
Slide 12 text
Example: Crater on PR
- #82565: Re-stabilize `integer::BITS`
Slide 13
Slide 13 text
Crater result
Slide 14
Slide 14 text
Example case: #82565
- Found one crate’s build failed
- Confirmed it was fixed on newer version
- The team confirmed that the change was safe
→ The PR has been merged with less pain
Slide 15
Slide 15 text
Check “Compatibility Notes” out on relnotes!
Slide 16
Slide 16 text
Example: Crater on Beta
#86597: Crater runs for 1.54
Slide 17
Slide 17 text
Example regression
#86831: infallible residual could not convert error
Slide 18
Slide 18 text
2. Performance monitoring
Slide 19
Slide 19 text
Rustc's compile-time issue
- Rustc is slow to compile (as you know)
- Well, don’t the team care about it?
→ Yes, there are some effort!
Slide 20
Slide 20 text
Performance monitoring
- perf.rust-lang.org
- Run benchmarks on all the merged PRs
- Triage them every week
- Run benchmarks on perf-sensitive PRs before merging
Slide 21
Slide 21 text
Performance monitoring
Slide 22
Slide 22 text
Performance triage
Slide 23
Slide 23 text
Performance triage
It’s also mentioned on TWiR
Slide 24
Slide 24 text
Example: benchmarks on PR
#86385: Use `AttrVec` for `Arm`, `FieldDef`, and `Variant`
Slide 25
Slide 25 text
Regression triage
Slide 26
Slide 26 text
Regression triage
- We’ve been doing our best but we cannot reduce the
number of regressions to 0.
- What should we do if we find them?
→ This is where the Prioritization WG comes in!
Slide 27
Slide 27 text
What’s the prioritization working group?
- Prioritize the regressions reported on GitHub issue tracker
- Monitor critical ones not to slip out
- Nominate some of them to t-compiler meetings if
necessary
- ...so that we do not ship regressions to stable
mistakenly
More details about WG can be found here
Slide 28
Slide 28 text
Prioritization?
- rust-lang/rust has four
priority levels
- Assign them to
regressions
Slide 29
Slide 29 text
Priority levels
- Critical
- A release blocker, must be resolved/discussed until
next release (e.g. unsound issue)
- High
- Not a blocker but should be resolved/triaged ASAP
(e.g. working -> broken regression)
- Medium
- Low
Slide 30
Slide 30 text
Prioritization workflow: prioritizing regressions
Slide 31
Slide 31 text
Put them on the t-compiler meeting agenda
Slide 32
Slide 32 text
Beta backport
Slide 33
Slide 33 text
How to contribute it?
Slide 34
Slide 34 text
How to contribute it?
Prioritization WG welcomes everyone!
- How to participate? Can I actually contribute?
- Anyone can join the Zulip stream and say “Hi”!
- See this blog post for more details
Slide 35
Slide 35 text
But it’s not the only way!
Slide 36
Slide 36 text
Other ways: “Cleanup Crew”
- Prepare Minimal Complete and Verifiable Example
(MCVE)
- Bisect a commit or nightly that introduced a regression
- Find related/duplicate issues if exists
More details can be found here
Slide 37
Slide 37 text
Other ways: Use Rust
- Everyone could find a regression by using Rust.
- Report it to the issue tracker if you find it and have some
time
Early reporting allows contributors more (earlier) time to
work on a fix 🚢
Slide 38
Slide 38 text
How to create a great bug report?
- Fill out the issue template as possible
- Try the latest Nightly if you’re on Stable or an old Nightly
- MCVE would be helpful!
- There’s a useful blog post by Felix S Klock II
- Backtrace would also be helpful
Slide 39
Slide 39 text
Conclusion
Slide 40
Slide 40 text
Conclusion
- Rust has several ways to prevent regressions
- Crater, performance monitoring, prioritization WG
- The discussion is open
- Everyone can read a log and leave a comment
- Prioritizing is one of good ways to get your feet wet at
contributing
Slide 41
Slide 41 text
Thanks!