Mining Gerrit Reviews for Contentious Reviews and Community Evolution

You ask me for a contribution Well, you know We’re
all doing what we can

Shane McIntosh Assistant Professor [email protected] @SoftwareREBELs rebels.ece.mcgill.ca RE Software s
EL B Mining Gerrit for Contentious Reviews & Community Evolution

RE Software s EL B RE Software s EL B

Repository Excavation

Development data is mainly kept for archival purposes 4

Development data is mainly kept for archival purposes 4 What
if your software repositories could talk?

My students and I mine through historical development data in
search of gold 5

search of gold 5 In which modules are future defects likely to appear? Baljinder Ghotra Chakkrit (Kla) T.

search of gold 5 In which modules are future defects likely to appear? Baljinder Ghotra Chakkrit (Kla) T. Can risky code be predicted at creation time? Takafumi Fukushima Jacob G. Barnett

Repository Excavation

report deploy build commit Build Engineering RE S B Repository Excavation

Release Engineering: Supporting release pipelines 7

Release Engineering: Supporting release pipelines Patch + + - -
7

Release Engineering: Supporting release pipelines 1. Integration Patch + +
- - 7

- - 2. Build 7

- - New! 2. Build 3. Deployment 7

- - New! 2. Build 3. Deployment 7 Decision support for modern release pipelines

1. Integration Patch 2. Build 3. Deployment Release Engineering: Supporting
release pipelines + + - - New! 8

release pipelines + + - - New! 8 Can we improve support for modern code review? Toshiki Hirao Ray Wen

release pipelines + + - - New! 8 How can we optimize available CI/CD resources? Keheliya Gallaba Can we improve support for modern code review? Toshiki Hirao Ray Wen

RE Software s EL B report deploy build commit Build
Engineering Repository Excavation

RE Software s EL B report deploy build commit Build
Engineering Repository Excavation We are looking to form industrial partnerships!

10 An overview of the modern code review process

Author Module A Code change 10 An overview of the
modern code review process

A code review tool (e.g., Gerrit) Author Module A Code
change 10 An overview of the modern code review process

Module A Qt Review #27977 11 A code review tool
(e.g., Gerrit) Author Reviewers may identify problems or suggest alternative approaches

(e.g., Gerrit) Reviewer Author Reviewers may identify problems or suggest alternative approaches

(e.g., Gerrit) Reviewer Shouldn't console.log() call the toString() method (where appropriate) on objects? Identifying a defect Author Reviewers may identify problems or suggest alternative approaches

(e.g., Gerrit) Reviewer Shouldn't console.log() call the toString() method (where appropriate) on objects? Identifying a defect Reviewer I think it’s better to do var s = "{}" console.log(s) Suggesting a solution Author Reviewers may identify problems or suggest alternative approaches

(e.g., Gerrit) Reviewer Shouldn't console.log() call the toString() method (where appropriate) on objects? Identifying a defect Reviewer I think it’s better to do var s = "{}" console.log(s) Suggesting a solution Author Update v2 Reviewers may identify problems or suggest alternative approaches

Author Module A Qt Review #35360 Providing updates to the
code change Note that running qutlook will probably crash. I will push a patch set to ﬁx the issue. Reviewer 12 A code review tool (e.g., Gerrit) Some reviewers even provide potential solutions to the issues they identify

Author Module A Qt Review #35360 Providing updates to the
code change Note that running qutlook will probably crash. I will push a patch set to ﬁx the issue. Reviewer 12 A code review tool (e.g., Gerrit) v2 Some reviewers even provide potential solutions to the issues they identify

Modern code review: Changes are guilty until proven innocent

(1) Upload change revision(s) (2) Execute sanity tests (3) Solicit
peer feedback (4) Initiate integration request (5) Execute integration tests (6) Final integration Version Control System Pass Fail Legend Modern code review: Changes are guilty until proven innocent

Code review takes a lot of developer time and is
not always useful 14

not always useful Impact of Peer Code Review on Peer Impression Formation: A Survey [Bosu and Carver, ESEM 2013] Devs spend 6 hours per week reviewing code 14

not always useful Impact of Peer Code Review on Peer Impression Formation: A Survey [Bosu and Carver, ESEM 2013] Devs spend 6 hours per week reviewing code Characteristics of Useful Code Reviews: An Empirical Study at Microsoft [Bosu et al., MSR 2015] 35% of review comments are “not useful” 14

Developers do not like to spend time on process

Mining Gerrit reviews to study reviewing processes

Mining Gerrit reviews to study reviewing processes Contentious reviews Toshiki
Hirao PhD Student

Mining Gerrit reviews to study reviewing processes Community evolution Ray
Wen Master’s Student Contentious reviews Toshiki Hirao PhD Student

And when the broken hearted people, Living in the world
agree, There will be an answer, Let it be

And when the broken hearted people, Living in the world
agree, There will be an answer, Let it be What happens when reviewers disagree?

Reviewer opinions about a patch may differ Author

Reviewer opinions about a patch may differ Reviewer 1 Author
+2

Reviewer 2 +2 –2

Reviewer 2 +2 –2 How do we proceed?

Understanding code review in contentious patches

Understanding code review in contentious patches How often?

Understanding code review in contentious patches How often? Trend over
time?

time? What drives abandonment? Start

time? What drives abandonment? Start How are concerns resolved?

Selecting subject systems for analysis or: why did we study
OpenStack? 0 22,500 45,000 67,500 90,000 Multi-reviewer patches Solo-reviewer patches 67% 21% 26% 17% 6%

time? What drives abandonment? Start How are concerns resolved?

Classifying contentious reviews Include SP (+2) Include SN (-2) Include
SN (-2) Yes No Yes No Yes No +2 +1 -2 -1 +2 +1 -1 +1 -2 -1 +1 -1 SP-SN SP-WN WP-SN WP-WN

Handling reviews with several revisions Contentious Pattern Non-contentious Pattern Revision
1 Revision 2 -1 B +2 A +1 A +1 C Revision 1 Revision 2 -2 A +1 B +1 C

Contentious reviews account for 26% of all OpenStack reviews 0
2,000 4,000 6,000 8,000 6,378 558 7,753 684 (1%) (13%) (1%) (11%) +2, -2 +2, -1 +1, -2 +1, -1 Number (and percentage) or reviews in each category

Contentious reviews with equal scores have a tendency to be
integrated 0% 22.5% 45% 67.5% 90% 76% 29% 88% 54% +2, -2 +2, -1 +1, -2 +1, -1 Rate at which patches are eventually integrated

time? What drives abandonment? Start How are concerns resolved? Contentious reviews are not rare Contention is not a show stopper

Contentious reviews are growing over time in absolute terms 1
10 100 1000 10000 1 3 5 7 9 11 15 19 23 27 31 35 • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • 1 10 100 1000 10000 • SP−SN SP−WN WP−SN WP−WN 11 011 11 12 12 012 12 012 12 13 13 013 13 013 13 14 14 014

Contentious reviews are also growing over time in comparative terms
0% 5% 10% 15% 20% 25% 30% 1 1 1 2 2 2 2 2 2 3 3 3 3 3 3 4 4 4 Weak Nega;ve (SP-WN, WP-WN) Strong Nega;ve (SP-SN, WP-SN)

Integration rates of contentious reviews have roughly stabilized in recent
periods 0.0 0.2 0.4 0.6 0.8 1.0 • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • 0% 20% 40% 60% 80% 100% • SP−SN SP−WN WP−SN WP−WN

time? What drives abandonment? Start How are concerns resolved? Contentious reviews are not rare Contention is not a show stopper Contentious reviews have been growing over time Integration rates of contentious reviews have stabilized

Qualitative analysis: Selecting a representative sample 684 reviews with +2
and -2 Sample of 362 reviews Random sampling for 95% CL 5% CI

Qualitative analysis: Manual classiﬁcation of reviews Sample of 362 reviews

Looks like a test coverage issue

Looks like a test coverage issue Agreed!

Looks like a test coverage issue Agreed! Test coverage

Qualitative analysis: Lifting tags to cohesive subhierarchies Test coverage Already
ﬁxed Side effects Alternative solution

ﬁxed Side effects Alternative solution Design Unnecessary contribution QA

ﬁxed Side effects Alternative solution Design Unnecessary contribution QA Internal concerns External concerns

Abandonment is often due to external concerns! Abandon External Concerns
Internal Concerns Integration Policy Compliance Unnecessary Fix Integration Planning Lack of Interest Testing Implementation Design Unclear Intention Not an Issue Already Fixed Blueprint Release Schedule Patch Dependency Lost by a reviewer Lost by an author Side Effect Shallow Fix Flawed Changes Alternative Solution Backward Compatibility Patch Size Test Failure Test Coverage 59% (102/173) 21 12 24 6 1 1 32 17 5 6 14 6 1 173 10 41% (71/173) 58% (59/105) 7% (7/105) 87% (62/71) 10% (7/71) 3% (2/71) 25% (25/105) 5 Branch Placement Squashing Commits 6 5 11% (11/105) 1

time? What drives abandonment? Start How are concerns resolved? Contentious reviews are not rare Contention is not a show stopper Contentious reviews have been growing over time Integration rates of contentious reviews have stabilized Concerns beyond patch scope are more strongly linked with abandonment than concerns within patch scope

Contention is more often resolved without altering the patch! Integrate
Directly Addressing Indirectly Addressing Legal Problems Unnecessary Fix Design Implementation Testing Integration Policy Compliance Alternative Solution Side Effect Shallow Fix Flawed Changes Backward Compatibility Test Coverage Test Failure Squashing Commits Branch Placement 44% (83/189) 2 1 3 9 21 7 4 65 12 3 3 3 70% (58/83) 189 Withdrawal of Negative Score Integration Planning Persuasion Self-change Release Schedule Blueprint Patch Dependency 10 56% (106/189) 4% (3/83) 7% (6/83) 5% (4/83) 28% (30/106) 72% (76/106) Legal Issues Unclear Intention 12% (10/83) 2% (2/83) 4 4 38

time? What drives abandonment? Start How are concerns resolved? Contentious reviews are not rare Contention is not a show stopper Contentious reviews have been growing over time Integration rates of contentious reviews have stabilized Concerns beyond patch scope are more strongly linked with abandonment than concerns within patch scope Concerns are often addressed through negotiation or scheduling

Life is very short, and there’s no time, for fussing
and ﬁghting, my friends

And in the end The love you take Is equal
to The love you make

And in the end The love you take Is equal
to The love you make You get what you put in(to code review)

You get what you put into code review Author

You get what you put into code review Author Looks
like code to me! +2 Rookie Reviewer

You get what you put into code review Author Looks
like code to me! +2 Rookie Reviewer Seasoned Reviewer Did you consider alternative designs? -1

How does reviewing feedback evolve with respect to…

How does reviewing feedback evolve with respect to… Community aging?

Reviewer experience?

Reviewer experience? Reviewer workload?

Automatically analyzing topics that are discussed in code reviews

Automatically analyzing topics that are discussed in code reviews Get
inline comments 506,950 inline comments

inline comments 506,950 inline comments Apply topic analysis Topic 1 Topic 2 Topic 3 Topic 1 Topic 2 Topic 3

inline comments 506,950 inline comments Apply topic analysis Topic 1 Topic 2 Topic 3 Topic 1 Topic 2 Topic 3 Topic score analysis

LDA automatically groups words into topics based on co-occurrence in
comments How Does Code Reviewing Feedback Evolve? 11 Table 1: The labelled topics with their corresponding words and topic share score CATEGORY TOPIC NAME SELECTED MOST RELEVANT WORDS TOPIC SHARE (%) Operations Logging log, debug, warn, little, anywhere, bit, level, dashboard 4.8 Release Engineering comment, version, package, install, repo, option, config, copyright 4.4 Performance big, core, timeout, performance, time, team, choice 5.5 Testing Testing test, miss, unit, sentence, character, functional, period 4.1 Improvement Request need, line, blank, pep, ref, alarm, insert, codeblock 4.1 Social Communication General Social Comm. ok, release, fix, yeah, liberty, thank, rid, mitaka, ah 6.8 Confirmation ditto, token, context, label, auth, align, hint, keystone 2.1 Server-related Discussion User and Roles service, role, admin, user, rule, ten- ant, security, snapshot 4.5 Network Issues port, network, ip, agent, router, ad- dress, neutron, subnet 2.8 Node Issues node, task, state, migration, cluster, disk, step, lock 4.1 Disk Image Issues image, class, glance, function, 3.3

We read comments with high topic scores and name the
topics How Does Code Reviewing Feedback Evolve? 11 Table 1: The labelled topics with their corresponding words and topic share score CATEGORY TOPIC NAME SELECTED MOST RELEVANT WORDS TOPIC SHARE (%) Operations Logging log, debug, warn, little, anywhere, bit, level, dashboard 4.8 Release Engineering comment, version, package, install, repo, option, config, copyright 4.4 Performance big, core, timeout, performance, time, team, choice 5.5 Testing Testing test, miss, unit, sentence, character, functional, period 4.1 Improvement Request need, line, blank, pep, ref, alarm, insert, codeblock 4.1 Social Communication General Social Comm. ok, release, fix, yeah, liberty, thank, rid, mitaka, ah 6.8 Confirmation ditto, token, context, label, auth, align, hint, keystone 2.1 Server-related Discussion User and Roles service, role, admin, user, rule, ten- ant, security, snapshot 4.5 Network Issues port, network, ip, agent, router, ad- dress, neutron, subnet 2.8 Node Issues node, task, state, migration, cluster, disk, step, lock 4.1 Disk Image Issues image, class, glance, function, 3.3 How Does Code Reviewing Feedback Evolve? 13 ging features are used to aid in debugging and understanding system behaviour. To ensure that the logs contain useful information, logging concerns are often raised during the code review process. How to enable TRACE log level in config file? Enabling the trace would enable debug also? Comment #136066, topic score(logging) = 0.94 The development of OPENSTACK involves contributions from several large soft-

Then we group related topics into categories How Does Code
Reviewing Feedback Evolve? 11 Table 1: The labelled topics with their corresponding words and topic share score CATEGORY TOPIC NAME SELECTED MOST RELEVANT WORDS TOPIC SHARE (%) Operations Logging log, debug, warn, little, anywhere, bit, level, dashboard 4.8 Release Engineering comment, version, package, install, repo, option, config, copyright 4.4 Performance big, core, timeout, performance, time, team, choice 5.5 Testing Testing test, miss, unit, sentence, character, functional, period 4.1 Improvement Request need, line, blank, pep, ref, alarm, insert, codeblock 4.1 Social Communication General Social Comm. ok, release, fix, yeah, liberty, thank, rid, mitaka, ah 6.8 Confirmation ditto, token, context, label, auth, align, hint, keystone 2.1 Server-related Discussion User and Roles service, role, admin, user, rule, ten- ant, security, snapshot 4.5 Network Issues port, network, ip, agent, router, ad- dress, neutron, subnet 2.8 Node Issues node, task, state, migration, cluster, disk, step, lock 4.1 Disk Image Issues image, class, glance, function, 3.3

PC1 PC2 Marginal topic distribution 2% 5% 10% 1 2
3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 20 19 18 Communication Testing Software Design Operations Server-oriented Version Control File System Documentation Formatting 1 - Documentation and Testing 2 - Social Communication 3 - Testing 4 - Key Errors 5 - Logging 6 - Default Parameters 7 - Object names ? 8 - Release Engineering & Ops 9 - Time and Performance 10 - User & Roles 11 - Exception Handling 12 - Image Issues 13 - Version Control 14 - File System 15 - Network Issues 16 - Node Issues 17 - Conﬁrmation 18 - Requests 19 - Function Header 20 - Formatting Legends

As the community has aged, design- oriented feedback has declined
Node Issues • • • • • • • • • • • • • • • * Disk Image Issues • • • • • • • • • • • • • • • • • • • • • Software Design Key Errors + • • • • • • • • • • • • • • • • • • • • • + Exception Han- dling + • • • • • • • • • • • • • • • • • • • • • Object Names • • • • • • • • • • • • • • • • • • • • • * Function Headers + • • • • • • • • • • • • • • • • • • • • • * Default and Pa- rameters • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •

On the other hand, discussion of networking and patch formatting
have grown Social Communication General Social Comm. • • • • • • • • * Conﬁrmation • • • • • • • • • • • • • • • • • • • • • Server-related Discussion User and Roles • • • • • • • • • • • • • • • • • • • • • + Network Issues * • • • • • • • • • • • • • • • • • • • • • Node Issues • • • • • • • • • • • • • • • • • • • • • * Disk Image Issues • • • • • • • • • • • • • • • • • • • • • Software Design Key Errors + • • • • • • • • • • • • • • • • • • • • • + • • • • • • • • • Function Headers + • • • • • • • • • • • • • • • • • • • • • * Default and Pa- rameters • • • • • • • • • • • • • • • • • • • • • - Version Control • • • • • • • • • • • • • • • • • • • • • - File System + • • • • • • • • • • • • • • • • • • • • • + - Formatting * • • • • • • • • • • • • • • • • • • • • • + - Documentation • • • • • • • • • • • • • • • • • • • •

Reviewer experience? Reviewer workload? As OpenStack has aged, the community has focused less on patch design and more on networking and patch formatting

System-oriented topics increase as reviewers accrue experience with OpenStack CATEGORY
TOPIC NAME TIME TREND TOPIC IMPACT (OVER TIME) EXP. TREND Operations Logging • • • • • • • • • • • • • • • • • • • • • * Release Engineer- ing • • • • • • • • • • • • • • • • • • • • • + Performance • • • • • • • • • • • • • • • • • • • • • * Quality Assurance Testing • • • • • • • • • • • • • • • • • • • • • Improvement Re- quest • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • E TIME TREND TOPIC IMPACT (OVER TIME) EXP. TREND TOPIC EXPERIENCE IMPACT WORKL TREND • • • • • • • • • • • • • • • • • • • • • * • • • • • •••••• • • • • • • •• • • • • • • • • • •• * ineer- • • • • • • • • • • • • • • • • • • • • • + • • • • • • • • • • •• • • •• •• • • • • •• ••• •• • + • • • • • • • • • • • • • • • • • • • • • * • • • •• • • • • • • •• • • • • • ••• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• •• • • • • • • ••• • • • • • t Re- • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • ••• • • • • • • • • • • • • • • • • • • • • • TREND (OVER TIME) TREND Operations Logging • • • • • • • • • • • • • • • • • • • • • * Release Engineer- ing • • • • • • • • • • • • • • • • • • • • • + Performance • • • • • • • • • • • • • • • • • • • • • * Quality Assurance Testing • • • • • • • • • • • • • • • • • • • • • Improvement Re- quest • • • • • • • • • • • • • • • • • • • • • Social Communication General Social Comm. • • • • • • • • • • • • • • • • • • • • • * • • • • • • TREND (OVER TIME) TREND IMPACT TREND • • • • • • • • • • • • • • • • • • • • • * • • • • • •••••• • • • • • • •• • • • • • • • • • •• * ineer- • • • • • • • • • • • • • • • • • • • • • + • • • • • • • • • • •• • • •• •• • • • • •• ••• •• • + e • • • • • • • • • • • • • • • • • • • • • * • • • •• • • • • • • •• • • • • • ••• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• •• • • • • • • ••• • • • • • nt Re- • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • ••• • • • • • • • • • • Social • • • • • • • • • • • • • • • • • • • • • * • ••• • •••• • ••• •• •• • • • • • •• • ••• • • * • • • • • • • • • • • •• • ••• •• • • • • • • ••• • • • • Communication Comm. • Conﬁrmation • • • • • • • • • • • • • • • • • • • • • Server-related Discussion User and Roles • • • • • • • • • • • • • • • • • • • • • + Network Issues * • • • • • • • • • • • • • • • • • • • • • Node Issues • • • • • • • • • • • • • • • • • • • • • * Disk Image Issues • • • • • • • • • • • • • • • • • • • • • Software Design Key Errors + • • • • • • • • • • • • • • • • • • • • • + Exception Han- + • • • • • • • • • • • • • • • • • • • • • • n • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • ••• •• • • • • • • ••• • • • • oles • • • • • • • • • • • • • • • • • • • • • + • • • • •• • • • •••• • • • •••••• • • • • • • • • + ues * • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • ••• • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • * • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • + Issues • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • •• • • • • • • • • • • ••• • • • • • • + • • • • • • • • • • • • • • • • • • • • • + • •• • • • • •• •• • • • • • • • • • • ••• •• • • • • Han- + • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • ••• • • * Improvement Re- quest • • • • • • • • • • • • • • • • • • • • Social Communication General Social Comm. • • • • • • • • • • • • • • • • • • • • • * Conﬁrmation • • • • • • • • • • • • • • • • • • • • • Server-related Discussion User and Roles • • • • • • • • • • • • • • • • • • • • • + Network Issues * • • • • • • • • • • • • • • • • • • • • • Node Issues • • • • • • • • • • • • • • • • • • • • • * Disk Image Issues • • • • • • • • • • • • • • • • • • • • • CATEGORY TOPIC NAME TIME TREND TOPIC IMPACT (OVER TIME) EXP. TREND Operations Logging • • • • • • • • • • • • • • • • • • • • • * Release Engineer- ing • • • • • • • • • • • • • • • • • • • • • + Performance • • • • • • • • • • • • • • • • • • • • • * Quality Assurance Testing • • • • • • • • • • • • • • • • • • • • • Improvement Re- quest • • • • • • • • • • • • • • • • • • • • • Social Communication General Social Comm. • • • • • • • • • • • • • • • • • • • • • * Server-related Discussion User and Roles • • • • • • • • • • • • • • • • • • • • + Network Issues * • • • • • • • • • • • • • • • • • • • • • Node Issues • • • • • • • • • • • • • • • • • • • • • * Disk Image Issues • • • • • • • • • • • • • • • • • • • • • Software Design Key Errors + • • • • • • • • • • • • • • • • • • • • • + Exception Han- dling + • • • • • • • • • • • • • • • • • • • • • Object Names • • • • • • • • • • • • • • • • • • • • • * • • • • • • • • • • • • • • • • • • • • • * • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • + Issues • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • •• • • • • • • • • • • ••• • • • • • • + • • • • • • • • • • • • • • • • • • • • • + • •• • • • • •• •• • • • • • • • • • • ••• •• • • • • Han- + • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • ••• • • • * es • • • • • • • • • • • • • • • • • • • • • * • • • ••• • • • ••••• • •• • •• •• • • • • • • • • * aders + • • • • • • • • • • • • • • • • • • • • • * • • • • ••• • •• • • • • • • • • • • • ••• • •• • • • * d Pa- • • • • • • • • • • • • • • • • • • • • •• •• •• • • • • • • • • • • • • • • • • • • • • • • • • • • • * • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • + Issues • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • •• • • • • • • • • • • ••• • • • • • • + • • • • • • • • • • • • • • • • • • • • • + • •• • • • • •• •• • • • • • • • • • • ••• •• • • • • Han- + • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • ••• • • • * es • • • • • • • • • • • • • • • • • • • • • * • • • ••• • • • ••••• • •• • •• •• • • • • • • • • * aders + • • • • • • • • • • • • • • • • • • • • • * • • • • ••• • •• • • • • • • • • • • • ••• • •• • • • * d Pa- • • • • • • • • • • • • • • • • • • • • • • ••• • • • • •• •• •• • • • • • • • • Node Issues • • • • • • • • • • • • • • • • • • • • • * Disk Image Issues • • • • • • • • • • • • • • • • • • • • • Software Design Key Errors + • • • • • • • • • • • • • • • • • • • • • + Exception Han- dling + • • • • • • • • • • • • • • • • • • • • • Object Names • • • • • • • • • • • • • • • • • • • • • * Function Headers + • • • • • • • • • • • • • • • • • • • • • * Default and Pa- • • • • • • • • • • • • • • • • • • E TIME TREND TOPIC IMPACT (OVER TIME) EXP. TREND TOPIC EXPERIENCE IMPACT WORKL TREND • • • • • • • • • • • • • • • • • • • • • * • • • • • •••••• • • • • • • •• • • • • • • • • • •• * ineer- • • • • • • • • • • • • • • • • • • • • • + • • • • • • • • • • •• • • •• •• • • • • •• ••• •• • + e • • • • • • • • • • • • • • • • • • • • • * • • • •• • • • • • • •• • • • • • ••• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• •• • • • • • • ••• • • • • • t Re- • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • ••• • • • • • • • • • • Social • • • • • • • • • • • • • • • • • • • • • * • ••• • •••• • ••• •• •• • • • • • •• • ••• • • * •

Low ROI topics like Formatting tend to decrease as reviewers
accrue experience rameters • • • • • • • - Version Control • • • • • • • • • • • • • • • • • • • • • - File System + • • • • • • • • • • • • • • • • • • • • • + - Formatting * • • • • • • • • • • • • • • • • • • • • • + - Documentation • • • • • • • • • • • • • • • • • • • • where D(m) represents the set of review comm other words, the topic impact metric measures • • • • • • • • • ••••• • • • • •• • •• •• • • • • • • • • • • Control • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • •• • •• • • • • • • •• • • • • • • • em + • • • • • • • • • • • • • • • • • • • • • + •• ••• •• • • • • • •• • • • • • • • • •• • • • • • • + ng * • • • • • • • • • • • • • • • • • • • • • + • • • • •• • • • •• •• •• • • • • • • •• • • • • • • • + tation • • • • • • • • • • • • • • • • • • • • • • • ••• • • • • •• • • • • • •• • • • • • • • • • • • esents the set of review comments that are written topic impact metric measures the proportion of r

Reviewer experience? Reviewer workload? As OpenStack reviewers accrue experience, they tend to comment more on system-oriented topics and less on low ROI topics As OpenStack has aged, the community has focused less on patch design and more on networking and patch formatting

Reviewers with high experience scores tend to also have a
heavy workload score ( ) , =0.68

Reviewer experience? Reviewer workload? Similar observations as were reported for experience As OpenStack has aged, the community has focused less on patch design and more on networking and patch formatting As OpenStack reviewers accrue experience, they tend to comment more on system-oriented topics and less on low ROI topics

I wanna hold your hand!

Mining Gerrit Reviews for Contentious Reviews and Community Evolution

Mining Gerrit Reviews for Contentious Reviews and Community Evolution

Other Decks in Programming

Featured

Transcript