Using Q&A Websites as a Method for Assessing Systematic Reviews

Transcript

1. Using Q&A Websites as a Method for Assessing Systematic Reviews Bruno Cartaxo Gustavo Pinto Danilo Ribeiro Fernando Kanei Ronnie E. S. Santos Fabio Q. B. da Silva Sérgio Soares
2. Questions and Answers (Q&A) websites enabled researchers to investigate the issues practitioners face in practice1 Systematic reviews (SRs) were presented as a way to integrate the best research evidence with practice2 However... some argue that there is a lack of connection between SRs and software engineering (SE) practice3,4 2 1. Mamykina, et al. Design lessons from the fastest q&a site in the west, CHI, 2011 2. Kitchenham et al. Evidence-Based Software Engineering, ICSE 2004 3. Hassler et al. Outcomes of a community workshop to identify and rank barriers to the systematic literature review process, EASE 2014 4. Santos and Da Silva. Motivation to perform systematic reviews and their impact on software engineering practice, ESEM 2013
3. RQ To what extent systematic reviews cover agile related questions posted on Q&A websites? Operational Definition of Coverage When at least one finding of a SR offers knowledge that helps to solve a question posted in SE related StackExchange websites 3
4. METHOD 4
5. 2. Systematic Reviews Selection 1. StackExchange Websites Selection Programmers (PROG) Project Management (PM) Quality Assurance & Testing (SQA) Reverse Engineering (RE) Software Recommendations (SREC) 5 120 SRs of a tertiary study1 32 SRs with guidelines1 24 SRs with search string 4 SRs about agile Practitioners guildelines filter 1. Da Silva et al. Six years of systematic literature reviews in software engineering: An updated tertiary study, IST, 2011 Search string filter Agile filter
6. 3. Related Questions Selection 6 46,743 Questions 26,687 Questions 641 Questions 293 Questions Questions’ score filter Search strings filter False-positive filter 4. Coverage Analysis Open Coding + Constant Comparison1 1. Stol et al. “Grounded theory in software engineering research: A critical review and guidelines. ICSE, 2016
7. RESULTS 7
8. Overall Coverage Coverage of the Agile Methods/Practices 8 Coverage of the Recurrent Questions
9. CONCLUDING REMARKS 9
10. Implications One can use our method to assess whether a research topic is demanded in practice Limitations The SRs came from a tertiary study published in 2011 Future Work Expand this analysis to SRs published till nowadays 10
11. THANK YOU Bruno Cartaxo bfsc@cin.ufpe.br 11