Example Overflow: Using Social Media for Code Recommendation

A talk I've given as part of the IBM Programming Languages and Software Engineering (PLSE) Seminar in 2012

Transcript

1. USING SOCIAL MEDIA FOR CODE RECOMMENDATION Alexey Zagalsky Ohad Barzilay Amiram Yehudai Tel Aviv University
2. Introduction  Present myself  Ongoing Research
3. Social Media Recommendations
4. Social Media “Architecture of Participation” Tim O’reilly
5. Social Media in SE Source Code Comments Tagging Micro- blogging Blogging Q&A Websites Wikies
6. Stack Overflow
7. Stack Overflow Stats (10.11.2012)  Total Questions: 3.9M  Total Answers: 7.6M  Comments: 16.9M  Questions per Min: 2.96  Answers per Min: 4.29  Views per Day: 2M
8. Related Work  Over 92% of the questions on StackOverflow are answered, and for those 92% the median answer time is 11 minutes. Design lessons from the fastest q&a site in the west L.Mamykina, B.Manoim, M.Mittal, G.Hripcsak, B.Hartmann (2011)  Linking StackOverflow data with API usage Crowd Documentation: Exploring the Coverage and the Dynamics of API Discussions on Stack Overflow Chris Parnin, Cristoph Teude, Lars Grammel and Margaret-Anne Storey (2012)
9. Sea Hawk Alberto Bacchelli, Luca Ponzanelli, Michele Lanza (2012)
10. Our Research Example Overflow
11. Research Question  Q1: Is it possible to use SO for code recommendations ?  Q2: Can we improve example usage by supporting actions (searching, comparing)?
12. Short Demo
13. Design Objectives  Comparing multiple examples  Display several code example at once  Display only the code snippet itself  Minimal context switching  If required show additional context at the same place  No need to go “forward” or “backwards”
14. Features
15. Example Overflow Populating the Repository
16. Example Overflow Searching  Keyword Search based on TF-IDF  Each example is represented as a Document with several parts.  Sdoc = [ Wtitle Stitle + Wtag Stag + Wanswer Sanswer + Wquestion Squestion + Wcode Scode ] Smetadata
18. Perliminary Evaluation Data Point Search Query Dynamic Dimension "jquery dynamic dimension" Hover "jquery hover div" Position "jquery position" Rounded Corners "jquery rounded corner" Draggable "jquery draggable" Droppable "jquery droppable" Autocomplete "jquery autocomplete from db" Accordion "jquery accordion" Date Picker "jquery datepicker" Image Scale "jquery image scale effect"
19. Rank of a suitable example at the returned search results Data Point Code Repository Tools Google Search Krugle Koders Stack Overflow Example Overflow Dynamic Dimension 4 Not found Not found 1 3 Hover 1 2 1 1 2 Position 3 Not found Not found 4 1 Rounded Corners 2 Not found 3 3 1 Draggable 1 Not found 3 2 1 Dropable 1 Not found 3 1 2 Autocomplete 1 Not found Not Found 1 1 Accordion 1 Not found 12 3 1 Date Picker 1 Not found 3 1 1 Image Scale 2 Not found Not found Not found 3 Avg. Rank 1.7 19.1 9.7778 3.8 1.6
20. Context Switching Comparison Data Point Code Repository Tools Google Search Krugle Koders Stack Overflow Example Overflow Dynamic Dimension 7 - - 1 0 Hover 1 3 1 1 0 Position 5 - - 7 0 Rounded Corners 3 - 5 5 0 Draggable 1 - 5 3 0 Dropable 1 - 5 2 0 Autocomplete 3 - - 1 0 Accordion 1 - 23 6 0 Date Picker 1 - 5 1 0 Image Scale 3 - - - 0 Avg. Mouse Clicks 2.6 3 7.3333 3 0
21. Small Pilot at ICSE12  The search is keyword dependent  Sequential Search vs Iterative Search (refine)  System performance matters  Feedback loop  Runnable code
22. Limitations  EO not only enables Example Embedding EcoSystem but also enabled by it.  It is unclear whether using SO can scale well  Preliminary evaluation is limited and a thorough user study should be conducted.
23. Future Work  User study:  Quantative  Example Overflow logs every action per user+session:  Query used  Results viewed  Additional pages loaded  Result chosen  Additional context needed  Qualitative  User observations
24. Questions ?