Example Overflow: Using Social Media for Code Recommendation

Example Overflow: Using Social Media for Code Recommendation

A talk I've given as part of the IBM Programming Languages and Software Engineering (PLSE) Seminar in 2012

419a48edd42158cd11d802f06b448e83?s=128

Alexey Zagalsky

November 12, 2012
Tweet

Transcript

  1. USING SOCIAL MEDIA FOR CODE RECOMMENDATION Alexey Zagalsky Ohad Barzilay

    Amiram Yehudai Tel Aviv University
  2. Introduction  Present myself  Ongoing Research

  3. Social Media Recommendations

  4. Social Media “Architecture of Participation” Tim O’reilly

  5. Social Media in SE Source Code Comments Tagging Micro- blogging

    Blogging Q&A Websites Wikies
  6. Stack Overflow

  7. Stack Overflow Stats (10.11.2012)  Total Questions: 3.9M  Total

    Answers: 7.6M  Comments: 16.9M  Questions per Min: 2.96  Answers per Min: 4.29  Views per Day: 2M
  8. Related Work  Over 92% of the questions on StackOverflow

    are answered, and for those 92% the median answer time is 11 minutes. Design lessons from the fastest q&a site in the west L.Mamykina, B.Manoim, M.Mittal, G.Hripcsak, B.Hartmann (2011)  Linking StackOverflow data with API usage Crowd Documentation: Exploring the Coverage and the Dynamics of API Discussions on Stack Overflow Chris Parnin, Cristoph Teude, Lars Grammel and Margaret-Anne Storey (2012)
  9. Sea Hawk Alberto Bacchelli, Luca Ponzanelli, Michele Lanza (2012)

  10. Our Research Example Overflow

  11. Research Question  Q1: Is it possible to use SO

    for code recommendations ?  Q2: Can we improve example usage by supporting actions (searching, comparing)?
  12. Short Demo

  13. Design Objectives  Comparing multiple examples  Display several code

    example at once  Display only the code snippet itself  Minimal context switching  If required show additional context at the same place  No need to go “forward” or “backwards”
  14. Features

  15. Example Overflow Populating the Repository

  16. Example Overflow Searching  Keyword Search based on TF-IDF 

    Each example is represented as a Document with several parts.  Sdoc = [ Wtitle Stitle + Wtag Stag + Wanswer Sanswer + Wquestion Squestion + Wcode Scode ] Smetadata
  17. None
  18. Perliminary Evaluation Data Point Search Query Dynamic Dimension "jquery dynamic

    dimension" Hover "jquery hover div" Position "jquery position" Rounded Corners "jquery rounded corner" Draggable "jquery draggable" Droppable "jquery droppable" Autocomplete "jquery autocomplete from db" Accordion "jquery accordion" Date Picker "jquery datepicker" Image Scale "jquery image scale effect"
  19. Rank of a suitable example at the returned search results

    Data Point Code Repository Tools Google Search Krugle Koders Stack Overflow Example Overflow Dynamic Dimension 4 Not found Not found 1 3 Hover 1 2 1 1 2 Position 3 Not found Not found 4 1 Rounded Corners 2 Not found 3 3 1 Draggable 1 Not found 3 2 1 Dropable 1 Not found 3 1 2 Autocomplete 1 Not found Not Found 1 1 Accordion 1 Not found 12 3 1 Date Picker 1 Not found 3 1 1 Image Scale 2 Not found Not found Not found 3 Avg. Rank 1.7 19.1 9.7778 3.8 1.6
  20. Context Switching Comparison Data Point Code Repository Tools Google Search

    Krugle Koders Stack Overflow Example Overflow Dynamic Dimension 7 - - 1 0 Hover 1 3 1 1 0 Position 5 - - 7 0 Rounded Corners 3 - 5 5 0 Draggable 1 - 5 3 0 Dropable 1 - 5 2 0 Autocomplete 3 - - 1 0 Accordion 1 - 23 6 0 Date Picker 1 - 5 1 0 Image Scale 3 - - - 0 Avg. Mouse Clicks 2.6 3 7.3333 3 0
  21. Small Pilot at ICSE12  The search is keyword dependent

     Sequential Search vs Iterative Search (refine)  System performance matters  Feedback loop  Runnable code
  22. Limitations  EO not only enables Example Embedding EcoSystem but

    also enabled by it.  It is unclear whether using SO can scale well  Preliminary evaluation is limited and a thorough user study should be conducted.
  23. Future Work  User study:  Quantative  Example Overflow

    logs every action per user+session:  Query used  Results viewed  Additional pages loaded  Result chosen  Additional context needed  Qualitative  User observations
  24. Questions ?