Example Overflow: Using Social Media for Code Recommendation

Example Overflow: Using Social Media for Code Recommendation

A talk I've given as part of the IBM Programming Languages and Software Engineering (PLSE) Seminar in 2012

419a48edd42158cd11d802f06b448e83?s=128

Alexey Zagalsky

November 12, 2012
Tweet

Transcript

  1. 7.

    Stack Overflow Stats (10.11.2012)  Total Questions: 3.9M  Total

    Answers: 7.6M  Comments: 16.9M  Questions per Min: 2.96  Answers per Min: 4.29  Views per Day: 2M
  2. 8.

    Related Work  Over 92% of the questions on StackOverflow

    are answered, and for those 92% the median answer time is 11 minutes. Design lessons from the fastest q&a site in the west L.Mamykina, B.Manoim, M.Mittal, G.Hripcsak, B.Hartmann (2011)  Linking StackOverflow data with API usage Crowd Documentation: Exploring the Coverage and the Dynamics of API Discussions on Stack Overflow Chris Parnin, Cristoph Teude, Lars Grammel and Margaret-Anne Storey (2012)
  3. 11.

    Research Question  Q1: Is it possible to use SO

    for code recommendations ?  Q2: Can we improve example usage by supporting actions (searching, comparing)?
  4. 13.

    Design Objectives  Comparing multiple examples  Display several code

    example at once  Display only the code snippet itself  Minimal context switching  If required show additional context at the same place  No need to go “forward” or “backwards”
  5. 14.
  6. 16.

    Example Overflow Searching  Keyword Search based on TF-IDF 

    Each example is represented as a Document with several parts.  Sdoc = [ Wtitle Stitle + Wtag Stag + Wanswer Sanswer + Wquestion Squestion + Wcode Scode ] Smetadata
  7. 17.
  8. 18.

    Perliminary Evaluation Data Point Search Query Dynamic Dimension "jquery dynamic

    dimension" Hover "jquery hover div" Position "jquery position" Rounded Corners "jquery rounded corner" Draggable "jquery draggable" Droppable "jquery droppable" Autocomplete "jquery autocomplete from db" Accordion "jquery accordion" Date Picker "jquery datepicker" Image Scale "jquery image scale effect"
  9. 19.

    Rank of a suitable example at the returned search results

    Data Point Code Repository Tools Google Search Krugle Koders Stack Overflow Example Overflow Dynamic Dimension 4 Not found Not found 1 3 Hover 1 2 1 1 2 Position 3 Not found Not found 4 1 Rounded Corners 2 Not found 3 3 1 Draggable 1 Not found 3 2 1 Dropable 1 Not found 3 1 2 Autocomplete 1 Not found Not Found 1 1 Accordion 1 Not found 12 3 1 Date Picker 1 Not found 3 1 1 Image Scale 2 Not found Not found Not found 3 Avg. Rank 1.7 19.1 9.7778 3.8 1.6
  10. 20.

    Context Switching Comparison Data Point Code Repository Tools Google Search

    Krugle Koders Stack Overflow Example Overflow Dynamic Dimension 7 - - 1 0 Hover 1 3 1 1 0 Position 5 - - 7 0 Rounded Corners 3 - 5 5 0 Draggable 1 - 5 3 0 Dropable 1 - 5 2 0 Autocomplete 3 - - 1 0 Accordion 1 - 23 6 0 Date Picker 1 - 5 1 0 Image Scale 3 - - - 0 Avg. Mouse Clicks 2.6 3 7.3333 3 0
  11. 21.

    Small Pilot at ICSE12  The search is keyword dependent

     Sequential Search vs Iterative Search (refine)  System performance matters  Feedback loop  Runnable code
  12. 22.

    Limitations  EO not only enables Example Embedding EcoSystem but

    also enabled by it.  It is unclear whether using SO can scale well  Preliminary evaluation is limited and a thorough user study should be conducted.
  13. 23.

    Future Work  User study:  Quantative  Example Overflow

    logs every action per user+session:  Query used  Results viewed  Additional pages loaded  Result chosen  Additional context needed  Qualitative  User observations