The Vesperin System

A Source code curation system

Transcript

1. SCC - SRI - 09 18, 2015 1 System for curating Java code examples on StackOverflow The Vesperin System Multistage (JSON) Source JSON Pack Text capacity Text Pack Kiwi Violette ( ( AST Multi-stage 1 : swap 2 : partition 4 : quicksort Code stages 3 : randomizedPartition import java.util.Random; public class Quicksort { private static Random rand = new Random(); public static void quicksort(int[] arr, int left, int right) { if (left < right) { int pivot = randomizedPartition(arr, left, right); quicksort(arr, left, pivot); quicksort(arr, pivot + 1, right); } } private static int randomizedPartition(int[] arr, int left, int right) { int swapIndex = left + rand.nextInt(right - left) + 1; swap(arr, left, swapIndex); return partition(arr, left, right); } private static int partition(int[] arr, int left, int right) { int pivot = arr[left]; int i = left - 1; int j = right + 1; while (true) { error path (0:warning, ...., n:warning) ok path Multistaging to Understand: Distilling code examples essence Code Examples Multistager (Sanchez et al., 2015) paper under review By Huascar Sanchez
2. SCC - SRI - 09 18, 2015 Research idea: Allow programmers to experiment with code modification ideas in the Web page of the Q&A system (in-place). Hypothesis: Intuitively experimenting with code modifications ideas hands-on can (1) help programmers deal with code with questionable quality upfront and (2) facilitate code understanding. 2 Vesperin
7. SCC - SRI - 09 18, 2015 7 Source object Source ID PK Description Text Content Text Notes Array curation request updated code (1) (2) browser plug-in RESTful service scratch space Vesperin page
8. SCC - SRI - 09 18, 2015 8 (1) Q&A page scratch space • The space where all in-place code modifications are made, via direct editing or via semi-automated code transformations. Vesperin page • reDOMed Q&A page drafts management • Drafts are snapshots of changed code for future recoveries. • Add error tolerance into curation process (Olsen, 2009) Vesperin actions • Make curation requests • Add notes in context • Check code syntax • Mark drafts notes (in context)
9. SCC - SRI - 09 18, 2015 9 mongo db (2) Curation request {“rename”: { “what”: “method”, “where”: [1, 6], “source”: { “name”: “..”, “content”: JSON requests A P I Reply {“draft”: { “before”: {}, “after”: { “name”: “..”, “content”: JSON replies twitter HTML page RESTful Incremental Java parser Publisher & Renderer Java code transformer: • Codepacking • Delete code member, • Rename code member, • Code cleanup, • Clip fragment, • Create new method
10. SCC- SRI - 09 18, 2015 10 Vesperin in the lab
11. SCC - SRI - 09 18, 2015 Considered the following research questions: • How will programmers use Vesperin? • Were the provided facilities sufficient? • Will programmers be able to better understand unfamiliar code examples via curation? Will Vesperin add value? 11 User Study
12. • 15 Participants, 3 tasks, 60 minutes • One group pretest posttest design • Participants are studied before and after the experimental manipulation (Babbie, 2015) • Variables • Independent variable: Vesperin system • Dependent variables: perception and experience SCC - SRI - 09 18, 2015 12 Study setup (Babbie, 2015)
13. Title - CONFYYY - MM DD, YYYY Results: 13 Participants Background experience. 40% of them visit StackOverflow multiple times a day. Moreover, nearly 70% of the participants were extremely familiar with Java and Refactoring. (a) Programming Experience. (b) StackOverflow Visit Frequency. (c) Level of Java Familiarity. (d) Level of Refactoring Familiarity. Figure 4.4: Summary of participants’ background information.
14. Title - CONFYYY - MM DD, YYYY Task 1 14 Task 1 Using client/server certificates for two way authentication SSL socket on Android
15. Title - CONFYYY - MM DD, YYYY Task 1 15 Task 2 How to add a push notification in my own android app
16. Title - CONFYYY - MM DD, YYYY Task 1 16 Task 3 Stop the Twitter stream and return List of status with twitter4j
17. Title - CONFYYY - MM DD, YYYY Procedure and manipulation 17 Procedure and manipulation (Babbie, 2015) Give Vesperin demo Measurement of observation e.g., Could such a system allow you to better understand code examples? Pretest Measurement of observation e.g., Did Vesperin allow you to better understand code examples? Posttest Application of Independent variable Use Vesperin Intervention final interview
18. SCC- SRI - 09 18, 2015 18 User experiences Obtained via 4 sources: observations, automated user interaction logging, pretest & posttest, and final interview
19. SCC - SRI - 09 18, 2015 Used in a hybrid comprehension strategy Mixed bottom-up and top-down strategies Used to explore control flow relationships Search and replaced; followed by annotation and cleanup Syntax checking often influenced curation. 19 How was Vesperin used?
20. SCC - SRI - 09 18, 2015 Editing activity surged early on, then subsided over time. This can be explained by looking at the assumptions of dual-process theories (Chaiken and Eagly, 1989) 20 How was Vesperin used? Edits 0 10 20 30 40 50 Minutes 2 4 6 8 10 12 14 16 18 20
21. SCC - SRI - 09 18, 2015 Its facilities were necessary, but not sufficient Unable to handle specific code examples: • Multiple orthogonal classes on a single scratch space • Multiple scratch spaces needed to work in concert Workarounds were used to address limitations • Used static nested classes • Combined content of all scratch spaces into single scratch space Limited aid for identifying code examples’ core parts 21 Were the set of facilities sufficient?
22. SCC - SRI - 09 18, 2015 Came in with high expectations, and left satisfied. (added value and better understanding as predicted) 22 How useful is Vesperin? Horizontal axes: 5-point Likert scale, ranging from strongly disagree (“- -”) to strongly agree (“++”). Vertical axes: number of participants. (a) Better Understanding participants 0 2 4 6 8 10 12 PRETEST POSTEST -- - O + ++ -- - O + ++ (b) Added Value participants 0 1 2 3 4 5 6 7 8 PRETEST POSTTEST -- - O + ++ -- - O + ++
23. SCC - SRI - 09 18, 2015 Used in a hybrid comprehension strategy Helped better understand source code: “It’s much easier to understand the code after its curation.” Its facilities were necessary, but not sufficient Limited aid for identifying prime sets of behavior 23 Overview of Results
24. SCC- SRI - 09 18, 2015 24 Epilogue
25. SCC - SRI - 09 18, 2015 Introduced a new paradigm (and its tools) for addressing code foraging’s challenges: The Vesperin System Our results confirmed thesis statement Addressed questionable quality of online code. Facilitated quick and accurate understanding of online code. We only scratched the surface … 25 Source code curation & tools
26. SCC - SRI - 09 18, 2015 Craft solutions by remixing curated code: Semi-automated resolution of delocalized code. Identification of the best chain of code stages to reuse Crowdsourcing program synthesis 26 Looking ahead
27. SCC - SRI - 09 18, 2015 27 Thank you Huascar Sanchez hsanchez@cs.ucsc.edu