Papers vs. Artifacts

Transcript

1. Papers vs. Artifacts
   Philipp Haller
   KTH Royal Institute of Technology
   Sweden
   ECOOP Doctoral Symposium
   Rome, Italy, 17 July 2016
   1
   

   View Slide

2. Haller Papers vs. Artifacts
   Artifacts are Important
   • Enable reproducing research results by independent
   researchers
   • Enable practically evaluating ideas and theories
   • Enable experimentation with variations and
   extensions
   • Not only by the original authors!
   2
   

   View Slide

3. Haller Papers vs. Artifacts
   “Artifacts are a
   hindrance!”
   • High-quality artifacts require a lot of development
   effort
   • Artifacts have no influence on acceptance of a
   research paper at the top PL/SE conferences
   • The main requirement for completing a PhD is
   producing publishable research results
   3
   “How to justify the effort developing artifacts?”
   

   View Slide

4. Haller Papers vs. Artifacts
   Potential Benefits
   • Increase quality of research
   • Increase visibility and impact of research
   • Support discovery of research problems
   4
   

   View Slide

5. Haller Papers vs. Artifacts
   Context: My PhD Project
   • Goal: high-level concurrent programming model that
   • simplifies avoiding data races;
   • (reduces risk of deadlocks;)
   • scales to large number of concurrent processes;
   • guarantees progress in presence of blocking ops;
   • integrates with thread model of mainstream VMs.
   • Main sources of inspiration: Erlang and Actor Model
   5
   

   View Slide

6. Haller Papers vs. Artifacts
   Actors
   • Actor = concurrent “process” that communicates via
   asynchronous messages
   • Asynchronous send, event-based receive
   • Avoiding low-level data races is straight-forward:
   • Exchange only immutable objects
   • Don’t capture mutable state
   • A lot more complex to ensure race safety for efficient
   by-reference message passing of mutable objects
   6
   

   View Slide

7. Haller Papers vs. Artifacts
   Artifacts
   • Here: limit to doctoral school (2005/2006—2010)
   • Participation in Scala team meetings from beginning
   to end of doctoral school (and beyond)
   • 2006: first release of Scala Actors
   • 2006—2010: partest compiler test framework
   • 2008: join patterns for Scala
   • 2009: uniqueness types for Scala (compiler plugin)
   7
   

   View Slide

8. Haller Papers vs. Artifacts
   Noteworthy events…
   8
   “We saw 5x normal tweets-per-second and about 4x
   tweets-per-minute as this chart illustrates. Overall,
   Twitter sailed smoothly through the inauguration [...]”
   Source: https://blog.twitter.com/2009/inauguration-day-twitter
   Other uses at The
   Guardian and others
   

   View Slide

9. Haller Papers vs. Artifacts
   Artifacts Annotated
   9
   • Participation in Scala team meetings from beginning
   to end of doctoral school (and beyond)
   • 2006: first release of Scala Actors
   • 2006—2010: partest compiler test framework
   • 2008: join patterns for Scala
   • 2009: uniqueness types for Scala (compiler plugin)
   Co-author of Scala specification,
   contributions to language, core
   library, and tools
   2008: Twitter starts using Scala
   Actors in production
   2006—2009: lots of
   development/maintenance
   End of 2008: start work on
   new research topic
   Most of 2010/2011 spent on
   writing a book on actors in
   Scala for developers
   

   View Slide

10. Haller Papers vs. Artifacts
    Scala Actors: Timeline
    • Entire year 2006 spent developing Scala Actors
    • 2007:
    • Development/maintenance, OSS community work,
    reading
    • Best student paper award (Coordination ’07)
    • 2008:
    • Part-time development/maintenance, OSS
    community work
    • Offer to write a book
    • 2009: minimal maintenance, talk at JavaOne
    10
    

    View Slide

11. Haller Papers vs. Artifacts
    Outcome
    • Several well-cited papers
    11
    Haller and Odersky. Scala Actors: Unifying thread-based and event-
    based programming. TCS ’09 (Google Scholar: 296 Citations)
    Haller and Odersky. Event-based programming without inversion of
    control. JMLC ’06 (Google Scholar: 119 Citations)
    Haller and Odersky. Capabilities for uniqueness and borrowing.
    ECOOP ’10 (Google Scholar: 69 Citations)
    • Published book (“Actors in Scala”, Jan 2012)
    • Opportunity to develop research further in context of
    Typesafe start-up company (2012–2014)
    • Wide visibility and reuse
    Haller and Van Cutsem. Implementing joins using extensible pattern
    matching. Coordination ’08 (Google Scholar: 45 Citations)
    

    View Slide

12. Haller Papers vs. Artifacts
    Another Artifact
    • Spent 2009 working on uniqueness types in Scala
    • Solely motivated by experience with actors
    • One of the most frequent questions of users:
    “how to ensure isolation of actors?”
    • At the same time developed a complete and detailed
    soundness proof
    • Research had clear priority: artifact (compiler plugin)
    developed “only” for experimental evaluation
    • Result: one user who tried it and gave up :-)
    12
    

    View Slide

13. Haller Papers vs. Artifacts
    Post Mortem
    • Making artifact useful in practice would have
    required a lot more engineering effort
    • Most time spent on type system design and
    soundness proof
    • Implementation technology was not ready
    • But also: experience has served as motivation to
    explore different approaches
    13
    

    View Slide

14. Haller Papers vs. Artifacts
    Increasing Impact
    • Your research produces performance results?
    • Making your artifact easily usable ensures many
    researchers compare to your work
    • (This even helps if your system is slow/limited/..)
    • If usable by practitioners: enables talks at industrial
    conferences
    • Your ideas influence industry: “wider impact”
    • Even better: useful to practitioners (requires
    engineering, documentation, community support)
    • Goal: establish your system as state of the art
    • Cannot be overlooked
    14
    

    View Slide

15. Haller Papers vs. Artifacts
    Conclusion
    Artifacts:
    • Not just a burden
    • Can inform research problems
    • Can dramatically increase impact
    • Can offer career opportunities: research
    collaborations, start-ups, consulting, …
    15
    

    View Slide