Papers vs. Artifacts

Transcript

1. Papers vs. Artifacts Philipp Haller KTH Royal Institute of Technology Sweden SPLASH Doctoral Symposium Pittsburgh, PA, USA, October 2015 1
2. Haller Papers vs. Artifacts Artifacts are Important • Enable reproducing research results by independent researchers • Enable practically evaluating ideas and theories • Enable experimentation with variations and extensions • Not only by the original authors! 2
3. Haller Papers vs. Artifacts “Artifacts are a hindrance!” • High-quality artifacts require a lot of development effort • Artifacts have no influence on acceptance of a research paper at the top PL/SE conferences • The main requirement for completing a PhD is producing publishable research results 3 “How to justify the effort developing artifacts?”
4. Haller Papers vs. Artifacts Potential Benefits • Increase quality of research • Increase visibility and impact of research • Support discovery of research problems 4
5. Haller Papers vs. Artifacts Context: My PhD Project • Goal: high-level concurrent programming model that • simplifies avoiding data races; • (reduces risk of deadlocks;) • scales to large number of concurrent processes; • guarantees progress in presence of blocking ops; • integrates with thread model of mainstream VMs. • Main sources of inspiration: Erlang and Actor Model 5
6. Haller Papers vs. Artifacts Actors • Actor = concurrent “process” that communicates via asynchronous messages • Asynchronous send, event-based receive • Avoiding low-level data races is straight-forward: • Exchange only immutable objects • Don’t capture mutable state • A lot more complex to ensure race safety for efficient by-reference message passing of mutable objects 6
7. Haller Papers vs. Artifacts Artifacts • Here: limit to doctoral school (2005/2006—2010) • Participation in Scala team meetings from beginning to end of doctoral school (and beyond) • 2006: first release of Scala Actors • 2006—2010: partest compiler test framework • 2008: join patterns for Scala • 2009: uniqueness types for Scala (compiler plugin) 7
8. Haller Papers vs. Artifacts Noteworthy events… 8 “We saw 5x normal tweets-per-second and about 4x tweets-per-minute as this chart illustrates. Overall, Twitter sailed smoothly through the inauguration [...]” Source: https://blog.twitter.com/2009/inauguration-day-twitter
9. Haller Papers vs. Artifacts Artifacts Annotated 9 • Participation in Scala team meetings from beginning to end of doctoral school (and beyond) • 2006: first release of Scala Actors • 2006—2010: partest compiler test framework • 2008: join patterns for Scala • 2009: uniqueness types for Scala (compiler plugin) Co-author of Scala specification, contributions to language, core library, and tools 2008: Twitter starts using Scala Actors in production 2006—2009: lots of development/maintenance End of 2008: start work on new research topic Most of 2010 spent on writing a book on actors in Scala for developers
10. Haller Papers vs. Artifacts Scala Actors: Timeline • Entire year 2006 spent developing Scala Actors • Every day: reading and programming • 2007: • Every day: programming, OSS community work, reading • Best student paper award • 2008: • Part-time development/maintenance • Offer to write a book • 2009: no development, minimal maintenance 10
11. Haller Papers vs. Artifacts Outcome • Several well-cited papers 11 Haller and Odersky. Scala Actors: Unifying thread-based and event- based programming. TCS ’09 (Google Scholar: 265 Citations) Haller and Odersky. Event-based programming without inversion of control. JMLC ’06 (Google Scholar: 108 Citations) Haller and Odersky. Capabilities for Uniqueness and Borrowing. ECOOP ’10 (Google Scholar: 61 Citations) • Published book (“Actors in Scala”, Jan 2012) • Opportunity to develop research further in context of Typesafe start-up company (2012-2014) • Opportunities in academic service, …
12. Haller Papers vs. Artifacts Another Artifact • Spent 2009 working on uniqueness types in Scala • Solely motivated by experience with actors • One of the most frequent questions we received: “how to ensure isolation of actors?” • At the same time developed a complete and detailed soundness proof • Research had clear priority: artifact (compiler plugin) developed for experimental evaluation • Result: one user who tried it and gave up :-) 12
13. Haller Papers vs. Artifacts Post Mortem • Making artifact useful in practice would have required a lot more engineering effort • Most time spent on type system design and soundness proof • Implementation technology was not ready • Motivation to explore different approaches 13
14. Haller Papers vs. Artifacts Increasing Impact • Your research produces performance results? • Making your artifact easily usable ensures many researchers compare to your work • (This even helps if your system is slow/limited/..) • If usable by practitioners can lead to talks at industrial conferences • Your ideas influence industry: “wider impact” • Even better: useful to practitioners (requires engineering, documentation, community support) • Goal: establish your system as state of the art • Cannot be overlooked 14
15. Haller Papers vs. Artifacts Conclusion Artifacts: • Not just a burden • Can inform research problems • Can dramatically increase impact • Can offer career opportunities: research collaborations, academic service, start-ups, consulting, … 15