Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Hitchhikers need free vehicles! Shared repositories for statistical analysis in SBST

Hitchhikers need free vehicles! Shared repositories for statistical analysis in SBST

Interested in learning more about this topic? Visit this web site to read the paper: https://www.gregorykapfhammer.com/research/papers/Kapfhammer2016/

4ae30d49c8cc07e42d5a871efb9bcfba?s=128

Gregory Kapfhammer

May 16, 2016
Tweet

Transcript

  1. Hitchhikers Need Free Vehicles! Gregory M. Kapfhammer Phil McMinn and

    Chris J. Wright shared by under a ickr photo BergsPix Creative Commons ( BY-ND ) license
  2. Randomization

  3. Randomization Inherent in SBST techniques

  4. Randomization Inherent in SBST techniques Necessitates careful experiment design

  5. Randomization Inherent in SBST techniques Necessitates careful experiment design Statistical

    analysis of results required!
  6. When I say "statistics" ...

  7. None
  8. None
  9. But, we need statistical analysis!

  10. None
  11. Statistical Analysis shared by creative commons licensed ( BY-NC )

    ickr photo Michael Kappel
  12. Statistical Analysis A Hitchhiker's Guide to Statistical Tests for Assessing

    Randomized Algorithms in Software Engineering shared by creative commons licensed ( BY-NC ) ickr photo Michael Kappel
  13. Statistical Analysis A Hitchhiker's Guide to Statistical Tests for Assessing

    Randomized Algorithms in Software Engineering Arcuri and Briand recommend statistical techniques shared by creative commons licensed ( BY-NC ) ickr photo Michael Kappel
  14. Statistical Analysis A Hitchhiker's Guide to Statistical Tests for Assessing

    Randomized Algorithms in Software Engineering Arcuri and Briand recommend statistical techniques Code snippets provided in the R language shared by creative commons licensed ( BY-NC ) ickr photo Michael Kappel
  15. Statistical Analysis A Hitchhiker's Guide to Statistical Tests for Assessing

    Randomized Algorithms in Software Engineering Arcuri and Briand recommend statistical techniques Code snippets provided in the R language A tremendous asset to the SBST community! shared by creative commons licensed ( BY-NC ) ickr photo Michael Kappel
  16. Subtleties of Statistical Analysis shared by creative commons licensed (

    BY-NC ) ickr photo Michael Kappel
  17. Subtleties of Statistical Analysis Well-meaning researchers may make small mistakes

    shared by creative commons licensed ( BY-NC ) ickr photo Michael Kappel
  18. Subtleties of Statistical Analysis Well-meaning researchers may make small mistakes

    Marco Torchiano revealed paradoxical e ect sizes shared by creative commons licensed ( BY-NC ) ickr photo Michael Kappel
  19. Hitchhikers Need Vehicles

  20. Hitchhikers Need Vehicles Shared repositories of statistical code

  21. Hitchhikers Need Vehicles Shared repositories of statistical code Well-tested implementations

    of procedures
  22. Hitchhikers Need Vehicles Shared repositories of statistical code Well-tested implementations

    of procedures Additional documentation and guidelines
  23. Hitchhikers Need Vehicles Shared repositories of statistical code Well-tested implementations

    of procedures Additional documentation and guidelines Replication packages for completed analyses
  24. Why is This Important?

  25. Enhance the Maturity of the SBST Field

  26. Supporting Tools and Platforms?

  27. None
  28. Suggestions shared by creative commons licensed ( BY-NC-ND ) ickr

    photo sunface13
  29. Suggestions Use GitHub to store data and analysis code shared

    by creative commons licensed ( BY-NC-ND ) ickr photo sunface13
  30. Suggestions Use GitHub to store data and analysis code Create

    R packages using devtools shared by creative commons licensed ( BY-NC-ND ) ickr photo sunface13
  31. Suggestions Use GitHub to store data and analysis code Create

    R packages using devtools Reveal your full analysis with RMarkdown shared by creative commons licensed ( BY-NC-ND ) ickr photo sunface13
  32. Suggestions Use GitHub to store data and analysis code Create

    R packages using devtools Reveal your full analysis with RMarkdown Use "best of breed" tools to support your work! shared by creative commons licensed ( BY-NC-ND ) ickr photo sunface13
  33. Carefully pick your analysis team ...

  34. None
  35. "Hadleyverse" shared by Publicly available photo Hadley Wickham

  36. "Hadleyverse" dplyr for fast data manipulation shared by Publicly available

    photo Hadley Wickham
  37. "Hadleyverse" dplyr for fast data manipulation tidyr for disciplined data

    restructuring shared by Publicly available photo Hadley Wickham
  38. "Hadleyverse" dplyr for fast data manipulation tidyr for disciplined data

    restructuring ggplot2 for impressive data visualization shared by Publicly available photo Hadley Wickham
  39. "Hadleyverse" dplyr for fast data manipulation tidyr for disciplined data

    restructuring ggplot2 for impressive data visualization Or, use the languages and packages you prefer shared by Publicly available photo Hadley Wickham
  40. "Hadleyverse" dplyr for fast data manipulation tidyr for disciplined data

    restructuring ggplot2 for impressive data visualization Or, use the languages and packages you prefer But, seriously, Hadley Wickham's code is awesome! shared by Publicly available photo Hadley Wickham
  41. Where do we go next?

  42. None
  43. Let's Talk shared by under a Stocksnap.io photo Alejandro Escamilla

    Public Domain license
  44. Let's Talk What statistical analysis do you regularly perform? shared

    by under a Stocksnap.io photo Alejandro Escamilla Public Domain license
  45. Let's Talk What statistical analysis do you regularly perform? What

    is needed to move the SBST community forward? shared by under a Stocksnap.io photo Alejandro Escamilla Public Domain license
  46. Let's Talk What statistical analysis do you regularly perform? What

    is needed to move the SBST community forward? What types of vehicles do hitchhikers really need? shared by under a Stocksnap.io photo Alejandro Escamilla Public Domain license
  47. Questions shared by under a Stocksnap.io photo Alejandro Escamilla Public

    Domain license
  48. Questions Sharing data sets larger than what GitHub supports? shared

    by under a Stocksnap.io photo Alejandro Escamilla Public Domain license
  49. Questions Sharing data sets larger than what GitHub supports? Use

    Git Large File Storage (LFS) shared by under a Stocksnap.io photo Alejandro Escamilla Public Domain license
  50. Questions Sharing data sets larger than what GitHub supports? Use

    Git Large File Storage (LFS) Why don't we release scripts for running experiments? shared by under a Stocksnap.io photo Alejandro Escamilla Public Domain license
  51. Questions Sharing data sets larger than what GitHub supports? Use

    Git Large File Storage (LFS) Why don't we release scripts for running experiments? They are often customized. But, yes, we should! shared by under a Stocksnap.io photo Alejandro Escamilla Public Domain license