The importance of experimentation - Juan M Lavista Ferres

3b085ba94fee217d7656971b0cb4cf00?s=47 PyCon Canada
August 10, 2013

The importance of experimentation - Juan M Lavista Ferres

Being a Data Scientist is not only about learning how to use Hadoop, SciPy, NumPy, etc, but also understanding what conclusions we can drive from the data. One of the most important job of a data scientist is to be able to answer if an idea works or not, and for this , understanding the design and use of randomize control experiments is key

To innovate we need to know what ideas work and what ideas do not work, and In order to do this , we need to establish causality. The Gold standard to prove casualty are randomize control experiment, but this is not enough... we also need to have a culture of experimentation, and in order to have this we first need to understand that when an idea fails is not an error.
Controlled experiments, also called randomized experiments and A/B tests, have had a profound influence on multiple fields, including medicine, agriculture, manufacturing, and advertising. Through randomization and proper design, experiments allow establishing causality scientifically, which is why they are the gold standard in drug tests. In software development, multiple techniques are used to define product requirements; controlled experiments provide a valuable way to assess the impact of new features on customer behavior.(Online Experimentation at Microsoft, Kohavi et al)
Web and the Internet allows a perfect place to run control experiments. We are going to review not only the need of randomize control experiments or A/B Testing , but the design, lessons learned , and the cultural challenges of organizations to become data driven.

3b085ba94fee217d7656971b0cb4cf00?s=128

PyCon Canada

August 10, 2013
Tweet

Transcript

  1. None
  2. None
  3. None
  4. Reference: http://www.exp-platform.com/Pages/Bloodletting.aspx

  5. None
  6. None
  7. None
  8. None
  9. None
  10. None
  11. Source: http://rosspw.com/obama-landing-page-design/

  12. Source: http://rosspw.com/obama-landing-page-design/

  13. Source: http://exp-platform.com

  14. Source: http://exp-platform.com

  15. Source: http://exp-platform.com

  16. Source: http://exp-platform.com

  17. None
  18. None
  19. None
  20. None
  21. Source:http://www.theaveragebody.com/average_hand_size.php

  22. None
  23. None
  24. None
  25. None
  26. Source: http://sciencenetlinks.com/student-teacher-sheets/nearsightedness-children/ Ohio State University Research News, March 9, 2000.

    Night lights don't lead to nearsightedness, study suggests 1999 – University of Pennsylvania Medical Center, Published in may Issue of Nature Later study at Ohio State University
  27. Source:http://www.nbcnews.com/id/39038581/ns/technology_and_science- back_to_school/t/facebook-use-can-lower-grades-percent-study- says/#.UgWwb21GY3w

  28. None
  29. None
  30. None
  31. None
  32. None
  33. None
  34. None
  35. None
  36. Source: http://glinden.blogspot.com/2006/04/early-amazon-shopping-cart.html it might distract people away from checking out

  37. None
  38. None
  39. None
  40. None
  41. None
  42. None
  43. None
  44. Email:jlavista@microsoft.com twitter:#bdatascientist