Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Once Upon a Data: Crunching Chaotic Numbers into True Stories

62321e5935c9c0731462b8178a7423f8?s=47 OmaymaS
November 03, 2018

Once Upon a Data: Crunching Chaotic Numbers into True Stories

Slides of a workshop given to non-data practitioners about dealing with data and understanding common traps.

62321e5935c9c0731462b8178a7423f8?s=128

OmaymaS

November 03, 2018
Tweet

Transcript

  1. Crunching Chaotic Numbers into Compelling Stories TRUE ONCE UPON A

    D T OMAYMA SAID
  2. None
  3. None
  4. People & Questions A COMPLEX Relationship ? ?

  5. Fireflies or Balls of Gases? Ever wondered what those sparkly

    dots are up there? Scene Link
  6. The Cult of Certainty The Beginner’s Mind “I don’t wonder

    I KNOW” “Have you ever WONDERED?”
  7. Go to School Grow Up Humans Default Setup: INQUIRY MODE

  8. Go to School Grow Up Humans Default Setup: INQUIRY MODE

    We go to SCHOOL We GROW UP BUT
  9. I position myself relentlessly as an IDIOT at IDEO Paul

    Bennett Chief Creative Officer “ ”
  10. Business Question Business Decision Taking Decisions The Classical Way

  11. Business Question Data Question Data Answer Business Decision Taking Decisions

    Data in the loop
  12. As I talk with companies about digital transformation, by far

    THE BIGGEST CHALLENGES they face are CULTURAL, not technical “ ” Doug Cutting Chief Architect
  13. Data IN Insights/ Solution OUT Machine Learning

  14. NOT in Real Life! Data IN Insights/ Solution OUT Machine

    Learning
  15. Algorithms

  16. It takes a BIG man to admit his data is

    small “ ” Joe Cheng CTO Big Data
  17. “What is the PROBLEM?”

  18. Data Traps and Trickery

  19. What Do Movie Ratings Hide?

  20. What Do Movie Ratings Hide?

  21. What Do Movie Ratings Hide?

  22. What Do Movie Ratings Hide?

  23. What Do Movie Ratings Hide?

  24. What Do Movie Ratings Hide?

  25. What Do Movie Ratings Hide?

  26. What Do Movie Ratings Hide?

  27. None
  28. 1 Single Measure/Score Data Traps and Trickery

  29. Do Storks Deliver Babies?

  30. Do Storks Deliver Babies? Correlation +ve -ve

  31. Do Storks Deliver Babies? Correlation

  32. Do Storks Deliver Babies? Recreated from data and code by

    Evelina Gabasova
  33. Do Storks Deliver Babies? Statistically Significant Recreated from data and

    code by Evelina Gabasova Correlation Coefficient = 0.62 P-value = 0.0079
  34. Do Storks Deliver Babies? STORKS BABIES

  35. Do Storks Deliver Babies? STORKS BABIES “Let’s INCREASE the storks”

  36. Do Storks Deliver Babies? STORKS BABIES Confounder?

  37. Do Storks Deliver Babies? Country Area? Recreated from data and

    code by Evelina Gabasova
  38. Do Storks Deliver Babies? Population? Recreated from data and code

    by Evelina Gabasova
  39. Correlation Between X & Y 0.816 Anscombe's Quartet

  40. ” Never trust summary statistics alone; always visualize your data

    “ Alberto Cairo The Functional Art Source:: The Functional Art Blog
  41. 2 Shallow Correlations Data Traps and Trickery

  42. Was There a Gender Bias In Graduate Admission at ?

  43. Was There a Gender Bias in Graduate Admission at Berkeley?

    was sued for gender bias 1973 vudlab.com/simpsons
  44. Was There a Gender Bias in Graduate Admission at Berkeley?

    vudlab.com/simpsons What about segmenting by DEPARTMENT?
  45. Recreated based on Rafael Irizaary educational gifs Reversing Trend

  46. 3 Simpson’s Paradox Data Traps and Trickery

  47. How Wealthy Would You Be ON AVERAGE ? What if

    Zuckerberg Walked into This Room? gadgetmatch.com
  48. How Wealthy Would You Be ON AVERAGE ? What if

    Zuckerberg Walked into This Room? gadgetmatch.com 100 $ 1000 $ 70 000 000 000 $ Median 600 $ Mean ~ 6 Billion $
  49. None
  50. None
  51. None
  52. None
  53. 4 Misuse of The Average Data Traps and Trickery

  54. None
  55. 5 Invalid Extrapolation Data Traps and Trickery

  56. Data Traps and Trickery 1- Single Measure/Score 2- Shallow Correlations

    3- Simpson's’ Paradox 4- Misuse of the Average 5- Invalid Extrapolation And more…
  57. Ask better questions Dealing with the complexity of real-life problems

  58. Test more and be skeptical Dealing with the complexity of

    real-life problems
  59. Have humility about our work Dealing with the complexity of

    real-life problems
  60. Avoid falling in love with our results Dealing with the

    complexity of real-life problems
  61. Consider the ethical implications Dealing with the complexity of real-life

    problems
  62. Crunching Chaotic Numbers into Compelling Stories TRUE ONCE UPON A

    D T OMAYMA SAID