Science-based Development

Science-based Development

Development, like science, is a messy endeavor. It’s near impossible to control all the variables. Testing code is a noble goal, but it’s very easy to test the wrong thing. Many experienced scientists are fooled by their senses and biases. To account for that they crafted and refined a Scientific Method. That method has been stress tested by centuries of experimentation, discovery and peer-review. It probably wouldn’t hurt to try to apply it to development and see what it can offer.

871fc089cd3cbbc18b9f868c8dd85cce?s=128

Olivier Lacan

January 17, 2013
Tweet

Transcript

  1. The Method Science-based Development

  2. process.

  3. Ain’t nobody got time for dat!

  4. You’re... doing it wrong?

  5. Science can help!

  6. It can make you be er.

  7. I dare ya! prove me wrong.

  8. None
  9. more hair hair not where I live photo non contractuelle

    @olivierlacan
  10. None
  11. None
  12. Things Science cares about: - me: nope - you: nope

    - truth: yep
  13. Science Use Do Things To

  14. Do Things With Science

  15. is a method Science

  16. is ok with wrong Science

  17. There was an elderly professor who had been passionately keen

    on a particular theory for a number of years. One day an American researcher came and u erly disproved our old man's hypothesis. The old man strode to the front, shook his hand and said, “My dear fellow, I wish to thank you, I have been wrong these fi een years”. And we all clapped our hands raw. That was the scientific ideal, of somebody who had a lot invested, a lifetime almost invested in a theory, and he was rejoicing that he had been shown wrong and that scientific truth had been advanced. Richard Dawkins in The Root of All Evil? Story Time
  18. doesn’t trust us Science

  19. liar

  20. pushed us forward Science

  21. None
  22. Scientific Method The

  23. 1 Inquiry

  24. curiosity it totally didn’t kill the cat, you guys!

  25. None
  26. None
  27. “I have no special talents. I am only passionately curious.”

    Albert Einstein
  28. seek knowledge

  29. find problems

  30. distrust authority

  31. Kary Mullis, biochemist

  32. “When I got the idea one night that I could

    amplify DNA (...), and that I could make lots of copies of some little piece of DNA, the thinking for that was about 20 minutes. (...) I did talk to people about it but if I’d listened to what I heard from all my friends who were molecular biologists I would have abandoned it.”
  33. too young don’t let ideas die they might be crazy

    enough to work
  34. 2 Hypothesis

  35. beware of assumptions

  36. this is what our users want we need this feature

    right now without a Like button, we’re nothing
  37. ask a question

  38. ask the right question

  39. it can be broad beware

  40. it can be specific beware too

  41. it must be testable

  42. it must be falsifiable able to be proven wrong

  43. it must predict “if Y, then X should happen”

  44. using AJAX here should improve our sign up rate by

    at least 10% falsifiable statement
  45. - down 1% sign ups: - up 0% - up

    5% - up 10% - up 20% hypothesis falsified hypothesis supported
  46. 3 Experiment

  47. reality where ideas come to die

  48. protocol experimental

  49. isolate variables

  50. enforce controls what was the real cause?

  51. check the null hypothesis try to prove yourself wrong

  52. check your bias at the door it’s easy to fool

    yourself
  53. participant doesn’t know experimenter knows single-blind

  54. double-blind participant doesn’t know experimenter doesn’t know

  55. triple-blind participant doesn’t know experimenter doesn’t know result analysts don’t

    know
  56. who’s who? you code participant experimenter user

  57. A/B Testing is OK. provided you don’t forget about: C

    D E F G H I J K L M N O P Q R S T U V W X Y Z
  58. Actual Testing is OK too.

  59. describe User do let(:user) { Factory.build(:user) } subject { user

    } it { should have_many(:achievements) } it { should have_many(:courses) } it { should have_many(:orders) } it { should have_many(:prizes) } it { should have_many(:videos) } end predictions
  60. context '#complete_course!' do it "increases count of completed courses" do

    expect { user.complete_course!(course) }.to change { user.courses.completed.count }.by(1) end end
  61. Rinse. Repeat. Forever. (continuous integration)

  62. 4 Analysis

  63. What happened? can other questions be asked?

  64. Failure is OK. improve the hypothesis or start from scratch

  65. Reproduction was the result isolated?

  66. Publication share your results and your tools!

  67. Science Inject some in your DEVELOPMENT

  68. None
  69. you’ll pay closer attention to worst case scenario: THINKING &

    ANALYZING DEVELOPING OBSERVING
  70. bye now.

  71. @olivierlacan feedback, rebuttals, experiences, etc.