Slide 1

Slide 1 text

#adchi / #ExperimentationMindset / @DocOnDev The Experimentation Mindset

Slide 2

Slide 2 text

#adchi / #ExperimentationMindset / @DocOnDev Doc Norton, CEO [email protected] @DocOnDev @adchi / #ExperimentationMindset / @DocOnDev

Slide 3

Slide 3 text

#adchi / #ExperimentationMindset / @DocOnDev Mindset

Slide 4

Slide 4 text

#adchi / #ExperimentationMindset / @DocOnDev A simple experiment

Slide 5

Slide 5 text

#adchi / #ExperimentationMindset / @DocOnDev Two Groups

Slide 6

Slide 6 text

#adchi / #ExperimentationMindset / @DocOnDev Small Difference; Big Result

Slide 7

Slide 7 text

#adchi / #ExperimentationMindset / @DocOnDev Mindset the ideas and attitudes with which a person approaches a situation

Slide 8

Slide 8 text

#adchi / #ExperimentationMindset / @DocOnDev fixed growth

Slide 9

Slide 9 text

#adchi / #ExperimentationMindset / @DocOnDev Dreyfus Model of skills acquisition

Slide 10

Slide 10 text

#adchi / #ExperimentationMindset / @DocOnDev Novice

Slide 11

Slide 11 text

#adchi / #ExperimentationMindset / @DocOnDev Advanced Beginner

Slide 12

Slide 12 text

#adchi / #ExperimentationMindset / @DocOnDev Competent

Slide 13

Slide 13 text

#adchi / #ExperimentationMindset / @DocOnDev Proficient

Slide 14

Slide 14 text

#adchi / #ExperimentationMindset / @DocOnDev Expert

Slide 15

Slide 15 text

#adchi / #ExperimentationMindset / @DocOnDev Where are you?

Slide 16

Slide 16 text

#adchi / #ExperimentationMindset / @DocOnDev Most think they’re Expert

Slide 17

Slide 17 text

#adchi / #ExperimentationMindset / @DocOnDev most stay at Proficient

Slide 18

Slide 18 text

#adchi / #ExperimentationMindset / @DocOnDev From Implementation

Slide 19

Slide 19 text

#adchi / #ExperimentationMindset / @DocOnDev Implementation Mindset get it right A+

Slide 20

Slide 20 text

#adchi / #ExperimentationMindset / @DocOnDev to experimentation

Slide 21

Slide 21 text

#adchi / #ExperimentationMindset / @DocOnDev experimentation Mindset explore different ways

Slide 22

Slide 22 text

#adchi / #ExperimentationMindset / @DocOnDev experimentation Mindset explore different ways

Slide 23

Slide 23 text

#adchi / #ExperimentationMindset / @DocOnDev but in a typical company…

Slide 24

Slide 24 text

#adchi / #ExperimentationMindset / @DocOnDev telephone sales

Slide 25

Slide 25 text

#adchi / #ExperimentationMindset / @DocOnDev Follow the script This will be great!

Slide 26

Slide 26 text

#adchi / #ExperimentationMindset / @DocOnDev It wasn’t Great Ptolemy Janet Raphael Susan Kathy Tommy Doc Chris P. Weekly Sales Rankings

Slide 27

Slide 27 text

#adchi / #ExperimentationMindset / @DocOnDev Watch The Master my name is ____________________ … Hello (sir/madam),

Slide 28

Slide 28 text

#adchi / #ExperimentationMindset / @DocOnDev Watch The Master my name is ____________________ … Pat Michaels Good Day!

Slide 29

Slide 29 text

#adchi / #ExperimentationMindset / @DocOnDev Experiment

Slide 30

Slide 30 text

#adchi / #ExperimentationMindset / @DocOnDev experiment

Slide 31

Slide 31 text

#adchi / #ExperimentationMindset / @DocOnDev experiment

Slide 32

Slide 32 text

#adchi / #ExperimentationMindset / @DocOnDev experiment

Slide 33

Slide 33 text

#adchi / #ExperimentationMindset / @DocOnDev top in sales! Doc Ptolemy Janet Kathy Tommy Raphael Susan Andy Weekly Sales Rankings

Slide 34

Slide 34 text

#adchi / #ExperimentationMindset / @DocOnDev 8 week review

Slide 35

Slide 35 text

#adchi / #ExperimentationMindset / @DocOnDev not the feedback I expected

Slide 36

Slide 36 text

#adchi / #ExperimentationMindset / @DocOnDev My last day

Slide 37

Slide 37 text

#adchi / #ExperimentationMindset / @DocOnDev Best Practices

Slide 38

Slide 38 text

#adchi / #ExperimentationMindset / @DocOnDev Implementation Mindset get it right A+

Slide 39

Slide 39 text

#adchi / #ExperimentationMindset / @DocOnDev Single Loop Assumptions Actions Outcomes Get it Right

Slide 40

Slide 40 text

#adchi / #ExperimentationMindset / @DocOnDev The Dark Side

Slide 41

Slide 41 text

#adchi / #ExperimentationMindset / @DocOnDev double Loop Assumptions Actions Outcomes Get it Right Explore Different Ways

Slide 42

Slide 42 text

#adchi / #ExperimentationMindset / @DocOnDev best practices

Slide 43

Slide 43 text

#adchi / #ExperimentationMindset / @DocOnDev leading practices

Slide 44

Slide 44 text

#adchi / #ExperimentationMindset / @DocOnDev from implementation to experimentation

Slide 45

Slide 45 text

#adchi / #ExperimentationMindset / @DocOnDev experimentation

Slide 46

Slide 46 text

#adchi / #ExperimentationMindset / @DocOnDev experimentation

Slide 47

Slide 47 text

#adchi / #ExperimentationMindset / @DocOnDev split (a/B) + Multivariate

Slide 48

Slide 48 text

#adchi / #ExperimentationMindset / @DocOnDev Cumulative Flow

Slide 49

Slide 49 text

#adchi / #ExperimentationMindset / @DocOnDev velocity trends

Slide 50

Slide 50 text

#adchi / #ExperimentationMindset / @DocOnDev code quality

Slide 51

Slide 51 text

#adchi / #ExperimentationMindset / @DocOnDev employee engagement

Slide 52

Slide 52 text

#adchi / #ExperimentationMindset / @DocOnDev team joy

Slide 53

Slide 53 text

#adchi / #ExperimentationMindset / @DocOnDev Retrospectives

Slide 54

Slide 54 text

#adchi / #ExperimentationMindset / @DocOnDev refine their process

Slide 55

Slide 55 text

#adchi / #ExperimentationMindset / @DocOnDev Orientation + On-Boarding

Slide 56

Slide 56 text

#adchi / #ExperimentationMindset / @DocOnDev Interest Leagues

Slide 57

Slide 57 text

#adchi / #ExperimentationMindset / @DocOnDev hackathons

Slide 58

Slide 58 text

#adchi / #ExperimentationMindset / @DocOnDev mentorships

Slide 59

Slide 59 text

#adchi / #ExperimentationMindset / @DocOnDev 360 / peer reviews

Slide 60

Slide 60 text

#adchi / #ExperimentationMindset / @DocOnDev managers institutionalize procedure - Gabe Newell

Slide 61

Slide 61 text

#adchi / #ExperimentationMindset / @DocOnDev being open about [pay] breeds trust - Leo Widrich

Slide 62

Slide 62 text

#adchi / #ExperimentationMindset / @DocOnDev managers [are] mentors - Kevin Goldsmith

Slide 63

Slide 63 text

#adchi / #ExperimentationMindset / @DocOnDev four things

Slide 64

Slide 64 text

#adchi / #ExperimentationMindset / @DocOnDev know your purpose; challenge assumptions

Slide 65

Slide 65 text

#adchi / #ExperimentationMindset / @DocOnDev make failure acceptable – Henry Ford “Failure is simply the opportunity to begin again, this time more intelligently.”

Slide 66

Slide 66 text

think big… start small… .

Slide 67

Slide 67 text

#adchi / #ExperimentationMindset / @DocOnDev be intolerant of mediocrity

Slide 68

Slide 68 text

#adchi / #ExperimentationMindset / @DocOnDev keep experimenting Assumptions Actions Outcomes Get it Right Explore Different Ways

Slide 69

Slide 69 text

Thank You!

Slide 70

Slide 70 text

#adchi / #ExperimentationMindset / @DocOnDev The Experimentation Mindset

Slide 71

Slide 71 text

#adchi / #ExperimentationMindset / @DocOnDev • Mad Scientist - https://www.flickr.com/photos/dzingeek/4587871752/ sizes/o/ • Stages of Growth - http://superkoopatroopa.deviantart.com/art/Terri- Growth-chart-125875917 • Elephant and Mother - http://upload.wikimedia.org/wikipedia/ commons/7/75/ Baby_asian_elephant_(with_his_mother)_born_in_the_Lao_Elephant_ Conservation_Center_in_Sayaboury.jpg Phone Script - http:// www.docstoc.com/docs/6629075/Sample-Sales-Script • Phones - • http://www.enumclawsda.org/uploads/ 1/3/3/9/13391741/1971539_orig.gif?119 • http://4.bp.blogspot.com/-qASUt2T7C7o/ULvJ8HVU3LI/ AAAAAAAAACw/LufW5HYhCyM/s1600/Tel%C3%A8fon+fixe +antic.png • https://c2.staticflickr.com/8/7443/8758001086_2692f91620_h.jpg • http://cdn2.hubspot.net/hub/147894/file-270530761-jpg/images/ phone-large.jpg • Mob Programming - https://techblog.betclicgroup.com/wp-content/ uploads/2014/09/mob-programming-1-1024.jpg • Spotify Matrix - http://blog.kevingoldsmith.com/wp-content/uploads/ 2014/03/Screen-Shot-2013-11-09-at-7.30.08-AM-1024x644.png • a/b | multivariate testing - https://www.dynamicyield.com/wp-content/ uploads/2014/05/testing.jpg • Hackathon - https://m2.behance.net/rendition/pm/22070355/disp/ f4d62818fcb09d43722a945486ca5300.png • Things - http://41.media.tumblr.com/ d721f204c69de1b5a6a4899429430c9e/ tumblr_n52o0matip1snioclo1_1280.jpg, http://cdn.instructables.com/ FIT/POMF/HKJTCKRQ/FITPOMFHKJTCKRQ.LARGE.jpg, http:// shadowgrail.deviantart.com/art/Rock-of-Ages-83993992, http:// vo5.deviantart.com/art/Fantastic-Four-163610854 Images

Slide 72

Slide 72 text

#adchi / #ExperimentationMindset / @DocOnDev • Students Test - https://img.washingtonpost.com/rf/image_1484w/ 2010-2019/WashingtonPost/2014/09/17/Others/Images/ 2014-09-17/DSC_54171410973389.jpg? uuid=h6VsJD6MEeSkMLgqPme3Yg • Woody Zuill - https://pbs.twimg.com/media/ A8NKQ9lCQAAdA7h.jpg • Code Quality (WTF) - http://www.sonarqube.org/wp-content/ uploads/2009/09/wtfm.jpg • 360 Feedback - http://screencrush.com/442/files/2012/07/360- review.jpg • Test Tube - http://fc09.deviantart.net/fs70/i/2010/018/2/1/ Vector_Test_Tube_by_Algerianoo.jpg • No Estimates - https://twitter.com/NoEstimates • Tesla - https://c2.staticflickr.com/ 4/3006/2932363100_5bf7ff5820_b.jpg • Smiley - https://blogs.dal.ca/csc/files/2012/07/MP900443129.jpg • Dilbert - http://www.stephenshapiro.com/images/dilbert-bp.gif • Manager’s Office - https://www.flickr.com/photos/islespunkfan/ 8281372547/sizes/l • Bad review - http://media.benefitspro.com/benefitspro/article/ 2014/02/06/negative%20reviews.jpg • Captain my Captain - http://startupeers.co/wp-content/uploads/ 2014/08/o-captain-e1407919367905.jpg • Vader in Suit - http://fc06.deviantart.net/fs71/f/2011/179/a/a/ darth_vader_by_iamthedarthvader-d3k9n2h.jpg • League - https://oliverwillis.com/wp-content/uploads/2015/08/ league-of-legends.jpg • Mentorship - http://humancapitalist.com/wordpress/wp-content/ uploads/2013/11/canstockphoto15167410.jpg more images

Slide 73

Slide 73 text

#adchi / #ExperimentationMindset / @DocOnDev • Dreyfus Model of Skills Acquisition - http://en.wikipedia.org/ wiki/Dreyfus_model_of_skill_acquisition, http:// files.eric.ed.gov/fulltext/ED384695.pdf#page=130, https:// www.dropbox.com/s/px1d5orrtjm5d5i/ADA084551.pdf?dl=0 • Learning through experimentation - http://lifehacker.com/ 5985565/how-and-why-to-intentionally-set-yourself-up-for- failure • HBR: Collective Genius - http://hbr.org/2014/06/collective- genius/ar/1 • How Organizations Learn - http://www.areteadventures.com/ articles/the_challenge_of_entering_the_green_room.pdf • Argyris: Teaching Smart People how to Learn - http://hbr.org/ 1991/05/teaching-smart-people-how-to-learn/ar/1, http:// pds8.egloos.com/pds/200805/20/87/ chris_argyris_learning.pdf • Argyris: Double Loop Learning in Organizations - http:// hbr.org/1977/09/double-loop-learning-in-organizations/ar/3 • A/B Testing and the Experimentation Culture - https:// hbr.org/2014/02/ab-testing-and-the-benefits-of-an- experimentation-culture/ • Enlightened Experimentation - https://hbr.org/2001/02/ enlightened-experimentation-the-new-imperative-for- innovation/ar/4 • Argyris: Single-Loop and Double-Loop Learning in Research on Decision Making - https:// 093#1c5e28761e25b79cfc10de03f5a188778a0.googledrive .com/host/0B_RO2ClD0V7AZHpWRS1mNUc2d1k/argyris.pdf • Ed Batista Blog - http://www.edbatista.com/2008/05/double- loop.html Reference Material

Slide 74

Slide 74 text

#adchi / #ExperimentationMindset / @DocOnDev • Lean Change - http://www.lean.org/LeanPost/ Posting.cfm?LeanPostId=225 • Scaling Up Excellence - http://www.amazon.com/ Scaling-up-Excellence-Hayagreeva-Rao-ebook/ dp/B00EGMQIDG • Start With Why - http://www.amazon.com/Start- Why-Leaders-Inspire-Everyone-ebook/dp/ B002Q6XUE4/ • Leading Change - http://www.amazon.com/gp/ product/0875847471/ • Woody Zuill - https://twitter.com/woodyzuill, http:// zuill.us/WoodyZuill/ • Mob Programming and No Estimates - http:// www.infoq.com/interviews/zuill-mob-programming • Democratizing Experimentation - https://hbr.org/ 2011/04/8-ways-to-democratize-experime/ • Failing by Design - https://hbr.org/2011/04/failing- by-design/ar/1 • Smart Business Experiments - https://hbr.org/ 2011/03/a-step-by-step-guide-to-smart-business- experiments/ar/1 • Evidence Based Management - https://hbr.org/ 2006/01/evidence-based-management/ar/5 more reference material

Slide 75

Slide 75 text

#adchi / #ExperimentationMindset / @DocOnDev • Buffer Transparency - http:// www.fastcompany.com/3024306/bottom- line/why-this-startup-made-their-salaries- radically-transparent • Buffer Transparency Fail - http:// www.fastcompany.com/3050682/lessons- learned/why-our-transparent-feedback- experiment-failed • Valve - https://blog.inf.ed.ac.uk/sapm/ 2014/03/14/management-at-valve-an- analysis-of-management-practices-at-valve- software/ • Valve Handbook - https:// www.valvesoftware.com/company/ Valve_Handbook_LowRes.pdf • Spotify White Paper - https:// dl.dropboxusercontent.com/u/1018963/ Articles/SpotifyScaling.pdf • Kevin Goldsmith on Spotify - http:// blog.kevingoldsmith.com/2014/03/14/ thoughts-on-emulating-spotifys-matrix- organization-in-other-companies/ yet more reference material