Business Impact Driven Development

Slide 1

Slide 1 text

1 Business impact driven development Dmitry Salahutdinov 12/10/209

Slide 2

Slide 2 text

Hey, my name is Dmitry 2 +

Slide 3

Slide 3 text

3 About myself :)

Slide 4

Slide 4 text

AMPLIFR 4

Slide 5

Slide 5 text

5 Vital project: 400m2

Slide 6

Slide 6 text

Continuous delivery

Slide 7

Slide 7 text

Canary releases A/B Testing, Green Blue Deployment

Slide 8

Slide 8 text

8 Feature Toggling

Slide 9

Slide 9 text

Experimenting

Slide 10

Slide 10 text

Measuring

Slide 11

Slide 11 text

Analysis & Reinforcement

Slide 12

Slide 12 text

12 Process is iterative and continuous Area, time, workers are limited   Needs are growing persistently

Slide 13

Slide 13 text

13 Продакт овнер? Growth hacker? Startup?

Slide 14

Slide 14 text

14 Продакт овнер? Growth hacker? Startup? Product owner?

Slide 15

Slide 15 text

15 Продакт овнер? Growth hacker? Startup? Product owner? Growth hacker?* * https://en.wikipedia.org/wiki/Growth_hacking

Slide 16

Slide 16 text

Yep! Run experiments to increase output Monitoring Analysis Changes

Slide 17

Slide 17 text

17 Using tools and techniques to turn data into meaningful business insights Basis: analytics

Slide 18

Slide 18 text

Business metrics Trial to Payment conversion Paid users outﬂow 18 High-level product performing indicators (money)

Slide 19

Slide 19 text

19 Product metrics ↗ Feature Usage Middle-level indicators in terms of product features efﬁciency

Slide 20

Slide 20 text

20 Tech metrics Low-level metrics showing how the technical underground perform

Slide 21

Slide 21 text

21 All fo them Are numbers

Slide 22

Slide 22 text

That are the only one truth Measure The only One Truth! Everyone in your team has own background and insights Product analytics works both: for developer & product owner 22

Slide 23

Slide 23 text

23 Why do we love metrics? - is the only one source of true data

Slide 24

Slide 24 text

24 Analytics/Metrics - is the only one source of true data - helps approve good ideas

Slide 25

Slide 25 text

25 Analytics/Metrics - is the only one source of true data - helps approve good ideas - helps reject bad ideas

Slide 26

Slide 26 text

26 Analytics/Metrics - is the only one source of true data - helps approve good ideas - helps reject bad ideas - works both ways

Slide 27

Slide 27 text

Reject idea: sticky header & footer Average user screen height is ~600px 27

Slide 28

Slide 28 text

Example: Main feature fails Error rates correlates with users outﬂow User outﬂow Let developers earn money by increasing code quality: Do refactoring legacy code and give back technical debt ☺ Process fails 28

Slide 29

Slide 29 text

29 Product Owner Developer Business Project: Role distribution

Slide 30

Slide 30 text

30 Product Owner Developer Business Startup: role distribution

Slide 31

Slide 31 text

Proper tools (requirements) 31

Slide 32

Slide 32 text

Tracking events 32

Slide 33

Slide 33 text

Show graphical data, compare charts Data visualisation Scheduled Post Created Draft 33

Slide 34

Slide 34 text

Metadata Event metadata stores “as is” User metadata associates within current event Pass extra user & event data to analytics 34

Slide 35

Slide 35 text

Metadata usage Scheduled Post  by user having Billing Plan “A” Scheduled Post  by user having Billing Plan “B” 35 Extra date for details analysis

Slide 36

Slide 36 text

Multiple platroms support Mobile application Web(frontend/backend) 36

Slide 37

Slide 37 text

Release tracking Release:  feature deployment Release:  feature improvement 37

Slide 38

Slide 38 text

Instrumentation 38

Slide 39

Slide 39 text

Amplitude 39 https://help.amplitude.com/hc/en-us/categories/200409887-Getting-Started

Slide 40

Slide 40 text

10 millions events for free 40

Slide 41

Slide 41 text

Amplitude API 41

Slide 42

Slide 42 text

Backend integration Ruby sample Node JS Mobile 42

Slide 43

Slide 43 text

Frontend integration 43

Slide 44

Slide 44 text

Share user properties Render “backend” User properties Pass actual user properties With events 44

Slide 45

Slide 45 text

Empirical tips 45

Slide 46

Slide 46 text

verb + noun (e.g. 'clicked signup’) noun + verb (e.g. 'signup clicked') Naming Scheduled Post Draft created Save post saved Fail to post ... ... ✅ ❌ Event naming convention prevents entropy 46

Slide 47

Slide 47 text

Separate environments To keep experiments pure and prevent testing events mixing Overall data Testing data Very signiﬁcant for low trafﬁc experiments! 47

Slide 48

Slide 48 text

Organise events & properties And keep it simple 48

Slide 49

Slide 49 text

Use cases 49

Slide 50

Slide 50 text

Existing feature analysis Start to collect metrics - measure feature performance - make a decision: improve or remove Measure business performance metrics before 50

Slide 51

Slide 51 text

New feature investigating/testing Start to collect metrics New feature deployment Ensure to have previous and next metrics collected 51

Slide 52

Slide 52 text

Incremental improvements Do small experiments with performance analysis 52

Slide 53

Slide 53 text

Impact driven? 53

Slide 54

Slide 54 text

54 Every change has to have an impact Impact driven?

Slide 55

Slide 55 text

Impact work cycle 55 - collect metrics

Slide 56

Slide 56 text

Impact work cycle 56 - collect metrics - analyse performance/proﬁt

Slide 57

Slide 57 text

Impact work cycle 57 - collect metrics - analyse performance/proﬁt - make a prediction (hypothesis)

Slide 58

Slide 58 text

Impact work cycle 58 - collect metrics - analyse performance/proﬁt - make a prediction (hypothesis) - deploy & monitor

Slide 59

Slide 59 text

Impact work cycle 59 - collect metrics - analyse performance/proﬁt - make a prediction (hypothesis) - deploy & monitor - repeat if ok, reject if not Experiment

Slide 60

Slide 60 text

Necessary conditions 60 - Statistical correctness of analytics

Slide 61

Slide 61 text

Necessary conditions 61 - Statistical correctness of analytics - Trafﬁc/Duration tradeoff

Slide 62

Slide 62 text

Necessary conditions 62 - Statistical correctness of analytics - Prevent interception - Trafﬁc/Duration tradeoff

Slide 63

Slide 63 text

Necessary conditions 63 - Statistical correctness of analytics - Prevent interception - Overhead for small experiments - Trafﬁc/Duration tradeoff

Slide 64

Slide 64 text

Necessary conditions 64 - Statistical correctness of analytics - Prevent interception - Overhead for small experiments - Time consuming for huge ones - Trafﬁc/Duration tradeoff

Slide 65

Slide 65 text

65 Hypothesis selection

Slide 66

Slide 66 text

66 Idea: ??? What is it for:??? How it will help:??? Voting

Slide 67

Slide 67 text

67 ICE scoring* Impact, Conﬁdence, Ease * https://tech.trello.com/ice-scoring/

Slide 68

Slide 68 text

68 * https://tech.trello.com/ice-scoring/ Doubt? Measure! ICE scoring* Impact, Conﬁdence, Ease

Slide 69

Slide 69 text

69 Experiment duration?

Slide 70

Slide 70 text

70 Hypothesis check: hard-code way 7 day trial

Slide 71

Slide 71 text

71 Hypothesis check: practical way Wait until data get stable Double check by reverting back

Slide 72

Slide 72 text

72 Hypothesis check: math way

Slide 73

Slide 73 text

73 Summary: - Metrics are good - Experiments are the best - Analysis is better

Slide 74

Slide 74 text

Slide 75

Slide 75 text

Feature availability Unexpected ideas could work feature visibility with ugly hack improved conversion ' 75

Slide 76

Slide 76 text

Unexpected ideas could not work 6%↗ post popup cancellation 76

Slide 77

Slide 77 text

Settings => Subscription ~20% Change Project Name => Activate Subscription Let just force people to change project name and earn 77

Slide 78

Slide 78 text

78 Why analytics/hypothesis?

Slide 79

Slide 79 text

Makes everyone happy 79 →Everyone on the team understands the core business model, i.e. who is paying for what and why

Slide 80

Slide 80 text

Makes everyone happy 80 →Everyone on the team understands the core business model, i.e. who is paying for what and why → Improves engineering team culture by building awareness of how customers use different features in real life

Slide 81

Slide 81 text

Makes everyone happy 81 →Everyone on the team understands the core business model, i.e. who is paying for what and why → Improves engineering team culture by building awareness of how customers use different features in real life → Improves team communication by building a common language

Slide 82

Slide 82 text

Makes everyone happy 82 →Everyone on the team understands the core business model, i.e. who is paying for what and why → Improves engineering team culture by building awareness of how customers use different features in real life → Improves team communication by building a common language → motivates the team by showing everyone their impact

Slide 83

Slide 83 text

Makes everyone happy 83 →Everyone on the team understands the core business model, i.e. who is paying for what and why → Improves engineering team culture by building awareness of how customers use different features in real life → Improves team communication by building a common language → motivates the team by showing everyone their impact → improves the product debate and arguments by providing a system

Slide 84

Slide 84 text

Makes everyone happy 84 →Everyone on the team understands the core business model, i.e. who is paying for what and why → Improves engineering team culture by building awareness of how customers use different features in real life → Improves team communication by building a common language → motivates the team by showing everyone their impact → improves the product debate and arguments by providing a system → summarises the perception of the project

Slide 85

Slide 85 text

Makes everyone happy 85 →Everyone on the team understands the core business model, i.e. who is paying for what and why → Improves engineering team culture by building awareness of how customers use different features in real life → Improves team communication by building a common language → motivates the team by showing everyone their impact → improves the product debate and arguments by providing a system → grows team experience, skill, and expertise. → summarises the perception of the project

Slide 86

Slide 86 text

86 Give it a try In your startup

Slide 87

Slide 87 text

Slide 88

Slide 88 text

STACHKA 15% discount coupon ❤ 88

Slide 89

Slide 89 text

89 Thank you @dsalahutdinov1 [email protected] https://dev.to/amplifr https://amplifr.com ♥