Product metrics for developers

What developers need to know about product metrics

Transcript

1. 1 Business impact driven development Dmitry Salahutdinov 12/10/209 Salahutdinov Dmitry Product metrics for developers What and why developers should get deep into product analytics
2. Hey, my name is Dmitry 2 +
3. AMPLIFR 3
4. 4 Vital project: 400m2
5. Continuous delivery
6. Canary releases A/B Testing
7. 7 Feature Toggling
8. Experimenting
9. Measuring
10. Analysis Reinforcement
11. 11 Process is iterative and continuous Area, time, workers are limited 
 Needs are growing Persistently
12. 12 Продакт овнер? Growth hacker? Typical weekday?
13. 13 Продакт овнер? Growth hacker? Startup? Product owner?
14. 14 Продакт овнер? Growth hacker? Startup? Product owner? Growth hacker?
15. Yep! Run experiments to increase impact Monitoring Analysis Changes
16. 16 Using tools and techniques to turn data into meaningful business insights Analytics
17. Business metrics Trial to Payment conversion Paid users outflow 17 Highest level metrics about performing product as “money maker”
18. 18 Product metrics ↗ Feature A Usage Middle-level performance metrics in terms of product features
19. 19 Tech metrics Low-level metrics metrics how the technical underground perform
20. 20 Are numbers
21. That are the only one truth Measure The only One Truth! Everyone in your team has own background and insights Product analytics works both: for developer & product owner 21
22. 22 Analytics/Metrics - is the only one source of true data
23. 23 Analytics/Metrics - is the only one source of true data - helps approve good ideas
24. 24 Analytics/Metrics - is the only one source of true data - helps approve good ideas - helps reject bad ideas
25. 25 Analytics/Metrics - is the only one source of true data - helps approve good ideas - helps reject bad ideas - works both ways
26. Example: main feature fails Error rates correlates with users outflow User outflow Let developers earn money by increasing code quality: Do refactoring legacy code and give back technical debt ☺ Fail Post 26
27. 27 Product Owner Developer Business Role distribution
28. 28 Product Owner Developer Business Startup role distribution
29. Proper tools 29
30. Tracking events 30
31. Show graphical data, compare Data visualisation Scheduled Post Created Draft 31
32. Metadata Event metadata stores “as is” User metadata associates within current event Pass extra user & event data to analytics 32
33. Metadata usage Scheduled Post
 by user having Billing Plan “A” Scheduled Post
 by user having Billing Plan “B” 33 Extra date for details analysis
34. Multiple platroms support Mobile application Web(frontend/backend) 34
35. Track releases Release:
 feature deployment Release:
 feature improvement 35
36. Instrumentation ⚒ 36
37. Amplitude 37
38. 10 millions events for free 38
39. Amplitude API 39
40. Backend integration Ruby sample Node JS Mobile 40
41. Frontend integration 41
42. Share user properties Render “backend” User properties Pass actual user properties With events 42
43. Empirical tips 43
44. verb + noun (e.g. 'clicked signup’) noun + verb (e.g. 'signup clicked') Naming Scheduled Post Draft created Save post saved Fail to post ... ... ✅ ❌ Event naming convention prevents entropy 44
45. Separate environments To keep experiments pure and prevent testing events mixing Overall data Testing data Very significant for low traffic experiments! 45
46. Organise events & properties And keep it simple 46
47. Use cases 47
48. Existing feature analysis Start to collect metrics - measure feature performance - make a decision: improve or remove Measure business performance metrics before 48
49. New feature investigating/testing Start to collect metrics New feature deployment Ensure to have previous and next metrics collected 49
50. Incremental improvements Do small experiments with performance analysis 50
51. Impact driven? For getting business impact 51
52. 52 Every change has to do an impact
53. Impact work cycle 53 - collect metrics
54. Impact work cycle 54 - collect metrics - analyse performance/profit
55. Impact work cycle 55 - collect metrics - analyse performance/profit - make a prediction (hypothesis)
56. Impact work cycle 56 - collect metrics - analyse performance/profit - make a prediction (hypothesis) - run & monitor
57. Impact work cycle 57 - collect metrics - analyse performance/profit - make a prediction (hypothesis) - run & monitor - repeat if ok, reject if not Experiment
58. 58 Example: health hacking :)
59. Necessary conditions 59 - Statistical correctness of analytics
60. Necessary conditions 60 - Statistical correctness of analytics - Traffic vs Duration
61. Necessary conditions 61 - Statistical correctness of analytics - Traffic vs Duration - Prevent interception
62. Necessary conditions 62 - Statistical correctness of analytics - Traffic vs Duration - Prevent interception - Overhead for small experiments
63. Necessary conditions 63 - Statistical correctness of analytics - Traffic vs Duration - Prevent interception - Overhead for small experiments - Time consuming for huge ones
64. Analytics makes devs happy get deeper to a business process essence to increase developer culture (awareness of feature benefits) to reduce communication blockers (distributed teams) to motivate yourself unique argumentation system to growth experience & expertise 64
65. Thank you ❤ @dsalahutdinov1 @dsalahutdinov https://dev.to/amplifr https://amplifr.com 65