Droidcon NYC 2023: Measuring and Improving Performance

Are you looking to track and improve the performance of specific customer flows in your app? We were at Cash App and in this talk I’ll walk through how we went about measuring, monitoring and improving the performance of one of Cash App’s main screens! I’ll discuss what metrics we tracked, how we set up alerting to ensure our product was healthy and how these checks set us up well for a future migration. Come join us to hear about what kind of instrumentation we considered and what made the final cut!

Transcript

1. Measuring and Improving Performance Insights from one of Cash App’s main screens
2. Hi, I’m Colin Marsch Android Engineer @ Cash App
3. Overview
4. Overview • Where we started
5. Overview • Where we started • What metrics we are tracking
6. Overview • Where we started • What metrics we are tracking • Collecting the data
7. Overview • Where we started • What metrics we are tracking • Collecting the data • Setting up monitoring
8. Overview • Where we started • What metrics we are tracking • Collecting the data • Setting up monitoring • Challenges we faced
9. Overview • Where we started • What metrics we are tracking • Collecting the data • Setting up monitoring • Challenges we faced • Discoveries
10. I work on Cash App’s Money Tab
11. Where we started
12. Only scroll performance tracked
13. Only scroll performance tracked
14. Why now?
15. Why now? • Compare data during a Kotlin Multiplatform migration
16. Why now? • Compare data during a Kotlin Multiplatform migration • Have visibility into the features on the screen
17. Why now? • Compare data during a Kotlin Multiplatform migration • Have visibility into the features on the screen • Ensure Cash App is performant for all customers
18. What metrics to track
19. What metrics are we tracking? Scroll performance Load times Stale Data ?? ?? ?? ??
20. What metrics are we tracking? Scroll performance Load times Stale Data ?? ?? ?? ??
21. What metrics are we tracking? Scroll performance Load times Stale Data ?? ?? ?? ??
22. What metrics are we tracking? Scroll performance Load times Stale Data ?? ?? ?? ??
23. Overall load time
24. Overall load time
25. Applet load times
28. data class Applet( val id: AppletId, val state: AppletState, )
29. data class Applet( val id: AppletId, val state: AppletState, )
30. sealed interface AppletState { data class Loading(...) : AppletState data class Unadopted(...) : AppletState data class Adopted(...) : AppletState data class Unavailable(...) : AppletState data class Error(...) : AppletState }
31. sealed interface AppletState { data class Loading(...) : AppletState data class Unadopted(...) : AppletState data class Adopted(...) : AppletState data class Unavailable(...) : AppletState data class Error(...) : AppletState }
32. sealed interface AppletState { data class Loading(...) : AppletState data class Unadopted(...) : AppletState data class Adopted(...) : AppletState data class Unavailable(...) : AppletState data class Error(...) : AppletState }
33. Savings: Adopted ✅ Bitcoin: Loading
 Stocks: Loading Taxes: Loading Savings: Adopted ✅ Bitcoin: Adopted ✅
 Stocks: Loading Taxes: Unavailable ✅ Savings: Adopted ✅ Bitcoin: Adopted ✅
 Stocks: Adopted ✅ Taxes: Unavailable ✅
34. Savings: Adopted ✅ Bitcoin: Loading
 Stocks: Loading Taxes: Loading Savings: Adopted ✅ Bitcoin: Adopted ✅
 Stocks: Loading Taxes: Unavailable ✅ Savings: Adopted ✅ Bitcoin: Adopted ✅
 Stocks: Adopted ✅ Taxes: Unavailable ✅
35. Savings: Adopted ✅ Bitcoin: Loading
 Stocks: Loading Taxes: Loading Savings: Adopted ✅ Bitcoin: Adopted ✅
 Stocks: Loading Taxes: Unavailable ✅ Savings: Adopted ✅ Bitcoin: Adopted ✅
 Stocks: Adopted ✅ Taxes: Unavailable ✅
36. Collecting the data
37. In-house analytics
38. In-house analytics Pros:
39. In-house analytics Pros: • Infrastructure already set up in Cash App
40. In-house analytics Pros: • Infrastructure already set up in Cash App • Data fl ows to visualization tools our teams are familiar with
41. In-house analytics Pros: • Infrastructure already set up in Cash App • Data fl ows to visualization tools our teams are familiar with Cons:
42. In-house analytics Pros: • Infrastructure already set up in Cash App • Data fl ows to visualization tools our teams are familiar with Cons: • Large cost for high volume events
43. In-house analytics Pros: • Infrastructure already set up in Cash App • Data fl ows to visualization tools our teams are familiar with Cons: • Large cost for high volume events • Visualization tools are mainly business focused
44. Datadog
45. Datadog Pros:
46. Datadog Pros: • Focused on engineering metrics
47. Datadog Pros: • Focused on engineering metrics • Used in Cash App backend services
48. Datadog Pros: • Focused on engineering metrics • Used in Cash App backend services • Many supported default metrics
49. Datadog Pros: • Focused on engineering metrics • Used in Cash App backend services • Many supported default metrics Cons:
50. Datadog Pros: • Focused on engineering metrics • Used in Cash App backend services • Many supported default metrics Cons: • External dependency risks (APK size, app startup, etc)
51. Datadog Pros: • Focused on engineering metrics • Used in Cash App backend services • Many supported default metrics Cons: • External dependency risks (APK size, app startup, etc) • Additional e ff ort to implement and maintain
52. Setting up monitoring
53. Automated alerting
54. Automated alerting • Main focus on overall and applet load times
55. Automated alerting • Main focus on overall and applet load times • Possible alerts on secondary metrics (e.g. scroll performance)
56. Early rollout to beta groups
57. Clear dashboards Easy to understand for our team and other stakeholders
58. Challenges we faced
59. Challenges we faced
60. Challenges we faced • Architecture
61. Challenges we faced • Architecture • Customer-based measurement
62. Challenges we faced • Architecture • Customer-based measurement • Platform consistency
63. Discoveries
64. Features perform differently
65. Some staleness is inevitable
66. Lessons learned
67. Lessons learned • Architecture choices have long term impact
68. Lessons learned • Architecture choices have long term impact • Performance monitoring is worth the e ff ort
69. Lessons learned • Architecture choices have long term impact • Performance monitoring is worth the e ff ort • Performance will always need to be top of mind
70. Stay tuned! code.cash.app or a future Droidcon talk