Slide 1
Slide 1 text
Measuring and Improving
Performance
Insights from one of Cash App’s main screens
Slide 2
Slide 2 text
Hi, I’m Colin Marsch
Android Engineer @ Cash App
Slide 3
Slide 3 text
Overview
Slide 4
Slide 4 text
Overview
• Where we started
Slide 5
Slide 5 text
Overview
• Where we started
• What metrics we are tracking
Slide 6
Slide 6 text
Overview
• Where we started
• What metrics we are tracking
• Collecting the data
Slide 7
Slide 7 text
Overview
• Where we started
• What metrics we are tracking
• Collecting the data
• Setting up monitoring
Slide 8
Slide 8 text
Overview
• Where we started
• What metrics we are tracking
• Collecting the data
• Setting up monitoring
• Challenges we faced
Slide 9
Slide 9 text
Overview
• Where we started
• What metrics we are tracking
• Collecting the data
• Setting up monitoring
• Challenges we faced
• Discoveries
Slide 10
Slide 10 text
I work on
Cash App’s Money Tab
Slide 11
Slide 11 text
Where we started
Slide 12
Slide 12 text
Only scroll
performance tracked
Slide 13
Slide 13 text
Only scroll
performance tracked
Slide 14
Slide 14 text
Why now?
Slide 15
Slide 15 text
Why now?
• Compare data during a Kotlin Multiplatform migration
Slide 16
Slide 16 text
Why now?
• Compare data during a Kotlin Multiplatform migration
• Have visibility into the features on the screen
Slide 17
Slide 17 text
Why now?
• Compare data during a Kotlin Multiplatform migration
• Have visibility into the features on the screen
• Ensure Cash App is performant for all customers
Slide 18
Slide 18 text
What metrics to track
Slide 19
Slide 19 text
What metrics are we tracking?
Scroll performance
Load times Stale Data
??
??
??
??
Slide 20
Slide 20 text
What metrics are we tracking?
Scroll performance
Load times Stale Data
??
??
??
??
Slide 21
Slide 21 text
What metrics are we tracking?
Scroll performance
Load times Stale Data
??
??
??
??
Slide 22
Slide 22 text
What metrics are we tracking?
Scroll performance
Load times Stale Data
??
??
??
??
Slide 23
Slide 23 text
Overall load time
Slide 24
Slide 24 text
Overall load time
Slide 25
Slide 25 text
Applet load times
Slide 26
Slide 26 text
No content
Slide 27
Slide 27 text
No content
Slide 28
Slide 28 text
data class Applet(
val id: AppletId,
val state: AppletState,
)
Slide 29
Slide 29 text
data class Applet(
val id: AppletId,
val state: AppletState,
)
Slide 30
Slide 30 text
sealed interface AppletState {
data class Loading(...) : AppletState
data class Unadopted(...) : AppletState
data class Adopted(...) : AppletState
data class Unavailable(...) : AppletState
data class Error(...) : AppletState
}
Slide 31
Slide 31 text
sealed interface AppletState {
data class Loading(...) : AppletState
data class Unadopted(...) : AppletState
data class Adopted(...) : AppletState
data class Unavailable(...) : AppletState
data class Error(...) : AppletState
}
Slide 32
Slide 32 text
sealed interface AppletState {
data class Loading(...) : AppletState
data class Unadopted(...) : AppletState
data class Adopted(...) : AppletState
data class Unavailable(...) : AppletState
data class Error(...) : AppletState
}
Slide 33
Slide 33 text
Savings: Adopted ✅
Bitcoin: Loading
Stocks: Loading
Taxes: Loading
Savings: Adopted ✅
Bitcoin: Adopted ✅
Stocks: Loading
Taxes: Unavailable ✅
Savings: Adopted ✅
Bitcoin: Adopted ✅
Stocks: Adopted ✅
Taxes: Unavailable ✅
Slide 34
Slide 34 text
Savings: Adopted ✅
Bitcoin: Loading
Stocks: Loading
Taxes: Loading
Savings: Adopted ✅
Bitcoin: Adopted ✅
Stocks: Loading
Taxes: Unavailable ✅
Savings: Adopted ✅
Bitcoin: Adopted ✅
Stocks: Adopted ✅
Taxes: Unavailable ✅
Slide 35
Slide 35 text
Savings: Adopted ✅
Bitcoin: Loading
Stocks: Loading
Taxes: Loading
Savings: Adopted ✅
Bitcoin: Adopted ✅
Stocks: Loading
Taxes: Unavailable ✅
Savings: Adopted ✅
Bitcoin: Adopted ✅
Stocks: Adopted ✅
Taxes: Unavailable ✅
Slide 36
Slide 36 text
Collecting the data
Slide 37
Slide 37 text
In-house analytics
Slide 38
Slide 38 text
In-house analytics
Pros:
Slide 39
Slide 39 text
In-house analytics
Pros:
• Infrastructure already set up in Cash App
Slide 40
Slide 40 text
In-house analytics
Pros:
• Infrastructure already set up in Cash App
• Data
fl
ows to visualization tools our teams are familiar with
Slide 41
Slide 41 text
In-house analytics
Pros:
• Infrastructure already set up in Cash App
• Data
fl
ows to visualization tools our teams are familiar with
Cons:
Slide 42
Slide 42 text
In-house analytics
Pros:
• Infrastructure already set up in Cash App
• Data
fl
ows to visualization tools our teams are familiar with
Cons:
• Large cost for high volume events
Slide 43
Slide 43 text
In-house analytics
Pros:
• Infrastructure already set up in Cash App
• Data
fl
ows to visualization tools our teams are familiar with
Cons:
• Large cost for high volume events
• Visualization tools are mainly business focused
Slide 44
Slide 44 text
Datadog
Slide 45
Slide 45 text
Datadog
Pros:
Slide 46
Slide 46 text
Datadog
Pros:
• Focused on engineering metrics
Slide 47
Slide 47 text
Datadog
Pros:
• Focused on engineering metrics
• Used in Cash App backend services
Slide 48
Slide 48 text
Datadog
Pros:
• Focused on engineering metrics
• Used in Cash App backend services
• Many supported default metrics
Slide 49
Slide 49 text
Datadog
Pros:
• Focused on engineering metrics
• Used in Cash App backend services
• Many supported default metrics
Cons:
Slide 50
Slide 50 text
Datadog
Pros:
• Focused on engineering metrics
• Used in Cash App backend services
• Many supported default metrics
Cons:
• External dependency risks (APK size, app startup, etc)
Slide 51
Slide 51 text
Datadog
Pros:
• Focused on engineering metrics
• Used in Cash App backend services
• Many supported default metrics
Cons:
• External dependency risks (APK size, app startup, etc)
• Additional e
ff
ort to implement and maintain
Slide 52
Slide 52 text
Setting up monitoring
Slide 53
Slide 53 text
Automated alerting
Slide 54
Slide 54 text
Automated alerting
• Main focus on overall and applet load times
Slide 55
Slide 55 text
Automated alerting
• Main focus on overall and applet load times
• Possible alerts on secondary metrics (e.g. scroll performance)
Slide 56
Slide 56 text
Early rollout to beta groups
Slide 57
Slide 57 text
Clear dashboards
Easy to understand for our
team and other stakeholders
Slide 58
Slide 58 text
Challenges we faced
Slide 59
Slide 59 text
Challenges we faced
Slide 60
Slide 60 text
Challenges we faced
• Architecture
Slide 61
Slide 61 text
Challenges we faced
• Architecture
• Customer-based measurement
Slide 62
Slide 62 text
Challenges we faced
• Architecture
• Customer-based measurement
• Platform consistency
Slide 63
Slide 63 text
Discoveries
Slide 64
Slide 64 text
Features perform differently
Slide 65
Slide 65 text
Some staleness is inevitable
Slide 66
Slide 66 text
Lessons learned
Slide 67
Slide 67 text
Lessons learned
• Architecture choices have long term impact
Slide 68
Slide 68 text
Lessons learned
• Architecture choices have long term impact
• Performance monitoring is worth the e
ff
ort
Slide 69
Slide 69 text
Lessons learned
• Architecture choices have long term impact
• Performance monitoring is worth the e
ff
ort
• Performance will always need to be top of mind
Slide 70
Slide 70 text
Stay tuned!
code.cash.app or a future Droidcon talk