From 0 to 100: Fine-tune all the things!

In this session we will talk about all the performance bottlenecks that we found along the way developing the Mapbox Navigation SDK for Android. We will learn how we monitored, made decisions based on numbers and tackled them: from designing and implementing a pipeline that runs automated tests, to selecting and analyzing the data obtained so issues can be detected and after evaluating their impact fixed. We will show the tools and platforms used, such as Systrace, test @Rules and AWS Device farm.

When you are developing SDKs or libraries, their performance is crucial to your customers, especially in the context of turn-by-turn navigation. The Mapbox Navigation SDK is built on top of different components and services and contains logic needed to get timed navigation instructions. To make the calculations for these instructions, the SDK compares the user's current location to the current route being traveled along.

The entire navigation experience includes voice announcements, real-time user progress to their destination, smooth map camera animations, and detecting when a user goes off-route. As you can imagine, all of these are critical when building a navigation app but at the same time pretty heavy in terms of performance.

By the end of the talk, you will be obsessed about the performance of your application and itching to fine-tune everything!

Transcript

1. From 0 to 100: Fine-tune all the things! Pablo Guardiola Mapbox @Guardiola31337
3. Mapbox Navigation SDK
4. Mapbox Navigation SDK • Maps • Location Services
6. Mapbox Navigation SDK • Maps • Location Services • Routing
8. Mapbox Navigation SDK • Maps • Location Services • Routing • Processing
10. Track performance over time as changes land in the SDK.
11. Which performance areas?
12. Which performance areas? • CPU
13. Which performance areas? • CPU • RAM
14. Which performance areas? • CPU • RAM • Bandwidth
15. Which performance areas? • CPU • RAM • Bandwidth • Battery
16. Android Studio Profiler How does this help us?
17. Android Studio Profiler How does this help us? • Real-time, readable data
19. Android Studio Profiler How does this help us? • Real-time, readable data • Session recording
20. Android Studio Profiler How does this help us? • Real-time, readable data • Session recording • Detailed analysis
21. Android Studio Profiler How does this help us? • Real-time, readable data • Session recording • Detailed analysis • Initial signal
22. Android Studio Profiler Does it solve our problem?
23. Android Studio Profiler Does it solve our problem? • Cannot download or send any recorded data
24. Android Studio Profiler Does it solve our problem? • Cannot download or send any recorded data • Android Studio resource consumption
25. Android Studio Profiler Does it solve our problem? • Cannot download or send any recorded data • Android Studio resource consumption • API 21+
26. Custom Test Rules How does this help us?
28. Custom Test Rules How does this help us? • Reproducible
30. Custom Test Rules How does this help us? • Reproducible • Record and store data
34. Custom Test Rules How does this help us? • Reproducible • Record and store data • Device farm
35. Custom Test Rules Does it solve our problem?
36. Custom Test Rules Does it solve our problem? • Needs reliable UI test
37. Custom Test Rules Does it solve our problem? • Needs reliable UI test • Takes a lot of time
38. Custom Test Rules Does it solve our problem? • Takes a lot of time • Needs reliable UI test • dumpsys output isn’t easily parsed
40. Custom Test Rules Does it solve our problem? • Takes a lot of time • Needs reliable UI test • dumpsys output isn’t easily parsed • Output data can be unreliable
41. Production Telemetry How does this help us?
42. Production Telemetry How does this help us? • Signal from “real world” usage
43. Production Telemetry How does this help us? • Signal from “real world” usage • Larger data pool
44. Production Telemetry How does this help us? • Signal from “real world” usage • Larger data pool • More devices
45. Production Telemetry Does it solve our problem?
46. Production Telemetry Does it solve our problem? • Production environment
47. Production Telemetry Does it solve our problem? • Production environment • Overhead of metric itself
48. Production Telemetry Does it solve our problem? • Production environment • Overhead of metric itself • Data consumption
49. Production Telemetry Does it solve our problem? • Production environment • Overhead of metric itself • Data consumption • Data normalization
50. Process
51. Process Test suite
52. Process Test suite @Rules Controlled routes Data generation
53. Process Test suite @Rules Controlled routes Data generation Data processing
54. Process Test suite @Rules Controlled routes Data generation Data processing Retrieve data Parse data Send data
55. Process Test suite @Rules Controlled routes Data generation Data processing Retrieve data Parse data Send data Data analysis
56. Process Test suite @Rules Controlled routes Data generation Data processing Retrieve data Parse data Send data Data analysis Query data Filter out data Present results
58. Pipeline
59. Pipeline Preparation
60. Pipeline Preparation Trigger CI Pull SDK code Build SDK Assemble apps
61. Pipeline Preparation Trigger CI Pull SDK code Build SDK AWS Device Farm Assemble apps
62. Pipeline Preparation Trigger CI Pull SDK code Build SDK AWS Device Farm Assemble apps Upload apps Run tests Download artifacts Completed check
63. Pipeline Preparation Trigger CI Pull SDK code Build SDK AWS Device Farm Assemble apps Upload apps Run tests Download artifacts Completed check Scripts
64. Pipeline Preparation Trigger CI Pull SDK code Build SDK AWS Device Farm Assemble apps Upload apps Run tests Download artifacts Parse data Publish data Completed check Scripts
65. Pipeline Preparation Trigger CI Pull SDK code Build SDK AWS Device Farm Assemble apps Upload apps Run tests Mode Download artifacts Parse data Publish data Completed check Scripts
66. Pipeline Preparation Trigger CI Pull SDK code Build SDK AWS Device Farm Assemble apps Upload apps Run tests Mode Download artifacts Parse data Publish data Completed check SQL (Presto) Build reports Scripts
67. What’s next?
68. What’s next? • More automation
69. What’s next? • More automation • Triggers / Thresholds
70. What’s next? • More automation • Triggers / Thresholds • Feature flags
71. Performance Takeaways
72. Performance Takeaways • Important
73. Performance Takeaways • Important • Tricky
74. Performance Takeaways • Important • Tricky • On-going effort
75. Performance Takeaways • Important • Tricky • On-going effort • Good amount of data
76. Questions?
77. We’re hiring!
78. Thanks! Pablo Guardiola @Guardiola31337