From 0 to 100: Fine-tune all the things!

From 0 to 100: Fine-tune all the things!

In this session we will talk about all the performance bottlenecks that we found along the way developing the Mapbox Navigation SDK for Android. We will learn how we monitored, made decisions based on numbers and tackled them: from designing and implementing a pipeline that runs automated tests, to selecting and analyzing the data obtained so issues can be detected and after evaluating their impact fixed. We will show the tools and platforms used, such as Systrace, test @Rules and AWS Device farm.

When you are developing SDKs or libraries, their performance is crucial to your customers, especially in the context of turn-by-turn navigation. The Mapbox Navigation SDK is built on top of different components and services and contains logic needed to get timed navigation instructions. To make the calculations for these instructions, the SDK compares the user's current location to the current route being traveled along.

The entire navigation experience includes voice announcements, real-time user progress to their destination, smooth map camera animations, and detecting when a user goes off-route. As you can imagine, all of these are critical when building a navigation app but at the same time pretty heavy in terms of performance.

By the end of the talk, you will be obsessed about the performance of your application and itching to fine-tune everything!

3c2bdd16c0ea8511dc254b8497a06f78?s=128

Pablo Guardiola

August 14, 2019
Tweet

Transcript

  1. From 0 to 100: Fine-tune all the things! Pablo Guardiola

    Mapbox @Guardiola31337
  2. None
  3. Mapbox Navigation SDK

  4. Mapbox Navigation SDK • Maps • Location Services

  5. None
  6. Mapbox Navigation SDK • Maps • Location Services • Routing

  7. None
  8. Mapbox Navigation SDK • Maps • Location Services • Routing

    • Processing
  9. None
  10. Track performance over time as changes land in the SDK.

  11. Which performance areas?

  12. Which performance areas? • CPU

  13. Which performance areas? • CPU • RAM

  14. Which performance areas? • CPU • RAM • Bandwidth

  15. Which performance areas? • CPU • RAM • Bandwidth •

    Battery
  16. Android Studio Profiler How does this help us?

  17. Android Studio Profiler How does this help us? • Real-time,

    readable data
  18. None
  19. Android Studio Profiler How does this help us? • Real-time,

    readable data • Session recording
  20. Android Studio Profiler How does this help us? • Real-time,

    readable data • Session recording • Detailed analysis
  21. Android Studio Profiler How does this help us? • Real-time,

    readable data • Session recording • Detailed analysis • Initial signal
  22. Android Studio Profiler Does it solve our problem?

  23. Android Studio Profiler Does it solve our problem? • Cannot

    download or send any recorded data
  24. Android Studio Profiler Does it solve our problem? • Cannot

    download or send any recorded data • Android Studio resource consumption
  25. Android Studio Profiler Does it solve our problem? • Cannot

    download or send any recorded data • Android Studio resource consumption • API 21+
  26. Custom Test Rules How does this help us?

  27. None
  28. Custom Test Rules How does this help us? • Reproducible

  29. None
  30. Custom Test Rules How does this help us? • Reproducible

    • Record and store data
  31. None
  32. None
  33. None
  34. Custom Test Rules How does this help us? • Reproducible

    • Record and store data • Device farm
  35. Custom Test Rules Does it solve our problem?

  36. Custom Test Rules Does it solve our problem? • Needs

    reliable UI test
  37. Custom Test Rules Does it solve our problem? • Needs

    reliable UI test • Takes a lot of time
  38. Custom Test Rules Does it solve our problem? • Takes

    a lot of time • Needs reliable UI test • dumpsys output isn’t easily parsed
  39. None
  40. Custom Test Rules Does it solve our problem? • Takes

    a lot of time • Needs reliable UI test • dumpsys output isn’t easily parsed • Output data can be unreliable
  41. Production Telemetry How does this help us?

  42. Production Telemetry How does this help us? • Signal from

    “real world” usage
  43. Production Telemetry How does this help us? • Signal from

    “real world” usage • Larger data pool
  44. Production Telemetry How does this help us? • Signal from

    “real world” usage • Larger data pool • More devices
  45. Production Telemetry Does it solve our problem?

  46. Production Telemetry Does it solve our problem? • Production environment

  47. Production Telemetry Does it solve our problem? • Production environment

    • Overhead of metric itself
  48. Production Telemetry Does it solve our problem? • Production environment

    • Overhead of metric itself • Data consumption
  49. Production Telemetry Does it solve our problem? • Production environment

    • Overhead of metric itself • Data consumption • Data normalization
  50. Process

  51. Process Test suite

  52. Process Test suite @Rules Controlled routes Data generation

  53. Process Test suite @Rules Controlled routes Data generation Data processing

  54. Process Test suite @Rules Controlled routes Data generation Data processing

    Retrieve data Parse data Send data
  55. Process Test suite @Rules Controlled routes Data generation Data processing

    Retrieve data Parse data Send data Data analysis
  56. Process Test suite @Rules Controlled routes Data generation Data processing

    Retrieve data Parse data Send data Data analysis Query data Filter out data Present results
  57. None
  58. Pipeline

  59. Pipeline Preparation

  60. Pipeline Preparation Trigger CI Pull SDK code Build SDK Assemble

    apps
  61. Pipeline Preparation Trigger CI Pull SDK code Build SDK AWS

    Device Farm Assemble apps
  62. Pipeline Preparation Trigger CI Pull SDK code Build SDK AWS

    Device Farm Assemble apps Upload apps Run tests Download artifacts Completed check
  63. Pipeline Preparation Trigger CI Pull SDK code Build SDK AWS

    Device Farm Assemble apps Upload apps Run tests Download artifacts Completed check Scripts
  64. Pipeline Preparation Trigger CI Pull SDK code Build SDK AWS

    Device Farm Assemble apps Upload apps Run tests Download artifacts Parse data Publish data Completed check Scripts
  65. Pipeline Preparation Trigger CI Pull SDK code Build SDK AWS

    Device Farm Assemble apps Upload apps Run tests Mode Download artifacts Parse data Publish data Completed check Scripts
  66. Pipeline Preparation Trigger CI Pull SDK code Build SDK AWS

    Device Farm Assemble apps Upload apps Run tests Mode Download artifacts Parse data Publish data Completed check SQL (Presto) Build reports Scripts
  67. What’s next?

  68. What’s next? • More automation

  69. What’s next? • More automation • Triggers / Thresholds

  70. What’s next? • More automation • Triggers / Thresholds •

    Feature flags
  71. Performance Takeaways

  72. Performance Takeaways • Important

  73. Performance Takeaways • Important • Tricky

  74. Performance Takeaways • Important • Tricky • On-going effort

  75. Performance Takeaways • Important • Tricky • On-going effort •

    Good amount of data
  76. Questions?

  77. We’re hiring!

  78. Thanks! Pablo Guardiola @Guardiola31337