Upgrade to Pro — share decks privately, control downloads, hide ads and more …

From 0 to 100: Fine-tune all the things!

From 0 to 100: Fine-tune all the things!

In this session we will talk about all the performance bottlenecks that we found along the way developing the Mapbox Navigation SDK for Android. We will learn how we monitored, made decisions based on numbers and tackled them: from designing and implementing a pipeline that runs automated tests, to selecting and analyzing the data obtained so issues can be detected and after evaluating their impact fixed. We will show the tools and platforms used, such as Systrace, test @Rules and AWS Device farm.

When you are developing SDKs or libraries, their performance is crucial to your customers, especially in the context of turn-by-turn navigation. The Mapbox Navigation SDK is built on top of different components and services and contains logic needed to get timed navigation instructions. To make the calculations for these instructions, the SDK compares the user's current location to the current route being traveled along.

The entire navigation experience includes voice announcements, real-time user progress to their destination, smooth map camera animations, and detecting when a user goes off-route. As you can imagine, all of these are critical when building a navigation app but at the same time pretty heavy in terms of performance.

By the end of the talk, you will be obsessed about the performance of your application and itching to fine-tune everything!

Pablo Guardiola

August 14, 2019
Tweet

More Decks by Pablo Guardiola

Other Decks in Programming

Transcript

  1. From 0 to 100: Fine-tune all
    the things!
    Pablo Guardiola Mapbox
    @Guardiola31337

    View Slide

  2. View Slide

  3. Mapbox Navigation SDK

    View Slide

  4. Mapbox Navigation SDK
    ● Maps
    ● Location Services

    View Slide

  5. View Slide

  6. Mapbox Navigation SDK
    ● Maps
    ● Location Services
    ● Routing

    View Slide

  7. View Slide

  8. Mapbox Navigation SDK
    ● Maps
    ● Location Services
    ● Routing
    ● Processing

    View Slide

  9. View Slide

  10. Track performance
    over time as changes
    land in the SDK.

    View Slide

  11. Which performance areas?

    View Slide

  12. Which performance areas?
    ● CPU

    View Slide

  13. Which performance areas?
    ● CPU
    ● RAM

    View Slide

  14. Which performance areas?
    ● CPU
    ● RAM
    ● Bandwidth

    View Slide

  15. Which performance areas?
    ● CPU
    ● RAM
    ● Bandwidth
    ● Battery

    View Slide

  16. Android Studio
    Profiler
    How does this help us?

    View Slide

  17. Android Studio
    Profiler
    How does this help us?
    ● Real-time, readable data

    View Slide

  18. View Slide

  19. Android Studio
    Profiler
    How does this help us?
    ● Real-time, readable data
    ● Session recording

    View Slide

  20. Android Studio
    Profiler
    How does this help us?
    ● Real-time, readable data
    ● Session recording
    ● Detailed analysis

    View Slide

  21. Android Studio
    Profiler
    How does this help us?
    ● Real-time, readable data
    ● Session recording
    ● Detailed analysis
    ● Initial signal

    View Slide

  22. Android Studio
    Profiler
    Does it solve our problem?

    View Slide

  23. Android Studio
    Profiler
    Does it solve our problem?
    ● Cannot download or send any
    recorded data

    View Slide

  24. Android Studio
    Profiler
    Does it solve our problem?
    ● Cannot download or send any
    recorded data
    ● Android Studio resource
    consumption

    View Slide

  25. Android Studio
    Profiler
    Does it solve our problem?
    ● Cannot download or send any
    recorded data
    ● Android Studio resource
    consumption
    ● API 21+

    View Slide

  26. Custom Test
    Rules
    How does this help us?

    View Slide

  27. View Slide

  28. Custom Test
    Rules
    How does this help us?
    ● Reproducible

    View Slide

  29. View Slide

  30. Custom Test
    Rules
    How does this help us?
    ● Reproducible
    ● Record and store data

    View Slide

  31. View Slide

  32. View Slide

  33. View Slide

  34. Custom Test
    Rules
    How does this help us?
    ● Reproducible
    ● Record and store data
    ● Device farm

    View Slide

  35. Custom Test
    Rules
    Does it solve our problem?

    View Slide

  36. Custom Test
    Rules
    Does it solve our problem?
    ● Needs reliable UI test

    View Slide

  37. Custom Test
    Rules
    Does it solve our problem?
    ● Needs reliable UI test
    ● Takes a lot of time

    View Slide

  38. Custom Test
    Rules
    Does it solve our problem?
    ● Takes a lot of time
    ● Needs reliable UI test
    ● dumpsys output isn’t easily
    parsed

    View Slide

  39. View Slide

  40. Custom Test
    Rules
    Does it solve our problem?
    ● Takes a lot of time
    ● Needs reliable UI test
    ● dumpsys output isn’t easily
    parsed
    ● Output data can be unreliable

    View Slide

  41. Production
    Telemetry
    How does this help us?

    View Slide

  42. Production
    Telemetry
    How does this help us?
    ● Signal from “real world” usage

    View Slide

  43. Production
    Telemetry
    How does this help us?
    ● Signal from “real world” usage
    ● Larger data pool

    View Slide

  44. Production
    Telemetry
    How does this help us?
    ● Signal from “real world” usage
    ● Larger data pool
    ● More devices

    View Slide

  45. Production
    Telemetry
    Does it solve our problem?

    View Slide

  46. Production
    Telemetry
    Does it solve our problem?
    ● Production environment

    View Slide

  47. Production
    Telemetry
    Does it solve our problem?
    ● Production environment
    ● Overhead of metric itself

    View Slide

  48. Production
    Telemetry
    Does it solve our problem?
    ● Production environment
    ● Overhead of metric itself
    ● Data consumption

    View Slide

  49. Production
    Telemetry
    Does it solve our problem?
    ● Production environment
    ● Overhead of metric itself
    ● Data consumption
    ● Data normalization

    View Slide

  50. Process

    View Slide

  51. Process
    Test suite

    View Slide

  52. Process
    Test suite
    @Rules
    Controlled routes
    Data generation

    View Slide

  53. Process
    Test suite
    @Rules
    Controlled routes
    Data generation
    Data processing

    View Slide

  54. Process
    Test suite
    @Rules
    Controlled routes
    Data generation
    Data processing
    Retrieve data
    Parse data
    Send data

    View Slide

  55. Process
    Test suite
    @Rules
    Controlled routes
    Data generation
    Data processing
    Retrieve data
    Parse data
    Send data
    Data analysis

    View Slide

  56. Process
    Test suite
    @Rules
    Controlled routes
    Data generation
    Data processing
    Retrieve data
    Parse data
    Send data
    Data analysis
    Query data
    Filter out data
    Present results

    View Slide

  57. View Slide

  58. Pipeline

    View Slide

  59. Pipeline
    Preparation

    View Slide

  60. Pipeline
    Preparation
    Trigger CI
    Pull SDK code
    Build SDK
    Assemble apps

    View Slide

  61. Pipeline
    Preparation
    Trigger CI
    Pull SDK code
    Build SDK
    AWS Device Farm
    Assemble apps

    View Slide

  62. Pipeline
    Preparation
    Trigger CI
    Pull SDK code
    Build SDK
    AWS Device Farm
    Assemble apps
    Upload apps
    Run tests
    Download artifacts
    Completed check

    View Slide

  63. Pipeline
    Preparation
    Trigger CI
    Pull SDK code
    Build SDK
    AWS Device Farm
    Assemble apps
    Upload apps
    Run tests
    Download artifacts
    Completed check
    Scripts

    View Slide

  64. Pipeline
    Preparation
    Trigger CI
    Pull SDK code
    Build SDK
    AWS Device Farm
    Assemble apps
    Upload apps
    Run tests
    Download artifacts
    Parse data
    Publish data
    Completed check
    Scripts

    View Slide

  65. Pipeline
    Preparation
    Trigger CI
    Pull SDK code
    Build SDK
    AWS Device Farm
    Assemble apps
    Upload apps
    Run tests
    Mode
    Download artifacts
    Parse data
    Publish data
    Completed check
    Scripts

    View Slide

  66. Pipeline
    Preparation
    Trigger CI
    Pull SDK code
    Build SDK
    AWS Device Farm
    Assemble apps
    Upload apps
    Run tests
    Mode
    Download artifacts
    Parse data
    Publish data
    Completed check
    SQL (Presto)
    Build reports
    Scripts

    View Slide

  67. What’s next?

    View Slide

  68. What’s next?
    ● More automation

    View Slide

  69. What’s next?
    ● More automation
    ● Triggers / Thresholds

    View Slide

  70. What’s next?
    ● More automation
    ● Triggers / Thresholds
    ● Feature flags

    View Slide

  71. Performance Takeaways

    View Slide

  72. Performance Takeaways
    ● Important

    View Slide

  73. Performance Takeaways
    ● Important
    ● Tricky

    View Slide

  74. Performance Takeaways
    ● Important
    ● Tricky
    ● On-going effort

    View Slide

  75. Performance Takeaways
    ● Important
    ● Tricky
    ● On-going effort
    ● Good amount of data

    View Slide

  76. Questions?

    View Slide

  77. We’re hiring!

    View Slide

  78. Thanks!
    Pablo Guardiola
    @Guardiola31337

    View Slide