Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Mobile Ops: Measuring Performance and Automating Workflow

Mobile Ops: Measuring Performance and Automating Workflow

Clay Smith

April 27, 2016
Tweet

More Decks by Clay Smith

Other Decks in Programming

Transcript

  1. MOBILE
    Clay Smith, New Relic
    Mobile+Web Developer Conference
    OPS
    4.27.2016

    View full-size slide

  2. "OH MOBILE OPS? THE
    THING PEOPLE DON'T DO?"
    - Anonymous Software Engineer
    Good background from @patrickdubois: http://www.jedi.be/blog/2016/01/28/Mobile
    %20Delivery%20with%20a%20devops%20mindset/

    View full-size slide

  3. "Very few [businesses] are
    well prepared to successfully
    deploy and maintain apps."
    -Accenture Mobile Survey
    https://adtmag.com/articles/2015/09/30/accenture-mobile-survey.aspx

    View full-size slide

  4. HOW DO WE
    GET THERE?

    View full-size slide

  5. MOBILE IS
    _________.

    View full-size slide

  6. MOBILE IS
    _________.
    the future

    View full-size slide

  7. MOBILE IS
    _________.
    vendor controlled

    View full-size slide

  8. MOBILE IS
    _________.
    constrained

    View full-size slide

  9. MOBILE IS
    _________.
    complicated

    View full-size slide

  10. DEPLOYMENT
    LIFECYCLE
    TESTING
    BUILDING
    DESIGNING
    DEFINING
    PLANNING

    View full-size slide

  11. DEPLOYMENT
    HARD PART
    TESTING
    BUILDING
    DESIGNING
    DEFINING
    PLANNING

    View full-size slide

  12. #1 DEV PAIN?
    According to Spotify Internal Survey at
    Mobile Delivery Days 2016.
    BUILDS

    View full-size slide

  13. SLOW GRADLE?
    ./gradlew assembleDebug --profile
    http://kevinpelgrims.com/blog/2015/06/11/speeding-up-your-gradle-builds/

    View full-size slide

  14. LET'S TALK
    AUTOMATION
    Automate the painful, error-prone things.

    View full-size slide

  15. CONTINOUS
    INTEGRATION
    Many, many options—Jenkins and beyond.
    https://gist.github.com/rmoriz/5296881

    View full-size slide

  16. RELEASE
    TRAIN (GIT)
    https://www.natashatherobot.com/train-facebooks-model-for-mobile-project-management/
    Master (in Play Store)
    Train/vNext
    Feature A

    View full-size slide

  17. SOME CI
    ADVICE
    Automatically build and test on push
    Explore distributing beta builds
    Integrate with Slack/HipChat/etc
    Automatic Play Store release... maybe*
    http://simpleprogrammer.com/2009/12/30/continuous-integration-best-practices/
    * https://wiki.jenkins-ci.org/display/JENKINS/Google+Play+Android+Publisher+Plugin
    https://github.com/Triple-T/gradle-play-publisher

    View full-size slide

  18. DEVICE
    TESTING
    Cloud vs Self Hosted?

    View full-size slide

  19. BUILDING IT YOURSELF?
    ‣ Consider http://square.github.io/spoon/
    ‣ Run tests on multiple devices in parallel +
    HTML Summary
    ‣ Pros: Device Labs look super cool
    ‣ Cons: Device labs are hard to maintain

    View full-size slide

  20. BECOME A FARMER?
    ‣ Many choices (search "mobile device
    testing")
    ‣ Ex: https://aws.amazon.com/device-farm/
    ‣ Can run tests or fuzz
    ‣ Most have build system integration + APIs
    (./gradlew devicefarmUpload)

    View full-size slide

  21. SPEED IS A
    Automating performance in Android
    FEATURE

    View full-size slide

  22. CASE STUDY
    #1
    A very large social network in Silicon Valley

    View full-size slide

  23. WHAT THEY DID
    ‣ Desire to avoid performance regressions
    ‣ Belief in the ability to detect performance
    issues before release by using advanced
    techniques and automation ("the lab")
    ‣ Advanced distributed system using
    zookeeper and custom racks of devices
    https://code.facebook.com/posts/924676474230092/mobile-performance-tooling-infrastructure-at-facebook/

    View full-size slide

  24. CASE STUDY
    #2
    Doing it with New Relic + TravisCI +
    GitHub + AWS Device Farm
    https://github.com/smithclay/AndroidPerfTestExample

    View full-size slide

  25. TRAVISCI
    GITHUB
    AWS DEVICE FARM
    NEW RELIC INSIGHTS
    ♥ SASS
    TravisCI runs on commits and
    uploads the latest build to
    AWS Device Farm,
    where performance analytics
    are collected from the app
    with New Relic.
    Build
    Test
    Collect

    View full-size slide

  26. AWS DEVICE FARM

    View full-size slide

  27. HI, PERF DATA
    https://newrelic.com/insights

    View full-size slide

  28. LAB ≠
    REAL WORLD

    View full-size slide

  29. MEASURE
    _________.
    real users

    View full-size slide

  30. MEASURE
    _________.
    KPIs

    View full-size slide

  31. SOME MOBILE KPIS
    ‣ Network requests (including failed)
    ‣ Startup time
    ‣ Blocked UI time
    ‣ Crashes, other errors
    ‣ Critical user interactions
    ‣ 3rd party services—what are they doing?
    https://medium.com/@mandrigin/ios-app-performance-instruments-beyond-48fe7b7cdf2

    View full-size slide

  32. DEPLOYMENT
    COLLECT DATA HERE
    TESTING
    BUILDING
    DESIGNING
    DEFINING
    PLANNING

    View full-size slide

  33. DEPLOYMENT
    AUTOMATE W/ CI HERE
    TESTING
    BUILDING
    DESIGNING
    DEFINING
    PLANNING

    View full-size slide

  34. WHY?
    Users don't care how you release apps,
    but they want them to work and be fast.

    View full-size slide

  35. THANK
    Slides posted shortly. @smithclay
    newrelic.com/mobile
    YOU
    4.27.2016

    View full-size slide