Mobile Ops: Measuring Performance and Automating Workflow

Mobile Ops: Measuring Performance and Automating Workflow

2ad55a66b8325eb097612c069ab76231?s=128

Clay Smith

April 27, 2016
Tweet

Transcript

  1. MOBILE Clay Smith, New Relic Mobile+Web Developer Conference OPS 4.27.2016

  2. "OH MOBILE OPS? THE THING PEOPLE DON'T DO?" - Anonymous

    Software Engineer Good background from @patrickdubois: http://www.jedi.be/blog/2016/01/28/Mobile %20Delivery%20with%20a%20devops%20mindset/
  3. "Very few [businesses] are well prepared to successfully deploy and

    maintain apps." -Accenture Mobile Survey https://adtmag.com/articles/2015/09/30/accenture-mobile-survey.aspx
  4. HOW DO WE GET THERE?

  5. MOBILE IS _________.

  6. MOBILE IS _________. the future

  7. MOBILE IS _________. vendor controlled

  8. MOBILE IS _________. constrained

  9. MOBILE IS _________. complicated

  10. DEPLOYMENT LIFECYCLE TESTING BUILDING DESIGNING DEFINING PLANNING

  11. DEPLOYMENT HARD PART TESTING BUILDING DESIGNING DEFINING PLANNING

  12. #1 DEV PAIN? According to Spotify Internal Survey at Mobile

    Delivery Days 2016. BUILDS
  13. SLOW GRADLE? ./gradlew assembleDebug --profile http://kevinpelgrims.com/blog/2015/06/11/speeding-up-your-gradle-builds/

  14. LET'S TALK AUTOMATION Automate the painful, error-prone things.

  15. CONTINOUS INTEGRATION Many, many options—Jenkins and beyond. https://gist.github.com/rmoriz/5296881

  16. RELEASE TRAIN (GIT) https://www.natashatherobot.com/train-facebooks-model-for-mobile-project-management/ Master (in Play Store) Train/vNext Feature

    A
  17. SOME CI ADVICE Automatically build and test on push Explore

    distributing beta builds Integrate with Slack/HipChat/etc Automatic Play Store release... maybe* http://simpleprogrammer.com/2009/12/30/continuous-integration-best-practices/ * https://wiki.jenkins-ci.org/display/JENKINS/Google+Play+Android+Publisher+Plugin https://github.com/Triple-T/gradle-play-publisher
  18. DEVICE TESTING Cloud vs Self Hosted?

  19. BUILDING IT YOURSELF? ‣ Consider http://square.github.io/spoon/ ‣ Run tests on

    multiple devices in parallel + HTML Summary ‣ Pros: Device Labs look super cool ‣ Cons: Device labs are hard to maintain
  20. BECOME A FARMER? ‣ Many choices (search "mobile device testing")

    ‣ Ex: https://aws.amazon.com/device-farm/ ‣ Can run tests or fuzz ‣ Most have build system integration + APIs (./gradlew devicefarmUpload)
  21. SPEED IS A Automating performance in Android FEATURE

  22. CASE STUDY #1 A very large social network in Silicon

    Valley
  23. WHAT THEY DID ‣ Desire to avoid performance regressions ‣

    Belief in the ability to detect performance issues before release by using advanced techniques and automation ("the lab") ‣ Advanced distributed system using zookeeper and custom racks of devices https://code.facebook.com/posts/924676474230092/mobile-performance-tooling-infrastructure-at-facebook/
  24. CASE STUDY #2 Doing it with New Relic + TravisCI

    + GitHub + AWS Device Farm https://github.com/smithclay/AndroidPerfTestExample
  25. TRAVISCI GITHUB AWS DEVICE FARM NEW RELIC INSIGHTS ♥ SASS

    TravisCI runs on commits and uploads the latest build to AWS Device Farm, where performance analytics are collected from the app with New Relic. Build Test Collect
  26. TRAVIS CI

  27. AWS DEVICE FARM

  28. HI, PERF DATA https://newrelic.com/insights

  29. LAB ≠ REAL WORLD

  30. MEASURE _________. real users

  31. MEASURE _________. KPIs

  32. SOME MOBILE KPIS ‣ Network requests (including failed) ‣ Startup

    time ‣ Blocked UI time ‣ Crashes, other errors ‣ Critical user interactions ‣ 3rd party services—what are they doing? https://medium.com/@mandrigin/ios-app-performance-instruments-beyond-48fe7b7cdf2
  33. DEPLOYMENT COLLECT DATA HERE TESTING BUILDING DESIGNING DEFINING PLANNING

  34. DEPLOYMENT AUTOMATE W/ CI HERE TESTING BUILDING DESIGNING DEFINING PLANNING

  35. WHY? Users don't care how you release apps, but they

    want them to work and be fast.
  36. THANK Slides posted shortly. @smithclay newrelic.com/mobile YOU 4.27.2016