Mobile Ops: Measuring Performance and Automating Workflow

Slide 1

Slide 1 text

MOBILE Clay Smith, New Relic Mobile+Web Developer Conference OPS 4.27.2016

Slide 2

Slide 2 text

"OH MOBILE OPS? THE THING PEOPLE DON'T DO?" - Anonymous Software Engineer Good background from @patrickdubois: http://www.jedi.be/blog/2016/01/28/Mobile %20Delivery%20with%20a%20devops%20mindset/

Slide 3

Slide 3 text

"Very few [businesses] are well prepared to successfully deploy and maintain apps." -Accenture Mobile Survey https://adtmag.com/articles/2015/09/30/accenture-mobile-survey.aspx

Slide 4

Slide 4 text

HOW DO WE GET THERE?

Slide 5

Slide 5 text

MOBILE IS _________.

Slide 6

Slide 6 text

MOBILE IS _________. the future

Slide 7

Slide 7 text

MOBILE IS _________. vendor controlled

Slide 8

Slide 8 text

MOBILE IS _________. constrained

Slide 9

Slide 9 text

MOBILE IS _________. complicated

Slide 10

Slide 10 text

DEPLOYMENT LIFECYCLE TESTING BUILDING DESIGNING DEFINING PLANNING

Slide 11

Slide 11 text

DEPLOYMENT HARD PART TESTING BUILDING DESIGNING DEFINING PLANNING

Slide 12

Slide 12 text

#1 DEV PAIN? According to Spotify Internal Survey at Mobile Delivery Days 2016. BUILDS

Slide 13

Slide 13 text

SLOW GRADLE? ./gradlew assembleDebug --profile http://kevinpelgrims.com/blog/2015/06/11/speeding-up-your-gradle-builds/

Slide 14

Slide 14 text

LET'S TALK AUTOMATION Automate the painful, error-prone things.

Slide 15

Slide 15 text

CONTINOUS INTEGRATION Many, many options—Jenkins and beyond. https://gist.github.com/rmoriz/5296881

Slide 16

Slide 16 text

RELEASE TRAIN (GIT) https://www.natashatherobot.com/train-facebooks-model-for-mobile-project-management/ Master (in Play Store) Train/vNext Feature A

Slide 17

Slide 17 text

SOME CI ADVICE Automatically build and test on push Explore distributing beta builds Integrate with Slack/HipChat/etc Automatic Play Store release... maybe* http://simpleprogrammer.com/2009/12/30/continuous-integration-best-practices/ * https://wiki.jenkins-ci.org/display/JENKINS/Google+Play+Android+Publisher+Plugin https://github.com/Triple-T/gradle-play-publisher

Slide 18

Slide 18 text

DEVICE TESTING Cloud vs Self Hosted?

Slide 19

Slide 19 text

BUILDING IT YOURSELF? ‣ Consider http://square.github.io/spoon/ ‣ Run tests on multiple devices in parallel + HTML Summary ‣ Pros: Device Labs look super cool ‣ Cons: Device labs are hard to maintain

Slide 20

Slide 20 text

BECOME A FARMER? ‣ Many choices (search "mobile device testing") ‣ Ex: https://aws.amazon.com/device-farm/ ‣ Can run tests or fuzz ‣ Most have build system integration + APIs (./gradlew devicefarmUpload)

Slide 21

Slide 21 text

SPEED IS A Automating performance in Android FEATURE

Slide 22

Slide 22 text

CASE STUDY #1 A very large social network in Silicon Valley

Slide 23

Slide 23 text

WHAT THEY DID ‣ Desire to avoid performance regressions ‣ Belief in the ability to detect performance issues before release by using advanced techniques and automation ("the lab") ‣ Advanced distributed system using zookeeper and custom racks of devices https://code.facebook.com/posts/924676474230092/mobile-performance-tooling-infrastructure-at-facebook/

Slide 24

Slide 24 text

CASE STUDY #2 Doing it with New Relic + TravisCI + GitHub + AWS Device Farm https://github.com/smithclay/AndroidPerfTestExample

Slide 25

Slide 25 text

TRAVISCI GITHUB AWS DEVICE FARM NEW RELIC INSIGHTS ♥ SASS TravisCI runs on commits and uploads the latest build to AWS Device Farm, where performance analytics are collected from the app with New Relic. Build Test Collect

Slide 26

Slide 26 text

TRAVIS CI

Slide 27

Slide 27 text

AWS DEVICE FARM

Slide 28

Slide 28 text

HI, PERF DATA https://newrelic.com/insights

Slide 29

Slide 29 text

LAB ≠ REAL WORLD

Slide 30

Slide 30 text

MEASURE _________. real users

Slide 31

Slide 31 text

MEASURE _________. KPIs

Slide 32

Slide 32 text

SOME MOBILE KPIS ‣ Network requests (including failed) ‣ Startup time ‣ Blocked UI time ‣ Crashes, other errors ‣ Critical user interactions ‣ 3rd party services—what are they doing? https://medium.com/@mandrigin/ios-app-performance-instruments-beyond-48fe7b7cdf2

Slide 33

Slide 33 text

DEPLOYMENT COLLECT DATA HERE TESTING BUILDING DESIGNING DEFINING PLANNING

Slide 34

Slide 34 text

DEPLOYMENT AUTOMATE W/ CI HERE TESTING BUILDING DESIGNING DEFINING PLANNING

Slide 35

Slide 35 text

WHY? Users don't care how you release apps, but they want them to work and be fast.

Slide 36

Slide 36 text

THANK Slides posted shortly. @smithclay newrelic.com/mobile YOU 4.27.2016