Mobile Ops: Measuring Performance and Automating Workflow

Transcript

1. MOBILE
   Clay Smith, New Relic
   Mobile+Web Developer Conference
   OPS
   4.27.2016
   

   View Slide

2. "OH MOBILE OPS? THE
   THING PEOPLE DON'T DO?"
   - Anonymous Software Engineer
   Good background from @patrickdubois: http://www.jedi.be/blog/2016/01/28/Mobile
   %20Delivery%20with%20a%20devops%20mindset/
   

   View Slide

3. "Very few [businesses] are
   well prepared to successfully
   deploy and maintain apps."
   -Accenture Mobile Survey
   https://adtmag.com/articles/2015/09/30/accenture-mobile-survey.aspx
   

   View Slide

4. HOW DO WE
   GET THERE?
   

   View Slide

5. MOBILE IS
   _________.
   

   View Slide

6. MOBILE IS
   _________.
   the future
   

   View Slide

7. MOBILE IS
   _________.
   vendor controlled
   

   View Slide

8. MOBILE IS
   _________.
   constrained
   

   View Slide

9. MOBILE IS
   _________.
   complicated
   

   View Slide

10. DEPLOYMENT
    LIFECYCLE
    TESTING
    BUILDING
    DESIGNING
    DEFINING
    PLANNING
    

    View Slide

11. DEPLOYMENT
    HARD PART
    TESTING
    BUILDING
    DESIGNING
    DEFINING
    PLANNING
    

    View Slide

12. #1 DEV PAIN?
    According to Spotify Internal Survey at
    Mobile Delivery Days 2016.
    BUILDS
    

    View Slide

13. SLOW GRADLE?
    ./gradlew assembleDebug --profile
    http://kevinpelgrims.com/blog/2015/06/11/speeding-up-your-gradle-builds/
    

    View Slide

14. LET'S TALK
    AUTOMATION
    Automate the painful, error-prone things.
    

    View Slide

15. CONTINOUS
    INTEGRATION
    Many, many options—Jenkins and beyond.
    https://gist.github.com/rmoriz/5296881
    

    View Slide

16. RELEASE
    TRAIN (GIT)
    https://www.natashatherobot.com/train-facebooks-model-for-mobile-project-management/
    Master (in Play Store)
    Train/vNext
    Feature A
    

    View Slide

17. SOME CI
    ADVICE
    Automatically build and test on push
    Explore distributing beta builds
    Integrate with Slack/HipChat/etc
    Automatic Play Store release... maybe*
    http://simpleprogrammer.com/2009/12/30/continuous-integration-best-practices/
    * https://wiki.jenkins-ci.org/display/JENKINS/Google+Play+Android+Publisher+Plugin
    https://github.com/Triple-T/gradle-play-publisher
    

    View Slide

18. DEVICE
    TESTING
    Cloud vs Self Hosted?
    

    View Slide

19. BUILDING IT YOURSELF?
    ‣ Consider http://square.github.io/spoon/
    ‣ Run tests on multiple devices in parallel +
    HTML Summary
    ‣ Pros: Device Labs look super cool
    ‣ Cons: Device labs are hard to maintain
    

    View Slide

20. BECOME A FARMER?
    ‣ Many choices (search "mobile device
    testing")
    ‣ Ex: https://aws.amazon.com/device-farm/
    ‣ Can run tests or fuzz
    ‣ Most have build system integration + APIs
    (./gradlew devicefarmUpload)
    

    View Slide

21. SPEED IS A
    Automating performance in Android
    FEATURE
    

    View Slide

22. CASE STUDY
    #1
    A very large social network in Silicon Valley
    

    View Slide

23. WHAT THEY DID
    ‣ Desire to avoid performance regressions
    ‣ Belief in the ability to detect performance
    issues before release by using advanced
    techniques and automation ("the lab")
    ‣ Advanced distributed system using
    zookeeper and custom racks of devices
    https://code.facebook.com/posts/924676474230092/mobile-performance-tooling-infrastructure-at-facebook/
    

    View Slide

24. CASE STUDY
    #2
    Doing it with New Relic + TravisCI +
    GitHub + AWS Device Farm
    https://github.com/smithclay/AndroidPerfTestExample
    

    View Slide

25. TRAVISCI
    GITHUB
    AWS DEVICE FARM
    NEW RELIC INSIGHTS
    ♥ SASS
    TravisCI runs on commits and
    uploads the latest build to
    AWS Device Farm,
    where performance analytics
    are collected from the app
    with New Relic.
    Build
    Test
    Collect
    

    View Slide

26. TRAVIS CI
    

    View Slide

27. AWS DEVICE FARM
    

    View Slide

28. HI, PERF DATA
    https://newrelic.com/insights
    

    View Slide

29. LAB ≠
    REAL WORLD
    

    View Slide

30. MEASURE
    _________.
    real users
    

    View Slide

31. MEASURE
    _________.
    KPIs
    

    View Slide

32. SOME MOBILE KPIS
    ‣ Network requests (including failed)
    ‣ Startup time
    ‣ Blocked UI time
    ‣ Crashes, other errors
    ‣ Critical user interactions
    ‣ 3rd party services—what are they doing?
    https://medium.com/@mandrigin/ios-app-performance-instruments-beyond-48fe7b7cdf2
    

    View Slide

33. DEPLOYMENT
    COLLECT DATA HERE
    TESTING
    BUILDING
    DESIGNING
    DEFINING
    PLANNING
    

    View Slide

34. DEPLOYMENT
    AUTOMATE W/ CI HERE
    TESTING
    BUILDING
    DESIGNING
    DEFINING
    PLANNING
    

    View Slide

35. WHY?
    Users don't care how you release apps,
    but they want them to work and be fast.
    

    View Slide

36. THANK
    Slides posted shortly. @smithclay
    newrelic.com/mobile
    YOU
    4.27.2016
    

    View Slide