Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Continuous Integration for Android Using Jenkins

Continuous Integration for Android Using Jenkins

This talk explains how Jenkins can be used to improve product quality, stability and delivery speed by embracing continuous integration. We’ll walk through how Android developers and testers can benefit from using Jenkins, and then give some pro tips for more advanced capabilities, including how to speed up your tests by running them in parallel across multiple Jenkins slaves.

It was presented at the Android Summit 2016: http://androidsummit.org/

VIDEO: https://www.youtube.com/watch?v=wNXJi75WHcA

Dan Jarvis: http://careers.stackoverflow.com/dj
Sam Edwards: https://twitter.com/handstandsam

Sam Edwards

August 26, 2016
Tweet

More Decks by Sam Edwards

Other Decks in Programming

Transcript

  1. CONTINUOUS
    INTEGRATION
    FOR ANDROID
    USING JENKINS

    View Slide

  2. 05
    About Us – Dan Jarvis & Sam Edwards
    • Tech Leads for Capital One Wallet on Android
    • Top rated finance app by a US bank
    • First US bank with native Android contactless payments
    • Each has 12+ years professional experience
    • Passionate about automation and knowledge sharing
    @HandstandSam

    View Slide

  3. • Continuous Integration (CI) fundamentals
    • Productivity tips
    • Capital One Wallet case study
    • Jenkins as a platform
    • Pro tips - how to run tests in parallel
    05
    Agenda

    View Slide

  4. CONTINUOUS
    INTEGRATION
    FUNDAMENTALS

    View Slide

  5. What is CI?

    View Slide

  6. Continuous Integration (CI) is a development practice
    that requires developers to integrate code into a shared
    repository several times a day. Each check-in is then
    verified by an automated build, allowing teams to
    detect problems early.

    View Slide

  7. Why do CI?

    View Slide

  8. View Slide

  9. Why?
    • Deliver faster and higher quality
    • Your product is always ready to deploy
    • Capture institutional knowledge in tests
    • Automation hugely improves productivity
    • Multiply your impact!

    View Slide

  10. What is Jenkins?

    View Slide

  11. A server that executes jobs
    based on triggers.

    View Slide

  12. Jenkins Terminology
    •Jobs
    •Plugins
    •Master, slaves and executors

    View Slide

  13. Jenkins Jobs – “A Job is essentially a script”

    View Slide

  14. Jenkins Jobs - Configuration

    View Slide

  15. Jenkins Jobs – Execution & History

    View Slide

  16. View Slide

  17. View Slide

  18. Jenkins Plugins
    • Plugins can be written to extend Jenkins
    capabilities
    Default Install Plugins Added

    View Slide

  19. 05
    Android Emulator Plugin

    View Slide

  20. Jenkins Master, Slaves and Executors
    • Jobs are configured on the master node
    • The master dispatches jobs to run on
    executors on slave nodes (which could
    be on the same machine as the master)

    View Slide

  21. What Makes Jenkins so Great?
    • Easy, installation, configuration & a web based interface
    • Open source https://github.com/jenkinsci/jenkins
    • >1000 plugins
    • Distributed and scalable
    • Long history and widespread adoption
    • Enterprise integrations

    View Slide

  22. PRODUCTIVITY
    TIPS

    View Slide

  23. Real-time Notifications

    View Slide

  24. View Slide

  25. View Slide

  26. Spoon - https://github.com/square/spoon

    View Slide

  27. View Slide

  28. View Slide

  29. View Slide

  30. Screenshots on Assertion Failure
    Spoon.screenshot(activity,"test_failure");

    View Slide

  31. Spoon & Screenshots

    View Slide

  32. More on Spoon, Espresso and Screenshots
    http://droidcon.nyc/
    http://www.devfestdc.org/

    View Slide

  33. How to Run Spoon

    View Slide

  34. Custom Links to Spoon Results

    View Slide

  35. Running Specific Instrumentation Tests (docs)
    Run By Instrumentation Argument(s) Code
    Method -e class com.FooTest#test1
    Class -e class com.FooTest
    Standard
    Annotations
    -e size small
    -e size medium
    -e size large
    @SmallTest
    @MediumTest
    @LargeTest
    Custom
    Annotations
    -e annotation
    -e notAnnotation
    @MyCustomAnnotation
    Package –e package

    View Slide

  36. Android Instrumentation Runner (docs)

    View Slide

  37. Spoon Arguments

    View Slide

  38. CASE STUDY
    WALLET

    View Slide

  39. View Slide

  40. The Testing Pyramid - Wallet Examples
    • Android Espresso Tests, Real Servers
    • Android Tests, Real Servers
    • JVM Tests, Mock Data
    • Android Component Tests, Mock Data
    Black
    Box Tests
    Integration Tests
    Component Tests
    Unit Tests

    View Slide

  41. Auto-run Tests for Every PR

    View Slide

  42. GitHub Pull Request Builder Plugin

    View Slide

  43. Capital One Wallet for Android on Jenkins Today
    Git Commit
    Pull Request (PR)
    Opened or Updated
    Unit Tests on PR Branch ✓
    PR Merged
    Assemble APKs
    and Test APKs
    Integration Tests
    Unit Tests
    Component Tests
    Blackbox Tests
    ✓ ✓




    View Slide

  44. JENKINS
    AS A PLATFORM

    View Slide

  45. Linked Jobs & Triggers

    View Slide

  46. Newbie – One Job
    Compile/Build
    Integration Tests
    Unit Tests
    Component Tests
    Blackbox Tests

    View Slide

  47. Better – Four Jobs
    Run Component Tests
    Run Unit Tests
    Compile/Build
    Compile/Build Compile/Build
    Run Blackbox Tests
    Run Integration Tests
    Compile/Build

    View Slide

  48. Better – Four Jobs
    Run Component Tests
    Run Unit Tests
    Compile/Build
    Compile/Build Compile/Build
    Run Blackbox Tests
    Run Integration Tests
    Compile/Build
    Limitations:
    • Not great if you want to repeatedly re-run the tests
    • Wastes time/money if you have less than four slaves/executors

    View Slide

  49. Better – Five Jobs
    Run Component Tests
    Run Unit Tests
    Compile/Build
    Run Blackbox Tests
    Run Integration Tests

    View Slide

  50. Configuration - Triggering

    View Slide

  51. Better – Five Jobs
    Run Component Tests
    Run Unit Tests
    Compile/Build
    Run Blackbox Tests
    Run Integration Tests

    View Slide

  52. Configuration – Copy Build Artifacts From Other Job

    View Slide

  53. Continuous Delivery

    View Slide

  54. Building a Release Candidate
    Compile/Build

    Manual Testing
    and Upload
    Alpha

    View Slide

  55. What if I told you every build
    that passes your automated tests
    is a release candidate?

    View Slide

  56. Continuous Delivery for Clients
    Compile/Build
    Integration Tests
    Unit Tests
    Component Tests
    Blackbox Tests





    Alpha

    View Slide

  57. Continuous Delivery for All

    View Slide

  58. Server
    Unit Tests
    Continuous Delivery for Servers


    Server
    Integration Tests

    View Slide

  59. Your automated client tests are
    now an API.

    View Slide

  60. iOS Stable
    Integration Tests
    Server
    Unit Tests
    Continuous Delivery for Servers

    Server
    Integration Tests
    Android Stable
    Integration Tests




    View Slide

  61. Continuous Delivery for Servers
    1. Server developer runs unit tests locally

    View Slide

  62. Continuous Delivery for Servers
    1. Server developer runs unit tests locally
    2. On pull request, unit tests and all server and client integration tests run
    on a transient server instance

    View Slide

  63. Continuous Delivery for Servers
    1. Server developer runs unit tests locally
    2. On pull request, unit tests and all server and client integration tests run
    on a transient server instance
    3. On merge, server code gets automatically deployed to shared QA
    region

    View Slide

  64. Pipeline Views

    View Slide

  65. Pipeline
    Jenkins Pipelines: https://jenkins.io/doc/pipeline/

    View Slide

  66. View Slide

  67. Manage Scripts in Separate Repository
    Benefits:
    • Versioned with great tooling to view history

    View Slide

  68. Manage Scripts in Separate Repository
    Benefits:
    • Versioned with great tooling to view history
    • Great when you have to update multiple or similar scripts like a library
    version number

    View Slide

  69. Manage Scripts in Separate Repository
    Benefits:
    • Versioned with great tooling to view history
    • Great when you have to update multiple or similar scripts like a library
    version number
    • Great when re-building Jenkins

    View Slide

  70. Manage Scripts in Separate Repository
    Benefits:
    • Versioned with great tooling to view history
    • Great when you have to update multiple or similar scripts like a library
    version number
    • Great when re-building Jenkins

    View Slide

  71. PRO TIPS
    RUNNING TESTS
    IN PARALLEL

    View Slide

  72. Recap
    Run Component Tests
    Run Unit Tests
    Compile/Build
    Run Blackbox Tests
    Run Integration Tests

    View Slide

  73. • Split them across devices
    and/or executors
    …or both!
    05
    Parallel Tests

    View Slide

  74. Sharding Your Tests
    Run Component Tests
    Run Unit Tests
    Compile/Build
    Run Blackbox Tests
    Run Integration Tests
    Blackbox Tests [0]
    Blackbox Tests [1]
    Blackbox Tests [2]
    Blackbox Tests [3]

    View Slide

  75. Parallel Tests Attempt #1

    View Slide

  76. View Slide

  77. View Slide

  78. • The main job uses up an executor while waiting for Phases to
    complete (which each use their own executor)
    • Phases need to do their own partitioning of the test suite
    • Test results are not aggregated back into a single report
    05
    Problems with MultiJob Plugin

    View Slide

  79. 05
    Android Instrumentation Sharding (docs)

    View Slide

  80. Matrix Project Plugin

    View Slide

  81. View Slide

  82. 05
    Configuring The Matrix

    View Slide

  83. 05
    Tell The Job Which Test Shard To Run

    View Slide

  84. • Easier to pass sharding index to build executors
    • Starts parallel jobs for all combinations of configurations
    05
    Benefits of Matrix Plugin

    View Slide

  85. Supercharging Your Matrix

    View Slide

  86. 05
    Android Emulator Plugin
    • Built in support for
    Multi-configuration (matrix)
    jobs!

    View Slide

  87. 05
    Android Emulator Plugin

    View Slide

  88. THE
    END

    View Slide

  89. ENCORE

    View Slide

  90. 05
    Android Emulator
    PROS
    • Multiple screen sizes are easy
    • FREE!
    • Fast (10x faster install speed then normal devices)
    CONS
    • “Snapshot” feature is unreliable
    • Workaround: Re-create the AVD every time

    View Slide

  91. 05
    Genymotion Emulator
    PROS
    • Multiple screen sizes are easy
    • Fast
    • Snapshot feature works pretty well
    CONS
    • Requires Licenses
    • Can be difficult to manage with sharding

    View Slide

  92. 05
    Real Devices
    PROS
    • They are REAL Devices
    • There are things like Google Test Cloud, Test Droid and Amazon
    Device Farm
    CONS
    • You have to buy all the devices & hook them up
    • OR you have to pay a lot for cloud based use of devices which
    can be slow

    View Slide

  93. 05
    Jenkins Architecture

    View Slide

  94. THAT’S ALL FOLKS

    View Slide