From Git Push, to Play store – A DevOps journey for Android Spiros I. Oikonomakis

Continuous Delivery

What is Continuous Delivery? Continuous Delivery is the ability to get a wide range of software changes – including new features, configuration changes, bug fixes, and experiments – into production, or into the hands of users, safely, quickly, and sustainably.

Continuous Delivery Principles ● Build quality in ● Work in small batches ● Computers perform repetitive tasks, people solve problems ● Relentlessly pursue continuous improvement ● Everyone is responsible

Continuous Delivery Foundations ● Configuration management ● Continuous integration ● Continuous testing

Configuration Management Automation plays a vital role in ensuring we can release software repeatedly and reliably. One key goal is to take repetitive manual processes like build, deployment, regression testing and automate them. Continuous Delivery

Continuous Integration Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily – leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Martin Fowler

Continuous Testing The key to building quality into our software is making sure we can get fast feedback on the impact of changes. This type of testing was normally done in a phase following “dev complete”. So the goal is to run many different types of tests—both manual and automated—continually throughout the delivery process Continuous Delivery

history

Tech Savvy Users #not

Minimize the risk of bad UX (1) The decision was simple: we decided to set up 10-15 “techie” users as a BETA group. Our continuous delivery flow became: ● Deploy updates to BETA group ● Updates remain in BETA only for at least 2 days ● Stage gradual rollout (10%) to STABLE channel ● Increase the rollout percentage on a daily basis ● Gradually achieve full deployment

Minimize the risk of bad UX (2) Two completely different build types with different names and icons

Initial CD flow (1) PUSH (feature branch) BUILD (qaRelease) TESTS QUALITY PACKAGING (unsigned)

Initial CD flow (2) git checkout develop -b feature-XXX //do some work git push origin feature-XXX // (QA)

Initial CD flow (3) git checkout develop -b feature-XXX //do some work git push origin feature-XXX // (QA)

Initial CD flow (4) git checkout develop -b feature-XXX //do some work git push origin feature-XXX // (QA)

Disadvantages ● QA manual work ○ Find unsigned build from the CI ○ Manual testing as we had only unit tests ● Manual release phase ○ Locally store Keystore & Credentials (to avoid keeping them in GIT repo) ○ Manually sign and generate builds ○ Manually uploading to Google Play ○ Update screenshots or texts as needed ○ Apply the same manual process after the two days in BETA for the STABLE release.

CD flow optimization (1) Decision to streamline the process of sharing builds to the QA team as the first step of the flow optimization with some automation magic with one more box: PUSH (feature branch) BUILD (qaRelease) TESTS QUALITY PACKAGING (unsigned) DELIVERY

CD flow optimization (2) DELIVERY QA? CRASHLYTICS SLACK QA SLACK SUCCESS YES NO

CD flow optimization (3)

Advantages ● One-click installation with the Crashlytics app ● Easier navigation to all available QA builds ● Notifying the QA channel in Slack

Automated delivery #not

CD pipeline

Release Plan (1) Releases follows “catch-the-train” approach (Software release train). No able to “catch the train” with your new changes, you need to wait for the next train (next release) to send your changes to the STABLE channel (except for hot fixes).

Release Plan (2) ● One daily release to the BETA channel (which is a separate build type) ● End of each week we send the current state of the BETA release to the STABLE channel (stage rollout 50%) The above is called Canary Release

Configuration management ● Obviously a Version Control System (eg. GIT) ● Store your secrets safely with Blackbox ○ Keystore file ○ Keystore alias ○ Passphrase ○ Crashlytics API key ● Deployment scripts (fastlane, shell scripts)

CI- Build (1) The following are key principles for a solid Continuous Integration process: ● Automated build ● Fail fast ● Constant testing ● Constant packaging ● Easier release

CI - Build (2) Select a build server to compile and build our application: ● Jenkins ● CircleCI ● TravisCI ● Bamboo

CT - Testing (1) Testing provides several important benefits: ● Boosts confidence of the team ● Increases code coverage testing ● Helps engineers find broken code quickly and easily ● Offloads QA work (especially for small changes)

CT - Testing (2) The key principles of a good testing strategy include: ● Testing business logic independently of a framework ● Testing Server Side (API, Database, WebSocket etc.) integration ● Testing the acceptance rules based on user actions as a Blackbox testing

CT - Testing (3) Following The Clean Architecture, the testing and TestDouble practice is easy: ● presentation[android]: ○ Mock Presenters – specifically the use cases – (Mockito) ○ Mocking data layer for acceptance tests (Espresso) ○ Unit testing for specific components (JUnit4) ● domain [java]: ○ Unit testing (JUnit4) ○ Mocking data layer ○ Independent of framework testing ● data[android lib]: ○ Unit testing (Robolectic) ○ Mock Server side (REST APIs & Firebase in our case)

Quality ● Code styles ● Security problems ● Code duplication ● Complexity ● Best programming practices ● Possible bugs

Quality Tools We use the following static program analysis tools to maintain high code quality: ● Checkstyle ● PMD ● FindBugs ● Lint ● Static program analysis insights (eg. SonarQube, Codacy) Fail as fast as possible approach

auto deploy

Final Flow PUSH BUILD TESTS QUALITY PACKAGING DELIVERY

Final Flow (1)

Final Flow (2)

Next steps ● More sophisticated static analysis tools (eg. Infer) ● Better tools for grading our codebase (eg. Upsource) ● Docker container configuration for building ● Real integration testing with communication to external components

Conclusion & Benefits ● Low-risk releases, so happy customers ● Faster time to market (product managers super happy) ● Higher-quality code ● Better products Happier teams

Thanks! We are hiring!!!! https://www.ezhome.com/careers @spirosoik github.com/spirosoik github.com/ezhome