Automate All The Things - Øredev 2019

Key takeaways:
- You will learn how to develop a more efficient and enjoyable development pipeline for your team
- You will understand how you can enforce code formatting, styling conventions, and best practices for your team
- You will see how automation can improve the code review process and overall code quality
- You will discover how to improve product quality through automated testing, releases, and issue tracking

Mobile moves fast, and so do you. To meet your deadlines, you want to build the right thing, at the right time; every time. Your continuous integration pipeline allows you to do this, but can it do more? - How can you enforce coding formatting, styling conventions, and best practices? - Do you build everything on every commit? Do you have a daily build or customize your build per branch? - What do you record and analyze? Are your tests passing? Do your screens look correct? Is your apk getting too large? - How do you distribute your apk for testing and release? - How do you track, record, and notify quickly when issues arise? In this talk, you'll what tools are available at every step of the continuous integration pipeline; allowing you to automate as much as possible and dedicate your time to what matters most.

Transcript

1. Automate All The Things Nate Ebel
2. @n8ebel #oredev Why Automate?
3. @n8ebel #oredev Consistently build the right thing at the right time
4. @n8ebel #oredev Save time and energy for the real challenges
5. @n8ebel #oredev Make projects more approachable
6. @n8ebel #oredev Slides & Repo • https://github.com/n8ebel/AutomationSandbox • https://github.com/n8ebel/GitHubActionsAutomationSandbox • Speakerdeck - @n8ebel
7. @n8ebel #oredev What to Automate?
8. @n8ebel #oredev Automate All The Things!
9. @n8ebel #oredev What To Automate? QUALITY REVIEW DEPLOY BUILD Where are you losing time, energy, or joy?
10. @n8ebel #oredev Automating Your Build
11. @n8ebel #oredev Automating Your Build BUILD BUILD Assemble Test Quality Deploy Educate
12. @n8ebel #oredev Automating Your Build CONFIG BUILD ASSEMBLE TEST DEPLOY
13. @n8ebel #oredev Automating Your Build CircleCI Bitrise GitHub Actions Codemagic
14. @n8ebel #oredev Building With GitHub Actions • Add “workflow” .yml file(s) • Define when to run • Define build/script steps • Push to default branch • Start building
15. @n8ebel #oredev Building With GitHub Actions ... on: [pull_request, issues] jobs: greeting: runs-on: ubuntu-latest steps: - uses: actions/first-interaction@v1 with: repo-token: ${{ secrets.GITHUB_TOKEN }} issue-message: 'Thank you for helping us make this project better ' pr-message: 'Thank you for submitting your first PR ' .github/workflows/greet_contributor.yml
16. @n8ebel #oredev Building With GitHub Actions
17. @n8ebel #oredev Building With GitHub Actions ... steps: - uses: actions/checkout@v1 - name: set up JDK 1.8 uses: actions/setup-java@v1 with: java-version: 1.8 - name: Build & Test Project run: ./gradlew assemble .github/workflows/build_project.yml
18. @n8ebel #oredev Building With CircleCI Workows run in response to triggers • Push, Pull_request, New Issue, Schedule, etc...
19. @n8ebel #oredev Enforcing Build Status Checks
20. @n8ebel #oredev Building With CircleCI
21. @n8ebel #oredev Enforcing Build Status Checks - Settings -> Branches - Click ‘Add Rule’ - Require Status Checks To Pass Before Merging
22. @n8ebel #oredev Enforcing Build Status Checks
23. @n8ebel #oredev Enforcing Build Status Checks
24. @n8ebel #oredev Now we have an automated build process
25. @n8ebel #oredev Validating Code Quality
26. @n8ebel #oredev Validating Code Quality ANDROID LINT CODE FORMATTING CODE SMELLS TESTS
27. @n8ebel #oredev Validating code quality with your automated build
28. @n8ebel #oredev Validating Code Quality - Add/Configure formatting/quality tool within project - Integrate with build script - Report issues
29. @n8ebel #oredev Validating Code Formatting With ktlint ... - name: Check Kotlin Formatting run: ./gradlew ktlintCheck --continue Update build workflow to incorporate new build task .github/workflows/build_project.yml
30. @n8ebel #oredev Validating Code Formatting With ktlint
31. @n8ebel #oredev What’s the problem here?
32. @n8ebel #oredev Failing builds require time and context switching
33. @n8ebel #oredev Locally validating code quality
34. @n8ebel #oredev Locally Validating Code Quality • Reformat/Lint code using IDE • Run ktlint, Android Lint, etc locally • Avoid pushing code which fails the build
35. @n8ebel #oredev Reformatting Code Using IDE • Set project code style from pre-defined style • Customize as desired • Update .gitignore • Ensure code styles included in repo • Optimize imports on the fly
36. @n8ebel #oredev Reformatting Code Using IDE # IDEA/Android Studio ignores *.iml /.idea/* # IDEA/Android Studio Ignore exceptions !/.idea/codeStyles/ !/.idea/fileTemplates/ !/.idea/inspectionProfiles/ !/.idea/scopes/ !/.idea/copyright/ ... .gitignore
37. @n8ebel #oredev Reformatting Code Using IDE - Add project code styles to git - Configure as desired to pass formatting checks
38. @n8ebel #oredev Reformatting Code Using IDE Bonus tip for enforcing ktlint rules within Android Studio & IntelliJ
39. @n8ebel #oredev Running ktlint Locally • Have to remember to run local checks before pushing code • Can use git-hooks to automatically run these checks
40. @n8ebel #oredev Git hooks are scripts that Git executes before or after events such as: commit, push, and receive.
41. @n8ebel #oredev Enforcing Code Quality With Git Hooks PRE-PUSH CHECK FORMATTING PRE-COMMIT DO SOMETHING
42. @n8ebel #oredev Enforcing Code Quality With Git Hooks #!/bin/sh # Adapted from https://proandroiddev.com/ooga-chaka-git-hooks-to-enforce-code-quality-11ce8d0d23cb echo "Checking code formatting" ./gradlew app:ktlintCheck --daemon status=$? if [ "$status" = 0 ] ; then echo "No formatting issues were found" exit 0 else echo 1>&2 "* There are code formatting issues that must be addressed" exit 1 fi .git/hooks/pre-push
43. @n8ebel #oredev Automating Git Hook Installation #!/bin/sh # Adapted from https://proandroiddev.com/ooga-chaka-git-hooks-to-enforce-code-quality-11ce8d0d23cb echo "Checking code formatting" ./gradlew app:ktlintCheck --daemon status=$? if [ "$status" = 0 ] ; then echo "No formatting issues were found" exit 0 else echo 1>&2 "* There are code formatting issues that must be addressed" exit 1 fi /scripts/git-hooks/pre-push.sh
44. @n8ebel #oredev Enforcing Code Quality With Git Hooks task copyGitHooks(type: Copy) { description 'Copies the git hooks from scripts/git-hooks to the .git folder.' ... } task installGitHooks(type: Exec) { description 'Installs the git hooks from scripts/git-hooks.' ... } afterEvaluate { tasks['clean'].dependsOn installGitHooks // We install the hook at the first occasion } git-hooks.gradle
45. @n8ebel #oredev Enforcing Code Quality With Git Hooks ... apply from: 'git-hooks.gradle' build.gradle
46. @n8ebel #oredev Now We Can Prevent Pushes That Will Fail, And Share With The Team
47. @n8ebel #oredev Reviewing Code
48. @n8ebel #oredev Reviewing Code Validate Changes Discuss Approaches Learn
49. @n8ebel #oredev Creating PR templates
50. @n8ebel #oredev Creating PR Templates Want to Avoid This Lack Of Detail
51. @n8ebel #oredev Creating PR Templates Favor and Encourage More Detail
52. @n8ebel #oredev Creating PR Templates ### Related To-Do ** link ** ### Proposed Changes * change 1 * change 2 ### Additional Info ** any additional useful context or info ** ### Checklist - [ ] Tests - [ ] Translations ### Screenshots Original | Updated :-------------------------:|:-------------------------: ** original screenshot ** | ** updated screenshot ** .github/pull_request_template.md • Link to issue • Describe changes • Additional context • Checklists • Screenshots
53. @n8ebel #oredev Creating PR Templates • PR Sections Pre-Populated Based On Template • Encourage Contributors To Provide Ample Detail
54. @n8ebel #oredev Creating issue templates
55. @n8ebel #oredev Creating Issue Templates --- name: Bug report about: Create a report to help us improve title: '' labels: bug assignees: '' --- **Description** A clear and concise description of what the bug is. **To Reproduce** Steps to reproduce the behavior: 1. Go to '...' 2. Click on '....' 3. See error **Expected behavior** ... .github/ISSUE_TEMPLATE/bug_report.md
56. @n8ebel #oredev Creating Issue Templates --- name: Feature request about: Suggest an idea for this project title: '' labels: enhancement assignees: '' --- **What problem are you trying to solve?** A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] **Describe the solution you'd like** A clear and concise description of what you want to happen. **Describe alternatives you've considered** A clear and concise description of any alternative solutions or features you've considered. **Additional context** Add any other context or screenshots about the feature request here. .github/ISSUE_TEMPLATE/feature_request.md
57. @n8ebel #oredev Using Issue Templates Choose a Template When Creating an Issue
58. @n8ebel #oredev Using Issue Templates • Issue Will Pre-Populate Based On Template • Encourage Contributors To Provide Enough Information
59. @n8ebel #oredev Referencing GitHub Issues Close issues via keyword in PR descriptions or commits • ‘Closes’ • ‘Fixes’ • ‘Resolves’ • etc...
60. @n8ebel #oredev Supercharging PRs with Danger
61. @n8ebel #oredev Danger runs during your CI process, and gives teams the chance to automate common code review chores.
62. @n8ebel #oredev What Is Danger? DANGER CHECK APK SIZE DEPENDENCY UPDATES INLINE LINT COMMENTS INFORMATIONAL MESSAGES ETC...
63. @n8ebel #oredev What Is Danger? Add Useful Messages, Statistics, Warnings, etc to Your Pull Requests
64. @n8ebel #oredev Adding Danger To Your Project source "https://rubygems.org" git_source(:github) {|repo_name| "https://github.com/#{repo_name}" } gem 'danger' Gemfile • Add/update project Gemfile • Run `bundle install` • Run `bundle exec danger init` • Follow the CLI setup process
65. @n8ebel #oredev Adding Danger To Your Project ... - name: Install Ruby uses: actions/setup-ruby@v1 with: ruby-version: '2.6' - name: Install Bundler run: | gem install bundler bundle install --jobs 4 --retry 3 - name: Run Danger run: bundle exec danger env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} .github/workflows/build_project.yml
66. @n8ebel #oredev Adding PR Comments With Danger # thank PR author message "Thanks @#{github.pr_author} " Dangerfile
67. @n8ebel #oredev Warning For WIP PRs With Danger # Make it more obvious that a PR is a work in progress and shouldn't be merged yet warn("PR is a Work in Progress and not ready to merge") if github.pr_title.include? "[WIP]" Dangerfile
68. @n8ebel #oredev Reporting APK Size With Danger # Notify of the release APK size. apk_size = (File.size('app/build/outputs/apk/release/app-release-unsigned.apk').to_f / 2**20).round(2) message "Release APK size: #{apk_size} MB" Dangerfile
69. @n8ebel #oredev Enforcing Useful PR Descriptions With Danger # Encourage contributors to write useful descriptions warn("Please provide a Pull Request description ...") if github.pr_body.length < 20 Dangerfile
70. @n8ebel #oredev Reporting Dependency Updates With Danger
71. @n8ebel #oredev Reporting Dependency Updates With Danger ... - name: Check Dependency Versions run: ./gradlew dependencyUpdates .github/workflows/build_project.yml plugins { id "org.jlleitschuh.gradle.ktlint" version "7.1.0" id "com.github.ben-manes.versions" version "0.20.0" } build.gradle Gradle Versions Plugin
72. @n8ebel #oredev Reporting Dependency Updates With Danger # Notify of outdated dependencies update_count = File.readlines("build/dependencyUpdates/report.txt").select { |line| line =~ /->/ }.count if update_count > 10 # More than 10 libraries to update is cumbersome in a comment, so summarize warn "There are #{update_count} dependencies with new milestone versions." elsif update_count > 0 file = File.open("build/dependencyUpdates/report.txt", "rb").read heading = "The following dependencies have later milestone versions:" warn file.slice(file.index(heading)..-1) end Dangerfile
73. @n8ebel #oredev Reporting Dependency Updates With Danger
74. @n8ebel #oredev Reporting Dependency Updates With Danger name: Check Dependency Versions on: schedule: - cron: '0 8 * * *' ... steps: - name: Checkout uses: actions/checkout@v1 - name: Check Dependency Versions run: ./gradlew dependencyUpdates .github/workflows/check_dependencies.yml
75. @n8ebel #oredev Reporting ktlint Errors With Danger
76. @n8ebel #oredev Reporting ktlint Errors With Danger gem 'danger' gem 'danger-checkstyle_format' Gemfile # Report inline ktlint issues checkstyle_format.base_path = Dir.pwd checkstyle_format.report 'app/build/reports/ktlint/ktlintMainSourceSetCheck.xml' Dangerfile
77. @n8ebel #oredev Reporting ktlint Errors With Danger # ignore inline messages that are outside of the current diff github.dismiss_out_of_range_messages Dangerfile
78. @n8ebel #oredev Reporting With Danger And More… • Android Lint • Test Coverage • String Translations • Change In APK Size • etc...
79. @n8ebel #oredev Deploying Your App
80. @n8ebel #oredev Deploying Your App • Daily build • Scheduled build • Triggered build • YOLO
81. @n8ebel #oredev Customizing app deployment
82. @n8ebel #oredev Customizing App Deployment Merge to Develop Distribute Test Build Merge to Master Upload to Google Play
83. @n8ebel #oredev Deploying Test Builds On Merge name: Deploy to Crashlytics on: push: branches: develop - name: Build project run: ./gradlew :app:assembleDebug --build-cache --stacktrace ... - name: Deploy to Crashlytics run: ./gradlew :app:crashlyticsUploadDistributionDebug --build-cache --stacktrace if: success() .github/workflows/deploy_debug.yml
84. @n8ebel #oredev Deploying Test Builds On Merge Automating Release Notes For Test Builds Generate internal release notes from git history
85. @n8ebel #oredev Deploying Release Builds On Merge name: Deploy to Play on: push: branches: master - name: Build project run: ./gradlew :app:assembleRelease --build-cache --stacktrace ... - name: Deploy to Play run: ./gradlew :app:publishReleaseApk --build-cache --stacktrace if: success() .github/workflows/deploy_release.yml
86. @n8ebel #oredev Deploying Release Builds On Merge • Deploy Release Builds With Gradle Play Publisher • Store Release Info In Repo • Track Changes Over Time • Full Internal Release By Merging To Master
87. @n8ebel #oredev Automate All The Things
88. @n8ebel #oredev What To Automate? QUALITY REVIEW DEPLOY BUILD
89. @n8ebel #oredev Losing time, energy, or joy to some task? … try automating it.
90. @n8ebel #oredev Resources • https://github.com/n8ebel/AutomationSandbox • https://github.com/n8ebel/GitHubActionsAutomationSandbox
91. @n8ebel #oredev Resources • https://proandroiddev.com/ooga-chaka-git-hooks-to-enforce-code-quality-11ce8d0d23cb • https://help.github.com/en/articles/creating-a-pull-request-template-for-your-repository • https://help.github.com/en/articles/closing-issues-using-keywords • https://danger.systems/ruby/ • https://danger.systems/guides/getting_started.html#including-danger • https://blog.bitrise.io/automating-code-review-tasks-for-multi-module-android-projects • https://github.com/Triple-T/gradle-play-publisher • https://try.crashlytics.com/beta/ • https://github.com/noboru-i/danger-checkstyle_format
92. @n8ebel #oredev Special Thanks • Ryan Harter • Joe Birch • Sebastiano Poggi
93. @n8ebel #oredev Thank You "n8ebel".apply { Twitter YouTube LinkedIn Instagram } Stay In Touch