Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Automate All The Things - Øredev 2019

Nate Ebel
November 06, 2019

Automate All The Things - Øredev 2019

Key takeaways:
- You will learn how to develop a more efficient and enjoyable development pipeline for your team
- You will understand how you can enforce code formatting, styling conventions, and best practices for your team
- You will see how automation can improve the code review process and overall code quality
- You will discover how to improve product quality through automated testing, releases, and issue tracking

Mobile moves fast, and so do you. To meet your deadlines, you want to build the right thing, at the right time; every time. Your continuous integration pipeline allows you to do this, but can it do more? - How can you enforce coding formatting, styling conventions, and best practices? - Do you build everything on every commit? Do you have a daily build or customize your build per branch? - What do you record and analyze? Are your tests passing? Do your screens look correct? Is your apk getting too large? - How do you distribute your apk for testing and release? - How do you track, record, and notify quickly when issues arise? In this talk, you'll what tools are available at every step of the continuous integration pipeline; allowing you to automate as much as possible and dedicate your time to what matters most.

Nate Ebel

November 06, 2019
Tweet

More Decks by Nate Ebel

Other Decks in Programming

Transcript

  1. Automate All The Things
    Nate Ebel

    View full-size slide

  2. @n8ebel #oredev
    Why Automate?

    View full-size slide

  3. @n8ebel #oredev
    Consistently build the
    right thing at the right
    time

    View full-size slide

  4. @n8ebel #oredev
    Save time and energy
    for the real challenges

    View full-size slide

  5. @n8ebel #oredev
    Make projects more
    approachable

    View full-size slide

  6. @n8ebel #oredev
    Slides & Repo
    ● https://github.com/n8ebel/AutomationSandbox
    ● https://github.com/n8ebel/GitHubActionsAutomationSandbox
    ● Speakerdeck - @n8ebel

    View full-size slide

  7. @n8ebel #oredev
    What to Automate?

    View full-size slide

  8. @n8ebel #oredev
    Automate
    All The Things!

    View full-size slide

  9. @n8ebel #oredev
    What To Automate?
    QUALITY REVIEW DEPLOY
    BUILD
    Where are you losing time, energy, or joy?

    View full-size slide

  10. @n8ebel #oredev
    Automating Your Build

    View full-size slide

  11. @n8ebel #oredev
    Automating Your Build
    BUILD
    BUILD
    Assemble
    Test
    Quality
    Deploy
    Educate

    View full-size slide

  12. @n8ebel #oredev
    Automating Your Build
    CONFIG
    BUILD
    ASSEMBLE
    TEST
    DEPLOY

    View full-size slide

  13. @n8ebel #oredev
    Automating Your Build
    CircleCI Bitrise
    GitHub
    Actions
    Codemagic

    View full-size slide

  14. @n8ebel #oredev
    Building With GitHub Actions
    ● Add “workflow” .yml file(s)
    ● Define when to run
    ● Define build/script steps
    ● Push to default branch
    ● Start building

    View full-size slide

  15. @n8ebel #oredev
    Building With GitHub Actions
    ...
    on: [pull_request, issues]
    jobs:
    greeting:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/first-interaction@v1
    with:
    repo-token: ${{ secrets.GITHUB_TOKEN }}
    issue-message: 'Thank you for helping us make this project better '
    pr-message: 'Thank you for submitting your first PR '
    .github/workflows/greet_contributor.yml

    View full-size slide

  16. @n8ebel #oredev
    Building With GitHub Actions

    View full-size slide

  17. @n8ebel #oredev
    Building With GitHub Actions
    ...
    steps:
    - uses: actions/checkout@v1
    - name: set up JDK 1.8
    uses: actions/setup-java@v1
    with:
    java-version: 1.8
    - name: Build & Test Project
    run: ./gradlew assemble
    .github/workflows/build_project.yml

    View full-size slide

  18. @n8ebel #oredev
    Building With CircleCI
    Workows run in response to triggers
    ● Push, Pull_request, New Issue, Schedule, etc...

    View full-size slide

  19. @n8ebel #oredev
    Enforcing Build Status Checks

    View full-size slide

  20. @n8ebel #oredev
    Building With CircleCI

    View full-size slide

  21. @n8ebel #oredev
    Enforcing Build Status Checks
    - Settings -> Branches
    - Click ‘Add Rule’
    - Require Status Checks To Pass Before Merging

    View full-size slide

  22. @n8ebel #oredev
    Enforcing Build Status Checks

    View full-size slide

  23. @n8ebel #oredev
    Enforcing Build Status Checks

    View full-size slide

  24. @n8ebel #oredev
    Now we have an
    automated build process

    View full-size slide

  25. @n8ebel #oredev
    Validating Code Quality

    View full-size slide

  26. @n8ebel #oredev
    Validating Code Quality
    ANDROID LINT
    CODE FORMATTING
    CODE SMELLS
    TESTS

    View full-size slide

  27. @n8ebel #oredev
    Validating code quality
    with your automated
    build

    View full-size slide

  28. @n8ebel #oredev
    Validating Code Quality
    - Add/Configure formatting/quality tool within project
    - Integrate with build script
    - Report issues

    View full-size slide

  29. @n8ebel #oredev
    Validating Code Formatting With ktlint
    ...
    - name: Check Kotlin Formatting
    run: ./gradlew ktlintCheck --continue
    Update build workflow to incorporate new build task
    .github/workflows/build_project.yml

    View full-size slide

  30. @n8ebel #oredev
    Validating Code Formatting With ktlint

    View full-size slide

  31. @n8ebel #oredev
    What’s the problem
    here?

    View full-size slide

  32. @n8ebel #oredev
    Failing builds require time and
    context switching

    View full-size slide

  33. @n8ebel #oredev
    Locally validating code
    quality

    View full-size slide

  34. @n8ebel #oredev
    Locally Validating Code Quality
    ● Reformat/Lint code using IDE
    ● Run ktlint, Android Lint, etc locally
    ● Avoid pushing code which fails the build

    View full-size slide

  35. @n8ebel #oredev
    Reformatting Code Using IDE
    ● Set project code style from pre-defined style
    ● Customize as desired
    ● Update .gitignore
    ● Ensure code styles included in repo
    ● Optimize imports on the fly

    View full-size slide

  36. @n8ebel #oredev
    Reformatting Code Using IDE
    # IDEA/Android Studio ignores
    *.iml
    /.idea/*
    # IDEA/Android Studio Ignore exceptions
    !/.idea/codeStyles/
    !/.idea/fileTemplates/
    !/.idea/inspectionProfiles/
    !/.idea/scopes/
    !/.idea/copyright/
    ...
    .gitignore

    View full-size slide

  37. @n8ebel #oredev
    Reformatting Code Using IDE
    - Add project code
    styles to git
    - Configure as
    desired to pass
    formatting checks

    View full-size slide

  38. @n8ebel #oredev
    Reformatting Code Using IDE
    Bonus tip for enforcing ktlint rules within Android Studio & IntelliJ

    View full-size slide

  39. @n8ebel #oredev
    Running ktlint Locally
    ● Have to remember to run local checks before pushing code
    ● Can use git-hooks to automatically run these checks

    View full-size slide

  40. @n8ebel #oredev
    Git hooks are scripts that Git executes
    before or after events such as:
    commit, push, and receive.

    View full-size slide

  41. @n8ebel #oredev
    Enforcing Code Quality With Git Hooks
    PRE-PUSH CHECK FORMATTING
    PRE-COMMIT DO SOMETHING

    View full-size slide

  42. @n8ebel #oredev
    Enforcing Code Quality With Git Hooks
    #!/bin/sh
    # Adapted from https://proandroiddev.com/ooga-chaka-git-hooks-to-enforce-code-quality-11ce8d0d23cb
    echo "Checking code formatting"
    ./gradlew app:ktlintCheck --daemon
    status=$?
    if [ "$status" = 0 ] ; then
    echo "No formatting issues were found"
    exit 0
    else
    echo 1>&2 "* There are code formatting issues that must be addressed"
    exit 1
    fi
    .git/hooks/pre-push

    View full-size slide

  43. @n8ebel #oredev
    Automating Git Hook Installation
    #!/bin/sh
    # Adapted from https://proandroiddev.com/ooga-chaka-git-hooks-to-enforce-code-quality-11ce8d0d23cb
    echo "Checking code formatting"
    ./gradlew app:ktlintCheck --daemon
    status=$?
    if [ "$status" = 0 ] ; then
    echo "No formatting issues were found"
    exit 0
    else
    echo 1>&2 "* There are code formatting issues that must be addressed"
    exit 1
    fi
    /scripts/git-hooks/pre-push.sh

    View full-size slide

  44. @n8ebel #oredev
    Enforcing Code Quality With Git Hooks
    task copyGitHooks(type: Copy) {
    description 'Copies the git hooks from scripts/git-hooks to the .git folder.'
    ...
    }
    task installGitHooks(type: Exec) {
    description 'Installs the git hooks from scripts/git-hooks.'
    ...
    }
    afterEvaluate {
    tasks['clean'].dependsOn installGitHooks // We install the hook at the first occasion
    }
    git-hooks.gradle

    View full-size slide

  45. @n8ebel #oredev
    Enforcing Code Quality With Git Hooks
    ...
    apply from: 'git-hooks.gradle'
    build.gradle

    View full-size slide

  46. @n8ebel #oredev
    Now We Can Prevent Pushes That Will
    Fail, And Share With The Team

    View full-size slide

  47. @n8ebel #oredev
    Reviewing Code

    View full-size slide

  48. @n8ebel #oredev
    Reviewing Code
    Validate Changes Discuss Approaches Learn

    View full-size slide

  49. @n8ebel #oredev
    Creating PR templates

    View full-size slide

  50. @n8ebel #oredev
    Creating PR Templates
    Want to Avoid This Lack Of Detail

    View full-size slide

  51. @n8ebel #oredev
    Creating PR Templates
    Favor and Encourage
    More Detail

    View full-size slide

  52. @n8ebel #oredev
    Creating PR Templates
    ### Related To-Do
    ** link **
    ### Proposed Changes
    * change 1
    * change 2
    ### Additional Info
    ** any additional useful context or info **
    ### Checklist
    - [ ] Tests
    - [ ] Translations
    ### Screenshots
    Original | Updated
    :-------------------------:|:-------------------------:
    ** original screenshot ** | ** updated screenshot **
    .github/pull_request_template.md
    ● Link to issue
    ● Describe changes
    ● Additional context
    ● Checklists
    ● Screenshots

    View full-size slide

  53. @n8ebel #oredev
    Creating PR Templates
    ● PR Sections
    Pre-Populated Based
    On Template
    ● Encourage
    Contributors To
    Provide Ample Detail

    View full-size slide

  54. @n8ebel #oredev
    Creating issue
    templates

    View full-size slide

  55. @n8ebel #oredev
    Creating Issue Templates
    ---
    name: Bug report
    about: Create a report to help us improve
    title: ''
    labels: bug
    assignees: ''
    ---
    **Description**
    A clear and concise description of what the bug is.
    **To Reproduce**
    Steps to reproduce the behavior:
    1. Go to '...'
    2. Click on '....'
    3. See error
    **Expected behavior**
    ...
    .github/ISSUE_TEMPLATE/bug_report.md

    View full-size slide

  56. @n8ebel #oredev
    Creating Issue Templates
    ---
    name: Feature request
    about: Suggest an idea for this project
    title: ''
    labels: enhancement
    assignees: ''
    ---
    **What problem are you trying to solve?**
    A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
    **Describe the solution you'd like**
    A clear and concise description of what you want to happen.
    **Describe alternatives you've considered**
    A clear and concise description of any alternative solutions or features you've considered.
    **Additional context**
    Add any other context or screenshots about the feature request here.
    .github/ISSUE_TEMPLATE/feature_request.md

    View full-size slide

  57. @n8ebel #oredev
    Using Issue Templates
    Choose a Template When Creating an Issue

    View full-size slide

  58. @n8ebel #oredev
    Using Issue Templates
    ● Issue Will
    Pre-Populate Based
    On Template
    ● Encourage
    Contributors To
    Provide Enough
    Information

    View full-size slide

  59. @n8ebel #oredev
    Referencing GitHub Issues
    Close issues via
    keyword in PR
    descriptions or
    commits
    ● ‘Closes’
    ● ‘Fixes’
    ● ‘Resolves’
    ● etc...

    View full-size slide

  60. @n8ebel #oredev
    Supercharging PRs with
    Danger

    View full-size slide

  61. @n8ebel #oredev
    Danger runs during your CI process, and gives
    teams the chance to automate common code
    review chores.

    View full-size slide

  62. @n8ebel #oredev
    What Is Danger?
    DANGER
    CHECK APK
    SIZE
    DEPENDENCY
    UPDATES
    INLINE LINT
    COMMENTS
    INFORMATIONAL
    MESSAGES
    ETC...

    View full-size slide

  63. @n8ebel #oredev
    What Is Danger?
    Add Useful Messages, Statistics, Warnings, etc to Your Pull Requests

    View full-size slide

  64. @n8ebel #oredev
    Adding Danger To Your Project
    source "https://rubygems.org"
    git_source(:github) {|repo_name| "https://github.com/#{repo_name}" }
    gem 'danger'
    Gemfile
    ● Add/update project Gemfile
    ● Run `bundle install`
    ● Run `bundle exec danger init`
    ● Follow the CLI setup process

    View full-size slide

  65. @n8ebel #oredev
    Adding Danger To Your Project
    ...
    - name: Install Ruby
    uses: actions/setup-ruby@v1
    with:
    ruby-version: '2.6'
    - name: Install Bundler
    run: |
    gem install bundler
    bundle install --jobs 4 --retry 3
    - name: Run Danger
    run: bundle exec danger
    env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
    .github/workflows/build_project.yml

    View full-size slide

  66. @n8ebel #oredev
    Adding PR Comments With Danger
    # thank PR author
    message "Thanks @#{github.pr_author} "
    Dangerfile

    View full-size slide

  67. @n8ebel #oredev
    Warning For WIP PRs With Danger
    # Make it more obvious that a PR is a work in progress and shouldn't be merged yet
    warn("PR is a Work in Progress and not ready to merge") if github.pr_title.include? "[WIP]"
    Dangerfile

    View full-size slide

  68. @n8ebel #oredev
    Reporting APK Size With Danger
    # Notify of the release APK size.
    apk_size = (File.size('app/build/outputs/apk/release/app-release-unsigned.apk').to_f / 2**20).round(2)
    message "Release APK size: #{apk_size} MB"
    Dangerfile

    View full-size slide

  69. @n8ebel #oredev
    Enforcing Useful PR Descriptions With Danger
    # Encourage contributors to write useful descriptions
    warn("Please provide a Pull Request description ...") if github.pr_body.length < 20
    Dangerfile

    View full-size slide

  70. @n8ebel #oredev
    Reporting Dependency Updates With Danger

    View full-size slide

  71. @n8ebel #oredev
    Reporting Dependency Updates With Danger
    ...
    - name: Check Dependency Versions
    run: ./gradlew dependencyUpdates
    .github/workflows/build_project.yml
    plugins {
    id "org.jlleitschuh.gradle.ktlint" version "7.1.0"
    id "com.github.ben-manes.versions" version "0.20.0"
    }
    build.gradle
    Gradle Versions Plugin

    View full-size slide

  72. @n8ebel #oredev
    Reporting Dependency Updates With Danger
    # Notify of outdated dependencies
    update_count = File.readlines("build/dependencyUpdates/report.txt").select { |line| line =~ /->/ }.count
    if update_count > 10
    # More than 10 libraries to update is cumbersome in a comment, so summarize
    warn "There are #{update_count} dependencies with new milestone versions."
    elsif update_count > 0
    file = File.open("build/dependencyUpdates/report.txt", "rb").read
    heading = "The following dependencies have later milestone versions:"
    warn file.slice(file.index(heading)..-1)
    end
    Dangerfile

    View full-size slide

  73. @n8ebel #oredev
    Reporting Dependency Updates With Danger

    View full-size slide

  74. @n8ebel #oredev
    Reporting Dependency Updates With Danger
    name: Check Dependency Versions
    on:
    schedule:
    - cron: '0 8 * * *'
    ...
    steps:
    - name: Checkout
    uses: actions/checkout@v1
    - name: Check Dependency Versions
    run: ./gradlew dependencyUpdates
    .github/workflows/check_dependencies.yml

    View full-size slide

  75. @n8ebel #oredev
    Reporting ktlint Errors With Danger

    View full-size slide

  76. @n8ebel #oredev
    Reporting ktlint Errors With Danger
    gem 'danger'
    gem 'danger-checkstyle_format'
    Gemfile
    # Report inline ktlint issues
    checkstyle_format.base_path = Dir.pwd
    checkstyle_format.report 'app/build/reports/ktlint/ktlintMainSourceSetCheck.xml'
    Dangerfile

    View full-size slide

  77. @n8ebel #oredev
    Reporting ktlint Errors With Danger
    # ignore inline messages that are outside of the current diff
    github.dismiss_out_of_range_messages
    Dangerfile

    View full-size slide

  78. @n8ebel #oredev
    Reporting With Danger
    And More…
    ● Android Lint
    ● Test Coverage
    ● String Translations
    ● Change In APK Size
    ● etc...

    View full-size slide

  79. @n8ebel #oredev
    Deploying Your App

    View full-size slide

  80. @n8ebel #oredev
    Deploying Your App
    ● Daily build
    ● Scheduled build
    ● Triggered build
    ● YOLO

    View full-size slide

  81. @n8ebel #oredev
    Customizing app
    deployment

    View full-size slide

  82. @n8ebel #oredev
    Customizing App Deployment
    Merge to Develop
    Distribute
    Test Build
    Merge to
    Master
    Upload to
    Google Play

    View full-size slide

  83. @n8ebel #oredev
    Deploying Test Builds On Merge
    name: Deploy to Crashlytics
    on:
    push:
    branches: develop
    - name: Build project
    run: ./gradlew :app:assembleDebug --build-cache --stacktrace
    ...
    - name: Deploy to Crashlytics
    run: ./gradlew :app:crashlyticsUploadDistributionDebug --build-cache
    --stacktrace
    if: success()
    .github/workflows/deploy_debug.yml

    View full-size slide

  84. @n8ebel #oredev
    Deploying Test Builds On Merge
    Automating Release
    Notes For Test Builds
    Generate internal release
    notes from git history

    View full-size slide

  85. @n8ebel #oredev
    Deploying Release Builds On Merge
    name: Deploy to Play
    on:
    push:
    branches: master
    - name: Build project
    run: ./gradlew :app:assembleRelease --build-cache --stacktrace
    ...
    - name: Deploy to Play
    run: ./gradlew :app:publishReleaseApk --build-cache --stacktrace
    if: success()
    .github/workflows/deploy_release.yml

    View full-size slide

  86. @n8ebel #oredev
    Deploying Release Builds On Merge
    ● Deploy Release Builds With Gradle Play
    Publisher
    ● Store Release Info In Repo
    ● Track Changes Over Time
    ● Full Internal Release By Merging To
    Master

    View full-size slide

  87. @n8ebel #oredev
    Automate All The Things

    View full-size slide

  88. @n8ebel #oredev
    What To Automate?
    QUALITY REVIEW DEPLOY
    BUILD

    View full-size slide

  89. @n8ebel #oredev
    Losing time, energy, or
    joy to some task? … try
    automating it.

    View full-size slide

  90. @n8ebel #oredev
    Resources
    ● https://github.com/n8ebel/AutomationSandbox
    ● https://github.com/n8ebel/GitHubActionsAutomationSandbox

    View full-size slide

  91. @n8ebel #oredev
    Resources
    ● https://proandroiddev.com/ooga-chaka-git-hooks-to-enforce-code-quality-11ce8d0d23cb
    ● https://help.github.com/en/articles/creating-a-pull-request-template-for-your-repository
    ● https://help.github.com/en/articles/closing-issues-using-keywords
    ● https://danger.systems/ruby/
    ● https://danger.systems/guides/getting_started.html#including-danger
    ● https://blog.bitrise.io/automating-code-review-tasks-for-multi-module-android-projects
    ● https://github.com/Triple-T/gradle-play-publisher
    ● https://try.crashlytics.com/beta/
    ● https://github.com/noboru-i/danger-checkstyle_format

    View full-size slide

  92. @n8ebel #oredev
    Special Thanks
    ● Ryan Harter
    ● Joe Birch
    ● Sebastiano Poggi

    View full-size slide

  93. @n8ebel #oredev
    Thank You
    "n8ebel".apply {
    Twitter
    YouTube
    LinkedIn
    Instagram
    }
    Stay In Touch

    View full-size slide