Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Make CI/CD More Comfortable Before/After Code Testing

Make CI/CD More Comfortable Before/After Code Testing

Talked at https://testnight.connpass.com/event/117578/

Not about testing methods but focusing on CI/CD

Matsuda Jumpei

March 07, 2019
Tweet

More Decks by Matsuda Jumpei

Other Decks in Programming

Transcript

  1. MAKE CI/CD MORE COMFORTABLE
    BEFORE/AFTER CODE TESTING
    7TH MAR. ANDROID TEST NIGHT #6

    View Slide

  2. Self Introduction
    Jumpei Matsuda @red_fat_daruma
    ▸ GitHub: jmatsu
    ▸ Software Architect @DeployGate
    ▸ A member of DroidKaigi
    ▸ Consists of craft beer (especially
    Belgium) and Japanese sake.
    2

    View Slide

  3. DroidKaigi 2019
    Thank You for Your Work and Those Who Attended!
    ▸ About 1000 people joined, over 90 speakers talked.
    ▸ We have already published session videos!
    ▸ Check it out! https://bit.ly/2N1DLIb
    3
    https://droidkaigi.jp/2019

    View Slide

  4. Agenda
    What I Will Talk About
    ▸ Motivation
    ▸ Definition of *comfortable*
    ▸ Tips to support your testing
    ▸ How to organize non-fragile CI
    ▸ Save/load intermediate results
    of the current/previous build
    ▸ Get advantages of DeployGate
    4

    View Slide

  5. Agenda
    What I Will NOT Talk About
    ▸ Technique for test code
    ▸ Testing method
    5

    View Slide

  6. Agenda
    What I Will Talk About
    ▸ Motivation
    ▸ Definition of *comfortable*
    ▸ Tips to support your testing
    ▸ How to organize non-fragile CI
    ▸ Save/load intermediate results
    of the current/previous build
    ▸ Get advantages of DeployGate
    6

    View Slide

  7. Motivation
    Before/After Code Testing?
    ▸ Testing is an important stage of dev. process
    ▸ There are many technique for testing and we are continuing
    to look for better practices
    ▸ However,
    ▸ It's a part of the whole process, and *code* is just one of
    testing factors
    ▸ Things around code testing are also important, and
    should be comfortable
    7

    View Slide

  8. Agenda
    What I Will Talk About
    ▸ Motivation
    ▸ Definition of *comfortable*
    ▸ Tips to support your testing
    ▸ How to organize non-fragile CI
    ▸ Save/load intermediate results
    of the current/previous build
    ▸ Get advantages of DeployGate
    8

    View Slide

  9. Definition
    What's *Comfortable* in This Talk?
    ▸ In short, it's to accelerate our development
    ▸ Not fragile, be stable
    ▸ Nobody likes CI/CD which fail frequently
    ▸ Be help for other development stages
    ▸ Finding/fixing bugs, QA, etc.
    9

    View Slide

  10. Agenda
    What I Will Talk About
    ▸ Motivation
    ▸ Definition of *comfortable*
    ▸ Tips to support your testing
    ▸ How to organize non-fragile CI
    ▸ Save/load intermediate results
    of the current/previous build
    ▸ Get advantages of DeployGate
    10

    View Slide

  11. How to organize non-fragile CI
    Not Fragile (Stable) Realized by Flexible
    ▸ No failure is allowed
    11

    View Slide

  12. Not fragile (stable) does not mean *robust*
    Flexible ⊂ Not Fragile (Stable)
    ▸ No failure is allowed
    ▸ Non-flexible with failures is an evil
    ▸ BAD: Fail CI anyway if any task fails
    ▸ BAD: Do not run a Test task if lint fails
    ▸ How to make it flexible?
    12

    View Slide

  13. Not fragile (stable) does not mean *robust*
    Flexible ⊂ Not Fragile (Stable)
    ▸ No failure is allowed
    ▸ Non-flexible with failures is an evil
    ▸ BAD: Fail CI anyway if any task fails
    ▸ BAD: Do not run a Test task if lint fails
    ▸ How to make it flexible?
    ▸ Check command status and make CI/CD fail if needed
    ▸ Separate jobs if possible
    13

    View Slide

  14. Not fragile (stable) does not mean *robust*
    Flexible ⊂ Not Fragile (Stable)
    ▸ No failure is allowed
    ▸ Non-flexible with failures is an evil
    ▸ BAD: Fail CI anyway if any task fails
    ▸ BAD: Do not run a Test task if lint fails
    ▸ How to make it flexible?
    ▸ Check command status and make CI/CD fail if needed
    ▸ Separate jobs if possible
    14

    View Slide

  15. Check command status and make CI/CD fail if needed
    Know Rules of Command Set To Be Executed
    ▸ Basically you have multiple roles in your dep. pipeline
    15
    Prepare
    dependencies
    Lints
    Reports for
    test-related res.
    Reports for
    others
    TESTING
    Assembling
    Deployment

    View Slide

  16. Check command status and make CI/CD fail if needed
    Consider *Required* Dependencies Between Command Set
    ▸ You can notice some of commands looks independent
    16
    Prepare
    dependencies
    Lints
    Reports for
    test-related res.
    Reports for
    others
    TESTING
    Assembling Deployment

    View Slide

  17. Check command status and make CI/CD fail if needed
    Check What Will Be Assured by Each Roles
    ▸ If you wanna assure nothing, you can eject CI/CD :p
    17
    Prepare
    dependencies
    Lints
    Reports for
    test-related res.
    Reports for
    others
    TESTING
    Assembling Deployment
    Code Quality
    Product Quality
    - Dev. experience
    - Product Quality
    through QA

    View Slide

  18. Check command status and make CI/CD fail if needed
    So, Which Command Should Fail CI/CD Immediately?
    ▸ Please note that this slide and your though may differ
    18
    Prepare
    dependencies
    Lints
    Reports for
    test-related res.
    Reports for
    others
    TESTING
    Assembling Deployment
    No.
    Basically, yes.
    As for assembling, yes.
    Conditional yes for dep.
    Yes!

    View Slide

  19. Examples
    Other Examples
    ▸ Use *continue* option when running Gradle tasks
    ▸ HSBEMFXMJOU%FCVHUFTU%FCVH6OJU5FTUDPOUJOVFP⒐JOF
    ▸ Use retry-logic to execute unstable commands
    ▸ For example, 500 error from maven repo. should be retried
    ▸ Execute reporting system like Danger anyway
    ▸ Basically we want to know test failures in a failure case
    19

    View Slide

  20. Not fragile (stable) does not mean *robust*
    Flexible ⊂ Not Fragile (Stable)
    ▸ No failure is allowed
    ▸ Non-flexible with failures is an evil
    ▸ BAD: Fail CI anyway if any task fails
    ▸ BAD: Do not run a Test task if lint fails
    ▸ How to make it flexible?
    ▸ Check command status and make CI/CD fail if needed
    ▸ Separate jobs if possible
    20

    View Slide

  21. Separate jobs if possible
    Built *Required* Graph Will Be Your Help
    ▸ Basically, jobs depend on execution time and CI services
    21
    Prepare
    dependencies
    Lints
    Reports for
    test-related res.
    Reports for
    others
    TESTING
    Assembling Deployment
    Vital job
    Test job
    Deployment job

    View Slide

  22. Agenda
    What I Will Talk About
    ▸ Motivation
    ▸ Definition of *comfortable*
    ▸ Tips to support your testing
    ▸ How to organize non-fragile CI
    ▸ Save/load intermediate results
    of the current/previous build
    ▸ Get advantages of DeployGate
    22

    View Slide

  23. Save/load intermediate results of the current/previous build
    How To Treat Intermediate Results of CI
    ▸ Major CI services have cache systems
    ▸ Key-value style cache, repo/branch-based cache
    ▸ They are useful for build intermediate results like artifact
    dependencies' cache
    ▸ But, how about finer-grained cache...?
    23

    View Slide

  24. Save/load intermediate results of the current/previous build
    Cache During a Build
    ▸ Create a new file right after executing a command
    ▸ If the file exists, the command exited with SUCCESS
    ▸ Otherwise, the command failed
    24

    View Slide

  25. Save/load intermediate results of the current/previous build
    Cache Between Builds on a Single PR
    ▸ Use the CI's cache system?
    25

    View Slide

  26. Save/load intermediate results of the current/previous build
    Cache Between Builds on a Single PR
    ▸ Use the CI's cache system?
    ▸ NO.
    ▸ You can use GitHub issues and HTML
    ▸ Fortunately(?), GitHub issues and comments can parse
    HTML
    ▸ It means you can use HTML comment syntax as well
    26
    *No issue found by ktlint!*

    View Slide

  27. Save/load intermediate results of the current/previous build
    Cache Between Builds on Multiple PRs
    ▸ Please let me know if there is a smart solution
    27

    View Slide

  28. Agenda
    What I Will Talk About
    ▸ Motivation
    ▸ Definition of *comfortable*
    ▸ Tips to support your testing
    ▸ How to organize non-fragile CI
    ▸ Save/load intermediate results
    of the current/previous build
    ▸ Get advantages of DeployGate
    28

    View Slide

  29. Get advantages of DeployGate
    DeployGate?
    ▸ A mobile app distribution service like Fabric Beta
    ▸ Annual BBQ planner in US
    ▸ https://deploygate.com
    29

    View Slide

  30. Get advantages of DeployGate
    Advantages of DeployGate (Require Paid Plans)
    ▸ Faster download through CDN
    ▸ Distribution tracks
    30

    View Slide

  31. Get advantages of DeployGate
    How To Have a Faster Download Experience
    ▸ CDN produces this advantage
    ▸ Therefore, first download from the signed URL is not fast,
    unfortunately
    31

    View Slide

  32. Get advantages of DeployGate
    How To Have a Faster Download Experience
    ▸ CDN produces this advantage
    ▸ Therefore, first download from the signed URL is not fast,
    unfortunately
    ▸ Download your artifact first on CI right after deployment
    ▸ Note that the URL will be expired, so this tip does not work if new URLs
    are published
    32
    artifact_url=$( | jq -r ".results.file")
    curl -SLs $artifact_url >/dev/null

    View Slide

  33. Get advantages of DeployGate
    Distribution Tracks
    ▸ Download pages for specific versions
    ▸ Like named internal track on Google Play Store
    33

    View Slide

  34. Get advantages of DeployGate
    Distribution Tracks
    ▸ Download pages for specific versions
    ▸ Like named internal track on Google Play Store
    ▸ Deploy release ver., latest develop ver., feature branch ver.
    ▸ GitHub:jmatsu/dpg will be your help
    ▸ GitHub:DroidKaigi/conference-app-2019 is a good practice
    34

    View Slide

  35. Matome dayo
    Be Comfortable Before/After Code Testing as Well
    ▸ How to organize non-fragile CI
    ▸ Reorganize jobs and handle exit statuses manually
    ▸ Save/load intermediate results of the current/previous
    build
    ▸ Use new files and/or html comments to save/load metadata
    ▸ Get advantages of DeployGate
    ▸ Download a first artifact on CI, use distribution tracks
    35

    View Slide

  36. Thank you for listening!
    Don't Hesitate To Ask Me any Questions
    ▸ Jumpei Matsuda
    ▸ @red_fat_daruma
    ▸ Software Architect @DeployGate
    ▸ A member of @DroidKaigi
    36

    View Slide