Make CI/CD More Comfortable Before/After Code Testing

Make CI/CD More Comfortable Before/After Code Testing

Talked at https://testnight.connpass.com/event/117578/

Not about testing methods but focusing on CI/CD

D4133d8efb46ae872d7a563b619bfc83?s=128

Matsuda Jumpei

March 07, 2019
Tweet

Transcript

  1. MAKE CI/CD MORE COMFORTABLE BEFORE/AFTER CODE TESTING 7TH MAR. ANDROID

    TEST NIGHT #6
  2. Self Introduction Jumpei Matsuda @red_fat_daruma ▸ GitHub: jmatsu ▸ Software

    Architect @DeployGate ▸ A member of DroidKaigi ▸ Consists of craft beer (especially Belgium) and Japanese sake. 2
  3. DroidKaigi 2019 Thank You for Your Work and Those Who

    Attended! ▸ About 1000 people joined, over 90 speakers talked. ▸ We have already published session videos! ▸ Check it out! https://bit.ly/2N1DLIb 3 https://droidkaigi.jp/2019
  4. Agenda What I Will Talk About ▸ Motivation ▸ Definition

    of *comfortable* ▸ Tips to support your testing ▸ How to organize non-fragile CI ▸ Save/load intermediate results of the current/previous build ▸ Get advantages of DeployGate 4
  5. Agenda What I Will NOT Talk About ▸ Technique for

    test code ▸ Testing method 5
  6. Agenda What I Will Talk About ▸ Motivation ▸ Definition

    of *comfortable* ▸ Tips to support your testing ▸ How to organize non-fragile CI ▸ Save/load intermediate results of the current/previous build ▸ Get advantages of DeployGate 6
  7. Motivation Before/After Code Testing? ▸ Testing is an important stage

    of dev. process ▸ There are many technique for testing and we are continuing to look for better practices ▸ However, ▸ It's a part of the whole process, and *code* is just one of testing factors ▸ Things around code testing are also important, and should be comfortable 7
  8. Agenda What I Will Talk About ▸ Motivation ▸ Definition

    of *comfortable* ▸ Tips to support your testing ▸ How to organize non-fragile CI ▸ Save/load intermediate results of the current/previous build ▸ Get advantages of DeployGate 8
  9. Definition What's *Comfortable* in This Talk? ▸ In short, it's

    to accelerate our development ▸ Not fragile, be stable ▸ Nobody likes CI/CD which fail frequently ▸ Be help for other development stages ▸ Finding/fixing bugs, QA, etc. 9
  10. Agenda What I Will Talk About ▸ Motivation ▸ Definition

    of *comfortable* ▸ Tips to support your testing ▸ How to organize non-fragile CI ▸ Save/load intermediate results of the current/previous build ▸ Get advantages of DeployGate 10
  11. How to organize non-fragile CI Not Fragile (Stable) Realized by

    Flexible ▸ No failure is allowed 11
  12. Not fragile (stable) does not mean *robust* Flexible ⊂ Not

    Fragile (Stable) ▸ No failure is allowed ▸ Non-flexible with failures is an evil ▸ BAD: Fail CI anyway if any task fails ▸ BAD: Do not run a Test task if lint fails ▸ How to make it flexible? 12
  13. Not fragile (stable) does not mean *robust* Flexible ⊂ Not

    Fragile (Stable) ▸ No failure is allowed ▸ Non-flexible with failures is an evil ▸ BAD: Fail CI anyway if any task fails ▸ BAD: Do not run a Test task if lint fails ▸ How to make it flexible? ▸ Check command status and make CI/CD fail if needed ▸ Separate jobs if possible 13
  14. Not fragile (stable) does not mean *robust* Flexible ⊂ Not

    Fragile (Stable) ▸ No failure is allowed ▸ Non-flexible with failures is an evil ▸ BAD: Fail CI anyway if any task fails ▸ BAD: Do not run a Test task if lint fails ▸ How to make it flexible? ▸ Check command status and make CI/CD fail if needed ▸ Separate jobs if possible 14
  15. Check command status and make CI/CD fail if needed Know

    Rules of Command Set To Be Executed ▸ Basically you have multiple roles in your dep. pipeline 15 Prepare dependencies Lints Reports for test-related res. Reports for others TESTING Assembling Deployment
  16. Check command status and make CI/CD fail if needed Consider

    *Required* Dependencies Between Command Set ▸ You can notice some of commands looks independent 16 Prepare dependencies Lints Reports for test-related res. Reports for others TESTING Assembling Deployment
  17. Check command status and make CI/CD fail if needed Check

    What Will Be Assured by Each Roles ▸ If you wanna assure nothing, you can eject CI/CD :p 17 Prepare dependencies Lints Reports for test-related res. Reports for others TESTING Assembling Deployment Code Quality Product Quality - Dev. experience - Product Quality through QA
  18. Check command status and make CI/CD fail if needed So,

    Which Command Should Fail CI/CD Immediately? ▸ Please note that this slide and your though may differ 18 Prepare dependencies Lints Reports for test-related res. Reports for others TESTING Assembling Deployment No. Basically, yes. As for assembling, yes. Conditional yes for dep. Yes!
  19. Examples Other Examples ▸ Use *continue* option when running Gradle

    tasks ▸ HSBEMFXMJOU%FCVHUFTU%FCVH6OJU5FTUDPOUJOVFP⒐JOF ▸ Use retry-logic to execute unstable commands ▸ For example, 500 error from maven repo. should be retried ▸ Execute reporting system like Danger anyway ▸ Basically we want to know test failures in a failure case 19
  20. Not fragile (stable) does not mean *robust* Flexible ⊂ Not

    Fragile (Stable) ▸ No failure is allowed ▸ Non-flexible with failures is an evil ▸ BAD: Fail CI anyway if any task fails ▸ BAD: Do not run a Test task if lint fails ▸ How to make it flexible? ▸ Check command status and make CI/CD fail if needed ▸ Separate jobs if possible 20
  21. Separate jobs if possible Built *Required* Graph Will Be Your

    Help ▸ Basically, jobs depend on execution time and CI services 21 Prepare dependencies Lints Reports for test-related res. Reports for others TESTING Assembling Deployment Vital job Test job Deployment job
  22. Agenda What I Will Talk About ▸ Motivation ▸ Definition

    of *comfortable* ▸ Tips to support your testing ▸ How to organize non-fragile CI ▸ Save/load intermediate results of the current/previous build ▸ Get advantages of DeployGate 22
  23. Save/load intermediate results of the current/previous build How To Treat

    Intermediate Results of CI ▸ Major CI services have cache systems ▸ Key-value style cache, repo/branch-based cache ▸ They are useful for build intermediate results like artifact dependencies' cache ▸ But, how about finer-grained cache...? 23
  24. Save/load intermediate results of the current/previous build Cache During a

    Build ▸ Create a new file right after executing a command ▸ If the file exists, the command exited with SUCCESS ▸ Otherwise, the command failed 24
  25. Save/load intermediate results of the current/previous build Cache Between Builds

    on a Single PR ▸ Use the CI's cache system? 25
  26. Save/load intermediate results of the current/previous build Cache Between Builds

    on a Single PR ▸ Use the CI's cache system? ▸ NO. ▸ You can use GitHub issues and HTML ▸ Fortunately(?), GitHub issues and comments can parse HTML ▸ It means you can use HTML comment syntax as well 26 *No issue found by ktlint!* <!-- { "is_successful": true, "last_build_num": 48 } -->
  27. Save/load intermediate results of the current/previous build Cache Between Builds

    on Multiple PRs ▸ Please let me know if there is a smart solution 27
  28. Agenda What I Will Talk About ▸ Motivation ▸ Definition

    of *comfortable* ▸ Tips to support your testing ▸ How to organize non-fragile CI ▸ Save/load intermediate results of the current/previous build ▸ Get advantages of DeployGate 28
  29. Get advantages of DeployGate DeployGate? ▸ A mobile app distribution

    service like Fabric Beta ▸ Annual BBQ planner in US ▸ https://deploygate.com 29
  30. Get advantages of DeployGate Advantages of DeployGate (Require Paid Plans)

    ▸ Faster download through CDN ▸ Distribution tracks 30
  31. Get advantages of DeployGate How To Have a Faster Download

    Experience ▸ CDN produces this advantage ▸ Therefore, first download from the signed URL is not fast, unfortunately 31
  32. Get advantages of DeployGate How To Have a Faster Download

    Experience ▸ CDN produces this advantage ▸ Therefore, first download from the signed URL is not fast, unfortunately ▸ Download your artifact first on CI right after deployment ▸ Note that the URL will be expired, so this tip does not work if new URLs are published 32 artifact_url=$(<response> | jq -r ".results.file") curl -SLs $artifact_url >/dev/null
  33. Get advantages of DeployGate Distribution Tracks ▸ Download pages for

    specific versions ▸ Like named internal track on Google Play Store 33
  34. Get advantages of DeployGate Distribution Tracks ▸ Download pages for

    specific versions ▸ Like named internal track on Google Play Store ▸ Deploy release ver., latest develop ver., feature branch ver. ▸ GitHub:jmatsu/dpg will be your help ▸ GitHub:DroidKaigi/conference-app-2019 is a good practice 34
  35. Matome dayo Be Comfortable Before/After Code Testing as Well ▸

    How to organize non-fragile CI ▸ Reorganize jobs and handle exit statuses manually ▸ Save/load intermediate results of the current/previous build ▸ Use new files and/or html comments to save/load metadata ▸ Get advantages of DeployGate ▸ Download a first artifact on CI, use distribution tracks 35
  36. Thank you for listening! Don't Hesitate To Ask Me any

    Questions ▸ Jumpei Matsuda ▸ @red_fat_daruma ▸ Software Architect @DeployGate ▸ A member of @DroidKaigi 36