Make CI/CD More Comfortable Before/After Code Testing

by Matsuda Jumpei

Slide 1

Slide 1 text

MAKE CI/CD MORE COMFORTABLE BEFORE/AFTER CODE TESTING 7TH MAR. ANDROID TEST NIGHT #6

Slide 2

Slide 2 text

Self Introduction Jumpei Matsuda @red_fat_daruma ▸ GitHub: jmatsu ▸ Software Architect @DeployGate ▸ A member of DroidKaigi ▸ Consists of craft beer (especially Belgium) and Japanese sake. 2

Slide 3

Slide 3 text

DroidKaigi 2019 Thank You for Your Work and Those Who Attended! ▸ About 1000 people joined, over 90 speakers talked. ▸ We have already published session videos! ▸ Check it out! https://bit.ly/2N1DLIb 3 https://droidkaigi.jp/2019

Slide 4

Slide 4 text

Agenda What I Will Talk About ▸ Motivation ▸ Deﬁnition of *comfortable* ▸ Tips to support your testing ▸ How to organize non-fragile CI ▸ Save/load intermediate results of the current/previous build ▸ Get advantages of DeployGate 4

Slide 5

Slide 5 text

Agenda What I Will NOT Talk About ▸ Technique for test code ▸ Testing method 5

Slide 6

Slide 6 text

Slide 7

Slide 7 text

Motivation Before/After Code Testing? ▸ Testing is an important stage of dev. process ▸ There are many technique for testing and we are continuing to look for better practices ▸ However, ▸ It's a part of the whole process, and *code* is just one of testing factors ▸ Things around code testing are also important, and should be comfortable 7

Slide 8

Slide 8 text

Slide 9

Slide 9 text

Definition What's *Comfortable* in This Talk? ▸ In short, it's to accelerate our development ▸ Not fragile, be stable ▸ Nobody likes CI/CD which fail frequently ▸ Be help for other development stages ▸ Finding/ﬁxing bugs, QA, etc. 9

Slide 10

Slide 10 text

Slide 11

Slide 11 text

How to organize non-fragile CI Not Fragile (Stable) Realized by Flexible ▸ No failure is allowed 11

Slide 12

Slide 12 text

Not fragile (stable) does not mean *robust* Flexible ⊂ Not Fragile (Stable) ▸ No failure is allowed ▸ Non-ﬂexible with failures is an evil ▸ BAD: Fail CI anyway if any task fails ▸ BAD: Do not run a Test task if lint fails ▸ How to make it ﬂexible? 12

Slide 13

Slide 13 text

Slide 14

Slide 14 text

Slide 15

Slide 15 text

Check command status and make CI/CD fail if needed Know Rules of Command Set To Be Executed ▸ Basically you have multiple roles in your dep. pipeline 15 Prepare dependencies Lints Reports for test-related res. Reports for others TESTING Assembling Deployment

Slide 16

Slide 16 text

Check command status and make CI/CD fail if needed Consider *Required* Dependencies Between Command Set ▸ You can notice some of commands looks independent 16 Prepare dependencies Lints Reports for test-related res. Reports for others TESTING Assembling Deployment

Slide 17

Slide 17 text

Check command status and make CI/CD fail if needed Check What Will Be Assured by Each Roles ▸ If you wanna assure nothing, you can eject CI/CD :p 17 Prepare dependencies Lints Reports for test-related res. Reports for others TESTING Assembling Deployment Code Quality Product Quality - Dev. experience - Product Quality through QA

Slide 18

Slide 18 text

Check command status and make CI/CD fail if needed So, Which Command Should Fail CI/CD Immediately? ▸ Please note that this slide and your though may differ 18 Prepare dependencies Lints Reports for test-related res. Reports for others TESTING Assembling Deployment No. Basically, yes. As for assembling, yes. Conditional yes for dep. Yes!

Slide 19

Slide 19 text

Examples Other Examples ▸ Use *continue* option when running Gradle tasks ▸ HSBEMFXMJOU%FCVHUFTU%FCVH6OJU5FTUDPOUJOVFP⒐JOF ▸ Use retry-logic to execute unstable commands ▸ For example, 500 error from maven repo. should be retried ▸ Execute reporting system like Danger anyway ▸ Basically we want to know test failures in a failure case 19

Slide 20

Slide 20 text

Slide 21

Slide 21 text

Separate jobs if possible Built *Required* Graph Will Be Your Help ▸ Basically, jobs depend on execution time and CI services 21 Prepare dependencies Lints Reports for test-related res. Reports for others TESTING Assembling Deployment Vital job Test job Deployment job

Slide 22

Slide 22 text

Slide 23

Slide 23 text

Save/load intermediate results of the current/previous build How To Treat Intermediate Results of CI ▸ Major CI services have cache systems ▸ Key-value style cache, repo/branch-based cache ▸ They are useful for build intermediate results like artifact dependencies' cache ▸ But, how about ﬁner-grained cache...? 23

Slide 24

Slide 24 text

Save/load intermediate results of the current/previous build Cache During a Build ▸ Create a new ﬁle right after executing a command ▸ If the ﬁle exists, the command exited with SUCCESS ▸ Otherwise, the command failed 24

Slide 25

Slide 25 text

Save/load intermediate results of the current/previous build Cache Between Builds on a Single PR ▸ Use the CI's cache system? 25

Slide 26

Slide 26 text

Save/load intermediate results of the current/previous build Cache Between Builds on a Single PR ▸ Use the CI's cache system? ▸ NO. ▸ You can use GitHub issues and HTML ▸ Fortunately(?), GitHub issues and comments can parse HTML ▸ It means you can use HTML comment syntax as well 26 *No issue found by ktlint!*

Slide 27

Slide 27 text

Save/load intermediate results of the current/previous build Cache Between Builds on Multiple PRs ▸ Please let me know if there is a smart solution 27

Slide 28

Slide 28 text

Slide 29

Slide 29 text

Get advantages of DeployGate DeployGate? ▸ A mobile app distribution service like Fabric Beta ▸ Annual BBQ planner in US ▸ https://deploygate.com 29

Slide 30

Slide 30 text

Get advantages of DeployGate Advantages of DeployGate (Require Paid Plans) ▸ Faster download through CDN ▸ Distribution tracks 30

Slide 31

Slide 31 text

Get advantages of DeployGate How To Have a Faster Download Experience ▸ CDN produces this advantage ▸ Therefore, ﬁrst download from the signed URL is not fast, unfortunately 31

Slide 32

Slide 32 text

Get advantages of DeployGate How To Have a Faster Download Experience ▸ CDN produces this advantage ▸ Therefore, ﬁrst download from the signed URL is not fast, unfortunately ▸ Download your artifact ﬁrst on CI right after deployment ▸ Note that the URL will be expired, so this tip does not work if new URLs are published 32 artifact_url=$( | jq -r ".results.file") curl -SLs $artifact_url >/dev/null

Slide 33

Slide 33 text

Get advantages of DeployGate Distribution Tracks ▸ Download pages for speciﬁc versions ▸ Like named internal track on Google Play Store 33

Slide 34

Slide 34 text

Get advantages of DeployGate Distribution Tracks ▸ Download pages for speciﬁc versions ▸ Like named internal track on Google Play Store ▸ Deploy release ver., latest develop ver., feature branch ver. ▸ GitHub:jmatsu/dpg will be your help ▸ GitHub:DroidKaigi/conference-app-2019 is a good practice 34

Slide 35

Slide 35 text

Matome dayo Be Comfortable Before/After Code Testing as Well ▸ How to organize non-fragile CI ▸ Reorganize jobs and handle exit statuses manually ▸ Save/load intermediate results of the current/previous build ▸ Use new ﬁles and/or html comments to save/load metadata ▸ Get advantages of DeployGate ▸ Download a ﬁrst artifact on CI, use distribution tracks 35

Slide 36

Slide 36 text

Thank you for listening! Don't Hesitate To Ask Me any Questions ▸ Jumpei Matsuda ▸ @red_fat_daruma ▸ Software Architect @DeployGate ▸ A member of @DroidKaigi 36