Slide 1

Slide 1 text

/ Advanced Code Coverage with Jenkins, GitHub and API Mocks Oleg Nenashev

Slide 2

Slide 2 text

/ > Hello from Lugano, CH @oleg_nenashev oleg-nenashev

Slide 3

Slide 3 text

/ bit.ly/open-roadmaps-for-your-open-communities

Slide 4

Slide 4 text

/ > whoami @oleg_nenashev oleg-nenashev Dr. Nenashev / Mr. Jenkins Developer tools hacker Community builder & DevRel consultant #RussiansAgainstPutin #StandWithUkraine

Slide 5

Slide 5 text

/ P.S: @jbaruch cosplay

Slide 6

Slide 6 text

/ Agenda ● Jenkins TL;DR ● Code Coverage in Jenkins. 2017 Edition ● Modern Code Coverage in Jenkins, done right ● Boosting coverage with Integration tests ● GitHub Checks API in Jenkins ● WireMock and Testcontainers These slides: speakerdeck.com/onenashev/advanced-code- coverage-with-jenkins-github-and-api-mocking

Slide 7

Slide 7 text

/ community.jenkins.io/t/coming-soon-advanced-code-coverage-with-jen kins-github-and-api-mocks/11874 Q&A:

Slide 8

Slide 8 text

/ @oleg_nenashev oleg-nenashev > whoami --jenkins

Slide 9

Slide 9 text

/ Disclaimer: Taking a break in Jenkins ● On sabbatical in the community ● Reasons: personal events, ongoing war ● I participate in governance and GSoC ● I will be back! CDF Statement We stand with the people of Ukraine. Please assist humanitarian efforts for the Ukrainian people and those affected by the military invasion of Ukraine by supporting international aid organizations, including UNICEF.

Slide 10

Slide 10 text

/ Image: dronetrest.com/t/diy-self-build-drones-or-ready-to-fly-rtf-drones/1414

Slide 11

Slide 11 text

/ Expectation

Slide 12

Slide 12 text

/ Expectation Reality

Slide 13

Slide 13 text

/ Jenkins is an AUTOMATION FRAMEWORK

Slide 14

Slide 14 text

/ Система Docs Plugins Configs Infrastruc ture Backend Samples Demos CI/CD System from Jenkins Infrastruc ture Infrastruc ture

Slide 15

Slide 15 text

/ Система Docs Plugins Configs Infra- structure Backend Samples Demos Infra- structure Infra- structure ? without proper maintenance Any Automation Framework

Slide 16

Slide 16 text

/ Team A(utomation) Jenkins is a part of the CI system, Dev Productivity Team’s Best Friend (?proof)

Slide 17

Slide 17 text

/ Code Coverage

Slide 18

Slide 18 text

/

Slide 19

Slide 19 text

/ A FEW PANDEMICS AGO… Jenkins was different

Slide 20

Slide 20 text

/ Back to 2017… Cobertura publishing In every job???

Slide 21

Slide 21 text

/ 1. Take library components 2. Build your own bicycle Pipeline Add some glue! Back to 2017… Using Pipeline libraries

Slide 22

Slide 22 text

/ Back to 2017… Global variable wrapper in a lib

Slide 23

Slide 23 text

/ Back to 2017… Using the global variable as a step

Slide 24

Slide 24 text

/ Back to 2017… Parallelizing tests

Slide 25

Slide 25 text

/ Back to 2017… Parallelizing coverage

Slide 26

Slide 26 text

/ Coverage Plugin

Slide 27

Slide 27 text

/ Coverage Plugin ● 2018 - Created as Code Coverage API Plugin during GSoC 2018 by Shenyu Zheng ● jenkins.io/projects/gsoc/2018/code-coverage-api-plugin ● 2020/2022 - Major UI/UX revamps ● 2023 - replaced by the Coverage Plugin plugins.jenkins.io/coverage

Slide 28

Slide 28 text

/ Code Coverage API Plugin (EoL). Supported Adapters ● JaCoCo ● Cobertura, gcov (Cobertura Plugin) ● llvm-cov (llvm-cov plugin) ● OpenCover (OpenCover Plugin) ● Istanbul - via Cobertura Reporter ● PIT Mutation Testing ● SonarQube Generic Coverage Adapter

Slide 29

Slide 29 text

/ Coverage Plugin. Supported Adapters ● JaCoCo ● Cobertura, gcov (Cobertura Plugin) ● llvm-cov (llvm-cov plugin) ● OpenCover (OpenCover Plugin) ● Istanbul - via Cobertura Reporter ● PIT Mutation Testing ● SonarQube Generic Coverage Adapter

Slide 30

Slide 30 text

/ Build Unit tests Publish Coverage

Slide 31

Slide 31 text

/ Usage ● Records coverage produced by your build tool ● Compatible with Jenkins Pipeline and Freestyle jobs

Slide 32

Slide 32 text

/ Coverage Plugin. Features ● Coverage charts and trends ● Fine grain reporting ● Source code navigation ● Parallel execution support ● Reports merging ● Flexible thresholds ● REST API

Slide 33

Slide 33 text

/ Features: Statistics

Slide 34

Slide 34 text

/ Features: Heatmaps

Slide 35

Slide 35 text

/ Features: Fine grain reporting

Slide 36

Slide 36 text

/ … and Code Browsing* * Disabled by default due to space/CPU consumption. Enable for projects when you need them

Slide 37

Slide 37 text

/ Examples Coverage Plugin ● Reference implementation & Dogfooding ● github.com/jenkinsci/coverage-plugin ● ci.jenkins.io/job/Plugins/job/coverage-plugin ● With sources ci.jenkins.io/job/Plugins/job/code-coverage-api-plugin/view/change-requests/job/PR-651/10 Jenkinsfile Core: ● github.com/jenkinsci/jenkins ● Cobertura ● Standard publishing

Slide 38

Slide 38 text

/ Examples Jenkinsfile Runner: ● https://github.com/jenkinsci/jenkinsfile-runner/ ● JaCoCo / JaCoCo Maven Plugin ● https://github.com/jenkinsci/jenkinsfile-runner/blob/master/pom.xml ● publishCoverage adapters: [jacocoAdapter('target/site/jacoco/jacoco.xml')]

Slide 39

Slide 39 text

/ How do we do it in Jenkins?

Slide 40

Slide 40 text

/ Jenkinsfile github.com/jenkinsci/code-coverage- api-plugin/blob/master/Jenkinsfile

Slide 41

Slide 41 text

/ And the Pipeline library ● buildPlugin() and buildPluginWithGradle in the shared Pipeline Library ● 400 codelines but easy for users ● Default settings to meet project quality standards github.com/jenkins-infra/pipeline-library/blob/ master/vars/buildPluginWithGradle.groovy

Slide 42

Slide 42 text

/ Let’s Integrate with GitHub!

Slide 43

Slide 43 text

/ ● “GitHub Checks API for Jenkins project” by Kezhi Xiong ● Google Summer of Code 2021 ● Implementation: Checks API Plugin + GitHub Adapter ● jenkins.io/projects/gsoc/2020/projects/github-checks/

Slide 44

Slide 44 text

/ Checks API 101 ● Status reporting to GitHub ● Available in GitHub and GitHub Enterprise ● Requires GitHub App authentication ● Checks API documentation: https://docs.github.com/en/rest/reference/checks

Slide 45

Slide 45 text

/ GitHub App Authentication ● Supported in GitHub Branch Source Plugin 2.7.1+. Patch by Tim Jacomb in April 2020 ● https://plugins.jenkins.io/github-branch-source ● Announcement: https://www.jenkins.io/blog/2020/04/16/github-app-auth entication/

Slide 46

Slide 46 text

/ Setting up GitHub App Authentication Guide: https://github.com/jenkinsci/github-branch-source-plugin/blob/m aster/docs/github-app.adoc 1. Register a new GitHub App in GitHub 2. Download access tokens 3. Install the App in your organization/repository 4. Configure Jenkins credentials 5. Use credentials in Branch Source / Org folders

Slide 47

Slide 47 text

/ Checks Per Commit

Slide 48

Slide 48 text

/ ● Multiple checks ● Status reporting ● Summary and Details ● GitHub Markdown support ● Annotations https://github.com/jenkinsci/gradle- plugin/pull/382/checks?check_run_i d=19876936412

Slide 49

Slide 49 text

/ https://github.com/jenkinsci/code-coverage-api-plugin/runs/2673064852

Slide 50

Slide 50 text

/ https://github.com/jenkinsci/coverag e-plugin/runs/20604906277 Summary Warnings for the diff

Slide 51

Slide 51 text

/

Slide 52

Slide 52 text

/ Integration testing ● Multiple technology stacks and toolchains ● Many teams involved ● Sacred knowledge ● [Almost] always too late

Slide 53

Slide 53 text

/ Build Unit tests Publish Reports Integration tests Publish Reports WAS

Slide 54

Slide 54 text

/

Slide 55

Slide 55 text

/ xkcd.com/303 www.jenkins.io/artwork

Slide 56

Slide 56 text

/

Slide 57

Slide 57 text

/

Slide 58

Slide 58 text

/ xkcd.com/303 www.jenkins.io/artwork

Slide 59

Slide 59 text

/ Source: https://en.wikipedia.org/wiki/Groundhog_Day_(film)

Slide 60

Slide 60 text

/ Build Unit tests Publish Coverage Integration tests Publish Coverage ● High infrastructure requirements ● Needs parallelization ● Reports merging ● Virtualization

Slide 61

Slide 61 text

/

Slide 62

Slide 62 text

/ Build Unit tests Publish Reports Integration tests Publish Reports NOW “Shift Left” Fast integrations tests are critical

Slide 63

Slide 63 text

/ Ways to do API integration testing 1. Testing against Production/Staging servers 2. Testing against a simplified portable instance (e.g. Testcontainers) 3. Mocking at the API provider level (e.g. WireMock) 4. Mocking at the code level Slow Fast Accu- rate Nope

Slide 64

Slide 64 text

/ Staging & QA Tests Build Integration tests Build Tests Gaining confidence in you software

Slide 65

Slide 65 text

/ Shifting ITs left with Testcontainers and WireMock

Slide 66

Slide 66 text

/

Slide 67

Slide 67 text

/ SDKs C/C++ testcontainers.org

Slide 68

Slide 68 text

/

Slide 69

Slide 69 text

/ Containers are not always slow! ● On-demand image build ● Caching Docker image builds ● Suspending containers between tests ● Graceful termination ○ github.com/testcontainers/moby-ryuk testcontainers.org

Slide 70

Slide 70 text

/

Slide 71

Slide 71 text

/ wiremock.org/docs/stubbing IF (request_url) THEN (response) * * it gets MUCH more complex WireMock Config JSON: Client library response request Client App Mock API Server HTTP/2

Slide 72

Slide 72 text

/ WireMock in Java wiremock.org/docs/stubbing

Slide 73

Slide 73 text

/ WireMock Ecosystem & Features wiremock.org/docs

Slide 74

Slide 74 text

/ WireMock Ecosystem & Features wiremock.org/docs

Slide 75

Slide 75 text

/ Comparison [1/2] 1. Testing against Production/Staging servers 2. Testing against a simplified/containerized instance (e.g. Testcontainers) 3. Mocking at the API provider level (e.g. WireMock) 4. Mocking at the code level Slow Fast Nope Accu- rate

Slide 76

Slide 76 text

/

Slide 77

Slide 77 text

/ WireMock has an official Testcontainers module! testcontainers.com/modules/wiremock

Slide 78

Slide 78 text

/ Example - JUnit 5

Slide 79

Slide 79 text

/

Slide 80

Slide 80 text

/ WireMock as a Proxy Tests Real API Provider ● Fault injection ● Recording ● Protocol Verification Testcontainers

Slide 81

Slide 81 text

/

Slide 82

Slide 82 text

/ Code Coverage: Parallel Pipeline Support

Slide 83

Slide 83 text

/ Code Coverage API: Merging reports ● Parameter in older versions ● Default behavior in new versions ● Keeps diffs and code browsing

Slide 84

Slide 84 text

/

Slide 85

Slide 85 text

/ Coverage Plugin. Supported Adapters ● JaCoCo ● Cobertura, gcov (Cobertura Plugin) ● llvm-cov (llvm-cov plugin) ● OpenCover (OpenCover Plugin) ● Istanbul - via Cobertura Reporter ● PIT Mutation Testing ● SonarQube Generic Coverage Adapter

Slide 86

Slide 86 text

/ wiremock.org/docs/solutions/testcontainers

Slide 87

Slide 87 text

/ wiremock/wiremock-testcontainers-go

Slide 88

Slide 88 text

/

Slide 89

Slide 89 text

/ Credits to ● All Jenkins contributors ● All WireMock contributors ● Code Coverage API / Coverage Plugin ○ Shenyu Zheng ○ Ulli Hafner ○ Florian Orendi - docs ❤ ● GitHub Checks API ○ Kezhi Xiong ○ Ulli Hafner ○ Tim Jacomb ● Google Summer of Code!

Slide 90

Slide 90 text

/ Takeaways - Coverage Reporting ● Jenkins is one of the strongest platforms for coverage reporting and analysis ● Code Coverage API supports parallelization and reports mergings ● Your technology stack can be supported OOTB or via report format converters ● Jenkins integrates with GitHub Checks

Slide 91

Slide 91 text

/ Integration Testing ● Great asset if done right ● Shift left your integration testing ● Mocking APIs and services, reducing external dependencies, containers ● There are tools for that, including WireMock and Testcontainers

Slide 92

Slide 92 text

/ What to do after the talk? 1. Check out our CI in Jenkins repositories ○ github.com/jenkinsci 2. Try out coverage reporting in Jenkins 3. Shift left your integration tests with WireMock and Testcontainers ○ wiremock.org ○ testcontainers.com 4. Share feedback with us! ○ www.jenkins.io/participate/connect ○ slack.wiremock.org

Slide 93

Slide 93 text

/ It’s [always] a great time to contribute ;) jenkins.io/participate wiremock.org/participate gradle.org/resources

Slide 94

Slide 94 text

/ It is a great time to contribute! https://jenkins.io/participate

Slide 95

Slide 95 text

/ Contacts: E-mail: [email protected] GitHub: oleg-nenashev Twitter: @oleg_nenashev QUESTIONS? community.jenkins.io/t/coming-soon-advanced-code -coverage-with-jenkins-github-and-api-mocks/11874