Slide 1

Slide 1 text

Advanced Code Coverage with Jenkins, GitHub and API Mocks Oleg Nenashev

Slide 2

Slide 2 text

> Hello from Lugano, CH @oleg_nenashev oleg-nenashev

Slide 3

Slide 3 text

bit.ly/open-roadmaps-for-your-open-communities

Slide 4

Slide 4 text

> whoami @oleg_nenashev oleg-nenashev Dr. Nenashev / Mr. Jenkins Developer tools hacker Community builder & DevRel consultant #RussiansAgainstPutin #StandWithUkraine

Slide 5

Slide 5 text

Agenda ● Jenkins TL;DR ● Code Coverage in Jenkins. 2017 Edition ● Modern Code Coverage in Jenkins, done right ● Boosting coverage with Integration tests ● GitHub Checks API in Jenkins ● WireMock and Testcontainers These slides: speakerdeck.com/onenashev/advanced-code- coverage-with-jenkins-github-and-api-mocking

Slide 6

Slide 6 text

community.jenkins.io/t/coming-soon-advanced-code-coverage-with-jen kins-github-and-api-mocks/11874 Q&A:

Slide 7

Slide 7 text

@oleg_nenashev oleg-nenashev > whoami --jenkins

Slide 8

Slide 8 text

Disclaimer: Taking a break in Jenkins ● On sabbatical in the community ● Reasons: personal events, ongoing war ● I participate in governance and GSoC ● I will be back! CDF Statement We stand with the people of Ukraine. Please assist humanitarian efforts for the Ukrainian people and those affected by the military invasion of Ukraine by supporting international aid organizations, including UNICEF.

Slide 9

Slide 9 text

Image: dronetrest.com/t/diy-self-build-drones-or-ready-to-fly-rtf-drones/1414

Slide 10

Slide 10 text

Expectation

Slide 11

Slide 11 text

Expectation Reality

Slide 12

Slide 12 text

Jenkins is an AUTOMATION FRAMEWORK

Slide 13

Slide 13 text

Система Docs Plugins Configs Infrastruc ture Backend Samples Demos CI/CD System from Jenkins Infrastruc ture Infrastruc ture

Slide 14

Slide 14 text

Система Docs Plugins Configs Infra- structure Backend Samples Demos Infra- structure Infra- structure ? without proper maintenance Any Automation Framework

Slide 15

Slide 15 text

Team A(utomation) Jenkins is a part of the CI system, Dev Productivity Team’s Best Friend (?proof)

Slide 16

Slide 16 text

Code Coverage

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

A FEW PANDEMICS AGO… Jenkins was different

Slide 19

Slide 19 text

Back to 2017… Cobertura publishing In every job???

Slide 20

Slide 20 text

1. Take library components 2. Build your own bicycle Pipeline Add some glue! Back to 2017… Using Pipeline libraries

Slide 21

Slide 21 text

Back to 2017… Global variable wrapper in a lib

Slide 22

Slide 22 text

Back to 2017… Using the global variable as a step

Slide 23

Slide 23 text

Back to 2017… Parallelizing tests

Slide 24

Slide 24 text

Back to 2017… Parallelizing coverage

Slide 25

Slide 25 text

Coverage Plugin

Slide 26

Slide 26 text

Coverage Plugin ● 2018 - Created as Code Coverage API Plugin during GSoC 2018 by Shenyu Zheng ● jenkins.io/projects/gsoc/2018/code-coverage-api-plugin ● 2020/2022 - Major UI/UX revamps ● 2023 - replaced by the Coverage Plugin plugins.jenkins.io/coverage

Slide 27

Slide 27 text

Code Coverage API Plugin (EoL). Supported Adapters ● JaCoCo ● Cobertura, gcov (Cobertura Plugin) ● llvm-cov (llvm-cov plugin) ● OpenCover (OpenCover Plugin) ● Istanbul - via Cobertura Reporter ● PIT Mutation Testing ● SonarQube Generic Coverage Adapter

Slide 28

Slide 28 text

Coverage Plugin. Supported Adapters ● JaCoCo ● Cobertura, gcov (Cobertura Plugin) ● llvm-cov (llvm-cov plugin) ● OpenCover (OpenCover Plugin) ● Istanbul - via Cobertura Reporter ● PIT Mutation Testing ● SonarQube Generic Coverage Adapter

Slide 29

Slide 29 text

Build Unit tests Publish Coverage

Slide 30

Slide 30 text

Usage ● Records coverage produced by your build tool ● Compatible with Jenkins Pipeline and Freestyle jobs

Slide 31

Slide 31 text

Coverage Plugin. Features ● Coverage charts and trends ● Fine grain reporting ● Source code navigation ● Parallel execution support ● Reports merging ● Flexible thresholds ● REST API

Slide 32

Slide 32 text

Features: Statistics

Slide 33

Slide 33 text

Features: Heatmaps

Slide 34

Slide 34 text

Features: Fine grain reporting

Slide 35

Slide 35 text

… and Code Browsing* * Disabled by default due to space/CPU consumption. Enable for projects when you need them

Slide 36

Slide 36 text

Examples Coverage Plugin ● Reference implementation & Dogfooding ● github.com/jenkinsci/coverage-plugin ● ci.jenkins.io/job/Plugins/job/coverage-plugin ● With sources ci.jenkins.io/job/Plugins/job/code-coverage-api-plugin/view/change-requests/job/PR-651/10 Jenkinsfile Core: ● github.com/jenkinsci/jenkins ● Cobertura ● Standard publishing

Slide 37

Slide 37 text

Examples Jenkinsfile Runner: ● https://github.com/jenkinsci/jenkinsfile-runner/ ● JaCoCo / JaCoCo Maven Plugin ● https://github.com/jenkinsci/jenkinsfile-runner/blob/master/pom.xml ● publishCoverage adapters: [jacocoAdapter('target/site/jacoco/jacoco.xml')]

Slide 38

Slide 38 text

How do we do it in Jenkins?

Slide 39

Slide 39 text

Jenkinsfile github.com/jenkinsci/code-coverage- api-plugin/blob/master/Jenkinsfile

Slide 40

Slide 40 text

And the Pipeline library ● buildPlugin() and buildPluginWithGradle()* in the shared Pipeline Library ● 400 codelines but easy for users ● Default settings to meet project quality standards github.com/jenkins-infra/pipeline-library/blob/ master/vars/buildPluginWithGradle.groovy * NOTE: Gradle® and the Gradlephant logo are registered trademarks of Gradle, Inc. "Gradle" means "Gradle Build Tool" and does not reference Gradle, Inc. and/or its subsidiaries.

Slide 41

Slide 41 text

Let’s Integrate with GitHub!

Slide 42

Slide 42 text

GitHub Checks in Jenkins ● “GitHub Checks API for Jenkins project” by Kezhi Xiong ● Google Summer of Code 2021 ● Implementation: Checks API Plugin + GitHub Adapter ● jenkins.io/projects/gsoc/2020/projects/github-checks/

Slide 43

Slide 43 text

Checks API 101 ● Status reporting to GitHub ● Available in GitHub and GitHub Enterprise ● Requires GitHub App authentication ● Checks API documentation: https://docs.github.com/en/rest/reference/checks

Slide 44

Slide 44 text

GitHub App Authentication ● Supported in GitHub Branch Source Plugin 2.7.1+. Patch by Tim Jacomb in April 2020 plugins.jenkins.io/github-branch-source ● Announcement: jenkins.io/blog/2020/04/16/github-app-authentication

Slide 45

Slide 45 text

Setting up GitHub App Authentication Guide: https://github.com/jenkinsci/github-branch-source-plugin/blob/m aster/docs/github-app.adoc 1. Register a new GitHub App in GitHub 2. Download access tokens 3. Install the App in your organization/repository 4. Configure Jenkins credentials 5. Use credentials in Branch Source / Org folders

Slide 46

Slide 46 text

Checks Per Commit 46

Slide 47

Slide 47 text

● Multiple checks ● Status reporting ● Summary and Details ● GitHub-Flavoured Markdown support ● Annotations 47 https://github.com/jenkinsci/je nkins/pull/11148/checks?chec k_run_id=52443616008

Slide 48

Slide 48 text

https://github.com/jenkinsci/code-coverage-api-plugin/runs/2673064852

Slide 49

Slide 49 text

https://github.com/jenkinsci/coverag e-plugin/runs/20604906277 Summary Warnings for the diff

Slide 50

Slide 50 text

No content

Slide 51

Slide 51 text

Integration testing ● Multiple technology stacks and toolchains ● Many teams involved ● Sacred knowledge ● [Almost] always too late

Slide 52

Slide 52 text

Build Unit tests Publish Reports Integration tests Publish Reports WAS

Slide 53

Slide 53 text

No content

Slide 54

Slide 54 text

xkcd.com/303 www.jenkins.io/artwork

Slide 55

Slide 55 text

No content

Slide 56

Slide 56 text

No content

Slide 57

Slide 57 text

xkcd.com/303 www.jenkins.io/artwork

Slide 58

Slide 58 text

Source: https://en.wikipedia.org/wiki/Groundhog_Day_(film)

Slide 59

Slide 59 text

Build Unit tests Publish Coverage Integration tests Publish Coverage ● High infrastructure requirements ● Needs parallelization ● Reports merging ● Virtualization

Slide 60

Slide 60 text

No content

Slide 61

Slide 61 text

Build Unit tests Publish Reports Integration tests Publish Reports NOW “Shift Left” Fast integrations tests are critical

Slide 62

Slide 62 text

Ways to do API integration testing 1. Testing against Production/Staging servers 2. Testing against a simplified portable instance (e.g. Testcontainers) 3. Mocking at the API provider level (e.g. WireMock) 4. Mocking at the code level Slow Fast Accu- rate Nope

Slide 63

Slide 63 text

Staging & QA Tests Build Integration tests Build Tests Gaining confidence in you software

Slide 64

Slide 64 text

Shifting ITs left with Testcontainers and WireMock

Slide 65

Slide 65 text

No content

Slide 66

Slide 66 text

SDKs C/C++ testcontainers.org

Slide 67

Slide 67 text

No content

Slide 68

Slide 68 text

Containers are not always slow! ● On-demand image build ● Caching Docker image builds ● Suspending containers between tests ● Graceful termination ○ github.com/testcontainers/moby-ryuk testcontainers.org

Slide 69

Slide 69 text

No content

Slide 70

Slide 70 text

wiremock.org/docs/stubbing IF (request_url) THEN (response) * * it gets MUCH more complex WireMock Config JSON: Client library response request Client App Mock API Server HTTP/2

Slide 71

Slide 71 text

WireMock in Java wiremock.org/docs/stubbing

Slide 72

Slide 72 text

WireMock Ecosystem & Features wiremock.org/docs

Slide 73

Slide 73 text

WireMock Ecosystem & Features wiremock.org/docs

Slide 74

Slide 74 text

Comparison [1/2] 1. Testing against Production/Staging servers 2. Testing against a simplified/containerized instance (e.g. Testcontainers) 3. Mocking at the API provider level (e.g. WireMock) 4. Mocking at the code level Slow Fast Nope Accu- rate

Slide 75

Slide 75 text

No content

Slide 76

Slide 76 text

WireMock has an official Testcontainers module! testcontainers.com/modules/wiremock

Slide 77

Slide 77 text

Example - JUnit 5

Slide 78

Slide 78 text

No content

Slide 79

Slide 79 text

WireMock as a Proxy Tests Real API Provider ● Fault injection ● Recording ● Protocol Verification Testcontainers

Slide 80

Slide 80 text

No content

Slide 81

Slide 81 text

Code Coverage: Parallel Pipeline Support

Slide 82

Slide 82 text

Code Coverage API: Merging reports ● Parameter in older versions ● Default behavior in new versions ● Keeps diffs and code browsing

Slide 83

Slide 83 text

No content

Slide 84

Slide 84 text

Coverage Plugin. Supported Adapters ● JaCoCo ● Cobertura, gcov (Cobertura Plugin) ● llvm-cov (llvm-cov plugin) ● OpenCover (OpenCover Plugin) ● Istanbul - via Cobertura Reporter ● PIT Mutation Testing ● SonarQube Generic Coverage Adapter

Slide 85

Slide 85 text

wiremock.org/docs/solutions/testcontainers

Slide 86

Slide 86 text

wiremock/wiremock-testcontainers-go

Slide 87

Slide 87 text

No content

Slide 88

Slide 88 text

Credits to ● All Jenkins contributors ● All WireMock contributors ● Code Coverage API / Coverage Plugin ○ Shenyu Zheng ○ Ulli Hafner ○ Florian Orendi - docs ❤ ● GitHub Checks API ○ Kezhi Xiong ○ Ulli Hafner ○ Tim Jacomb ● Google Summer of Code!

Slide 89

Slide 89 text

Takeaways - Coverage Reporting ● Jenkins is one of the strongest platforms for coverage reporting and analysis ● Code Coverage API supports parallelization and reports mergings ● Your technology stack can be supported OOTB or via report format converters ● Jenkins integrates with GitHub Checks

Slide 90

Slide 90 text

Integration Testing ● Great asset if done right ● Shift left your integration testing ● Mocking APIs and services, reducing external dependencies, containers ● There are tools for that, including WireMock and Testcontainers

Slide 91

Slide 91 text

What to do after the talk? 1. Check out our CI in Jenkins repositories ○ github.com/jenkinsci 2. Try out coverage reporting in Jenkins 3. Shift left your integration tests with WireMock and Testcontainers ○ wiremock.org ○ testcontainers.com 4. Share feedback with us! ○ www.jenkins.io/participate/connect ○ slack.wiremock.org

Slide 92

Slide 92 text

It’s [always] a great time to contribute ;) jenkins.io/participate wiremock.org/participate

Slide 93

Slide 93 text

It is a great time to contribute! https://jenkins.io/participate

Slide 94

Slide 94 text

Contacts: E-mail: [email protected] GitHub: oleg-nenashev Twitter: @oleg_nenashev QUESTIONS? community.jenkins.io/t/coming-soon-advanced-code -coverage-with-jenkins-github-and-api-mocks/11874