Advanced Code Coverage with
Jenkins, GitHub and API Mocks
Oleg Nenashev
Slide 2
Slide 2 text
> Hello from Lugano, CH
@oleg_nenashev
oleg-nenashev
Slide 3
Slide 3 text
bit.ly/open-roadmaps-for-your-open-communities
Slide 4
Slide 4 text
> whoami
@oleg_nenashev
oleg-nenashev
Dr. Nenashev / Mr. Jenkins
Developer tools hacker
Community builder & DevRel consultant
#RussiansAgainstPutin #StandWithUkraine
Slide 5
Slide 5 text
Agenda
● Jenkins TL;DR
● Code Coverage in Jenkins. 2017 Edition
● Modern Code Coverage in Jenkins, done right
● Boosting coverage with Integration tests
● GitHub Checks API in Jenkins
● WireMock and Testcontainers
These slides:
speakerdeck.com/onenashev/advanced-code-
coverage-with-jenkins-github-and-api-mocking
Disclaimer: Taking a break in Jenkins
● On sabbatical in the community
● Reasons: personal events, ongoing war
● I participate in governance and GSoC
● I will be back!
CDF Statement
We stand with the people of Ukraine.
Please assist humanitarian efforts for the
Ukrainian people and those affected by the
military invasion of Ukraine by supporting
international aid organizations, including
UNICEF.
Система
Docs
Plugins Configs
Infrastruc
ture
Backend
Samples
Demos
CI/CD System from Jenkins
Infrastruc
ture
Infrastruc
ture
Slide 14
Slide 14 text
Система
Docs
Plugins Configs
Infra-
structure
Backend
Samples
Demos
Infra-
structure
Infra-
structure
?
without proper
maintenance
Any Automation Framework
Slide 15
Slide 15 text
Team A(utomation)
Jenkins is a part of the CI system,
Dev Productivity Team’s Best Friend (?proof)
Slide 16
Slide 16 text
Code Coverage
Slide 17
Slide 17 text
No content
Slide 18
Slide 18 text
A FEW PANDEMICS AGO…
Jenkins was different
Slide 19
Slide 19 text
Back to 2017… Cobertura publishing
In every
job???
Slide 20
Slide 20 text
1. Take library
components
2. Build your own
bicycle Pipeline
Add some
glue!
Back to 2017… Using Pipeline libraries
Slide 21
Slide 21 text
Back to 2017… Global variable wrapper in a lib
Slide 22
Slide 22 text
Back to 2017… Using the global variable as a step
Slide 23
Slide 23 text
Back to 2017… Parallelizing tests
Slide 24
Slide 24 text
Back to 2017…
Parallelizing coverage
Slide 25
Slide 25 text
Coverage Plugin
Slide 26
Slide 26 text
Coverage Plugin
● 2018 - Created as Code Coverage API Plugin
during GSoC 2018 by Shenyu Zheng
● jenkins.io/projects/gsoc/2018/code-coverage-api-plugin
● 2020/2022 - Major UI/UX revamps
● 2023 - replaced by the Coverage Plugin
plugins.jenkins.io/coverage
And the Pipeline library
● buildPlugin() and buildPluginWithGradle()* in the shared Pipeline Library
● 400 codelines but easy for users
● Default settings to meet project quality standards
github.com/jenkins-infra/pipeline-library/blob/
master/vars/buildPluginWithGradle.groovy
* NOTE: Gradle® and the Gradlephant logo are registered trademarks of Gradle, Inc. "Gradle"
means "Gradle Build Tool" and does not reference Gradle, Inc. and/or its subsidiaries.
Slide 41
Slide 41 text
Let’s Integrate with GitHub!
Slide 42
Slide 42 text
GitHub Checks in Jenkins
● “GitHub Checks API for Jenkins project” by Kezhi Xiong
● Google Summer of Code 2021
● Implementation: Checks API Plugin + GitHub Adapter
● jenkins.io/projects/gsoc/2020/projects/github-checks/
Slide 43
Slide 43 text
Checks API 101
● Status reporting to GitHub
● Available in GitHub and GitHub Enterprise
● Requires GitHub App authentication
● Checks API documentation:
https://docs.github.com/en/rest/reference/checks
Slide 44
Slide 44 text
GitHub App Authentication
● Supported in GitHub Branch Source Plugin 2.7.1+.
Patch by Tim Jacomb in April 2020
plugins.jenkins.io/github-branch-source
● Announcement:
jenkins.io/blog/2020/04/16/github-app-authentication
Slide 45
Slide 45 text
Setting up GitHub App Authentication
Guide:
https://github.com/jenkinsci/github-branch-source-plugin/blob/m
aster/docs/github-app.adoc
1. Register a new GitHub App in GitHub
2. Download access tokens
3. Install the App in your organization/repository
4. Configure Jenkins credentials
5. Use credentials in Branch Source / Org folders
Slide 46
Slide 46 text
Checks Per
Commit
46
Slide 47
Slide 47 text
● Multiple checks
● Status reporting
● Summary and Details
● GitHub-Flavoured
Markdown support
● Annotations
47
https://github.com/jenkinsci/je
nkins/pull/11148/checks?chec
k_run_id=52443616008
Build
Unit tests
Publish Reports
Integration tests
Publish Reports
NOW
“Shift Left”
Fast integrations
tests are critical
Slide 62
Slide 62 text
Ways to do API integration testing
1. Testing against Production/Staging servers
2. Testing against a simplified portable
instance (e.g. Testcontainers)
3. Mocking at the API provider level (e.g.
WireMock)
4. Mocking at the code level
Slow
Fast
Accu-
rate
Nope
Slide 63
Slide 63 text
Staging & QA
Tests
Build
Integration tests
Build
Tests
Gaining confidence in you software
Slide 64
Slide 64 text
Shifting ITs left
with Testcontainers and WireMock
Slide 65
Slide 65 text
No content
Slide 66
Slide 66 text
SDKs
C/C++ testcontainers.org
Slide 67
Slide 67 text
No content
Slide 68
Slide 68 text
Containers are not always slow!
● On-demand image build
● Caching Docker image builds
● Suspending containers between tests
● Graceful termination
○ github.com/testcontainers/moby-ryuk
testcontainers.org
Slide 69
Slide 69 text
No content
Slide 70
Slide 70 text
wiremock.org/docs/stubbing
IF (request_url)
THEN (response) *
* it gets MUCH more complex
WireMock Config JSON:
Client
library response
request
Client App Mock API Server
HTTP/2
Slide 71
Slide 71 text
WireMock in Java
wiremock.org/docs/stubbing
Slide 72
Slide 72 text
WireMock Ecosystem & Features
wiremock.org/docs
Slide 73
Slide 73 text
WireMock Ecosystem & Features
wiremock.org/docs
Slide 74
Slide 74 text
Comparison [1/2]
1. Testing against Production/Staging servers
2. Testing against a simplified/containerized
instance (e.g. Testcontainers)
3. Mocking at the API provider level (e.g.
WireMock)
4. Mocking at the code level
Slow
Fast
Nope
Accu-
rate
Slide 75
Slide 75 text
No content
Slide 76
Slide 76 text
WireMock has an official Testcontainers module!
testcontainers.com/modules/wiremock
Slide 77
Slide 77 text
Example
- JUnit 5
Slide 78
Slide 78 text
No content
Slide 79
Slide 79 text
WireMock as a Proxy
Tests
Real
API Provider
● Fault injection
● Recording
● Protocol Verification
Testcontainers
Slide 80
Slide 80 text
No content
Slide 81
Slide 81 text
Code Coverage: Parallel Pipeline Support
Slide 82
Slide 82 text
Code Coverage API: Merging reports
● Parameter in older versions
● Default behavior in new versions
● Keeps diffs and code browsing
Credits to
● All Jenkins contributors
● All WireMock contributors
● Code Coverage API / Coverage Plugin
○ Shenyu Zheng
○ Ulli Hafner
○ Florian Orendi - docs ❤
● GitHub Checks API
○ Kezhi Xiong
○ Ulli Hafner
○ Tim Jacomb
● Google Summer of Code!
Slide 89
Slide 89 text
Takeaways - Coverage Reporting
● Jenkins is one of the strongest platforms for
coverage reporting and analysis
● Code Coverage API supports parallelization and
reports mergings
● Your technology stack can be supported OOTB or
via report format converters
● Jenkins integrates with GitHub Checks
Slide 90
Slide 90 text
Integration Testing
● Great asset if done right
● Shift left your integration testing
● Mocking APIs and services, reducing
external dependencies, containers
● There are tools for that, including
WireMock and Testcontainers
Slide 91
Slide 91 text
What to do after the talk?
1. Check out our CI in Jenkins repositories
○ github.com/jenkinsci
2. Try out coverage reporting in Jenkins
3. Shift left your integration tests with
WireMock and Testcontainers
○ wiremock.org
○ testcontainers.com
4. Share feedback with us!
○ www.jenkins.io/participate/connect
○ slack.wiremock.org
Slide 92
Slide 92 text
It’s [always] a great time to contribute ;)
jenkins.io/participate
wiremock.org/participate
Slide 93
Slide 93 text
It is a great time to contribute!
https://jenkins.io/participate