/
Advanced Code Coverage with
Jenkins, GitHub and API Mocks
Oleg Nenashev
Slide 2
Slide 2 text
/
> Hello from Lugano, CH
@oleg_nenashev
oleg-nenashev
Slide 3
Slide 3 text
/
bit.ly/open-roadmaps-for-your-open-communities
Slide 4
Slide 4 text
/
> whoami
@oleg_nenashev
oleg-nenashev
Dr. Nenashev / Mr. Jenkins
Developer tools hacker
Community builder & DevRel consultant
#RussiansAgainstPutin #StandWithUkraine
Slide 5
Slide 5 text
/
P.S: @jbaruch cosplay
Slide 6
Slide 6 text
/
Agenda
● Jenkins TL;DR
● Code Coverage in Jenkins. 2017 Edition
● Modern Code Coverage in Jenkins, done right
● Boosting coverage with Integration tests
● GitHub Checks API in Jenkins
● WireMock and Testcontainers
These slides:
speakerdeck.com/onenashev/advanced-code-
coverage-with-jenkins-github-and-api-mocking
/
Disclaimer: Taking a break in Jenkins
● On sabbatical in the community
● Reasons: personal events, ongoing war
● I participate in governance and GSoC
● I will be back!
CDF Statement
We stand with the people of Ukraine.
Please assist humanitarian efforts for the
Ukrainian people and those affected by the
military invasion of Ukraine by supporting
international aid organizations, including
UNICEF.
/
Система
Docs
Plugins Configs
Infrastruc
ture
Backend
Samples
Demos
CI/CD System from Jenkins
Infrastruc
ture
Infrastruc
ture
Slide 15
Slide 15 text
/
Система
Docs
Plugins Configs
Infra-
structure
Backend
Samples
Demos
Infra-
structure
Infra-
structure
?
without proper
maintenance
Any Automation Framework
Slide 16
Slide 16 text
/
Team A(utomation)
Jenkins is a part of the CI system,
Dev Productivity Team’s Best Friend (?proof)
Slide 17
Slide 17 text
/
Code Coverage
Slide 18
Slide 18 text
/
Slide 19
Slide 19 text
/
A FEW PANDEMICS AGO…
Jenkins was different
Slide 20
Slide 20 text
/
Back to 2017… Cobertura publishing
In every
job???
Slide 21
Slide 21 text
/
1. Take library
components
2. Build your own
bicycle Pipeline
Add some
glue!
Back to 2017… Using Pipeline libraries
Slide 22
Slide 22 text
/
Back to 2017… Global variable wrapper in a lib
Slide 23
Slide 23 text
/
Back to 2017… Using the global variable as a step
Slide 24
Slide 24 text
/
Back to 2017… Parallelizing tests
Slide 25
Slide 25 text
/
Back to 2017…
Parallelizing coverage
Slide 26
Slide 26 text
/
Coverage Plugin
Slide 27
Slide 27 text
/
Coverage Plugin
● 2018 - Created as Code Coverage API Plugin
during GSoC 2018 by Shenyu Zheng
● jenkins.io/projects/gsoc/2018/code-coverage-api-plugin
● 2020/2022 - Major UI/UX revamps
● 2023 - replaced by the Coverage Plugin
plugins.jenkins.io/coverage
/
And the Pipeline library
● buildPlugin() and buildPluginWithGradle in the shared Pipeline Library
● 400 codelines but easy for users
● Default settings to meet project quality standards
github.com/jenkins-infra/pipeline-library/blob/
master/vars/buildPluginWithGradle.groovy
Slide 42
Slide 42 text
/
Let’s Integrate with GitHub!
Slide 43
Slide 43 text
/
● “GitHub Checks API for Jenkins project” by Kezhi Xiong
● Google Summer of Code 2021
● Implementation: Checks API Plugin + GitHub Adapter
● jenkins.io/projects/gsoc/2020/projects/github-checks/
Slide 44
Slide 44 text
/
Checks API 101
● Status reporting to GitHub
● Available in GitHub and GitHub Enterprise
● Requires GitHub App authentication
● Checks API documentation:
https://docs.github.com/en/rest/reference/checks
Slide 45
Slide 45 text
/
GitHub App Authentication
● Supported in GitHub Branch Source Plugin 2.7.1+.
Patch by Tim Jacomb in April 2020
● https://plugins.jenkins.io/github-branch-source
● Announcement:
https://www.jenkins.io/blog/2020/04/16/github-app-auth
entication/
Slide 46
Slide 46 text
/
Setting up GitHub App Authentication
Guide:
https://github.com/jenkinsci/github-branch-source-plugin/blob/m
aster/docs/github-app.adoc
1. Register a new GitHub App in GitHub
2. Download access tokens
3. Install the App in your organization/repository
4. Configure Jenkins credentials
5. Use credentials in Branch Source / Org folders
Slide 47
Slide 47 text
/
Checks Per
Commit
Slide 48
Slide 48 text
/
● Multiple checks
● Status reporting
● Summary and Details
● GitHub Markdown support
● Annotations
https://github.com/jenkinsci/gradle-
plugin/pull/382/checks?check_run_i
d=19876936412
/
Build
Unit tests
Publish Reports
Integration tests
Publish Reports
NOW
“Shift Left”
Fast integrations
tests are critical
Slide 63
Slide 63 text
/
Ways to do API integration testing
1. Testing against Production/Staging servers
2. Testing against a simplified portable
instance (e.g. Testcontainers)
3. Mocking at the API provider level (e.g.
WireMock)
4. Mocking at the code level
Slow
Fast
Accu-
rate
Nope
Slide 64
Slide 64 text
/
Staging & QA
Tests
Build
Integration tests
Build
Tests
Gaining confidence in you software
Slide 65
Slide 65 text
/
Shifting ITs left
with Testcontainers and WireMock
Slide 66
Slide 66 text
/
Slide 67
Slide 67 text
/
SDKs
C/C++ testcontainers.org
Slide 68
Slide 68 text
/
Slide 69
Slide 69 text
/
Containers are not always slow!
● On-demand image build
● Caching Docker image builds
● Suspending containers between tests
● Graceful termination
○ github.com/testcontainers/moby-ryuk
testcontainers.org
Slide 70
Slide 70 text
/
Slide 71
Slide 71 text
/
wiremock.org/docs/stubbing
IF (request_url)
THEN (response) *
* it gets MUCH more complex
WireMock Config JSON:
Client
library response
request
Client App Mock API Server
HTTP/2
Slide 72
Slide 72 text
/
WireMock in Java
wiremock.org/docs/stubbing
Slide 73
Slide 73 text
/
WireMock Ecosystem & Features
wiremock.org/docs
Slide 74
Slide 74 text
/
WireMock Ecosystem & Features
wiremock.org/docs
Slide 75
Slide 75 text
/
Comparison [1/2]
1. Testing against Production/Staging servers
2. Testing against a simplified/containerized
instance (e.g. Testcontainers)
3. Mocking at the API provider level (e.g.
WireMock)
4. Mocking at the code level
Slow
Fast
Nope
Accu-
rate
Slide 76
Slide 76 text
/
Slide 77
Slide 77 text
/
WireMock has an official Testcontainers module!
testcontainers.com/modules/wiremock
Slide 78
Slide 78 text
/
Example
- JUnit 5
Slide 79
Slide 79 text
/
Slide 80
Slide 80 text
/
WireMock as a Proxy
Tests
Real
API Provider
● Fault injection
● Recording
● Protocol Verification
Testcontainers
Slide 81
Slide 81 text
/
Slide 82
Slide 82 text
/
Code Coverage: Parallel Pipeline Support
Slide 83
Slide 83 text
/
Code Coverage API: Merging reports
● Parameter in older versions
● Default behavior in new versions
● Keeps diffs and code browsing
/
Credits to
● All Jenkins contributors
● All WireMock contributors
● Code Coverage API / Coverage Plugin
○ Shenyu Zheng
○ Ulli Hafner
○ Florian Orendi - docs ❤
● GitHub Checks API
○ Kezhi Xiong
○ Ulli Hafner
○ Tim Jacomb
● Google Summer of Code!
Slide 90
Slide 90 text
/
Takeaways - Coverage Reporting
● Jenkins is one of the strongest platforms for
coverage reporting and analysis
● Code Coverage API supports parallelization and
reports mergings
● Your technology stack can be supported OOTB or
via report format converters
● Jenkins integrates with GitHub Checks
Slide 91
Slide 91 text
/
Integration Testing
● Great asset if done right
● Shift left your integration testing
● Mocking APIs and services, reducing
external dependencies, containers
● There are tools for that, including
WireMock and Testcontainers
Slide 92
Slide 92 text
/
What to do after the talk?
1. Check out our CI in Jenkins repositories
○ github.com/jenkinsci
2. Try out coverage reporting in Jenkins
3. Shift left your integration tests with
WireMock and Testcontainers
○ wiremock.org
○ testcontainers.com
4. Share feedback with us!
○ www.jenkins.io/participate/connect
○ slack.wiremock.org
Slide 93
Slide 93 text
/
It’s [always] a great time to contribute ;)
jenkins.io/participate
wiremock.org/participate
gradle.org/resources
Slide 94
Slide 94 text
/
It is a great time to contribute!
https://jenkins.io/participate