Creating Faster and More Reliable Web Tests with Blended Testing

Jan Molak Lead Developer of Serenity/JS

Contributor to the Screenplay Pattern

Co-author of "BDD in Action, 2nd edition"
Trainer, Consultant

janmolak.com
Blended Testing

View Slide

Jan Molak | serenity-js.org
Blended Testing
Interacting with multiple interfaces of the system under test 
to improve their stability, reliability, and performance  
without reducing their coverage.

View Slide

Tool-centric E2E automation
How picking "the best tool for the job" can get us in trouble

View Slide

• Scenarios focus on interacting with one interface at the time
System 
Under Test
Web UI
System 
State

View Slide

System 
Under Test
Web UI
System 
State
Web Integration

Tool

View Slide

System 
Under Test
Web

Test Suite
Web UI
System 
State
Web Integration

Tool

View Slide


• Test suites organised per-interface
System 
Under Test
Web

Test Suite
Web UI
REST API System 
State
Web Integration

Tool

View Slide


System 
Under Test
Web

Test Suite
Web UI
REST API System 
State
Web Integration

Tool
API Integration
Tool

View Slide


System 
Under Test
Web

Test Suite
Web UI
REST API
API

Test Suite
System 
State
Web Integration

Tool
API Integration
Tool

View Slide


• Little or no code reuse => high maintenance or poor coverage
System 
Under Test
Web

Test Suite
Web UI
REST API
API

Test Suite
System 
State
Web Integration

Tool
API Integration
Tool

View Slide


System 
Under Test
Web

Test Suite
Web UI
Mobile App
REST API
API

Test Suite
System 
State
Web Integration

Tool
API Integration
Tool
?

View Slide


System 
Under Test
Web

Test Suite
Web UI
Mobile App
REST API
API

Test Suite
System 
State
Web Integration

Tool
API Integration
Tool
Messaging API
?
?

View Slide


System 
Under Test
Web

Test Suite
Web UI
Mobile App
REST API
API

Test Suite
System 
State
Web Integration

Tool
API Integration
Tool
Batch API
Messaging API
?
?
?

View Slide

Pick "the best tool" for the job - too many jobs, too many tools

View Slide

UI-centric E2E automation
Check the UI, like the user would

View Slide

• Scenarios only ever interact with the web UI
System 
Under Test
Web UI
Mobile App
REST API
System 
State
Batch API
Messaging API

View Slide

System 
Under Test
Web UI
Mobile App
REST API
System 
State
Web Integration

Tool
Batch API
Messaging API

View Slide

System 
Under Test
Web

Test Suite
Web UI
Mobile App
REST API
System 
State
Web Integration

Tool
Batch API
Messaging API

View Slide

• Interactions with other interfaces accidental, ad-hoc, or missed
System 
Under Test
Web

Test Suite
Web UI
Mobile App
REST API
System 
State
Web Integration

Tool
Batch API
Messaging API

View Slide

System 
Under Test
Web

Test Suite
Web UI
Mobile App
REST API
System 
State
Web Integration

Tool
Batch API
Messaging API
?
?

View Slide


• Slow and unreliable tests => high failure diagnosis costs
System 
Under Test
Web

Test Suite
Web UI
Mobile App
REST API
System 
State
Web Integration

Tool
Batch API
Messaging API
?
?

View Slide

• Slow and unreliable tests => high failure diagnosis costs
System 
Under Test
Web

Test Suite
Web UI
Mobile App
REST API
System 
State
Web Integration

Tool
Batch API
Messaging API
?
?
API

Test Suite
API Integration
Tool

View Slide

Check the UI, like the user would - users don't care about the UI

View Slide

Blended testing
Check the UI, like the user would - users don't care about the UI
Focus on actors, blend interfaces and tools

View Slide

UCD mental model for Blended Testing
Roles - Who?

View Slide

Roles - Who?
↳ Goals - Why?

View Slide

Roles - Who?
↳ Goals - Why?
↳ Tasks - What?

View Slide

Roles - Who?
↳ Goals - Why?
↳ Tasks - What?
↳ Interactions - How?
Mental model inspired by: 
- "In praise of abstraction" by Kevin Lawrence, 2007

- "A bit of UCD for BDD and ATDD" by Antony Marcano, 2011

View Slide

Roles - Who?
↳ Goals - Why?
↳ Tasks - What?
↳ Interactions - How?
↳ Integration tools
↳ Interfaces - Implementation detail

View Slide

Blended testing
• Acceptance tests focus on actors, goals, and tasks

• Use only interfaces necessary to do a meaningful test
System 
Under Test
Acceptance

Test Suite
System 
State
Integration Tools Interfaces
Test
Automation
Framework

View Slide

Blended testing

• Use only interfaces necessary to do a meaningful test
• Building multimodal automation frameworks is really hard
System 
Under Test
Acceptance

Test Suite
System 
State
Integration Tools Interfaces
Test
Automation
Framework
?

View Slide

Actor-centred blended testing
How to blend with Serenity/JS Screenplay Pattern

View Slide

Screenplay Pattern
Using your domain language to implement test scenarios 
that model what tasks the actors need to perform  
in order to accomplish their goals  
when interacting with the system under test.

View Slide

Screenplay Pattern: Actors and roles
Feature: Deals on apples

Scenario: 3-for-2 on apples

Given a pack of apples costs $2.50

And there’s a 3-for-2 deal on apples

When the shopper wants to buy 3 packs of apples

Then they should be requested to pay $5.00

Scenario: 3-for-2 deal reminder





But they should be reminded about the 3-for-2 deal

View Slide












import {actorCalled} from '@serenity-js/core'

actorCalled('Shopper')

View Slide













//
Shopper role:

//
- finds products of interest

//
- manages basket

//
- makes payments


View Slide













//
TODO: who's managing the shop?

//
Shopper role:

//

//
- manages basket

//
- makes payments


View Slide













//
Shopkeeper role:

//
- manages stock

//
- manages prices

//
- manages deals

actorCalled('Shopkeeper')

//
Shopper role:

//

//
- manages basket

//
- makes payments


View Slide













//
Shopkeeper goals?


//
Shopper goals?

Screenplay Pattern: Goals

View Slide













//
Shopkeeper goals:

//
- price apples at $2.50

//
- enable 3-for-2 deal on apples


//
Shopper goals:

//
- find apples

//
- check price

//
- check missed deal suggestions

Screenplay Pattern: Goals

View Slide













actorCalled('Shopkeeper').attemptsTo(
 
setProductPrice('apples', '$2.50'),

enableDeal('3-for-2', 'apples'),
 
)

actorCalled('Shopper').attemptsTo(

findProduct('apples'),

addProductToBasket('apples', 3),

verifyBasketTotal('$5.00'),

verifyMissedDeals([ ]),

)
Screenplay Pattern: Tasks

View Slide













 

 
)






)
Task names based on your domain

View Slide













 

 
)






)
High-level tasks 
unconstrained by 
interface implementation

View Slide

 

 
)






)
Implementing Tasks
import {Task} from '@serenity-js/core'

const setProductPrice = (name, price)
=>

Task.where(`#actor sets price of ${ name } at ${ price }`,
 
//
…sub-tasks
 
)
Tasks are composites 
of activities

View Slide

 

 
)
Implementing API Tasks

import {Send, PostRequest} from '@serenity-js/rest'

=>

 
Send.a(PostRequest.to('/products').with({

name: name,
 
price: price,

}),

)
Interacting with 
REST APIs

View Slide

 

 
)

import {Send, LastResponse, PostRequest} from '@serenity-js/rest'

import {Ensure, equals} from '@serenity-js/assertions'

=>

 

name: name,
 
price: price,

}),

Ensure.that(LastResponse.status(), equals(200))

)
Composing assertions 
into tasks

View Slide

import {

CallAnApi

} from '@serenity-sj/rest'


.whoCan(

CallAnApi.using(axios)

)

.attemptsTo(
 
setProductPrice('apples','$2.50'),

 
)



=>

 

name: name,
 
price: price,

}),


)
Abilities enable interactions

View Slide

import {

CallAnApi



.whoCan(


)

.attemptsTo(
 

 
)



=>

setProductPriceViaAdminApi(name, price)

const setProductPriceViaAdminApi = (name, price)
=>

 

name: name,
 
price: price,

}),


)

View Slide

import {

CallAnApi



.whoCan(


)

.attemptsTo(
 

 
)
Alternative implementations



=>

setProductPriceViaAdminApi(name, price)

=>

//
…

const setProductPriceViaAdminUI = (name, price)
=>


authenticateWithStoreAdminPanel(adminCredentials()),

openProductCatalog(),

findProductByName(name),

changeProductPrice(price),
 
)

of the same task

View Slide

import {

BrowseTheWebWithWebdriverIO

} from '@serenity-sj/webdriverio'


.whoCan(

CallAnApi.using(axios),

 
.using(browser)

)

.attemptsTo(
 

 
)



=>

setProductPriceViaAdminUI(name, price)

=>

//
…

const setProductPriceViaAdminUI = (name, price)
=>


authenticateWithStoreAdminPanel(adminCredentials()),

openProductCatalog(),

findProductByName(name),

changeProductPrice(price),
 
)
The same test scenario,

interacting through

the web UI

View Slide

import {




.whoCan(


 
.using(browser)

)

.attemptsTo(
 

 
)
Implementing web UI tasks

import {Enter, Click, Text} from '@serenity-js/web'


const authenticateWithStoreAdminPanel = ({username, password})
=>

Task.where(`#actor authenticates with StoreAdmin`,

Enter.theValue(username).into(LoginForm.usernameField()),

Enter.theValue(password).into(LoginForm.passwordField()),

Click.on(LoginForm.submitButton()),

Ensure.that(Text.of(AdminPanel.user()), equals(username))

)
Web interactions

View Slide

import {




.whoCan(


 
.using(browser)

)

.attemptsTo(
 

 
)
Implementing web UI tasks

import {Enter,Click,Text,PageElement,By} from '@serenity-js/web'


const authenticateWithStoreAdminPanel = ({username, password})
=>

Task.where(`#actor authenticates with StoreAdmin`,

Enter.theValue(username).into(LoginForm.usernameField()),

Enter.theValue(password).into(LoginForm.passwordField()),

Click.on(LoginForm.submitButton()),

Ensure.that(Text.of(AdminPanel.user()), equals(username))

)

const LoginForm = {

usernameField: ()
=>

PageElement.located(By.css('input.username'))

.describedAs('username field)

passwordField:
//
…

}
Locating page

elements

View Slide

Blending like a pro
with Serenity/JS

View Slide

Blended testing like a pro

await actorCalled('Shopkeeper').attemptsTo(
 

 
)

await actorCalled('Shopper').attemptsTo(





)

View Slide


• Tasks provide alternative implementations and enable code reuse

await actorCalled('Shopkeeper').attemptsTo(
 

 
)

await actorCalled('Shopper').attemptsTo(





)
Serenity/JS
Custom 
task libraries
findProduct
 
setProductPrice
 
enableDeal
Tasks

View Slide


System 
Under Test
Acceptance 
Tests
System 
State
Serenity/JS 
Modules
Custom 
task libraries
Targeted 
Tests

View Slide



• Standardised patterns
System 
Under Test
Acceptance 
Tests
System 
State
Serenity/JS 
Modules
Custom
Integrations
Custom 
task libraries
Targeted 
Tests

View Slide



• Standardised patterns mean consistent reporting

View Slide

Start automating
with Serenity/JS in 15 minutes 🚀
serenity-js.org

View Slide

Creating Faster and More Reliable Web Tests with Blended Testing

Creating Faster and More Reliable Web Tests with Blended Testing

Jan Molak

Other Decks in Programming

Featured

Transcript