Slide 1
Slide 1 text
No content
Slide 2
Slide 2 text
Self Introduction
Takuya Suemura
Software Developer / Tester
Test Automation Specialist @
Autify, Inc.
Committer of CodeceptJS
Slide 3
Slide 3 text
No content
Slide 4
Slide 4 text
Features
Record & Playback
Self Healing
Parallel Execution
Cross-Browser Compatibility Test
Slide 5
Slide 5 text
Agenda
About Testability and why we need it
Adding new small test into the low testability code
Building a stable product with high testability
How to make more testable an app on the E2E level
Slide 6
Slide 6 text
Developing vs Testing
There is a tendency to focus on small details during development:
Incomplete logic
Effect on other components
DB becomes locked for a long period
Cannot think about what will bring great value to the user
Testing is an effective approach for various concerns, big and small
Slide 7
Slide 7 text
Testing Small Concern
Using Complex UI
Slide 8
Slide 8 text
TESTABILITY
Slide 9
Slide 9 text
Testing Small Concern
Using Complex UI
...it means the system has low testability
Slide 10
Slide 10 text
What's the testability?
A part of software quality
Leverage for quality
improvement
A basis for various tests
Slide 11
Slide 11 text
Difficulity of E2E Automation
Slide 12
Slide 12 text
Difficulity of E2E Automation
Limited operating point – everything needs to be operated through UI
Functions cannot be split – cannot split functions which is not useful for the user
Slide 13
Slide 13 text
Ice Cream Cone
Slide 14
Slide 14 text
Test Pyramid
And
Ice Cream Cone
Slide 15
Slide 15 text
Why does this icecream occur?
Lack of understanding of testing
Doesn’t know testing other than acceptance testing
“Unit test? What is that? Can you eat it?”
Misunderstanding of refactoring
Starts off prioritizing implementation without any automated testing
“We can just refactor it later, right? :D”
Slide 16
Slide 16 text
Icecream = Value of product
Slide 17
Slide 17 text
Functional
Integration
Unit
Integration
Unit
Use Case
External
Integration
Performance
/ Security
Use Case
External
Integration
Performance
/ Security
UI Integration /
Backend API
Big Cone
Slide 18
Slide 18 text
Example:
A simple new user registration form
1. Display an error if a required field is empty
2. Display an error if the email address does not
contain “@”
3. [Submit] button cannot be clicked if there is an error
4. Email addresses that are already registered cannot
be registered
5. Registration is complete when the user clicks the link
in the new registration email
Slide 19
Slide 19 text
Separating
the client
and the
server-
side
Slide 20
Slide 20 text
Does UI testing always
need a real backend?
https://unsplash.com/photos/xGtHjC_QNJM
Slide 21
Slide 21 text
UI Integration Test
Use lightweight & flexible browser automation tool
Puppeteer(https://pptr.dev/)
Cypress(https://www.cypress.io/)
Use mock instead of the real backend
MockServer(https://www.mock-server.com/)
PollyJS(https://netflix.github.io/pollyjs/#/)
Slide 22
Slide 22 text
Feature('mock');
Scenario('Interrupt suggestion', (I) => {
I.amOnPage('https://www.google.com/')
I.mockRequest('GET', 'https://www.google.com/complete/search?*', 404)
I.fillField('.gLFyf.gsfi', 'OnlineTestConf')
});
Slide 23
Slide 23 text
Testing backend APIs
public function testBasicExample()
{
$response = $this->withHeaders([
'X-Header' => 'Value',
])->json('POST', '/user', ['name' => 'Sally']);
$response
->assertStatus(201)
->assertJson([
'created' => true,
]);
}
https://laravel.com/docs/7.x/http-tests
Slide 24
Slide 24 text
Other tools for testing backend APIs
Karate(https://github.com/intuit/karate)
Tavern(https://github.com/taverntesting/tavern)
Postman(https://www.postman.com/)
Slide 25
Slide 25 text
Cone To Cup
Integration
Unit
Use Case
External
Integration
Performance
/ Security
UI Component /
Backend API
Integration
Unit
Use Case
External
Integration
Performance
/ Security
UI Integration /
Backend API
Slide 26
Slide 26 text
Use
Architectures
https://en.wikipedia.org/wiki/Model–view–
viewmodel
Slide 27
Slide 27 text
Testing UI Component
Slide 28
Slide 28 text
Get More Icecream
https://unsplash.com/photos/Wpg3Qm0zaGk
Slide 29
Slide 29 text
Efficiently create test
data from UI
Using Record & Playback Tool
WildFire
iMacros
Slide 30
Slide 30 text
Reuse your automation code during a manual test
$ npx codeceptjs shell
I.loginAs('admin')
I.addAllItemToCart()
I.proceedOrder()
Slide 31
Slide 31 text
Automate test
preparation
Add APIs for preparing test data
& preconditions
Combinate those APIs with a
process automation tool
n8n(nodemation)
(https://n8n.io/)
Slide 32
Slide 32 text
Testing EMail & SMS
Mail
MailCatcher(https://mailcatcher.me/)
MailHog(https://github.com/mailhog/MailHog)
MailTrap(https://mailtrap.io/)
Mail & SMS
Mailosaur(https://mailosaur.com/)
Slide 33
Slide 33 text
Automatability for E2E Testing
https://unsplash.com/photos/FTfjMijq-Ws
Slide 34
Slide 34 text
Semantic Locator
I.FillField('Username', 'takuyasuemura')
I.FillField('Email', '[email protected]')
I.FillField('Password', 'P@ssword')
I.click('Sign up for GitHub')
https://github.com/
Slide 35
Slide 35 text
Utilize the meaning
within a structure
const modal = label =>
locate('.modal').withText(label)
within(modal('Modal title'), () => {
I.see('Modal body text goes here.')
I.click('Save changes')
})
https://getbootstrap.com/docs/4.0/components/modal/
Slide 36
Slide 36 text
Using Multiple
Locators
Selenium IDE's fallback locator
Slide 37
Slide 37 text
ID: not matched
class: matched
alter text: matched
coodinate: not matched
image source: matched
.... Confidence: 80%
AI for
Automatability
Slide 38
Slide 38 text
Why we need to improve the testability?
Slide 39
Slide 39 text
To assure the value for customers
https://unsplash.com/photos/oqStl2L5oxI
Slide 40
Slide 40 text
Conclusion: Larger cup, more icecream
Have more tests focusing on targeted concern by effectively using the existing
interface and mock.
Remake applications protected by the test so that they are even easier to test.
Maximize the product’s appeal by testing products high in internal quality.
Slide 41
Slide 41 text
Enjoy Testing!