Build Self-Organized Team
through Test Automation
Hiroyuki Ito, Isao Takahashi
LINE Corporation.
DevOpsDays Taipei 2019
with
Karate
Slide 2
Slide 2 text
Key Message
Building Self-Organized Team
by Complemental Relationship
between Product Development Team and SET
Slide 3
Slide 3 text
Agenda
1. Case Study : Introduce API Testing
2. Karate : API Testing Framework
3. Develop the tool to accelerate team
Slide 4
Slide 4 text
1. Case Study :
Introduce API Testing
Slide 5
Slide 5 text
Background Story - Issues
Some team’ product tends to be in trouble
Bug
Mis-configuration
Mis-operation
The team doesn’t aware troubles until customer claims
It’s huge loss for business (income, reputation)
Slide 6
Slide 6 text
Background Story - Why?
QA resources were not enough to cover their product
They created E2E test cases in past, but obsoleted
They understand importance of test,
but less knowledge/resources
Slide 7
Slide 7 text
B.K. – Write & Maintain API Test by SET
We‘ve written API Test from API doc and hearing
With JUnit, which the team used to use for Unit Test
It worked, but less contribute from the team !
Slide 8
Slide 8 text
B.K. – Write & Maintain API Test by SET
It worked, but less contribute from the team
!
Still difficult
to write?
We do
too much?
Slide 9
Slide 9 text
!
Still difficult
to write?
Slide 10
Slide 10 text
2. Karate :
API Testing Framework
Slide 11
Slide 11 text
What is Karate?
API Testing
Framework
written in Java
Write like
Cucumber
Run like JUnit
Testing
Mocking
Perf-Testing https://github.com/intuit/karate
Native Support
for JSON-based
REST API
How Good is Karate?
Simple
for
Simple Case
Cucumber
Compatible
Reusable
Test Case
https://github.com/intuit/karate
Extensible
for
Complex Case
Slide 15
Slide 15 text
Demo (Live Coding)
Write -> Execute -> Show Report
Slide 16
Slide 16 text
Demo (Live Coding)
(
Live Coding
)
Slide 17
Slide 17 text
No content
Slide 18
Slide 18 text
No content
Slide 19
Slide 19 text
No content
Slide 20
Slide 20 text
No content
Slide 21
Slide 21 text
Demo (Live Coding)
(Live Coding
)
Slide 22
Slide 22 text
A.K. – Rewrite API Test with Karate
Rewrite all test cases by SET
Introduce Karate to the team
B.K. : Before Karate
A.K : After Karate_
Slide 23
Slide 23 text
How We Did
Slide 24
Slide 24 text
How We Did
Not Enforce, Just Suggest
Slide 25
Slide 25 text
How We Did
Not Enforce, Just Suggest
Discussion
Closely
Slide 26
Slide 26 text
A.K. – Rewrite API Test with Karate
Rewrite all test cases by SET
Introduce Karate to the team
The team started writing test cases by themselves! !
Slide 27
Slide 27 text
New Issue Emerged
The team write test cases, and SET reviews
SET maintains CI Job
Resolve unmatched version between deployed product and
test
Exclude specific test case temporarily
High cost when trouble occurs
Slide 28
Slide 28 text
! We do
too much?
Slide 29
Slide 29 text
3. Develop the Tool
to Accelerate Team
Slide 30
Slide 30 text
Chatbot for Managing CI Job
Confirm current configuration
Modify configuration
Target branch
Target label for test
Execute Job on demand
Slide 31
Slide 31 text
No content
Slide 32
Slide 32 text
Why We Did
Slide 33
Slide 33 text
Why We Did
Educating Manual Ops is Not Scaled
Slide 34
Slide 34 text
Why We Did
Educating Manual Ops is Not Scaled
Demotivate
Them
Slide 35
Slide 35 text
A.B. – Write & Maintain Test by Team
The team writes/reviews test cases by their own
The team configures/runs CI Job how/when they want
Now we SET don’t support what did B.K. !
Slide 36
Slide 36 text
What was Changed?
Reduce trouble
Find trouble earlier
In Development, Release, and Operation
1 week -> 10 min. to detect regression
Think and act by their own
Slide 37
Slide 37 text
Episodes
Test found Regression Bug!
But, the team didn’t aware it was bug !
Test failed randomly.
And, the team utilized it for improvement "
Slide 38
Slide 38 text
Episodes
Test found Regression Bug!
But, the team didn’t aware it was bug !
Test failed randomly.
And, the team utilized it for improvement
Slide 39
Slide 39 text
Episodes
Test found Regression Bug!
But, the team didn’t aware it was bug
Test failed randomly.
And, the team utilized it for improvement !
Slide 40
Slide 40 text
How SET Works
Spend time for communication rather than
getting domain knowledge
Delegate actions gradually
Develop if necessary
Slide 41
Slide 41 text
Self-Organized Teams Make …
Proactive behavior !
Think, Decide and Act
Have confidence that they can solve issues
Ownership to contribute for Product, and Business !
Slide 42
Slide 42 text
Current Action – Reduce MTTR More Quickly
Now, we can notice troubles from test report
However, we are still struggling for …
“where is cause hidden?” !
Slide 43
Slide 43 text
Sebas-Report
Test Log
Slide 44
Slide 44 text
Test Overview
Slide 45
Slide 45 text
Test Detail
Slide 46
Slide 46 text
HTTP Call Trace
Slide 47
Slide 47 text
Key Message (Again)
Building Self-Organized Team
by Complemental Relationship
between Product Development Team and SET