Build Self-Organized Team through Test Automation with Karate

「DevOpsDays Taipei 2019」（2019-10-16,17,18）での発表資料です。
(Agenda上のタイトルは"Karate for DevOps, with DevOps")
https://devopsdays.tw/
​
我々SET(Software Engineer in Test)が、テスト自動化支援活動を通してLINE社内のあるAPI開発チームの自己組織化をなし得るに至った物語を、ケーススタディとしてお伝えいたします。
また、その中で重要な役割を担ったKarateというAPI Testing Frameworkについてご紹介いたします。
​
- テストの自動化をしてみたものの、なかなかチームに定着しない
- APIの自動テストを作ってみたいけど、どうすればいいか分からない
- (LINEの)SETはどういう活動をしているのか
​
といった悩みや疑問をお持ちの方々の一助になれれば幸いです。

Transcript

1. Build Self-Organized Team through Test Automation Hiroyuki Ito, Isao Takahashi LINE Corporation. DevOpsDays Taipei 2019 with Karate
2. Key Message Building Self-Organized Team by Complemental Relationship between Product Development Team and SET
3. Agenda 1. Case Study : Introduce API Testing 2. Karate : API Testing Framework 3. Develop the tool to accelerate team
4. 1. Case Study : Introduce API Testing
5. Background Story - Issues šSome team’ product tends to be in trouble šBug šMis-configuration šMis-operation šThe team doesn’t aware troubles until customer claims šIt’s huge loss for business (income, reputation)
6. Background Story - Why? šQA resources were not enough to cover their product šThey created E2E test cases in past, but obsoleted šThey understand importance of test, but less knowledge/resources
7. B.K. – Write & Maintain API Test by SET šWe‘ve written API Test from API doc and hearing šWith JUnit, which the team used to use for Unit Test šIt worked, but less contribute from the team !
8. B.K. – Write & Maintain API Test by SET šIt worked, but less contribute from the team ! Still difficult to write? We do too much?
9. ! Still difficult to write?
10. 2. Karate : API Testing Framework
11. What is Karate? API Testing Framework written in Java Write like Cucumber Run like JUnit Testing Mocking Perf-Testing https://github.com/intuit/karate Native Support for JSON-based REST API
12. Latest Technology Rader https://www.thoughtworks.com/radar/languages-and-frameworks/karate
13. Example of Karate
14. How Good is Karate? Simple for Simple Case Cucumber Compatible Reusable Test Case https://github.com/intuit/karate Extensible for Complex Case
15. Demo (Live Coding) šWrite -> Execute -> Show Report
16. Demo (Live Coding) š(
    Live Coding )
21. Demo (Live Coding) š(Live Coding  
    
    )
22. A.K. – Rewrite API Test with Karate šRewrite all test cases by SET šIntroduce Karate to the team B.K. : Before Karate A.K : After Karate_
23. How We Did
24. How We Did Not Enforce, Just Suggest
25. How We Did Not Enforce, Just Suggest Discussion Closely
26. A.K. – Rewrite API Test with Karate šRewrite all test cases by SET šIntroduce Karate to the team šThe team started writing test cases by themselves! !
27. New Issue Emerged šThe team write test cases, and SET reviews šSET maintains CI Job šResolve unmatched version between deployed product and test šExclude specific test case temporarily šHigh cost when trouble occurs
28. ! We do too much?
29. 3. Develop the Tool to Accelerate Team
30. Chatbot for Managing CI Job šConfirm current configuration šModify configuration šTarget branch šTarget label for test šExecute Job on demand
32. Why We Did
33. Why We Did Educating Manual Ops is Not Scaled
34. Why We Did Educating Manual Ops is Not Scaled Demotivate Them
35. A.B. – Write & Maintain Test by Team šThe team writes/reviews test cases by their own šThe team configures/runs CI Job how/when they want šNow we SET don’t support what did B.K. !
36. What was Changed? šReduce trouble šFind trouble earlier šIn Development, Release, and Operation š1 week -> 10 min. to detect regression šThink and act by their own
37. Episodes šTest found Regression Bug! But, the team didn’t aware it was bug ! šTest failed randomly. And, the team utilized it for improvement "
38. Episodes šTest found Regression Bug! But, the team didn’t aware it was bug ! šTest failed randomly. And, the team utilized it for improvement
39. Episodes šTest found Regression Bug! But, the team didn’t aware it was bug šTest failed randomly. And, the team utilized it for improvement !
40. How SET Works šSpend time for communication rather than getting domain knowledge šDelegate actions gradually šDevelop if necessary
41. Self-Organized Teams Make … šProactive behavior ! šThink, Decide and Act šHave confidence that they can solve issues šOwnership to contribute for Product, and Business !
42. Current Action – Reduce MTTR More Quickly šNow, we can notice troubles from test report šHowever, we are still struggling for … “where is cause hidden?” !
43. Sebas-Report Test Log
44. Test Overview
45. Test Detail
46. HTTP Call Trace
47. Key Message (Again) Building Self-Organized Team by Complemental Relationship between Product Development Team and SET
48. Thank You!