Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Build Self-Organized Team through Test Automation with Karate

Build Self-Organized Team through Test Automation with Karate

「DevOpsDays Taipei 2019」(2019-10-16,17,18)での発表資料です。
(Agenda上のタイトルは"Karate for DevOps, with DevOps")
https://devopsdays.tw/

我々SET(Software Engineer in Test)が、テスト自動化支援活動を通してLINE社内のあるAPI開発チームの自己組織化をなし得るに至った物語を、ケーススタディとしてお伝えいたします。
また、その中で重要な役割を担ったKarateというAPI Testing Frameworkについてご紹介いたします。

- テストの自動化をしてみたものの、なかなかチームに定着しない
- APIの自動テストを作ってみたいけど、どうすればいいか分からない
- (LINEの)SETはどういう活動をしているのか

といった悩みや疑問をお持ちの方々の一助になれれば幸いです。

53850955f15249a1a9dc49df6113e400?s=128

LINE Developers
PRO

October 16, 2019
Tweet

Transcript

  1. Build Self-Organized Team through Test Automation Hiroyuki Ito, Isao Takahashi

    LINE Corporation. DevOpsDays Taipei 2019 with Karate
  2. Key Message Building Self-Organized Team by Complemental Relationship between Product

    Development Team and SET
  3. Agenda 1. Case Study : Introduce API Testing 2. Karate

    : API Testing Framework 3. Develop the tool to accelerate team
  4. 1. Case Study : Introduce API Testing

  5. Background Story - Issues šSome team’ product tends to be

    in trouble šBug šMis-configuration šMis-operation šThe team doesn’t aware troubles until customer claims šIt’s huge loss for business (income, reputation)
  6. Background Story - Why? šQA resources were not enough to

    cover their product šThey created E2E test cases in past, but obsoleted šThey understand importance of test, but less knowledge/resources
  7. B.K. – Write & Maintain API Test by SET šWe‘ve

    written API Test from API doc and hearing šWith JUnit, which the team used to use for Unit Test šIt worked, but less contribute from the team !
  8. B.K. – Write & Maintain API Test by SET šIt

    worked, but less contribute from the team ! Still difficult to write? We do too much?
  9. ! Still difficult to write?

  10. 2. Karate : API Testing Framework

  11. What is Karate? API Testing Framework written in Java Write

    like Cucumber Run like JUnit Testing Mocking Perf-Testing https://github.com/intuit/karate Native Support for JSON-based REST API
  12. Latest Technology Rader https://www.thoughtworks.com/radar/languages-and-frameworks/karate

  13. Example of Karate

  14. How Good is Karate? Simple for Simple Case Cucumber Compatible

    Reusable Test Case https://github.com/intuit/karate Extensible for Complex Case
  15. Demo (Live Coding) šWrite -> Execute -> Show Report

  16. Demo (Live Coding) š( Live Coding )

  17. None
  18. None
  19. None
  20. None
  21. Demo (Live Coding) š(Live Coding   )

  22. A.K. – Rewrite API Test with Karate šRewrite all test

    cases by SET šIntroduce Karate to the team B.K. : Before Karate A.K : After Karate_
  23. How We Did

  24. How We Did Not Enforce, Just Suggest

  25. How We Did Not Enforce, Just Suggest Discussion Closely

  26. A.K. – Rewrite API Test with Karate šRewrite all test

    cases by SET šIntroduce Karate to the team šThe team started writing test cases by themselves! !
  27. New Issue Emerged šThe team write test cases, and SET

    reviews šSET maintains CI Job šResolve unmatched version between deployed product and test šExclude specific test case temporarily šHigh cost when trouble occurs
  28. ! We do too much?

  29. 3. Develop the Tool to Accelerate Team

  30. Chatbot for Managing CI Job šConfirm current configuration šModify configuration

    šTarget branch šTarget label for test šExecute Job on demand
  31. None
  32. Why We Did

  33. Why We Did Educating Manual Ops is Not Scaled

  34. Why We Did Educating Manual Ops is Not Scaled Demotivate

    Them
  35. A.B. – Write & Maintain Test by Team šThe team

    writes/reviews test cases by their own šThe team configures/runs CI Job how/when they want šNow we SET don’t support what did B.K. !
  36. What was Changed? šReduce trouble šFind trouble earlier šIn Development,

    Release, and Operation š1 week -> 10 min. to detect regression šThink and act by their own
  37. Episodes šTest found Regression Bug! But, the team didn’t aware

    it was bug ! šTest failed randomly. And, the team utilized it for improvement "
  38. Episodes šTest found Regression Bug! But, the team didn’t aware

    it was bug ! šTest failed randomly. And, the team utilized it for improvement
  39. Episodes šTest found Regression Bug! But, the team didn’t aware

    it was bug šTest failed randomly. And, the team utilized it for improvement !
  40. How SET Works šSpend time for communication rather than getting

    domain knowledge šDelegate actions gradually šDevelop if necessary
  41. Self-Organized Teams Make … šProactive behavior ! šThink, Decide and

    Act šHave confidence that they can solve issues šOwnership to contribute for Product, and Business !
  42. Current Action – Reduce MTTR More Quickly šNow, we can

    notice troubles from test report šHowever, we are still struggling for … “where is cause hidden?” !
  43. Sebas-Report Test Log

  44. Test Overview

  45. Test Detail

  46. HTTP Call Trace

  47. Key Message (Again) Building Self-Organized Team by Complemental Relationship between

    Product Development Team and SET
  48. Thank You!