Technology supporting the release of HOME'S

Slide 1

Slide 1 text

Technology supporting the release of HOME'S Naoto Kishino (Software Engineer in Test at LIFULL) April 19th, 2019 at Selenium Conf Tokyo

Slide 2

Slide 2 text

© LIFULL Co.,Ltd. 本書の無断転載、複製を固く禁じます。 2 Introduction Naoto Kishino (木住野奈夫人) Software Engineer in Test Group (3 members) Test Automation/CI/CD Group vision : 自動化と高速化を進め、プロとして最高のDXを実現する Create the best DX (Developer Experience) in the world. @naotospace NA070

Slide 3

Slide 3 text

© LIFULL Co.,Ltd. 本書の無断転載、複製を固く禁じます。 3 Overview We accelerated release frequency from two times a week to four times a week. We achieved this goal by solving some problems in the automated system testing which is executed before release.

Slide 4

Slide 4 text

Slide 5

Slide 5 text

Slide 6

Slide 6 text

Slide 7

Slide 7 text

Slide 8

Slide 8 text

Slide 9

Slide 9 text

Slide 10

Slide 10 text

Slide 11

Slide 11 text

Slide 12

Slide 12 text

Slide 13

Slide 13 text

© LIFULL Co.,Ltd. 本書の無断転載、複製を固く禁じます。 13 Release! Test About development process Hundreds of new features release per month. It was able to release on Tuesday and Thursday. Monday Tuesday Wednesday Thursday Friday ◯ ◯

Slide 14

Slide 14 text

© LIFULL Co.,Ltd. 本書の無断転載、複製を固く禁じます。 14 About development problem Large-scale features can only released on Tuesday. Monday Tuesday Wednesday Thursday Friday X (Delay) → → → → Release Release! Too Slow….

Slide 15

Slide 15 text

© LIFULL Co.,Ltd. 本書の無断転載、複製を固く禁じます。 15 About development Team Change the number of releasable day in a week Monday Tuesday Wednesday Thursday Friday ◯ ◯ Monday Tuesday Wednesday Thursday Friday ◯ ◯ ◯ ◯

Slide 16

Slide 16 text

Slide 17

Slide 17 text

© LIFULL Co.,Ltd. 本書の無断転載、複製を固く禁じます。 17 Problems in our automated system testing Release! • E2E(End-to-End) test using Selenium. • Executed before release. • To make sure no incident is increasing by degrades. System testing

Slide 18

Slide 18 text

Slide 19

Slide 19 text

Slide 20

Slide 20 text

Slide 21

Slide 21 text

© LIFULL Co.,Ltd. 本書の無断転載、複製を固く禁じます。 21 2. Too flaky ● Test code were implemented in complex processing ● Some cases include unsuitable step in automated testing Problems in our automated system testing

Slide 22

Slide 22 text

Slide 23

Slide 23 text

Slide 24

Slide 24 text

Slide 25

Slide 25 text

© LIFULL Co.,Ltd. 本書の無断転載、複製を固く禁じます。 25 Solution 1 Recreate test plan Define the scopes about manual testing and automated testing • Manual：Specification based – New features and regression test • Automated：Risk based – Comprehensive regression test

Slide 26

Slide 26 text

Slide 27

Slide 27 text

Slide 28

Slide 28 text

Slide 29

Slide 29 text

Slide 30

Slide 30 text

Slide 31

Slide 31 text

Slide 32

Slide 32 text

Slide 33

Slide 33 text

© LIFULL Co.,Ltd. 本書の無断転載、複製を固く禁じます。 33 Solution 2 New testing framework “Bucky” $ bucky run -case case_1 Data base Save test result Execute test TestC ode TestC ode Test Case Page object Element Locater Element Location Page object Make test report Bucky-core Bucky-management

Slide 34

Slide 34 text

Slide 35

Slide 35 text

Slide 36

Slide 36 text

© LIFULL Co.,Ltd. 本書の無断転載、複製を固く禁じます。 36 Solutions. 1. Too many test cases -> Test plan 2. Too flaky -> Retest 3. Maintainability was too low-> Test code in YAML Solution 2 New testing framework “Bucky”

Slide 37

Slide 37 text

Slide 38

Slide 38 text

© LIFULL Co.,Ltd. 本書の無断転載、複製を固く禁じます。 38 What’s YAML? https://yaml.org/spec/1.2/spec.html Solution 2 New testing framework “Bucky” # YAML example american: - Boston Red Sox - Detroit Tigers - New York Yankees national: - New York Mets - Chicago Cubs - Atlanta Braves

Slide 39

Slide 39 text

© LIFULL Co.,Ltd. 本書の無断転載、複製を固く禁じます。 39 Solution 2 New testing framework “Bucky” - case_name: favorite_1 func: Save to favorite. desc: Save Real Estate to favorite from detail page procs: - proc: Open Real Estate’s detail page. exec: operate: go url: www.homes.co.jp/chintai/b-0000000000000/ - proc: Save to favorite. exec: operate: click page: bukken_detail part: favorite - proc: Check whether target Real Estate is saved as favorite. exec: verify: assert_exist_part page: bukken_detail part: complete_favorite

Slide 40

Slide 40 text

© LIFULL Co.,Ltd. 本書の無断転載、複製を固く禁じます。 40 Test code in YAML ● Very simple ○ No programming skills dependence ○ No if and loop ○ Reduce complexity -> Improved maintainability -> Stabilization test execution Solution 2 New testing framework “Bucky”

Slide 41

Slide 41 text

© LIFULL Co.,Ltd. 本書の無断転載、複製を固く禁じます。 41 Solution 2 New testing framework “Bucky” # Open Real Estate’s detail page. self.driver.get('www.homes.co.jp/chintai/b-0000000000000/’) # Save to favorite. page = bukkenluist.Page(self.driver) page.favorite.click # Check whether target Real Estate is saved as favorite. self.verify.is_true(self.is_exist_part(page, 'complete_favorite’)) def is_exist_part(page, part_name): try: getattr(page, part_name) return True except NoSuchElementException: return False except Exception as e: raise Exception(e.message) - case_name: favorite_1 func: Save to favorite. desc: Save Real Estate to favorite from detail page procs: - proc: Open Real Estate’s detail page. exec: operate: go url: www.homes.co.jp/chintai/b-0000000000000/ - proc: Save to favorite. exec: operate: click page: bukken_detail part: favorite - proc: Check whether target Real Estate is saved as favorite. exec: verify: assert_exist_part page: bukken_detail part: complete_favorite Bucky-core Old（Python)

Slide 42

Slide 42 text

Slide 43

Slide 43 text

Slide 44

Slide 44 text

Slide 45

Slide 45 text

Slide 46

Slide 46 text

© LIFULL Co.,Ltd. 本書の無断転載、複製を固く禁じます。 46 Bucky have many useful functions! We published as OSS. Please take a look and have a try of Bucky! https://github.com/lifull-dev/bucky-core https://github.com/lifull-dev/bucky-management

Slide 47

Slide 47 text